Search indexing broken on ckan.net

[dread]

e.g. searching for 'buddhist' or 'sanskrit', you don't get this newly created package: ​http://ckan.net/package/digitalsanskritbuddhistcanon

[pudo]

This is triggered by an issue where having local blinker notifications without asynchronous notifications will break indexing. The issues cause is still unknown, but one possible fix is running CKAN with a queue enabled, synchronous indexing on and no queue consumers attached. Not a real solution, but made possible in cset:a065dbc8041c

[ollyc]

This is down to blinker not persisting the signal objects. Although the docs claim:

"Every call to signal('name') returns the same signal object, allowing unconnected parts of code (different modules, plugins, anything) to all use the same signal without requiring any code sharing or special imports"

This isn't true unless you maintain a reference to the signal object. To demonstrate this:

>>> import gc
>>> from blinker import signal
>>> signal_id = id(signal('Package'))
>>> gc.collect()
0
>>> id(signal('Package')) == signal_id
False

The synchronous notifications code connects to signals without storing the signal objects, which are subsequently garbage collected and hence never fire. The async notifications stores references to all signals as a class attribute, so this problem is not seen when async notifications are enabled.

[dread]

Superb finding Friedrich! Any chance of a quick fix for metastable then?

[dread]

Sorry, I meant Olly!

[ollyc]

Fix and tests for this are here:

• ​http://bitbucket.org/ollyc/ckan/changeset/48234e36ee61
• ​http://bitbucket.org/ollyc/ckan/changeset/f1447bbc9d65
• ​http://bitbucket.org/ollyc/ckan/changeset/1d3cb5378a2f
• ​http://bitbucket.org/ollyc/ckan/changeset/e8409c84683d
• ​http://bitbucket.org/ollyc/ckan/changeset/8e89cbce15c7
• ​http://bitbucket.org/ollyc/ckan/changeset/d1d63db64585
• ​http://bitbucket.org/ollyc/ckan/changeset/3d9eb4172f48

[dread]

These changesets do the trick. I've:

• merged Ollie's changes into default and metastable branches.
• updated ckan.net to latest metastable and reindexed it.
• ckan.net now searches on latest packages, including sanskrit example.
• test-hmg.ckan.net also updated to latest metastable for testing with UAT.
• ckanhmg is to be updated next week and DGU ticket remains for this.