CKAN: Ticket #695: Search indexing broken on ckan.net

description changed

dread — Wed, 13 Oct 2010 17:34:09 GMT

description modified (diff)

pudo — Mon, 18 Oct 2010 08:49:11 GMT

This is triggered by an issue where having local blinker notifications without asynchronous notifications will break indexing. The issues cause is still unknown, but one possible fix is running CKAN with a queue enabled, synchronous indexing on and no queue consumers attached. Not a real solution, but made possible in cset:a065dbc8041c

status changed; resolution set

pudo — Mon, 18 Oct 2010 10:57:36 GMT

status changed from new to closed
resolution set to wontfix

status changed; resolution deleted

ollyc — Thu, 21 Oct 2010 15:22:17 GMT

status changed from closed to reopened
resolution wontfix deleted

ollyc — Thu, 21 Oct 2010 16:06:09 GMT

This is down to blinker not persisting the signal objects. Although the docs claim:

"Every call to signal('name') returns the same signal object, allowing unconnected parts of code (different modules, plugins, anything) to all use the same signal without requiring any code sharing or special imports"

This isn't true unless you maintain a reference to the signal object. To demonstrate this:

>>> import gc
>>> from blinker import signal
>>> signal_id = id(signal('Package'))
>>> gc.collect()
0
>>> id(signal('Package')) == signal_id
False

The synchronous notifications code connects to signals without storing the signal objects, which are subsequently garbage collected and hence never fire. The async notifications stores references to all signals as a class attribute, so this problem is not seen when async notifications are enabled.

dread — Thu, 21 Oct 2010 16:11:21 GMT

Superb finding Friedrich! Any chance of a quick fix for metastable then?

dread — Thu, 21 Oct 2010 16:12:04 GMT

Sorry, I meant Olly!

ollyc — Fri, 22 Oct 2010 11:50:40 GMT

Fix and tests for this are here:

status changed; resolution set

dread — Fri, 22 Oct 2010 17:02:53 GMT

status changed from reopened to closed
resolution set to fixed

These changesets do the trick. I've:

merged Ollie's changes into default and metastable branches.
updated ckan.net to latest metastable and reindexed it.
ckan.net now searches on latest packages, including sanskrit example.
test-hmg.ckan.net also updated to latest metastable for testing with UAT.
ckanhmg is to be updated next week and DGU ticket remains for this.