CKAN: Ticket Query

#2231: Base harvester broken on 1.6.1

amercader — Thu, 15 Mar 2012 12:44:59 GMT

On its current version, ckanext-harvest passed a custom schema to package_create and package_update to allow defining an id for new packages (the same as in remote ckan instances) [1]

Recent changes in the logic layer have made that the context object is no longer directly read from the context, but only get from plugins [2]. This makes the schema sent from ckanext-harvest to be ignored and the harvesting fails because the id field is not expected

ValidationError: {'Id': u'The input field id was not expected.'}

We need a way to either:

Be more flexible accepting schemas on the logic functions
Modify the harvester plugin to implement the necessary interface

[1] https://github.com/okfn/ckanext-harvest/blob/master/ckanext/harvest/harvesters/base.py#L115

[2] https://github.com/okfn/ckan/blob/0c07d3be846777318b35c31325215dff00e99313/ckan/logic/action/create.py#L38

#2266: Can't delete all of a package's resources over REST API

dread — Wed, 28 Mar 2012 11:01:44 GMT

Nothing happens if you set resources=[] or resources=null.

#1616: Catch exceptions when rebuilding the search index

amercader — Fri, 06 Jan 2012 10:11:09 GMT

Right now if an exception is found while reindexing, the whole process stops and the remaining datasets are left out of the index. The process should continue after logging the exception. If more than a certain number of exceptions occur in a row, the process should stop.

#1559: Comments Extension / Disqus

jilly mathews — Mon, 19 Dec 2011 10:48:40 GMT

Polish off comments extension dev and test. estimate 2 days.

#2240: Document CityData ideas

ross — Mon, 19 Mar 2012 15:09:21 GMT

Document the ideas raised on 14/03 for CityData? and distribute to team for discussion

#2249: Fix google spreadsheet to work with DataStore.

kindly — Wed, 21 Mar 2012 11:44:02 GMT

Need google spreadsheet app to work with the new datastore. This should simplify it a lot.

#2241: Look into making test_activity.py faster

seanh — Mon, 19 Mar 2012 15:52:34 GMT

...but only if it can be done reasonably quickly.

#2242: Research buildbot possibilities for CKAN

seanh — Mon, 19 Mar 2012 15:53:25 GMT

#2224: Simplify javascript and css dependencies and add minified version

rgrp — Sat, 10 Mar 2012 13:20:23 GMT

Get rid of jquery UI
- Requires switching to bootstrap lookahead plus a small bit of coding for ajax part (see https://gist.github.com/2007241)
Remove modernizr (what do we use it for?)
Convert jquery.tmpl to jquery mustache (?)
Remove ckanjs moving core parts of it directly into our code.

Compress everything for releases and add minified version to build (with switch to enable use of this?)

#2216: Translation of selected facets on search pages

seanh — Wed, 07 Mar 2012 19:30:48 GMT

#2232: Unicode Exception when rebuilding the search index

amercader — Thu, 15 Mar 2012 13:03:02 GMT

In some cases this exception is fired when building the search index. As explained in #1616 this makes the whole process stop.

  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/solr/core.py", line 326, in wrapper
    return self._update(content, query)
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/solr/core.py", line 550, in _update
    rsp = self._post(selector, request, self.xmlheaders)
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/solr/core.py", line 639, in _post
    return check_response_status(self.conn.getresponse())
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/solr/core.py", line 1096, in check_response_status
    raise ex
SolrException: HTTP code=400, reason=ParseError at [row,col]:[1,2354] Message: An invalid XML character (Unicode: 0x1) was found in the element content of the document.
Traceback (most recent call last):
  File "/var/lib/ckan/pdeu/pyenv/bin/paster", line 9, in <module>
    load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 84, in run
    invoke(command, command_name, options, args[1:])
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 123, in invoke
    exit_code = runner.run(args)
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 218, in run
    result = self.command()
  File "/var/lib/ckan/pdeu/pyenv/src/ckan/ckan/lib/cli.py", line 298, in command
    rebuild()
  File "/var/lib/ckan/pdeu/pyenv/src/ckan/ckan/lib/search/__init__.py", line 134, in rebuild
    {'id': pkg.id}
  File "/var/lib/ckan/pdeu/pyenv/src/ckan/ckan/lib/search/index.py", line 54, in insert_dict
    return self.update_dict(data)
  File "/var/lib/ckan/pdeu/pyenv/src/ckan/ckan/lib/search/index.py", line 79, in update_dict
    self.index_package(pkg_dict)
  File "/var/lib/ckan/pdeu/pyenv/src/ckan/ckan/lib/search/index.py", line 153, in index_package
    raise SearchIndexError(e)
ckan.lib.search.common.SearchIndexError: HTTP code=400, reason=ParseError at [row,col]:[1,2354] Message: An invalid XML character (Unicode: 0x1) was found in the element content of the document.

#2248: Update docs for creating custom forms

johnglover — Wed, 21 Mar 2012 11:17:10 GMT

Should describe implementing IDatasetForm, subclassing controllers is deprecated

#2245: Update documentation for writing CKAN extensions

johnglover — Mon, 19 Mar 2012 17:35:46 GMT