{22} Trac tickets (2647 matches)

Results (1401 - 1500 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1378 enhancement zephod zephod ckan-sprint-2011-10-24 closed fixed New to TheDataHub? - StackOverflow style banner

Implement a banner across the top of the screen giving newbies a link to the about page, and making them feel welcome. See stackoverflow for reference?

1318247329000000 1324047835000000
#1379 defect zephod zephod ckan-sprint-2011-11-07 closed fixed Markdown conversion of linked references broken

eg. http://thedatahub.org/dataset/us-gov-tarp

[This Paper][my-paper] is very interesting.

[my-paper]: http://somelink.com/paper

Write a test to check this markdown feature works - if not, use external markdown lib rather than webhelpers.markdown. Probably need to add markdown lib to list of requirements.

1318247719000000 1320161439000000
#1380 enhancement zephod zephod ckan-sprint-2011-11-07 closed fixed Making customizing CSS easier

While integrating the new theme and making a large number of UX tweaks, I've allowed the CSS file to bloat out of control with some very specific rules and subtle hacks, as well as a couple of clearfix <div> objects in the markup which isn't really good practice.

In general we want a logical CSS structure which is as easy to modify as possible. You should not find yourself being overwritten by highly specific rules all the time!


  1. Clean up css
  2. Add a config option ckan.template_head_end - arbitrary string inserted at end of <head> tag so you can add custom css etc (may also want to update theming docs to reflect possibility of using this)
  3. (While we're at it): Add ckan.template_footer_end config option to deployment.ini_template. (This already exists, we just haven't told anybody)
  4. The /users index page isn't linked anywhere. Clean it up and link it in the footer next to Groups etc.
  5. package/new, package/edit, group/new and group/edit all include their forms differently. Use h.literal in all cases.
  6. Fix the padding on minornavigation.


David Read sent me the following as background:

  1. basic theming is described here: http://docs.ckan.org/en/latest/theming.html

Deep customisation of the templates is done by creating an extension, with this as a template: https://bitbucket.org/okfn/ckanext-exampletheme/overview

That's what Augusto and the Brazilians have done http://alpha.dados.gov.br/dados/ and their code is here: hg clone http://dev.dados.gov.br/codigo/dev/tema-ckan <snip> Another thing of interest was us becoming compatible with Wordpress themes last December. Richard Pope worked with Rufus on this. I believe the base theme is 'twentyten' and I'm guessing that one would swap that css file with another to change theme, but I don't know. I don't know if this feature has been dropped since then. http://lists.okfn.org/pipermail/ckan-dev/2010-December/000073.html


1318263956000000 1324048324000000
#1381 enhancement amercader pudo ckan-sprint-2011-10-24 closed fixed Managing groups via package entity API

It is not currently possible to assign or remove groups by adding or removing their group name from the "groups" list on the REST API. This should be allowed, as it is easier than first adding a package and then editing each group it must belong to.

1318266706000000 1324037815000000
#1383 enhancement kindly kindly ckan-sprint-2011-10-24 closed fixed Add IResourceChange IResourceUrlChange Interfaces.

Need to add way for extensions to trigger based on resource change and just resource url change.

1318342153000000 1320142305000000
#1385 enhancement dread dread closed fixed Resolve postgres permissions issues

Currently there is a problem because the docs guide us to set the sqlalchemy url to use 'localhost' i.e. loopback, whereas paster commands don't specify '-h localhost' so use unix sockets (you need to do 'sudo -u postgres'). These should be the same.

Also do we need to tell people to add a line to their postgres authentication config /etc/postgresql/8.4/main/pg_hba.conf to help things? Florian suggests:

local   std         std                          md5


1318418537000000 1326218703000000
#1386 enhancement rgrp rgrp ckan-sprint-2011-10-24 closed fixed Disallow account creation via openid

Superticket: #1343

Creation of accounts from OpenID causes large number of problems:

  • No guarantee of other profile info
  • Poor username (just from openid)

We therefore will:

  1. Require creation of user account via register (no auto-creation of accounts via openid)
    • sidebar of register page should point to login page rather than openid signin

  1. Permit association of an openid with an account ...
    • How: paste in your openid url (requires change to edit form to have this)
      • won't work for google with their weird openid urls
    • [future] Nicer way is to have login via openid while logged in (which association then happening in background)
  1. Migration (for 1+2):
    • (?) Generate decent usernames for all existing accounts
    • As this is nontrivial suggest instead we allow editing of usernames (by account owner and sysadmins). This is useful in its own right and is a reasonable 80/20 solution.
      • This does present the problem of re-associating commits with the new username. Best solution to this would be to switch revisions to point to userid rather than username. See #1534
  1. [optional] Change display_name to always be username
    • Change My Account to Username at top right of all pages (once we know usernames are short ...)
      • could do this straight away by truncating long usernames (e.g. truncate at 20 chars ...)
      • Also should we lose the icon?
1318425493000000 1324036930000000
#1387 defect toby johnglover ckan-v1.8 closed fixed Not specifying a protocol in dataset url causes broken link to be served on dataset view page

If a user enters a url with no protocol (eg: www.data.gov.uk) in the 'url' field of a dataset, a broken link is created on the view page.

CKAN should probably either put in a http:// if no protocol is given, or validation should fail.

1318428893000000 1343042311000000
#1388 defect dread ckan-backlog closed fixed etags caching on home page

Needs to update on:

  1. language - It's broken on thedatahub.org if you change to German and then click on the CKAN icon top-left to go to thedatahub.org again.


  1. whether signed in or not - #1373

BUT not latest revisions (which is what it was)

Or get rid of etag caching on this page and others?

1318429443000000 1320174229000000
#1389 defect johnglover johnglover ckan-sprint-2011-10-24 closed fixed Update ckanext-datacatalogs for ckan 1.5 1318497052000000 1319625799000000
#1390 enhancement johnglover johnglover ckan-sprint-2011-10-24 closed fixed Update ckanext-moderatededits for ckan 1.5 on datacatalogs.org 1318497099000000 1319625810000000
#1391 enhancement johnglover johnglover ckan-sprint-2011-10-24 closed fixed Update datacatalogs.org to ckan 1.5 1318497186000000 1319625819000000
#1392 enhancement johnglover johnglover ckan-sprint-2011-11-07 closed fixed Move datacatalogs.org to an OKFN server 1318497251000000 1320143150000000
#1394 defect dread dread ckan-sprint-2012-01-09 closed fixed Resource validation error messages misleading

(Editing a dataset) If the second resource contains any validation error then it says "Resources: Package resource(s) incomplete" and "Resource 1:".

1318515262000000 1325604784000000
#1395 defect seanh closed fixed "Import Error: cannot import name UnicodeMultiDict" when installing ckan from source

At the paster db init command when installing ckan from source I get the error "Import Error: cannot import name UnicodeMultiDict?" (happens with both ckan 1.4.2 and today's latest bleeding edge code, on Ubuntu 10.04.3).

UnicodeMultiDict? has been removed in a recent version of python-webob, and the pip install ... lucid_missing.txt causes a too-new version of python-webob to be installed into ckan's virtualenv (the new webob gets installed as a dependency of formalchemy).

I manually did pip uninstall webob and then ran paster db init again and it worked.

1318520183000000 1320857823000000
#1396 enhancement rgrp rgrp ckan-sprint-2012-02-06 closed fixed Improvements to user page UX - 1d

Super ticket: #1506

  • Show member since
  • Display email (in obfuscated form)
  • List all owned datasets (in standard manner)
    • Will need change to logic layer to show datasets on user dict (this is useful for the API as well ...)
  • (Re)move list of changes
    • Should have activity (once activity available)


  • ?? Allow users to provide a home page, organization (requires migration)
1318527967000000 1330086237000000
#1397 enhancement kindly rgrp ckan-v1.7 closed fixed [super] Resource archiving

We want to cache/archive data associated to a resource so it is available if the resource url disappears (and in order to support other processing we may wish to do e.g. webstorer ...)

Etherpad: http://ckan.okfnpad.org/queue (most relevant parts inlined here)


  • Add task_status table to store qa/archiever/webstore information that does not need to be versioned. - #1363 (and #1371 - related logic functions)

Configuration setup for daemons

Pass config through to workers i.e site_url, user, api_key. Need to make site user account. #1408

celeryd config:

All providers of tasks will add an item to the following entry point:

name = ckanext.{name}.tasks:....


from pkg_resources import iter_entry_points
for entry in iter_entry_points:

CELERY_IMPORTS = celeryimports

Work Items

  • Resource change notifications in core - Make an IResourceChange and IResourceUrlChange. [1d] [0.75d] - #1383
  • Generate archiving request on resource url change [0.25d][0.25d] - #1399
  • Make site user account.
  • Make entry point system for celery config
  • Archiver daemon #891
    1. implement link-check function and task (point 2 from Archiver.update above) [1d] [0.5d]
    2. Rewrite archiver to use external storage. (decide how!)[3d][~2d]
  • Write to resource and task status table.[1d][0.75d]
  • [Required?] Make archived data available in WUI - #892
  • Documentation - #1400
1318529593000000 1338202920000000
#1398 enhancement kindly rgrp ckan-sprint-2011-11-21 closed fixed Automated conversion of resource data into webstore
  • Possible sibling / requirement: #1397 (resource archiving)
  • Setup test environment. 1d
  • Make task to upload files to webstorer. 3d
    • This includes writing back webstore_url and updated date. 0.5d
    • Type guessing of file format. 1d
    • Make cron to upload all un webstored resources. 0.5d
  • Testing for errors by running cron 2d
  • Make nice way to deploy celery 1d.
  • Documentation. 1d
  • Deployment. 1d
1318529724000000 1321827970000000
#1399 enhancement johnglover rgrp ckan-sprint-2011-10-24 closed fixed Generate archiving request on resource url change

Superticket: #1397

Generate archiving request on resource url change by implementing IResourceUrlChange and sending tasks to celery. [0.25d][0.25d]

Will be part of of ckanext-archiver extension.


  1. A resource is added to CKAN
  2. IResourceCreate event generated
  3. IF: resource url points to ckan storage or falls within some other set of exclusion conditions then END else continue
  4. Generate a Archiver.Update task with resource.id
1318600123000000 1319625829000000
#1400 enhancement johnglover rgrp ckan-sprint-2012-03-05 closed fixed Document Archiver Extension
  • Create page on http://wiki.ckan.org/ about setting up a queue (this will merge into core docs when this is stable) - having a queue will now become a generic requirement for a bunch of extensions
  • README with install instructions in ckanext-archiver
  • Create an entry in http://wiki.ckan.org/Extensions
  • Write a blog post
1318600705000000 1331544816000000
#1401 defect rgrp flofokus ckan-sprint-2012-01-23 closed fixed Some links don't take account when CKAN is mounted at a non-root URL

if you want your ckan to reside not at http://yourhost.org/ but rather e.g. at http://yourhost.org/ckan/ you can use this hint http://wiki.pylonshq.com/display/pylonsdocs/Configuration+Files and set ckan.site_url accordingly in your .ini. However, there are some instances where internal links are created without the required prefix.

1318621586000000 1328001014000000
#1402 enhancement kindly rgrp ckan-v1.6 closed fixed Migrate repository from mercurial to git

Plan to migrate from mercurial git


  • Do trial run
  • Announce conversion date / time
  • Require everyone to have pushed all outstanding changes at that time
  • Do conversion
  • Test
  • Announce on list and ckan.org/
1318811651000000 1324334011000000
#1404 enhancement zephod zephod ckan-v1.7 closed wontfix Search Page UI improvements

[Refactored] :: Follows on from #1506 UX changes.

  • Declutter the sidebar. No yellow box.
  • Facets to go on the left, rather than the right. More logical flow.
  • Did you know you can search by author? Probably not. Find a nice way of presenting extended search options.
  • Make Datasets in the search page look more like Datasets on the groups pages (ie. like awesome sexy search results).
    • Update that look-and-feel to include the new resource icons created in #1506
1318847818000000 1338203639000000
#1405 defect dread [email protected] ckan-sprint-2011-11-07 closed fixed You can add a package twice to a group

To reproduce (from Jonathan Gray):

  • I go to add an item to the group
  • As I type, I get autocomplete options (cool!)
  • When I click on one, it adds this, and copies the same autocomplete option in the box
  • When I click submit, it seems to add the item twice
  • On the main group page I think it just shows up once, but when I click edit it shows two identical boxes, both ticked - and the count includes two copies of the same item

Basically the problem is allowing the same package associated multiple times with a group.

This is an issue on CKAN 1.5b and probably earlier.

1318856354000000 1320152291000000
#1407 defect rgrp dread ckan-sprint-2011-12-19 closed fixed Stats extension not working

Graphs don't show any more at http://thedatahub.org/stats

1319014605000000 1323764239000000
#1408 enhancement kindly kindly ckan-sprint-2011-10-24 closed fixed Make site user, so tasks can post information back to ckan.

Give site user system administrative rights.

1319019626000000 1320141847000000
#1409 enhancement zephod zephod ckan-sprint-2011-11-07 closed fixed Add Gravatars to user profiles

Since we now require email addresses, it's sensible to request users' gravatars to add a little flavour to their user profile (and, potentially, other places eg. comments threads?)

[Reopened: Additional] Use gravatars in the HTML generated by helpers.py to create lists of users. Eg. on user/list and dataset/history.

1319200773000000 1324047722000000
#1410 enhancement zephod zephod ckan-v1.5 closed duplicate Add Gravatars to user profiles

Since we now require email addresses, it's sensible to request users' gravatars to add a little flavour to their user profile (and, potentially, other places eg. comments threads?)

1319200774000000 1319213205000000
#1412 defect rgrp seanh ckan-sprint-2012-02-06 closed worksforme application.js crashes when viewing a dataset with no resources

Create a new dataset, don't add any resources to the dataset yet, view the dataset in ckan, if you have firebug enabled you'll see the error "resource is undefined" at line 699 of application.js.

1319450625000000 1328226385000000
#1413 enhancement seanh shevski ckan-backlog closed fixed Ask users to 'add email address' when logged in

I'd like us to display a one-off banner that prompts users who log into thedatahub.org to go and update & their profile & fill in their email address (if we could do it just for those who don't have an email address, then even better) That is, once they log in, they see a banner which says: Please update your profile [here] and add your email address, so you can receive notifications, be able to reset your password and get updates

Then when we build notifications or if we decide to survey people, we can actually contact them. Email address should be required as standard

DR: Also, mention their Full Name too - v. useful for selecting correct user in group curation

1319453108000000 1330020486000000
#1415 enhancement thejimmyg nils.toedtmann ckan-sprint-2011-12-05 closed fixed Comments on current status of ckan deb packages

This is a scratch pad ticket with some comments on the current status of our ckan deb packages. I know that some of it is the deb packaging roadmap anyway, please forgive me if i mention them here again.

Rufus and me re-deployed some community ckan instances onto s022 (see http://trac.okfn.org/ticket/926). We followed the documentation http://docs.ckan.org/en/latest/install-from-package.html

  • Deb package version number: the version of the deb package is "python-ckan 1309471251~149be76faabc+lucid-1", and it's hard to guess from there that it contains a ckan 1.4.2a
  • When is 1.4.3/1.5.x expected as deb package?
  • There was a bug in the DB upgrade script /usr/share/pyshared/ckan/migration/versions/029_version_groups.py (line 150) which looks like it was fixed 1.4.1==>1.4.2 but was nevertheless present in this deb package.
  • The current script /usr/bin/ckan-std-install
    • does not set the Apache ServerName? according to the $INSTANCE variable
    • automatically configures a ckan extension named after $INSTANCE
    • depends on local postgres
    • could be replaced with "/usr/bin/ckan-deploy --name=ckan-std --domain=ckan-std.localhost (see next point)
  • (i think this is exactly James' plan): have more generic deployment script /usr/bin/ckan-deploy as part of python-ckan which takes arguments like
    • --domain=cc.ckan.net
    • --aliases=$list-of-domains
    • --name=cc (defaults to "domain")
    • --no-db (does not configure a DB)
    • --sql-alchemy=$DB_CONFIG_STRING (also runs "paster --plugin ckan db upgrade --config")
    • --extension $list-of-extesions
    • ...
1319457069000000 1323167941000000
#1416 defect dread dread ckan-sprint-2011-11-07 closed fixed Registering with blank email can cause exception

This might well be a bot, but not POSTing the email field should not cause an exception - it should be 400 and then the sysadmin doesn't get these nags.

Module ckan.controllers.user:79 in register
<<              c.login = request.params.getone('login')
                   c.fullname = request.params.getone('fullname')
                   c.email = request.params.getone('email')
                   if not model.User.check_name_available(c.login):
                       h.flash_error(_("That username is not available."))
>>  c.email = request.params.getone('email')
Module webob.multidict:335 in getone
<<          values were found.
               return self._decode_value(self.multi.getone(self._encode_key(key)))
           def mixed(self):
>>  return self._decode_value(self.multi.getone(self._encode_key(key)))
Module webob.multidict:99 in getone
<<          v = self.getall(key)
               if not v:
                   raise KeyError('Key not found: %r' % key)
               if len(v) > 1:
                   raise KeyError('Multiple values match %r: %r' % (key, v))
>>  raise KeyError('Key not found: %r' % key)
KeyError: "Key not found: 'email'"
HTTP_USER_AGENT	'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MRA 4.3 (build 01218); .NET CLR 1.1.4322)'
webob._parsed_post_vars	(MultiDict([('login', 'ahusernames9'), ('fullname', 'ahusernames9'), ('password1', '******'), ('password2', '******'), ('s', 'Sign up')]), <FakeCGIBody at 0xb9d7c5ec viewing MultiDict([('lo...p')])>)
1319533989000000 1320153507000000
#1417 defect dread dread ckan-sprint-2011-10-24 closed fixed Browser language detection doesn't work

In Firefox:

  • Edit | Preferences | Content | Languages - put 'de' at top language
  • Clear cookies
  • Load thedatahub.org
  • it should be in German, but it's in the default language - English.
1319539010000000 1319651617000000
#1418 defect dread dread ckan-sprint-2011-10-28 closed duplicate If default language is not 'en' then you can't switch language to 'en'

e.g. cz.ckan.net has lang=cz but when you click on Home page link "English" it flashes message "Language has been set to: English" but the rest of the page is still Czech.

1319539146000000 1319539691000000
#1419 enhancement dread ckan-sprint-2011-11-07 closed invalid Can't log in via OpenID

I couldn't log into theDataHub with OpenID today. I tried both Google ID and MyOpenID. Both times the login on the remote auth server went fine, but when it returns you to theDataHub you get error "Login failed. Bad username or password."

1319543013000000 1319796164000000
#1420 enhancement johnglover johnglover ckan-sprint-2011-11-21 closed fixed Refactor ckanext-qa to work as a celery task

Currently can run as a celery task launched by paster command.

Still have to add a CKAN plugin so that QA will run when a resource is added or URL changed, and then update the extension controller/templates to look for QA data from the task_status table instead of directly on the resource.

1319626138000000 1320843789000000
#1421 defect dread dread ckan-sprint-2011-10-28 closed fixed Tag cloud recipe broken

home page doesn't have c.facets setup.

Only affects 1.5 branch - ok Was broken in cset:05f60f2d26e8.

1319638575000000 1319639311000000
#1422 enhancement dread dread closed fixed Remove proxy & etag caching

Rufus decided this was more trouble than it was worth.

Removes ckan/lib/cache.py

1319639663000000 1319645722000000
#1425 enhancement zephod dread closed fixed Dataset delete button

From Jonathan gray:

At the moment to delete a package you have to go to 'state' and change from 'active' to 'deleted'.

For me this was not very obvious, and someone recently wrote to me about this too.

Its a small issue - but unless you know what 'state: active' refers to, its not obvious what this means, nor that this is where to go if you want to delete a package.

From UI perspective this could be replaced by either a trash can or some text that says 'delete', perhaps with some prompt that explains what happens when you delete a package - explaining that non-admins won't be able to see it, and how you purge, etc.

1319649860000000 1330085282000000
#1426 defect dread dread ckan-sprint-2011-10-28 closed invalid Validation errors cause 409 not 400 response

Validation error is incorrect arguments, so should be a "400 Bad Request" response. "409 Conflict" is for when there is a conflict creating two resources with the same identifier. Yet we get 409 for all validation errors.

1319708619000000 1319709236000000
#1427 enhancement dread dread ckan-sprint-2011-10-28 closed fixed Tags validation needs to limit length to 100 characters

The tag name is limited in length in the db table to 100 characters. If you try and create a longer one you get an exception. This should cause a ValidationError? and tell the user this sensibly.

1319708698000000 1319709925000000
#1428 defect dread dread ckan-sprint-2011-11-07 closed fixed config options not used or documented

Need to ensure config options in the deployment.ini_tmpl match up with doc/configuration.rst. Remove any unused ones too.

1319803594000000 1319804773000000
#1430 defect amercader ckan-sprint-2011-11-07 closed fixed Documents get mixed between SOLR cores

On some occasions (apparently random), the documents indexed in a specific SOLR core get mixed with different site_ids.

E.g: We look for all documents in the testing.iatiregistry.org core, faceted by site_id. We would expect all documents to have site_id = iati_testing, but some of them have site_id = iatiregistry.org


<lst name="facet_fields">
<lst name="site_id">
<int name="iati_testing">265</int>
<int name="iatiregistry.org">255</int>

If we compare one of the records which disappeared from the "iati_testing" site_id in both the production and testing SOLR cores of the server, the records are exactly the same, including the indexed_ts property:



Note that the response from the URLs shown may vary, as the testing site could have been reindexed.

1320068076000000 1324033923000000
#1431 defect dread dread ckan-v1.5 closed fixed Captcha field - foreign chars cause exception

During registering a user, the user inputs foreign chars into the captcha field.

URL: http://thedatahub.org/user/register
Module ckan.lib.captcha:22 in check_recaptcha
<<                                     remoteip=client_ip_address,
           f = urllib2.urlopen(recaptcha_server_name, params)
           data = f.read()
>>  response=recaptcha_response_field))
Module urllib:1267 in urlencode
<<          for k, v in query:
                   k = quote_plus(str(k))
                   v = quote_plus(str(v))
                   l.append(k + '=' + v)
>>  v = quote_plus(str(v))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xea' in position 0: ordinal not in range(128)
1320078849000000 1320084104000000
#1433 enhancement kindly rgrp ckan-sprint-2011-11-21 closed fixed Support SQLAlchemy 0.7

Why: current stable version of sqlalchemy. geoalchemy stuff required 0.7 and likely that some other things will require it soon.

Probably requires work on vdm https://bitbucket.org/okfn/vdm

NB: should have discussion before making 0.7 the default required version in CKAN core.

1320143453000000 1324472583000000
#1434 enhancement seanh dread ckan-v1.8 closed fixed Add i18n strings from extensions

Decide which extensions are important.

1320145648000000 1341245889000000
#1435 enhancement rgrp ckan-v1.6 closed wontfix Switch to continuous.io for buildbot (?) 1320145831000000 1323283538000000
#1436 defect johnglover johnglover ckan-sprint-2011-11-07 closed fixed Packages with a 'pending' state not listed on groups page 1320155227000000 1320243278000000
#1437 defect dread dread ckan-sprint-2011-11-07 closed fixed JSONP parameter in Action API

Action API needs JSONP support - be able to return responses encapsulated in a function of a supplied name. This is important for remote sites running javascript to interact with a CKAN site.

Specifying the callback parameter is the way we've achieved JSONP with the RESTful and Search APIs. It should work like this:

curl http://test.ckan.net/api/action/package_show?callback=jsoncallback -d '{"id": "fd788e57-dce4-481c-832d-497235bf9f78"}'

Or maybe the callback should be specified in the payload in the context or data_dict?

(My understanding is that CORS is similar - when more browsers support it, can we drop JSONP?)

1320161088000000 1320173795000000
#1440 defect dread dread ckan-sprint-2011-11-07 closed fixed 'paster db load' doesn't update search index

paster db load ckan.pgdump will load in the database but not update the search index, no we're using SOLR.


We can't tack on 'rebuild' to this command, because it needs a 'db upgrade' first. I'm tempted to make 'db load' do the upgrade and rebuild all in one command.

1320232415000000 1320235084000000
#1441 defect dread dread closed fixed Use local ckan logo image

Using local image rather than the same image at assets.okfn.org reduces dependency on another server that may be down, slow or remote. James agrees.

1320234597000000 1320235062000000
#1442 defect dread dread ckan-sprint-2011-11-07 closed fixed You can't GET then POST a dataset as new

If you try to GET a dataset, purge it then POST it again, it fails checking authz for the group:

Module ckan.controllers.api:278 in create
<<                  (register, subregister))
                   response_data = action(context, data_dict)
                   location = None
                   if "id" in data_dict:
>>  response_data = action(context, data_dict)
Module ckan.logic.action.create:240 in package_create_rest
<<      api = context.get('api_version') or '1'
           check_access('package_create_rest', context, data_dict)
           dictized_package = package_api_to_dict(data_dict, context)
>>  check_access('package_create_rest', context, data_dict)
Module ckan.logic:129 in check_access
<<          #    log.debug('Valid API key needed to make changes')
               #    raise NotAuthorized
               logic_authorization = new_authz.is_authorized(action, context, data_dict)
               if not logic_authorization['success']:
                   msg = logic_authorization.get('msg','')
>>  logic_authorization = new_authz.is_authorized(action, context, data_dict)
Module ckan.new_authz:16 in is_authorized
<<      auth_function = _get_auth_function(action)
           if auth_function:
               return auth_function(context, data_dict)
               raise ValueError(_('Authorization function not found: %s' % action))
>>  return auth_function(context, data_dict)
Module ckan.logic.auth.create:115 in package_create_rest
<<          return {'success': False, 'msg': _('Valid API key needed to create a package')}
           return package_create(context, data_dict)
       def group_create_rest(context, data_dict):
>>  return package_create(context, data_dict)
Module ckan.logic.auth.create:15 in package_create
<<      else:
               check2 = check_group_auth(context,data_dict)
               if not check2:
                   return {'success': False, 'msg': _('User %s not authorized to edit these groups') % str(user)}
>>  check2 = check_group_auth(context,data_dict)
Module ckan.logic.auth.create:90 in check_group_auth
<<      groups = set()
           for group_dict in group_dicts:
               id = group_dict.get('id')
               if not id:
>>  id = group_dict.get('id')
AttributeError: 'unicode' object has no attribute 'get'

(using RESTful Model API)

The reason is the POST is expecting the group to be expressed as a dict, not the list of names that GET returns.

1320263318000000 1320275623000000
#1443 defect dread dread ckan-sprint-2011-11-07 closed fixed language detection

If language detected doesn't match any of the offered CKAN languages, you get the default language, but see an error flash up for every page, saying:

Could not change language to 'None': IOError: [Errno 2] No translation file found for domain: 'ckan'

Affects 1.5b only.

1320425588000000 1320432511000000
#1444 defect dread dread ckan-future closed wontfix Language detection defaults to en_us?

Using curl you always get English, even if ckan.locale_default=de. Find out why. (1.5b)

Maybe we should disable locale detection for this release if lots of people's browsers don't have it set correctly (e.g. chinese with us internet explorer)

1320432634000000 1340190852000000
#1445 enhancement zephod rgrp ckan-sprint-2011-12-05 closed fixed Resource View page in WUI

Super ticket: #1032

  • Locate at: /dataset/{dataset}/resource/{id}

See: http://wiki.ckan.org/Dataset_View_Page

Implemented in branch feature-1445-resource-view.

Still to do:

  • Add inline data explorer to page
1320665049000000 1330019916000000
#1446 enhancement rgrp rgrp ckan-sprint-2012-01-09 closed fixed Data Explorer v2

Super ticket: #1602 (Data Previewer v2)

We already have first pass of Data Explorer that was released as part of #1357.

Tickets include (* indicates improvement over current explorer)

Est: 10-15d (should be broken down -- partly is in recline issues)

1320665596000000 1326281658000000
#1448 enhancement kindly kindly closed fixed Set up nice way to do celery deployment.

Celery is awkward to deply, need to find a way to do it more simply. i.e using celery-pylons and supervisor. A modified version of celery-pylons may be the best solution. 1d

1320666977000000 1325774155000000
#1449 enhancement johnglover johnglover ckan-sprint-2011-11-21 closed fixed Improved resource listing on dataset view page

Should list resources in the form of search results rather than as a table.

1320677693000000 1321873008000000
#1450 enhancement zephod johnglover ckan-sprint-2011-12-05 closed fixed Simplify dataset view page

See http://wiki.ckan.org/Dataset_and_Resource_Pages_Proposal

  • Create a collapsible section for the README
  • Reduce the number of things on the page
  • Tidy up groups and tags in the sidebar
  • Resource view becomes a list of search results


  • Resource navbar item links to first resource
  • Resource navbar has a dropdown menu of all resources in the dataset
  • Edit and Authz navbar items should be in a separate logical group to the right

Very rough summary: create more info section and move stuff out (relayout page e.g. README, tags etc)

1320677788000000 1330019974000000
#1451 enhancement johnglover johnglover ckan-sprint-2012-01-09 closed fixed Reintegrate download stats on dataset and resource view page - 0.5d
  • css class: resource-url-analytics
  • Assign to the link tag (a) everywhere we want to count (dataset, resource view) - 0.25d
  • Display counts in same place ...

And deploy on http://thedatahub.org/ - 0.25d

Possible: Also move analytics extension into core (decided not to).

1320677859000000 1324401792000000
#1452 enhancement dread dread closed wontfix Offer detected browser language, rather than auto-switch

There are issues with setting the site's language according to the browser detection:

  • Many browsers have the language setting incorrectly set. e.g. people who download US versions of Mozilla, pirated US copies in the far east.
  • Although thedatahub.org has a worldwide audience, many CKAN instances are designed for a particular country's citizens. So Open Berlin should probably be in German as default for all.

I suggest the site should have a default language saved in the config. The browser language *should* be detected, and that prompts a flash message offering to change to that language. And if you change language that is saved in the cookie (as we currently do).

Then do some testing to see if this suits people.

1320682652000000 1340190587000000
#1453 enhancement icmurray icmurray ckan-sprint-2011-12-05 closed fixed Flexible tag names

Allowing more flexible tag names:

  • allowing spaces
  • allow capital letters (search is case in-sensitive)
  • allow all punctuation except for commas and double-quotes '"'
  • allow unicode
  • commas delimit tag names in the package create/edit form


  • package creation/edit form.
  • /tag/{tagname} uri
  • search action
  • api controller (search/package-create/edit)
  • web controller (search/package-create/edit)
  • search api documentation
  • autocomplete for tag names
1320771850000000 1329395697000000
#1454 enhancement dread dread ckan-sprint-2011-11-21 closed fixed postgres search code/data removal

We made SOLR the main and only supported way to do CKAN search three months ago (#1275), but I notice that we still have remnants of the postgres full-text search hanging around:

  • model code (ckan/model/search_index.py)
  • table in the database (package_search)

The presence of these still create sqlalchemy warnings and add to our db, (& db backups etc), so it would be good to get rid of them if we are not using them. John says this can probably be removed

1320838999000000 1320849527000000
#1455 defect johnglover dread ckan-sprint-2011-12-05 closed fixed Search results when 'all_fields' don't include 'extra' fields

When you do a search like this:


the "extra" fields (e.g. "triples", "shortname") get missed off the results. The docs say it should be a "full record" and I don't see any reason why this is missed off.

This is a problem because search all_fields is the only way for clients and front-ends to get packages in bulk. They end up (like lodcloud) doing thousands of requests to get packages individually.

The full record is:

{"count": 1, "results": [{"res_description": ["Download", "XML Sitemap", "SPARQL enpdoint", "Example (RDF/XML)"], "name": "2000-us-census-rdf", "license": "Non-OKD Compliant::Creative Commons Non-Commercial (Any)", "author": "Joshua Tauberer", "author_email": "http://razor.occams.info/", "ckan_url": "http://thedatahub.org/dataset/2000-us-census-rdf", "notes": "2000 U.S. Census converted into over a billion RDF triples.\n\nPopulation statistics at various geographic levels, from the U.S. as a whole, down through states, counties, sub-counties (roughly, cities and incorporated towns)\n\nNotes: also found in the of SPARQL Endpoints.\n\nFrom home page:\n\n> * For the detailed Census statistics, you'll have to download the raw Census data files from the Census Bureau, my Perl script and the patch file below and run it yourself because the files are too big for me to offer as a download!\n> \n> * The data and scripts can be reused under Creative Commons Attribution-NonCommercial-ShareAlike.\n", "entity_type": "package", "site_id": "www.ckan.net", "download_url": "http://www.rdfabout.com/demo/census/", "indexed_ts": "2011-11-01T12:52:36.034Z", "url": "http://www.rdfabout.com/demo/census/", "state": "active", "title": "2000 U.S. Census in RDF (rdfabout.com)", "groups": ["lod", "lodcloud"], "res_format": ["", "meta/sitemap", "api/sparql", "example/rdf+xml"], "license_id": "cc-nc", "revision_id": "fcbad0de-79ea-41bd-8e01-eb832a05b732", "res_url": ["http://www.rdfabout.com/demo/census/", "http://www.rdfabout.com/sitemap.xml", "http://www.rdfabout.com/sparql", "http://www.rdfabout.com/rdf/usgov/geo/us/ny"], "id": "551ec435-f198-4d52-9b56-ec0b0be6aec9", "tags": ["census", "data", "demographics", "deref-vocab", "format-dc", "format-geonames", "format-politico", "format-rdf", "geographic", "linkeddata", "lod", "lodcloud.nolinks", "no-license-metadata", "no-provenance-metadata", "no-vocab-mappings", "population", "published-by-third-party", "rdf", "statistics", "us"]}]}
1320858265000000 1324474466000000
#1456 enhancement amercader amercader ckan-sprint-2011-11-21 closed fixed Use resource description instead of name if both are present

If a resource has both description and name the name is used. Descriptions are generally more, well, descriptive, so let's use those.

1320862619000000 1324472178000000
#1461 defect pudo closed fixed CkanClient doesn't submit auth headers for GET requests

e.g. package_register_get.

1321354037000000 1321359503000000
#1462 defect amercader thejimmyg ckan-sprint-2011-12-19 closed fixed Multi-Instance CKAN Solr Search Issues

With the 1.5 package release, CKAN instances over-write each other's indexes. Adria has a fix.

I'd like to see a new CKAN 1.5.1 release with the Solr changes merged and the Solr site ID set up as part of the install ckan.site_id = dgu-os.okfn.org. I'd also like the ability to search on ID and to have the ID directly visible on the package page.

We need to decide how to do this upgrade without affecting existing CKAN solr indexes.

1321375470000000 1330083671000000
#1463 defect thejimmyg ckan-sprint-2011-11-21 closed wontfix QA extension no-longer works with packaged CKAN 1.5

The extension needs upgrading:

No module named webstore.database

The other option is to delay this functionality and merge with the publisher dashboards after the group refactor.

John, David Raznick, David Read, Ira, what do you think?

1321375614000000 1321874259000000
#1464 enhancement thejimmyg ckan-v1.6 closed wontfix Replace RabbitMQ with Celeryd to support running multiple instances

The current harvesting implementation can only have one instance per server.

We could either:

  • Accept this
  • Change the RabbitMQ code to support multiple instances
  • Switch the entire harvesting to use Celery
  • Wait for James's redis based live feedback based system
1321375709000000 1328529042000000
#1465 enhancement thejimmyg ckan-v1.7 closed fixed Upgrade the harvester to support publishers properly

At the moment most of the harvesting functionality is only available to sysadmins. This is fine for most cases but going forward we will need to grant each publisher access to their own sources, harvest objects and harvest object errors. This will probably be done based on the groups refactor.

One more pressing concern is giving DGU publishers the ability to make calls to /harvest/object/id. We want them to be able to see documents that faild validation and also to see *all* errors in each document.

1321375923000000 1338203858000000
#1467 defect thejimmyg thejimmyg ckan-sprint-2012-01-09 closed worksforme CKAN dumps dgu miss certain publisher information

Pawel knows about this so David Read, Pawel and I need to find time to discuss it.

1321376042000000 1326120319000000
#1468 defect johnglover thejimmyg ckan-sprint-2011-12-05 closed fixed Update the search index for individual packages

When I delete a package from the pylons shell I need to remove that package's entry from the Solr search index. That doesn't happen at the moment.

It would be useful if the search-index rebuild command could take a list of package IDs to rebuild.

1321376195000000 1322591997000000
#1469 enhancement amercader thejimmyg ckan-sprint-2012-03-05 closed fixed Geo-spatial search on the front end

Now that we have a geospatial search API, we should build a front-end that integrates with the existing search for displaying results.

1321376324000000 1330102878000000
#1470 defect dread amercader ckan-sprint-2011-11-21 closed fixed Check user name in the profile form 1321446143000000 1324473955000000
#1471 enhancement seanh dread ckan-v1.8 closed fixed Be compatible with new ubuntu release in April

Nils flagged up that Ubuntu launches another LTS in April which we should probably move to. This ticket is to get the ball rolling with what we do. We can always include in our package the exact package versions of deps that we want, but it is good to be in line with the release if possible.

Here's how to see the packages included with it: http://packages.ubuntu.com/search?keywords=python-pylons&searchon=names&suite=all&section=all

Pylons 0.9.7 -> 1.0-2 Requires a small amount of work SQLAlchemy 0.7.3-2 OK Babel 0.9.6-1 OK Genshi 0.6-2 Perfect

Another alternative is to move to another web framework, especially minding that Pylons is not being further developed.

1321552345000000 1340810270000000
#1472 enhancement rgrp rgrp ckan-sprint-2011-11-21 closed fixed Convert ckanclient to git and github from mercurial 1321624905000000 1321721874000000
#1473 defect dread dread ckan-sprint-2011-11-21 closed fixed max name length validators missing

Max length of package name and version field is 100 characters, but the validators seem to enforce this, although in the web interface the javascript limits typing in these fields, so it just affects the API really.

1321631452000000 1321872717000000
#1474 enhancement kindly ckan-sprint-2011-11-21 closed fixed fix up navl tests

navl tests are being skipped unskip them!

1321825892000000 1321826753000000
#1475 enhancement kindly kindly ckan-sprint-2012-01-23 closed fixed Create documentation showing how to make an extension that use celery. 1321826681000000 1327576068000000
#1477 enhancement kindly kindly ckan-v1.6 closed fixed [Super] Group refactor

This is the implementation of the final draft in


Steps to complete this will follow (somewhat) in the following order.

Change package_group table to membership. (#1478) 3d

  • model migration. (done)
  • simplify revisioning (done)
  • Remove any sqlalchemy releations from groups. (done)
  • change logic functions (done)

Single type per group (#1531)

  • migration add type to group table. (done) 1d

Add capacity table (#1522)

  • model migration. (done)
  • add to logic functions.

Allow other objects to be added to groups. (#1531)

  • change logic functions (done)

Adapt thedatahub.org to use new model.

  • move auth groups over to groups/members
  • change user_object roles to use groups? keep user object roles for time being.
  • make create update package/group put the correct items in the group/members/capacity tables.
  • work out how ui works.

--not doing the below as part of this super ticker now #1669.

Create model for dgu.

  • adapt auth functions to use auth.
  • adapt package_create/update group_create/update to fill in the correct capacties/members.

Adapt Iati, and potentially other custom instances.


  • #1279 Deprecate Authorization Groups
  • #1198 Publisher hierarchy
1321959320000000 1328016209000000
#1478 enhancement kindly kindly ckan-sprint-2011-12-05 closed fixed change package_group to members table

This will comprise of the following steps.

  • model migration.
  • simplify revisioning
  • Remove any sqlalchemy relations from groups.
  • change logic functions

estimate 2d

1321959973000000 1323172654000000
#1479 defect dread dread ckan-sprint-2011-12-05 closed fixed Can't edit a user with a unicode email address
  1. Register User with an email address with a unicode char (e.g. u'\u044e')
  2. View the User in the UI (/user/) or with 'user_show' Action API


Module ckan.controllers.user:98 in read
<<          try:
                   user_dict = get_action('user_show')(context,data_dict)
               except NotFound:
                   h.redirect_to(controller='user', action='login', id=None)
>>  user_dict = get_action('user_show')(context,data_dict)
Module ckan.logic.action.get:488 in user_show
<<      check_access('user_show',context, data_dict)
           user_dict = user_dictize(user_obj,context)
           if not (Authorizer().is_sysadmin(unicode(user)) or user == user_obj.name):
>>  user_dict = user_dictize(user_obj,context)
Module ckan.lib.dictization.model_dictize:189 in user_dictize
           result_dict['display_name'] = user.display_name
           result_dict['email_hash'] = user.email_hash
           result_dict['number_of_edits'] = user.number_of_edits()
           result_dict['number_administered_packages'] = user.number_administered_packages()
>>  result_dict['email_hash'] = user.email_hash
Module ckan.model.user:59 in email_hash
<<          if self.email:
                   e = self.email.strip().lower()
               return hashlib.md5(e).hexdigest()
           def get_reference_preferred_for_uri(self):
>>  return hashlib.md5(e).hexdigest()
UnicodeEncodeError: 'ascii' codec can't encode character u'\u044e' in position 17: ordinal not in range(128)
1321960486000000 1321961592000000
#1480 enhancement dread dread ckan-sprint-2011-12-05 closed fixed Munge functions in util

munge_title_to_name, munge_tag v. useful for client software importing things into CKAN. I want it for ONS.

1321965837000000 1321978546000000
#1481 defect johnglover dread ckan-sprint-2011-12-05 closed fixed License not shown if no resources

The license is not displayed any more, unless there are resources e.g. http://test.ckan.net/dataset/dtest#section-basic-information is OGL, but that's not shown.

1321983266000000 1323169343000000
#1482 defect johnglover zephod ckan-sprint-2011-12-05 closed fixed Minor JS bugs

Three JS bugs have appeared:

  • The slug editor is not updating the text field again. [Fixed; commit:fadda17383]
  • The View Dataset page will stop executing javascript if there are no resources. [Fixed; commit:e54b089f6e]
  • The Add-Dataset and Add-Group URL editor shows the wrong URL for datasets and groups. Broken in this commit:

https://github.com/okfn/ckan/commit/5a01e67 by 'map.explicit=true'. JohnG: Can you look into this? [Fixed; commit:88e88380f6]

1321991091000000 1322062576000000
#1483 defect johnglover johnglover ckan-sprint-2011-12-19 closed fixed Tasks sometimes fail to update task_status table when trying to report a failure

The error reports do not have a "value". The original task in the task status table is not made and it tries to make a new task, but can't.

Issue: why is the original task_status entry not made?

kindly: there is a chance of course that the task is quicker than the plugin to get to the point where it needs to update the table.

kindly: so we may have to find a way to update the task status before we run the task.

kindly: I think you can supply a task_id to celery. i.e we make our own guid as the task id. So we can use that one to populate the task status.

1322050605000000 1323279151000000
#1484 defect dread dread ckan-sprint-2011-12-05 closed fixed Creating package with duplicate tags causes exception


curl http://localhost:5000/api/action/package_create -d '{"name":"test5", "tags":[{"name": "tag6"}, {"name": "tag6"}]}' -H 'Authorization:tester'

Also broken in Web UI (adding a package).

Affects API and Web UI in 1.5 and I think all versions from 1.4.1

1322052046000000 1322059006000000
#1485 enhancement kindly kindly closed fixed Package/Group form extension mechanism so you can add forms for particular package_types

We want to be able to change form depending on package type or group type.

This is dependent on a type field being added to the Package and the Group.

1322059169000000 1340034422000000
#1486 defect seanh seanh closed fixed "allow_partial_update" option does not work with resource lists of packages
  1. Call ckan.logic.action.update:package_update()
  2. Pass in a context dict with "allow_partial_update":True
  3. Pass in a partial data dict containing only the package's ID and those fields that you want to update (e.g. title).

=> All of the package's resource will have their statuses changed to "deleted".

The "allow_partial_update" option should apply to the resource lists of packages, but it does not.

1322061318000000 1323172027000000
#1487 enhancement kindly ckan-sprint-2011-12-05 closed fixed Fix group ordering on homepage

ordering on homepage by name instead of group count

1322094280000000 1324474147000000
#1488 enhancement dread dread ckan-sprint-2011-11-21 closed fixed List of installed extensions - add to the status_show

Add the list of extensions install (config's ckan.plugins) to status_show logic action (See #1087)

1322130555000000 1322241966000000
#1490 enhancement amercader amercader ckan-sprint-2011-12-05 closed fixed Standardize output from package listings coming from the logic layer

Right now, the two logic functions that return a list of packages (package_search [1] and group_packages_list [2])use custom functions to generate the output dict. That's suboptimal because:

  • The template function that renders the package listing (package_list_from_dict) could end up receiving different dicts.
  • Not all package properties are available at the template level (e.g extras, tags...). These can be really useful to customize the listing from an extension.

In general only the functions present in lib/dictization/model_dictize.py should be used to build the output of a logic function, in that case package_dictize. If necessary, they can be modified to include missing properties, like on this particular case the "isopen" property, needed by the template renderer.

[1] https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L685 [2] https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L442

1322225239000000 1324033557000000
#1491 defect seanh dread closed fixed Visible strings need internationalisation

Sean spotted that some strings need internationalising, such as flash messages. Generally all strings that appear on the web front-end should be internationalised.

Particular places that need i18n:

  • ckan/controllers/admin
  • ckan/lib/base

Note: there are some exceptions, such as in i18n.py, very obscure error messages and stuff that only appears on the API.

NB: there is a cost in making a string internationalisable (all our volunteers have to translate it), so we should not be too zealous. We should also look at the i18n/ckan.pot to see if any of the existing strings can be reused.

1322481000000000 1330020630000000
#1492 defect kindly dread closed wontfix Interference between extra and relationship fields in API

From Jan Kučera <[email protected]…>

I have also tried to use extras fields to store information about relationships between datasets. I found out that if you first add relationship using CKAN API into dataset relationships attribute and than you add extras field for the same relationship with key in format "parent_of:dataset_name" or "child_of:dataset_name", storing this extras field will clear all relationships in relationships attribute stored earlier.

I might be some kind of bug but I suppose you will know better. Is it possible to use relationships attribute and extras field for relationships at the same time?

1322481496000000 1340034528000000
#1493 defect dread dread ckan-sprint-2011-12-05 closed fixed 'search-index rebuild/clear' doesn't work if no ckan.site_id

You can't delete things from the SOLR search index if the ckan.site_id and ckan.site_url are blank.

Should assert that one of these are set up.

1322484422000000 1324474360000000
#1494 enhancement seanh seanh ckan-sprint-2012-01-09 closed wontfix API call for getting a user's public activity stream as rendered rtext

This could be implemented as a separate API call, or the rendered text versions of the activities could be added into the JSON returned by the existing API call.

This requires setting up templates for rendering activity streams items and detail items as nice, human-readable text.

There are some open questions, e.g.: Do we want the entire activity stream rendered as a block of plain text? As HTML? Or do we want a list of JSON objects, where each object contains its textual and/or HTML representations as fields?

Activity stream items and their related detail items are separate objects that each have their own textual representations.

For a mockup of the kind of text messages we want, see:


but note that this ticket is just for creating the text snippets themselves, not rendering then in an HTML page or RSS feed. Also the mockup only show activity items and not their detail items.

1322495447000000 1326109757000000
#1495 enhancement kindly seanh ckan-sprint-2012-01-23 closed fixed API call for getting a user's public activity stream

An API call that returns a user's public activity stream in JSON format.

1322495549000000 1327576099000000
#1496 enhancement seanh seanh ckan-v1.7 closed fixed Render a user's activity stream on her user page 1322495588000000 1334582338000000
#1497 enhancement dread dread ckan-sprint-2011-12-05 closed fixed Suppress benign warnings

We really shouldn't have lots of benign stuff printed on stderr when we run every command. It confuses users and hides important stuff.

1322569640000000 1322569805000000
#1498 enhancement amercader amercader ckan-sprint-2011-12-05 closed fixed Support multiple SOLR schema versions

At some point the SOLR schema used by CKAN will need to be changed and the changes won't be always backward compatible, so we need a mechanism to support instances that use older versions of the schema. This process will involve:

  • Better documentation on how to deploy SOLR, as single instance or multicore
  • A convention on how to tag schema versions and SOLR endpoints to ensure that CKAN instances point to the correct SOLR URL.
  • (If possible) Checks in CKAN core that deal with different schema versions.
1322585264000000 1323165876000000
Note: See TracReports for help on using and creating reports.