{22} Trac tickets (2647 matches)

Results (301 - 400 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13 14
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1288 defect dread ckan-backlog new Package edit/creation can't include 'relationships' field

When you create or edit a package (via the API), you aren't able to specify the relationships it has. (If you do you get 409 {"__junk": ["The input field __junk was not expected."]} )

The normal way to create relationships is via /api/rest/relationships/ and this works. But when you GET a package, the dictionary lists all relationship details. So this bug creates a problem for editing a package that has relationships - you want to GET it, make any edits and then PUT it back. The work-around is to delete the 'relationships' key from the dict before you PUT it back.


Ideally, CKAN would read the 'relationships' key and make the necessary changes. This is a chunk of work.

Another good option is to allow an unchanged 'relationships' value, but barf it is edited. This is also a chunk of work.

A bad option would be to just ignore the 'relationships' value, since users will get frustrated changing this value and wonder why it never saves, not understanding it is different to all the rest, without error message.

A final option is to get rid of relationships altogether.

1314203695000000 1339774098000000
#1289 CREP dread ckan-backlog closed wontfix Remove 'relationships'


Package Relationships have not taken off in the 18 months we've had them in the API. There are some issues with them and we need to spend more time improving them or consider getting rid of them.

The Problem

Original use cases are expressed here: #253 Here are comments about how we could handle these specific examples better:

  1. groups of packages - maybe better with a custom tag?
  2. fragment resources - soon to be covered by 'kind' resource field #957

3&5. derived resource - better to have some sort of resource relationship perhaps?

  1. linked resource - again better to have some sort of resource relationship perhaps?

Outstanding issues needing serious effort to fix:

  • #256 Editing them in Web UI (not done yet)
  • #1288 Package edit/creation can't include 'relationships' field


Remove relationships from model, API, tests, Web UI. Data migration to remove from db.

Why do it this way

Getting frustrated having problems with the code, when it's not used much. Often asked about what it's for, but rarely used. Seems an overly complicated design.

Backwards Compatibility


Implementation plan


See Specification

Risks and mitigations

Risk: a customer suddenly wants this, and the new ways to relate resources are not in place yet.

Mitigation: discuss this decision thoroughly to make sure we are confident the use cases are not important. Discuss with team, ckan-discuss and specifically the LOD people who have some related packages on thedatahub.org.


David Read


Not yet.

1314206502000000 1317315211000000
#1314 enhancement dread ckan-backlog assigned ckanclient search - generator improvements

Apparently the search generator always makes two requests, even if you don't want to see the search results, which might be slow. Can this be optimised?

Maybe we should also provide a second search function that doesn't use the generator - the original simple search function (that leaves the user to deal with limit & offset).

1315395410000000 1340191233000000
#1317 defect dread ckan-backlog assigned password reset - improve user search

In password reset, it gets confused if you have two similar users. This is because with the string the user provides, it searches several fields, not just name but also fullname and email address, allowing you to search for these. But only name is unique.

Specific problem: Ira searches for "Irina" then it finds both: <User name=irina fullname=Irina [email protected] ] and <User name=shevski fullname=Ira email=> (I think)

Maybe need to choose which field it searches?

1315415539000000 1340191221000000
#1322 enhancement dread assigned Action API improvements

Focusing on improving Action API as the v3 API:

  • have an optional parameter of the data_dict called "options". Options would contain items that would get passed into the context. e.g. {"options" : {"ref_package_by": "id"}}.
  • instead of using API version to change the way packages are referenced, use the ref_package_by.
  • All package_show, group_show etc. to accept an object 'name' as an alternative to object 'id'.
  • Action API is v3 of api, replacement for v1 & v2. Default for most urls is still v1, but if url is /api/action then default to v3.

Next steps:

  • Add search API (package, resource,
  • Add Util API
  • Clarify JSONP still applies
  • Add doc strings, clarifying parameters
1315474749000000 1340191088000000
#1326 enhancement thejimmyg ckan-backlog new Write a set of auth plugin functions to integrate with Druapl

Ticket #787 described join auth between CKAN and Drupal. The authentication part is live and implemented. This ticket is a placeholder for work that will be needed in the new auth system to link authorization functions to Drupal. It is dependent on the groups refactor.

1315821084000000 1315821084000000
#1327 enhancement rgrp ckan-v1.6 closed duplicate [super] Dataset Archiving

Split out of #852. Automated archiving of datasets (related to QA).

Automated archiving using worker process

  • #890 - Timed actions in ckanext-queue
  • #891 - Resource download worker daemon
  • #892 - Make stored data available in WUI
1315821490000000 1320662446000000
#1328 defect [email protected] assigned Unicode & paster commands

A possible bug in CKAN when I tried deleting users using "paster --plugin=ckan user delete" command.

To reproduce the bug do the following:

  1. Create a user with an ID (which in my case was a user's full name)

that contains non-unicode caracters like Norwegian "æ", "ø", or "å".

  1. Make sure that you can see something like the example below:

(pyenv) [email protected]:$ paster --plugin=ckan user Users: name=Rustæm

  1. Then try deleting the user with following command:

(pyenv) [email protected]:$ paster --plugin=ckan user delete "Rustæm"

You should now get a python encoding error. I know that this is quite rare case, but in our case it caused some trouble. Could you guys have a look at this bug?

CKAN ver. 1.3.3.

1315823110000000 1340191065000000
#1330 enhancement rgrp ckan-backlog closed invalid Deprecate / Remove test_authz.py

test_authz.py appears to test in great detail some very specific additional authz (related to total site lock-down it seems -- introduced I think for hri project).

I think there are simpler ways to get total site lockdown (use external auth!) and this test is slow and delicate (e.g. depends on specific words in templates). Suggest removing. If we don't remove we should at least refactor tests for access to certain pages to use a proper method of testing (e.g. agreed html comments in each page) rather than being depending on the presence of absence of specific wording.

1315899129000000 1327060201000000
#1332 defect dread closed fixed i18n IndexError exceptions

We get this i18n error for the URL http://no.ckan.net/authorizationgroup (which runs 1.4.2 with Norwegian translation). I'm not sure why it occurs. Fixing it is low importance though, because this site doesn't use Authorization Group feature

Usually caused by bots. The user does not appear to be logged in.

WebApp Error: <type 'exceptions.IndexError'>: list index out of range
Module genshi.filters.i18n:177 in _generate
<<                  msgbuf.append(*previous)
                       previous = None
                   for event in msgbuf.translate(gettext(msgbuf.format())):
                       yield event
                   if previous:
>>  for event in msgbuf.translate(gettext(msgbuf.format())):
Module genshi.filters.i18n:1029 in translate
<<                      )
               parts = parse_msg(string)
               parts_counter = {}
               for order, string in parts:
>>  parts = parse_msg(string)
Module genshi.filters.i18n:1143 in parse_msg
<<      if string:
               parts.append((stack[-1], string))
           return parts
>>  parts.append((stack[-1], string))
IndexError: list index out of range
1315906429000000 1315999104000000
#1352 enhancement amercader ckan-backlog new Use logic functions instead of as_dict when indexing entities

The current search implementation uses the output of the the as_dict method of the domain Package object to update the index


It also uses package_to_api1 in the SynchronousSearch? plugin:


This prevents extensions from being able to index custom properties (e.g. faceting by custom extras not included in the model).

The search should use the logic function to get the package properties:

1316615397000000 1339774086000000
#1353 defect nickstenning ckan-v1.5 closed fixed No UI to remove resources

I have no idea whether this was a deliberate decision or not, but there is a total absence of any UI with which to delete resources from the currently deployed version of thedatahub.org.

1316729765000000 1317075904000000
#1355 defect amercader ckan-backlog new Package extras property does not include the newly created ones

The extras in the package object sent to the extensions after editing (https://bitbucket.org/okfn/ckan/src/01efd5649c10/ckan/logic/action/update.py#cl-226) do not include the newly added.

1317034126000000 1339774056000000
#1366 defect dread ckan-future assigned Search inside extra fields

SOLR search doesn't support searching for part of an extra field, but it does for other fields.

i.e. title="One Two Three" matches q=one AND q=title:one and geographic_coverage="England Scotland" matches q=England BUT NOT q=geographic_coverage:England

This problem emerged when we went to SOLR in #1275 (CKAN 1.5a). Tests were skipped.

This is could be a problem for DGU and maybe elsewhere.

1317290992000000 1338206707000000
#1373 defect shevski ckan-sprint-2011-11-07 closed fixed home page view does not react to logging in / out

Either: thedatahub.org does not display 'add a dataset' top navigation link or the 'create dataset' main central link to logged in user - instead showing 'sign up' link in the central box. It does however display the 'my account' and 'logout' link in the top right correctly.

Navigating to another part of the site (e.g. search or about or my account) does bring the 'add a dataset' navigation link back and the functionality works.

Works fine on test.ckan.net so only on thedatahub.

Not obviously a caching problem, since I did try clearing cache.

OR: it displays as if you're logged in even when you log out. The links doesn't work and prompt you to log in though.

1317829479000000 1320174240000000
#1388 defect dread ckan-backlog closed fixed etags caching on home page

Needs to update on:

  1. language - It's broken on thedatahub.org if you change to German and then click on the CKAN icon top-left to go to thedatahub.org again.


  1. whether signed in or not - #1373

BUT not latest revisions (which is what it was)

Or get rid of etag caching on this page and others?

1318429443000000 1320174229000000
#1395 defect seanh closed fixed "Import Error: cannot import name UnicodeMultiDict" when installing ckan from source

At the paster db init command when installing ckan from source I get the error "Import Error: cannot import name UnicodeMultiDict?" (happens with both ckan 1.4.2 and today's latest bleeding edge code, on Ubuntu 10.04.3).

UnicodeMultiDict? has been removed in a recent version of python-webob, and the pip install ... lucid_missing.txt causes a too-new version of python-webob to be installed into ckan's virtualenv (the new webob gets installed as a dependency of formalchemy).

I manually did pip uninstall webob and then ran paster db init again and it worked.

1318520183000000 1320857823000000
#1406 enhancement zephod ckan-backlog new Re-enable RSS subscriptions

RSS 'subscribe' buttons appeared in many places on the site but were not very helpful. They took (confused) users pointed to the raw feed code, and Google Reader could not understand the feed. Safari, however, could interpret it correctly.

Their presentation needs to be clear and consistent; the RSS feed really needs testing in a variety of readers; and we need to decide exactly which items should get a feed. (Package updates? Groups?)

1318861327000000 1320930088000000
#1414 enhancement shevski ckan-backlog new track user log-ins on thedatahub.org

Set up tracking for user logins so that we have stats about how many active users of thedatahub exist want to be able to see who logged in the the last x months

1319454782000000 1319454782000000
#1419 enhancement dread ckan-sprint-2011-11-07 closed invalid Can't log in via OpenID

I couldn't log into theDataHub with OpenID today. I tried both Google ID and MyOpenID. Both times the login on the remote auth server went fine, but when it returns you to theDataHub you get error "Login failed. Bad username or password."

1319543013000000 1319796164000000
#1423 enhancement markbrough ckan-backlog new Edit resources suggestions
  • Description vs Name - Edit Resources view is showing the name of the package rather than the description, and a lot (all?) of the packages before the upgrade don't have names, so might be good to swap this round again, e.g.: http://thedatahub.org/dataset/edit/iati-registry
  • Moving resources - Moving them up or down the list used to be quite useful if you had a lot of resources that you might want to leave on the resources page, but only one or two that were actually current and that you wanted to draw attention to. This doesn't exist any more on CKAN but I think it would be good to add it back in.
1319641906000000 1338203678000000
#1424 enhancement dread ckan-backlog new Openness notice should be clearer

ckan-discuss discussion suggests changes to the 'openness' indicator ( http://lists.okfn.org/pipermail/ckan-discuss/2011-October/001786.html )

Dataset view page:

  • If there is an explicit but non-OKD compliant license, such as CC-BY-NC, then this should be stated explicitly, perhaps: “This dataset is Not Open. License: Creative Commons Attribution Noncommerical. This is not an open license as it does not meet the Open Knowledge Definition.”
  • If the license is marked as “Other::License Not Specified”, then this should be stated explicitly, perhaps: “This dataset is Not Open. It is published without an explicit license, the publisher reserves all rights to the dataset.”
  • 3. If the license field was left empty by the contributor of the Data Hub record, then again this should be stated explicitly, perhaps: “This dataset is Not Open. The license of this dataset is unknown or unspecified. Start an enquiry on IsItOpenData? »
  • There is a bug so that non-open licenses doesn't have an openness notice.
  • If downloadable resources are not available, this should not affect 'openness' - check this has been removed.
1319648089000000 1319648089000000
#1430 defect amercader ckan-sprint-2011-11-07 closed fixed Documents get mixed between SOLR cores

On some occasions (apparently random), the documents indexed in a specific SOLR core get mixed with different site_ids.

E.g: We look for all documents in the testing.iatiregistry.org core, faceted by site_id. We would expect all documents to have site_id = iati_testing, but some of them have site_id = iatiregistry.org


<lst name="facet_fields">
<lst name="site_id">
<int name="iati_testing">265</int>
<int name="iatiregistry.org">255</int>

If we compare one of the records which disappeared from the "iati_testing" site_id in both the production and testing SOLR cores of the server, the records are exactly the same, including the indexed_ts property:



Note that the response from the URLs shown may vary, as the testing site could have been reindexed.

1320068076000000 1324033923000000
#1432 enhancement rgrp ckan-backlog new [super] Data processing system for CKAN and Webstore

Super ticket: #1190

A data processing system which utilizes the Webstore. One could get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules)

1320142747000000 1339774041000000
#1435 enhancement rgrp ckan-v1.6 closed wontfix Switch to continuous.io for buildbot (?) 1320145831000000 1323283538000000
#1438 enhancement dread ckan-future new Action API - parameter discovery/checking

Many actions in the Action API require parameters. What params are needed should be listed and checked. Because currently, if you get them wrong you simply get a useless 500 error.

Currently they are listed in the docs, extracted from the code manually.

So you could GET /action/api/package_list to receive not only the help text, but a list of arguments.

And if you send an extra or missing argument then an intelligent error message can be returned.


How about some sort of decorator on the action function:

@logic_params(id, offset, limit)
def get_package_list(context, data_dict):

This would do the param checking, and is there a way to extract these params from the function? Or do a registration of the logic function?

I'd certainly like to keep the list of the list of params for the function with the function, for ease of reading the code.

Another good thing would be to pass in the params named as themselves, rather than having them contained in the data_dict.

1320161890000000 1338205050000000
#1439 enhancement dread ckan-backlog new Action API discoverablility

A good service API needs to be discoverable, so you are not always having to refer to the documentation html.

Maybe /api/action should return a list of actions available? (Currently this returns a 404.)

  • It would be nice to sort these into get/create/update/delete.
  • #1438 Parameters for each of the actions must be discoverable too

/api/action/{action_name} should also return the help text / parameters allowable. (Currently this returns 400 error)

1320161970000000 1325474974000000
#1457 defect jilly.mathews ckan-future new Bug with DataNL instance

"when logging into http://register.data.overheid.nl/ with OpenID, the /user/me page gives a 404. CKAN version 1.3.4."

n the manual it says an API key kan be created via http://test.ckan.net/user/me /. However, when I try the corresponding http://register.data.overheid.nl/user/me, I get a 404 error (not found).

1320925041000000 1320925041000000
#1460 enhancement rgrp assigned Improve extensions documentation

Current extensions documentation needs some work: http://docs.ckan.org/en/latest/plugins.html

  • Queue extension section may now be out of date (?)
  • Think about how it integrates with https://github.com/okfn/ckanext-example (especially tutorial and example extension)
  • Document all plugin points (auto-extract from CKAN source??)
1321114523000000 1340191001000000
#1461 defect pudo closed fixed CkanClient doesn't submit auth headers for GET requests

e.g. package_register_get.

1321354037000000 1321359503000000
#1463 defect thejimmyg ckan-sprint-2011-11-21 closed wontfix QA extension no-longer works with packaged CKAN 1.5

The extension needs upgrading:

No module named webstore.database

The other option is to delay this functionality and merge with the publisher dashboards after the group refactor.

John, David Raznick, David Read, Ira, what do you think?

1321375614000000 1321874259000000
#1464 enhancement thejimmyg ckan-v1.6 closed wontfix Replace RabbitMQ with Celeryd to support running multiple instances

The current harvesting implementation can only have one instance per server.

We could either:

  • Accept this
  • Change the RabbitMQ code to support multiple instances
  • Switch the entire harvesting to use Celery
  • Wait for James's redis based live feedback based system
1321375709000000 1328529042000000
#1465 enhancement thejimmyg ckan-v1.7 closed fixed Upgrade the harvester to support publishers properly

At the moment most of the harvesting functionality is only available to sysadmins. This is fine for most cases but going forward we will need to grant each publisher access to their own sources, harvest objects and harvest object errors. This will probably be done based on the groups refactor.

One more pressing concern is giving DGU publishers the ability to make calls to /harvest/object/id. We want them to be able to see documents that faild validation and also to see *all* errors in each document.

1321375923000000 1338203858000000
#1466 enhancement thejimmyg ckan-backlog new Need to support https login for multiple instances as part of the CKAN package install 1321375978000000 1328529062000000
#1474 enhancement kindly ckan-sprint-2011-11-21 closed fixed fix up navl tests

navl tests are being skipped unskip them!

1321825892000000 1321826753000000
#1487 enhancement kindly ckan-sprint-2011-12-05 closed fixed Fix group ordering on homepage

ordering on homepage by name instead of group count

1322094280000000 1324474147000000
#1489 enhancement dread ckan-backlog assigned Updating example theme/extension

ckanext-example needs updating for CKAN 1.5:

  • theme changes
  • new forms

About: 'ckanext-exampletheme' was created in Spring 2011 as an example CKAN extension that showed how to customise the look & operation of CKAN. This moved to github and renamed 'ckanext-example'.

1322137920000000 1324292384000000
#1509 defect dread assigned Mis-dated old revisions of datasets

e.g. http://thedatahub.org/dataset/osm%402011-07-12T12%3A16%3A47.590358 gives:

This is the current revision of this dataset, as edited at 2011-07-12 12:16.

but it should say the date 2011-07-12 (which is the data being displayed).

The problem is looking at PackageRevision?, when it might be a tag or extra.

1323090398000000 1340190814000000
#1514 defect dread ckan-v1.6 closed duplicate Modifying user name loses connection with revisions

If you edit your user name, the number of revisions you made returns to 0. This is because in the Revision object, the user's name is stored, rather than the user's ID.

rgrp: We can reconnect that pretty easily (and have a longer term solution that involves not using the usernames but the userids in in the Revision objects so we don't hvae this problem in future!)

1323100659000000 1323279018000000
#1520 task dread ckan-sprint-2011-12-19 closed fixed Disable name changing

Because of #1514 we should just disable name changing, until #1514 is done.

1323169663000000 1323280999000000
#1532 defect dread ckan-sprint-2011-12-19 closed fixed Registration with OpenID has misleading error message

The log-in page says "Login using Open ID" and gives instructions for signing up. YET this is only available to users who've already added openid to their account. If you have not done this and then sign-in via OpenID (which is successful from the OpenID end) then you are told "Login failed. Bad username or password." in a flash message.

Proposed solution (i don't know if this is possible):

  • When you log-in for the first time via OpenID, it doesn't actually log you in in CKAN. It just sends you to the 'Create User' page with the OpenID field pre-filled, and puts up a flash message "This OpenID account is not yet registered on thedatahub. Please complete your details.". This allows you to complete the registration and logs you in, and allows you to log-in directly with OpenID in the future.


  • Just change the error message to be 'You need to register in CKAN first. Quote your OpenID in the registration form to use it in future.'
  • Remove OpenID altogether
1323276392000000 1323956027000000
#1534 enhancement rgrp ckan-backlog new Change revisions to record userid rather than username

The use of username is problematic because username's can change.

  • Change all revision creation code to use user id (simplest is to change c.author field in lib/base.py (?))
    • (?) Add a field ipaddr for ip address of anonymous users? (or just keep putting this in author field on Revision and then acception that those won't match when we do a look up against user table)
  • Change user view page to look up against user id rather than name
  • Perform migration on existing Revision objects
    • Match should probably be against both openid and username when searching Revisions' author field (especially true on CKAN where some people have already changed their username from being their openid)
1323278790000000 1338205050000000
#1535 enhancement dread ckan-backlog new Plump for auth header of: X-CKAN-API-KEY

When using the API, the apikey needs to be supplied in a header called 'Authorization'. Because some proxys / deployments use this header for other things, a configurable header was provided as an alternative, with default "X-CKAN-API-KEY".

Rufus suggests having *one* way for this. a) making this not configurable any more b) making X-CKAN-API-KEY the default

(keep Authorization allowed, but not documented, for backwards compatibility)

1323279082000000 1339774019000000
#1542 enhancement dread ckan-backlog new Buttons to purge spam datasets and groups

A sysadmin should be able to easily examine a suspect group or package, determine if it was created by a spammer (as opposed to being a legitimate object that has been graffitied by a spammer) and purge it.

The existing two-stage revision delete is currently unreliable and perhaps too laborious.

Olav and Richard have needs along this line.

1323364930000000 1339774000000000
#1544 task dread ckan-backlog new delete old git branches

We have 150 odd branches (git branch -a) - most of them old - we should prune them. At very least, branches that have been merged in should be deleted. Look at old branches that haven't been merged in and wonder why.

May be of some use:

git branch --merged
git branch --no-merged
1323702610000000 1323702610000000
#1545 enhancement amercader ckan-sprint-2012-01-09 closed wontfix Remove external asset dependencies

CKAN is pulling a number of resources from external locations. This causes problems when connectivity is limited and you have to work locally. Maybe some of them cold be moved to CKAN source to avoid external requests.

Quick search:

./ckan/templates/layout_base.html:            <img src="http://assets.okfn.org/images/logo/okf_logo_white_and_green_tiny.png" id="footer-okf-logo" />
./ckan/templates/layout_base.html:            <a href="http://opendefinition.org/"><img alt="This Content and Data is Open" src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" style="border: none ; margin-bottom: -4px;"/></a>
./ckan/templates/package/resource_read.html:                <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" />
./ckan/templates/package/read.html:          <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" /></a>
./ckan/templates/_util.html:                    <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" />
./ckan/templates/_util.html:                  <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" />
./ckan/public/scripts/vendor/ckanjs/1.0.0/ckanjs.js:      this.$dialog.html('<h2>Loading results...</h2><img src="http://assets.okfn.org/images/icons/ajaxload-circle.gif" />');
./ckan/public/scripts/vendor/ckanjs/1.0.0/ckanjs.js:          self.setMessage('Uploading file ... <img src="http://assets.okfn.org/images/icons/ajaxload-circle.gif" class="spinner" />');
./ckan/public/scripts/vendor/ckanjs/1.0.0/ckanjs.js:      self.setMessage('Checking upload permissions ... <img src="http://assets.okfn.org/images/icons/ajaxload-circle.gif" class="spinner" />');
Binary file ./ckan/lib/app_globals.pyc matches
./ckan/lib/app_globals.py:                                  'http://assets.okfn.org/p/ckan/img/ckan.ico')
./ckan/config/deployment.ini_tmpl:ckan.favicon = http://assets.okfn.org/p/ckan/img/ckan.ico
1323702635000000 1325260051000000
#1549 enhancement ross ckan-backlog closed wontfix [super] Short link tool

It would be great to have a CKAN extension that allowed users (or CKAN itself) to generate short links to other URIs (both internal and external). Once created, shortlinks made by CKAN should be changeable. This would allow uploaded content to be moved without the user's link changing at all. The tool itself might also be of use as a general link-shortener to users other than the CKAN system itself.

Another useful feature would be for this to also collect some simple analytics such as the referrer and client IP for future reference. I'm not yet sure what we would do with the analytics other than some sort of popularity metric.


  • Core, or Extension, or Self-hosted?
1324036998000000 1325474219000000
#1550 enhancement ross ckan-backlog assigned Allow simple auth via the API

It should be possible to pass userid/username and api key and obtain a response from CKAN for external services that use CKAN auth. Those services shouldn't be talking to the DB directly.

1324049369000000 1346670055000000
#1578 enhancement rgrp ckan-backlog new [super] Re-enable and refactor ratings 1324322443000000 1325473015000000
#1585 enhancement dread closed fixed Security fix

(details embargoed until 31/1/2012)

1324473465000000 1340633128000000
#1596 enhancement dread ckan-future new Refactor authz roles

Suggestions from rgrp:

  • Get rid of Roles, and replace them with direct assignment of actions, even though there are many actions, and extensions can add arbitrary ones.
  • Debatable whether we should cut the number of actions to correspond to the three roles defined by the base system.
  • Have a method of finding roles (or, in future, actions) relevant to a given protection object (e.g. FILE-UPLOAD(ER) not relevant to Packages)

(This ticket is split off from #1065)

1324549888000000 1338205019000000
#1597 enhancement dread ckan-sprint-2012-01-23 closed fixed Tag search - filter by group

I want to browse tags, but filtered for a particular group. Currently our tag API doesn't allow for filtering by group.

This is important for improving groups as communities within a site #1521. It would be easy to do this by adding an option to filter by a group. BUT are there any other use cases that would warrant a more complete faceted tag search?


BTW I can currently draw a tag cloud for a group - I can get the top tags used in a group like this:

curl http://thedatahub.org/api/action/package_search -d '{"q":"groups:country-ca", "facet.field":"tags", "rows":"0"}'

but it only contains the top 20 tags.

1324550492000000 1326821156000000
#1598 enhancement rgrp ckan-backlog new Reinstate Ratings

Ratings were disabled approximately a year ago because:

  • Unclear purpose and UX. What did ratings tell you? How useful were they?
  • Spamming (esp by bots: you could submit an anonymous rating via a GET request which caused problems)

Both problems are solvable and it would be nice to have this feature reinstated.

  • Purpose: can make this more purposable by limiting to logged in users (or at least distinguishing logged in from non-logged in users)
    • Even better we could allow ratings to be made public (I'm interested in what someone else I respect finds important)
  • Spamming: limit to logged in users and / or use AJAX over an API to submit ...
1325177524000000 1325474818000000
#1604 enhancement dread ckan-backlog new Get ckanext-moderatededits working with CKAN 1.5+ templates

ckanext-moderatededits requires an old and possibly development version of CKAN. It would be good to update it for later CKAN versions.

According to the README, you need CKAN from branch feature-1141-moderated-edits-ajax but the changelog suggests this branch went into version 1.4.2. So it possibly works with 1.4.2 and 1.4.3(.1). But CKAN 1.5 has revamped templates, so the genshi stream filters definitely don't work.

BTW history_ajax/read_ajax calls have been deprecated in CKAN since 1.5.2a and will need fixing up to use the Action API too as part of this.

1325352429000000 1325352429000000
#1606 enhancement dread ckan-backlog new metadata license config option

Add a config option to choose the metadata licence. Set default to Open Database License.

Currently the dataset edit form says "Important: By submitting content, you agree to release your contributions under the Open Database License." This is hard-coded, but not suitable for when DGU uses the CKAN form - they use the OGL.

1325501130000000 1339773981000000
#1615 enhancement thejimmyg ckan-v1.6 closed worksforme CKAN Should work behind a proxy server

This would allow deployment via Nginx or Apache using proxy to Paster, uWSGI. At the moment CKAN isn't aware of the proxy's IP address so when you perform an action which does a redirect (such as adding a package), CKAN redirects you to the *internal IP* not the external *proxy IP*.

We would like this work to facilitate testing within VMs as part of our new build infrastructure.

It would also be nice if CKAN worked when mounted at a path other than /. That could be dealt with in another ticket because it isn't a problem at the moment.

1325687841000000 1328888870000000
#1628 defect dread ckan-sprint-2012-02-06 closed wontfix get ckanext-dgu working with ckan 1.5.1

johnglover said: I can confirm that even with the mapping fix, the ckanext-dgu dgu_form plugin template does not work properly with 1.5.1, so should probably not be installed at present. The edit page is ugly but should work (eg: http://dgu-os.okfn.org/dataset/edit/abandoned-vehicles), but the 'add a dataset' page is broken (eg: http://dgu-os.okfn.org/dataset/new)

1326214762000000 1328526967000000
#1642 defect pudo ckan-backlog new Extra link generators generate garbled HTML

I had a package descriptions with URLs that contain "group:foo". This produces garbled output as the system tries to generate two sets of links: the outer link and an inner link.

Need to fix the parser.


Webdienst basierende Bereitstellung von Geobasisdaten der Freien und Hansestadt Hamburg. Folgende Geobasisdaten werden als WebMapTileService? (WMT-S) für die Dauer des Wettbewerbs netzbasiert unter der Creative Commons Lizenz zur Verfügung gestellt: Digitale Orthophotos 40 cm Auflösung (Layer: apps4d_DOP40), Digitale Stadtkarte (Layer: apps4d_DISK), Digitale Regionalkarte (Layer: apps4d_DIRK), Digitale Karte 1:5000 (Layer: apps4d_DK5).

Metadateneinträge zu den Daten im PortalU:

One fix is quoting the URLs

1326382171000000 1339773967000000
#1643 enhancement shevski ckan-backlog new Add fixed tags to thedatahub for better browsing

Similar to publicdata.eu, want to have themed areas such as finance, environment, census, etc and country tags

1326393293000000 1326393293000000
#1644 enhancement shevski ckan-backlog new Order default dataset page by most downloaded resources on thedatahub

Instead of alphabetically as we do currently, alternatively by most viewed datasets

for http://thedatahub.org/dataset

1326393542000000 1326393542000000
#1647 enhancement shevski ckan-backlog new add links to ckan discuss & dev to thedatahub

In the footer as well as more clearly & directly on the About page

1326673852000000 1326707383000000
#1648 enhancement shevski ckan-backlog closed fixed Clarify that additional info = extra fields + add guidance

Super ticket: #1506

Need to decide which term to use and then have the same for editing as well as viewing a dataset.

In creating/editing a dataset, want more explanation about adding extra fields (probably as a tooltip or similar).. i.e. that this let's you add extra custom metadata such as 'location: uk' which is then searchable etc

1326674843000000 1330632344000000
#1656 enhancement ross closed duplicate Configuration for reverse proxying

Provide configuration for reverse proxying that will correctly handle the mapping of a URL to a sub-folder (using X-SCRIPT-NAME)

  • Analysis of the best solution [1d]
  • Implement reverse proxying in tandem with #1653 [2d]
  • Document and store the configuration files. [1d]
1326711575000000 1326711659000000
#1661 defect dread ckan-backlog assigned Wrong Routes version installed by CKAN package

Jaakko Louhio reported that the wrong Routes version got installed during CKAN's package install.

He is using Ubuntu 10.04 and he believes it install Routes 1.12.3 instead of what we use which is 1.11.

1326718061000000 1339773949000000
#1662 defect dread closed wontfix OpenID not compatible with mounting CKAN at non-root URL

Mounting CKAN at a non-root URL was made to work properly here: #1659

Unfortunately OpenID doesn't play nicely and would require some work to get working.

1326730366000000 1326730414000000
#1668 defect dread ckan-backlog new repoze version discrepency

There's a discrepency in repoze.who versions between the source and package installs:

  • repoze.who - package 1.0.18 vs source 1.0.19
  • repoze.who-friendlyform - package 1.0b3 vs source 1.0.8

We get a test failure [1] with the 1.0b3 version (from the ubuntu 10.04 python-repoze.who-plugins package). But we've not noticed any problems on s057 instances (br, no, ie etc) which have the package versions of repoze.who.

The reason the package install uses the earlier packaged versions rather than the ones we'd like is that repoze uses all sorts of horrendous import hacks, making it too difficult to put into our 'ckan-conflict' source package.

James suggests we 'do something horrible like dynamically patch repoze on CKAN import'.

[1] http://buildbot.okfn.org/builders/builder-ckan/builds/1371/steps/shell/logs/stdio ERROR: ckan.tests.functional.test_user.TestUserController?.test_user_create_unicode

1326801746000000 1326801746000000
#1677 enhancement amercader ckan-v1.6 closed duplicate Make synchronous search the default behaviour

Right now you need to explicitly load the synchronous_search plugin in your ini file, when this is probably the behaviour that all users expect by default. We could keep a config flag to deactivate it, but synchronous search should be the default behaviour.

1326807604000000 1326807655000000
#1679 enhancement dread ckan-backlog new Default roles problem

The 'editor', 'anon_editor' and 'reader' roles are intended to have immutable actions. This was designed to prevent their names being subverted - e.g. an editor should always be able to edit! It also meant that when we add Actions (e.g. DELETE-PACKAGE) then it can be added sensibly to these roles in an upgrade just by changing the defaults table (ckan/model/authz.py).

The problem is that this immutability is only enforced on 'db upgrade'. So you can happily change the editor role using the paster command and it works, right up until you do an upgrade and realise permissions are different.

We should stop the paster commands being able to edit these roles. Or get rid of the immutability completely. Views?

1326823042000000 1339773923000000
#1684 enhancement ross ckan-backlog assigned Remove all config from ckanext-archiver

ckanext-archiver currently has a settings file (and a default) and it should be passed in all relevant information from the context.

Remove all settings (ARCHIVE_DIR and MAX_CONTENT_LENGTH and others) and pass them in from CKAN.

1326983821000000 1346670037000000
#1692 enhancement rgrp ckan-v1.7 closed wontfix Add image attribute to Dataset and Group

Add image attribute for Dataset and Group which would be shown in prominent place.

For dataset this could be a relevant image and for Group this would be more of a logo.

I think this has very high value from a UX point of view.


  • Do we need to support uploading and storing the image (or do users store elsewhere)
    • Could use something like transloadit service
    • Use our own storage facility with a reserved directory
  • Do we need to do resizing etc?
    • Maybe. Perhaps can prefer this in reasonably small size and then just size down in css for thumbnails etc.
1327108761000000 1338204395000000
#1697 enhancement rgrp ckan-backlog new A Configurable list of states for a Dataset

Currently have 'active' and 'deleted' suggest also:

  • 'draft'
  • 'hidden'

(Do we need both). Also write out workflows related to these.

1327400630000000 1338204189000000
#1717 enhancement shevski ckan-backlog new [super] Search UX improvements
  1. Make it possible to search by tag (e.g. by typing tag:csv into the search bar and clicking enter, it should add the 'csv' tag facet to the search)
  1. Rename and standardise the list of format tags, on search page this should also be called 'Format' instead of 'res_format' (in the right hand side bar on search page).
  1. Make it possible to view full list of tags, formats and groups by clicking on the name. From here you should be table to click on a classification and go back to a search page faceted by that classification. E.g. from search page, click on 'tags', on tag page click on 'london' or whatever, and be navigated back to search page with search within 'london' tag only. Or y'know, a better way of doing it.
  1. More standard classifications, such as 'Location' and 'Theme' - like on publicdata.eu
  1. Blue search button should be displayed in line with the search bar, not underneath
  1. Datasets should be displayed in order of most viewed or downloaded instead of alphabetically. For alphabetic search we could consider adding a way to facet by first letter of dataset name
1327603981000000 1330088539000000
#1740 enhancement seanh ckan-future new Get rid of `from module import ...`

It's really bad to do from module import * and CKAN has a lot of them. I suggest a three-pronged approach:

  1. Don't add any more of them.
  1. When you're programming if you see an easy opportunity to remove one then do so.
  1. At some point we should task someone to go through the code and remove them all (which is what this ticket is for), but this will be a big job and may break things.

We should also get rid of most or all of the from module import foo and from module import foo, bar statements.

I think the right thing to do is just import module and then use module.foo in your code. But if you find yourself doing module.foo.bar then you may have a code smell.

See: http://docs.python.org/dev/howto/doanddont.html

1328094369000000 1328094884000000
#1745 enhancement rgrp ckan-v1.9 new Dataset search UX improvements as of Jan 2012

Changes to make search both more exploratory and more satisfying to use

  • Search query build - #1603
    • Ability to add new facet fields "live"
    • That is add fields which then contain faceted options (a bit like data.hri.fi)
  • (??) Autocomplete / drop down on search (i.e. search while you type)
    • Dubious about value / cost ratio here

Probably would involve to pure JS and HTML implementation.


Probably require

  • API changes to expose solr style API directly #1737
1328224941000000 1340033358000000
#1746 enhancement seanh ckan-backlog closed wontfix Activity streams pagination

Currently user, package and group activity streams only return the most recent 15 activities, even though all activities are kept in the db. Do we want to add pagination - to both the API and the HTML pages - to support retrieving older activities?

1328446488000000 1355141062000000
#1747 enhancement seanh ckan-backlog new Expire old activities

Currently the activity streams database tables just get longer and longer over time. Do we want to eventually delete the oldest activities, to keep the length of the table within limits?

1328446589000000 1339773859000000
#1749 enhancement seanh ckan-future assigned Allow creating activity details through API

Currently the activity_create() logic action function only lets you create top-level activity stream items, and not their related activity details. It should handle activity details via nested dicts.

1328465406000000 1338213933000000
#1750 enhancement seanh ckan-backlog new Move ckan/lib/activity.py into the model

Move ckan/lib/activity.py moved to into the model - say ckan/model/activity_extension.py, because it's so tightly knit with the model code, whereas most of the lib code is used in the controllers.

1328465888000000 1339773840000000
#1787 enhancement dread ckan-future new [super] Improve RESTful API
  • Lists of entities should be full URLs, rather than just the names
  • Discoverability - /api/v3/rest should list the entity types that can be listed

This could be v3 of the RESTful interface.

1328702082000000 1328702082000000
#1789 enhancement seanh ckan-backlog new Implement a tag_update() logic action function

So users can rename a tag and/or move it between vocabularies using the API.

Currently we have create_tag() and delete_tag(), but if you were to 'update' a tag by deleting it and then recreating it all the datasets that had that tag will have lost it and you'll have to re-add it to them all.

What should happen to datasets that have the tag, if the tag gets moved between vocabularies? All the datasets just keep the tag with the new vocabulary? This will become a problem if/when we support 'radio button'-style vocabularies (where each dataset can only have one tag from the vocabulary).

1328805413000000 1339773812000000
#1790 enhancement dread ckan-future new Click to delete tags, rather than have all existing tags in the tag text box

From Pablo:

Editing the tags field is clumsy when there are too many tags. Could show existing effectively as tags (like delicious), then allow clicks to delete. New tags added via text box.

1328888674000000 1328888674000000
#1798 defect dread closed fixed API search in non-q fields has exception for unicode characters

You get an exception if you use the API to search packages and specify a non-ascii character in a field other than q.

For example:


This "N-dash" (Unicode character 2013) causes this exception:

Module ckan.controllers.api:460 in search
<<                      if ver in u'12':
                               # Otherwise, put all unrecognised ones into the q parameter
                               params = convert_legacy_parameters_to_solr(params)
                           query = query_for(model.Package)
                           results = query.run(params)
>>  params = convert_legacy_parameters_to_solr(params)
Module ckan.lib.search.query:38 in convert_legacy_parameters_to_solr
<<      for search_key in non_solr_params:
               value_obj = legacy_params[search_key]
               value = str(value_obj).replace('+', ' ')
               if search_key == 'all_fields':
                   if value:
>>  value = str(value_obj).replace('+', ' ')
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 30: ordinal not in range(128)

This problem affects CKAN 1.5 and 1.5.1 only.

1329395420000000 1329395560000000
#1800 refactor seanh ckan-future new Tidy up *_list() and *_search() functions in ckan/logic/action/get.py

For consistency all the *_list() functions should list objects only and not accept an optional search query. There should be *_search() functions whenever search is needed.

Currently it is inconsistent, e.g. package_list() and resource_list() do not accept a search query option and there are package_search() and resource_search(), but user_list() does accept a query and there is no user_search(). tag_list() also accepts a query, and there is also a tag_search() that apparently duplicates the search functionality.

1329405129000000 1338204886000000
#1801 enhancement dread ckan-backlog closed fixed No links to password reset

You can reset your password (#1186) but you have to know the URI (/user/reset) - there is no link to this page!

1329405290000000 1343144718000000
#1805 enhancement toby closed fixed Error pages do not translate

eg 404 page

create a 404 error via a bad url - translation links go to /document/error not the actual bad url

1329489216000000 1329760795000000
#1823 enhancement rgrp ckan-backlog new Spring clean bin directory

Huge number of accumulated (and likely unnecessary) scripts in /bin directory.

1329773331000000 1338203554000000
#1824 enhancement seanh ckan-future new Add vocabulary pages

For a free tag foo you can visit the page at /tag/foo and see a list of all the datasets that have the tag foo, and when the tag appears on dataset view pages etc. it's linked to this tag page.

We should do the same thing for vocabulary tags. A tag bar in vocabulary baz should be hyperlinked to a page /tag/baz/bar, or perhaps /vocab/baz/bar.

1329845089000000 1338204958000000
#1827 enhancement dread ckan-backlog new 'Register' link should be hidden if you not allowed to register
I have just deny visitors the create-user permission:
sudo -u ckanstd paster --plugin=ckan roles deny reader create-user -c /etc/ckan/std/std.ini
sudo -u ckanstd paster --plugin=ckan roles deny anon_editor create-user -c /etc/ckan/std/std.ini

and after restarting, the register link is *not* hidden, but now when you access the register page, it shows you this message "Unauthorized to create a user" (when not logged in). But anyway that is an improvement.
1329924939000000 1339773730000000
#1831 enhancement ross ckan-backlog assigned Login with email address

Users should be able to log in using either their username, or their email address, both of which are unique.

Will require a change to UsernamePasswordAuthenticator? in ckan.lib.authenticator.py and possible a useful User.by_email in the user model if it doesn't already exist.

1330073906000000 1346670504000000
#1832 enhancement dread ckan-backlog assigned dataset purge API

Purging datasets (deleting them fully, not just changing the state to 'deleted') is important for users testing dataset creation over the API on a test CKAN instance.

Without this, they need to resort to more difficult methods such as:

  • cleaning and reloading the database
  • setting the test datasets to state 'deleted' and also appending a suffix '_00' and incrementing the number until there is no clash of names.

Requested for NHSIC.


  • This could slot into the Action API.
  • Of course we would need to ensure the user's had been given the specific right to purge.
  • I suggest we log the full details of the dataset being purged.
1330077745000000 1339773706000000
#2197 defect zydio ckan-backlog assigned Storage Metadata API: add/update not working with local file storage (Pairtree)

If OFS is configured with Pairtree to use a local file storage, all POST requests to add/update metadata ( /api/storage/metadata/{label} ) will fail.

This is due to the use of BotoOFS specific private methods in StorageAPIController.set_metadata(), eg: self.ofs._require_bucket(bucket), self.ofs._get_key(b, label), self.ofs._update_key_metadata(k, metadata) ... those methods can't be found in POTFS and this causes errors. The API should use only OFSInterface methods, or should conditionally make calls based off the actual type of self.ofs.

PS: I did set "ckan" as "Component" in the ticket because storage has been integrated back into the core in CKAN 1.6

1330421377000000 1346662128000000
#2198 enhancement zydio closed fixed API documentation is missing Storage Metadata API info

Now that ckanext-storage is back into the core (v1.6), CKAN documentation should probably contain info on Storate Metadata API.

1330421743000000 1339771453000000
#2201 enhancement rgrp ckan-v1.7 closed duplicate Add citation info to Dataset and Resource page


%title. %author. Retrieved %date. %site_title.

For resource: %title = %dataset_title. %resource_name.

1330765080000000 1330765400000000
#2202 enhancement rgrp ckan-future reopened Display page view count on dataset and resource pages

Just like we display download counts we should display view counts.

1330765455000000 1338204929000000
#2230 refactor seanh ckan-v1.7 closed fixed Tidy up of search facets code duplication

Because of a clash between two development branches there is some duplication of code to do with code facets (note: at the time of writing the code duplication exists only on the feature-1821-multilingual-extension branch, but this will be merged into master at some point):

The package_search() function is adding the search facets to the search results twice with two different data structures, with keys "facets" and "new_facets". It should be reduced to just the new facets (with the key changed to "facets").

Also the group and package controllers are adding both facets and new_facets to the context, should be new_facets only (but renamed to facets).

The facet_items() function in helpers.py should be removed, it uses the old facets structure and shouldn't be needed anymore with the new facets structure.

In facets.html, facet_sidebar() should be removed as it uses the old facets structure and facet_div() implements the same functionality but uses the new facets.

In facets.html, facet_list_items() will have to be updated to not use the facet_items() helper and to use the new facets structure instead.

Anywhere that "new_facets" appears it will have to be changed to "facets" (e.g. in the ckanext/multilingual/plugin.py.

This is the merge commit that introduced the duplication: https://github.com/okfn/ckan/commit/1153aa876f54c22289e460aeececea22d1d4d51d

This is the earlier commit where the search facets were refactored: https://github.com/okfn/ckan/commit/3970e52008b75933fda1be1d488bed2578d98c9c

1331744085000000 1338204707000000
#2234 enhancement seanh ckan-future assigned Write a CKAN extension for pulling items from RSS/Atom feeds into CKAN templates

You configure the extension with some RSS and/or Atom feeds, it automatically reads items from these feeds and makes them available in the template context, you write a custom template to e.g. display 'news' items from a Wordpress blog on your front page.

This extension might be simpler and less fragile than ckanext-wordpresser, and also more generally useful.


  • Mark Pilgrim's Universal Feed Parser might be useful for reading the feeds
  • Feed items should probably be cached somewhere


  • The news item 'widget' should be wrapped in a known class so that it can be styled easily regardless of the format of any HTML entry.
  • For non-HTML formatted items (Atom should tell you the content type of the entry) maybe we should have a template for rendering each item along with any enclosures that it might reference
  • Caching is pretty crucial and should probably obey the ttl of the feed.
1331902755000000 1346669567000000
#2235 enhancement rgrp ckan-future new Group drop down on dataset edit should use chosen and sort groups by name 1331907357000000 1338204726000000
#2247 enhancement zephod ckan-backlog new Resource preview glitch in some browsers

From Ira: Preview for google spreadsheets are not displaying correctly for me in Firefox v.10.0.02, fine in Chrome. http://i.imgur.com/KJaqz.png

1332246614000000 1332246614000000
#2258 enhancement rgrp ckan-future new Customizable contributor agreement
  • Customize text at bottom of forms
  • Also need to make clear that this does not apply to the data itself (that is covered by the license you choose on your dataset ...)
1332751549000000 1338204747000000
#2265 enhancement dread ckan-future new 'More Like This' for a dataset

When viewing a dataset, it would be nice to show a couple of 'Related Datasets'. i.e. ones that are similar.

SOLR has a feature for finding documents similar to a particular document, called 'More Like This'.

We would like this for DGU.

1332865220000000 1339771350000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Note: See TracReports for help on using and creating reports.