{22} Trac tickets (2647 matches)

Results (1901 - 2000 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1584 enhancement johnglover johnglover ckan-backlog new QA report improvements - 2.5d

Super: #1594

  • qa/{username}
  • qa/{groupname}
  • paginate QA results
  • search / filter QA results
  • list organisation report by default, but can disable via config option (done)
  • UX tidy up of report pages - hide border if no sidebar, etc
1324459433000000 1338981975000000
#1588 enhancement johnglover johnglover ckan-backlog new QA - Give SPARQL endpoints a 4 star rating

Super: #1594

From Richard Cyganiak on the CKAN Discuss list:

Besides considering the media type of resources, it would also make sense to check for the presence of a SPARQL endpoint. SPARQL endpoints are recorded for more than 300 datasets on the Data Hub using the pseudo-type "api/sparql". A few more are recorded with the format "SPARQL". I suggest that datasets with such resources should also be considered for the fourth star.

1324480405000000 1325475178000000
#1589 enhancement johnglover johnglover ckan-backlog new QA - Give 5 star rating to datasets with link metadata

Super: #1594

From Richard Cyganiak on the CKAN Discuss list:

Regarding the fifth star (is the dataset linked to others?). This cannot be automatically determined just by looking at the format. It either requires inspection of the actual data, or information about links in the metadata. As you're probably aware, we've established conventions for recording information on data links in CKAN [1], as part of the work of the lodcloud group on the Data Hub. Link information is captured for hundreds of datasets. I would claim that we have the majority of four-star datasets covered there, and hence you can determine if they should get the fifth star by checking for the presence of a links:xxx field.

1324480600000000 1325475095000000
#1587 enhancement johnglover johnglover ckan-sprint-2012-06-25 closed fixed QA - Check license as part of 5 star rating

Super: #1594

From Richard Cyganiak on the CKAN discuss list:

According to (most of) the definitions of the five-star scheme, the first star already requires an open license, so a dataset shouldn't get any stars at all if it's not under an open license.

1324480208000000 1339433296000000
#1533 defect rgrp amercader ckan-backlog closed wontfix Make sure ckanext-wordpresser works on CKAN 1.5.1

Superceded by #2234

Super ticket: #1508

Etherpad: http://ckan.okfnpad.org/wordpresser

The wordpresser extension does not work with ckan 1.5.1. The wordpress page is not loaded.

Looks like this changeset may be involved https://github.com/okfn/ckanext-wordpresser/commit/0efa0649896a7339865f178a302be639ddc561e1

I think that even before that changeset, the page was shown without any margins.

In any case, the extension seems to be too sensible to changes in the ckan layout so perhaps it should take into account the CKAN version and have a fixed set of xpaths and templates that work for each version

1323277660000000 1338202676000000
#1386 enhancement rgrp rgrp ckan-sprint-2011-10-24 closed fixed Disallow account creation via openid

Superticket: #1343

Creation of accounts from OpenID causes large number of problems:

  • No guarantee of other profile info
  • Poor username (just from openid)

We therefore will:

  1. Require creation of user account via register (no auto-creation of accounts via openid)
    • sidebar of register page should point to login page rather than openid signin

  1. Permit association of an openid with an account ...
    • How: paste in your openid url (requires change to edit form to have this)
      • won't work for google with their weird openid urls
    • [future] Nicer way is to have login via openid while logged in (which association then happening in background)
  1. Migration (for 1+2):
    • (?) Generate decent usernames for all existing accounts
    • As this is nontrivial suggest instead we allow editing of usernames (by account owner and sysadmins). This is useful in its own right and is a reasonable 80/20 solution.
      • This does present the problem of re-associating commits with the new username. Best solution to this would be to switch revisions to point to userid rather than username. See #1534
  1. [optional] Change display_name to always be username
    • Change My Account to Username at top right of all pages (once we know usernames are short ...)
      • could do this straight away by truncating long usernames (e.g. truncate at 20 chars ...)
      • Also should we lose the icon?
1318425493000000 1324036930000000
#1319 enhancement rgrp dread ckan-sprint-2011-10-24 closed fixed Require an email address on User registration

Superticket: #1343

This avoids difficulties when they forget their password.

  • (?) Email validation (navl does not have one!)
  • Require email confirmation to be activated (?) (Answer: no, not at present -- see discussion below)
1315415932000000 1324036947000000
#1399 enhancement johnglover rgrp ckan-sprint-2011-10-24 closed fixed Generate archiving request on resource url change

Superticket: #1397

Generate archiving request on resource url change by implementing IResourceUrlChange and sending tasks to celery. [0.25d][0.25d]

Will be part of of ckanext-archiver extension.


  1. A resource is added to CKAN
  2. IResourceCreate event generated
  3. IF: resource url points to ckan storage or falls within some other set of exclusion conditions then END else continue
  4. Generate a Archiver.Update task with resource.id
1318600123000000 1319625829000000
#891 task johnglover pudo ckan-sprint-2011-11-07 closed fixed Resource download worker daemon

Superticket: #1397

Write a worker daemon to download all resources from a CKAN instance to a local repository.


  • Do we only want to download openly licensed information? ANS: no, we do everything (though do need to think about this re. IP issues)
  • Should we have clever ways to dump APIs? ANS: no.
  • Do we respect robots.txt even for openly licensed information? ANS: No (we're not crawling we're archiving)
  • Use HTTP/1.1 Caching headers? ANS: if not changed since we last updated don't bother to recache.
    • Complete support for ETags
    • Expires, Max-Age etc.
  • Check


  • Download files via HTTP, HTTPS (will not do FTP)


  1. [Archiver.Update checks queue (automated as part of celery)]
  2. Open url and get any info from resource on cache / content-length etc
    1. If FAILURE status: update task_status table (could retry if not more than 3 failures so far). Report task failure in celery
    2. Check headers for content-length and content-type ...
      • IF: content-length > max_content_length: EXIT (store outcomes on task_status, and update resource with size and content-type and any other info we get?)
      • ELSE: check content-type.
        • IF: NOT data stuff (e.g. text/html) then EXIT. (store outcomes and info on resource)
        • ELSE: archive it (compute md5 hash etc)
      • IF: get content-length and content-length unchanged GOTO step 4
  3. Archive it: connect to storage system and store it. Bucket: from config, Key: /archive/{timestamp}/{resourceid}/filename.ext
    • Add cache url to resource and updated date
    • Add other relevant info to resource such as md5, content-type etc
  4. Update task_status

Optional functionality

  • If result object is HTML, search for references to "proper data" (CSV download pages etc.)
  • Download from POST forms (accepting licenses or weird proprietary systems)
  • Support running on Google Apps Engine to save traffic costs.

Existing work

1294052979000000 1320149841000000
#88 enhancement rgrp rgrp v0.11 closed fixed Download (link) metadata

Support for download metadata (file size, file formats, version information).

See also: ticket:87 (multiple download links)

1248693385000000 1265891491000000
#95 enhancement rgrp rgrp closed fixed Add manifest support

Support for listing files (manifest) contained within a package.

Traditional manifests just list the files. Suggest in addition we have support for optional metadata in form of key, value pairs.

1249981842000000 1251454716000000
#307 defect rgrp rgrp datapkg-0.7 closed fixed datapkg upload command

Support uploading a distribution somewhere.

  • New cli command: upload
  • Pluggable uploader framework. Suggest initial support

Depends on ticket:306

1272474224000000 1288027815000000
#877 enhancement rgrp rgrp ckan-v1.4-sprint-2 closed fixed File upload in WUI (extension)

Support uploading files to a storage system from WUI with a simple upload form.


  • Implement as an Extension
  • Locate at /upload
  • Add new system action/permission (FILE_UPLOAD) and restrict by default to logged in users
  • Use OFS + boto -> generate form key (and create the bucket if it doesn't exist? Or do we assume bucket already exists)
  • This wants to be tied in to general package workflow. See parent ticket:852.
1292868146000000 1298624165000000
#69 enhancement rgrp rgrp v0.9 closed fixed Change to text-only license field and use external license repo

Switch from license domain object to a simple license field and use license list from new centralised license repo:


  • This will require a migration

Cost: 4h (plus migration ...)

1245687449000000 1246437494000000
#64 enhancement rgrp rgrp v0.10 closed fixed Switch to repoze.who for authentication

Switch to repoze.who + openid plugin for authentication from authkit.

Already did this in microfacts so should be fairly easy.

1245168853000000 1246127063000000
#715 defect rgrp johnbywater ckan-v1.4-sprint-2 closed fixed The group list page shall show the group title *or* group name

Symptom #1: Groups without titles don't get a clickable link, because there is no text. Defaulting to the group name would solve this trouble. Symptom #2: Package groups drop-down list show's 'None' when a group doesn't have a title.

Therefore, we need to centralize presentation of group title, so it defers to group name. And we need tests.

Also please sort groups by title on group list page.

1287583089000000 1297796784000000
#2254 enhancement toby toby closed fixed better hooks for templates

TD look at issues and then email ckan-dev

meeting notes from 21-3-2012 general ideas non specific


  • We want to remove the Transformer usage due to performance and its close tie to genshi
  • Want a standard way for core code and extensions to put code into the templates where *templates request it* (templates control insertion rather than extensions ...)
  • Stacking versus overwriting on entry points


  • placeholders or helper methods
  • {{user _info}} {{sub_menu}} is this equivalent to <span class="insert-user-info"></span>
    • [xyz arguments] -> h.load_snippet('xyz', ...)
  • ${h.snipit()}
1332341333000000 1338210316000000
#795 story thejimmyg johnbywater ckan-v1.3 closed duplicate "Match up" GEMINI2 "metadata point of contact" with registered entities

TM wants to know how this will be "matched up". I don't know whether this always the data provider, or sometimes the publisher.

1289228025000000 1296593361000000
#2651 enhancement icmurray ross ckan-v1.8 closed fixed Check support for TSV which doesn't appear to work well.

TSV support doesn't seem to work very well, may be the mimetype ( text/tab-separated-values )

See http://thedatahub.org/dataset/wikipedia-e3-timestamp-position-modification/resource/d883ab44-07f4-4992-800a-3e4bf5d53a96

1341923318000000 1343209784000000
#1120 enhancement tsm ckan-backlog new Atom feeds of each tag

Tags could/should have an Atom feed. This would mean that every edit to relevant packages could be easily monitored. See [1].

[1] http://lists.okfn.org/pipermail/ckan-discuss/2011-May/001162.html

1304309285000000 1339774568000000
#223 enhancement nickstenning dread v1.0 closed fixed UI Review - Tag

Tags landing page tag

  • Both search for tags *and* browse the list of tags right here
  • Big cloud of tags (sorted by popularity/usage?) filterable with a search box.
  • Tags search page tag/search and list tag/list are both redundant and should be merged straight into tag.
1260880276000000 1273596111000000
#2562 enhancement johnmartin aron.carroll demo phase 5 closed invalid Demo theme tag improvements

Tags should be implemented in two sections.

  • Custom predefined tags using vocabularies
  • Provide suggestions, auto completion then fallback to free text
1340035714000000 1352206599000000
#118 enhancement dread dread v0.10 closed fixed Use paginate in webhelpers

Take out import of paginate in setup.py. Use paginate in webhelpers instead. Make changes to take account of any i/f changes.

1253273657000000 1253784902000000
#2413 enhancement ross ross ckan-sprint-2012-05-29 closed fixed Replace/Fork autoneg

Take revelvant parts of autoneg ( https://github.com/wwaites/autoneg.git ) so that we can remove the dependency (just for two functions) and thereby remove flup as well.

1337761125000000 1337766546000000
#1139 enhancement lucychambers lucychambers ckan-backlog closed fixed Create CKAN Theme Gallery

Take screenshots of existing ckan instances esp those mentioned <http://wiki.ckan.net/Theming> and put on flickr in ckan or ckan-theme group so we can create a gallery ... (both to illustrate theming but also to show ckan instances that are around -- could add to http://wiki.ckan.net/Instances)

1305645859000000 1306941356000000
#1372 refactor johnglover johnglover ckan-sprint-2011-10-24 closed fixed Refactor archiver to work as a celery task

Take the resource archiver (currently part of ckanext-qa) and make it into a standalone extension that works as a celery task.

1317808970000000 1319625783000000
#2410 enhancement ross ross ckan-sprint-2012-05-29 closed wontfix New RDF serialization required VOID vocab for related datasets

Taken from http://lists.okfn.org/pipermail/ckan-dev/2012-May/002218.html

A few months ago, the RDF provided when dereferencing a dataset's URI was very useful. It described the datasets using many of the "right" RDF vocabularies (dcterms, void, etc).

The RDF that is returned now no longer has these nice descriptions. Instead, it seems to be a mindless transcription of the CKAN record.

For example, the older descriptions gave:


void:subset [

void:target <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>, <http://thedatahub.org/dataset/geonames-semantic-web> ; void:triples 50 ; a void:Linkset

], [

void:target <http://thedatahub.org/dataset/dbpedia>, <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> ; void:triples 52 ; a void:Linkset

], [

void:target <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>, <http://thedatahub.org/dataset/govtrack> ; void:triples 52 ; a void:Linkset

] ; void:triples 130005 ;

but now it's simply:


dct:relation [

rdf:value "130005" ; rdfs:label "triples"

], [

rdf:value "52" ; rdfs:label "links:dbpedia"

], [

rdf:value "50" ; rdfs:label "links:geonames-semantic-web"

], [

rdf:value "52" ; rdfs:label "links:govtrack"

] ;

My old copies of the dataset descriptions mentioned a "GoLD CKAN 1.0" that was credited for the nicer modeling.

Where did that go? Can we get it back?

Thanks, Tim Lebo


dc:source <http://ckan.net/> ; opmv:wasGeneratedBy [

opmv:used <http://ckan.net/api/rest/package/farmers-markets-geographic-data-united-states>, <http://ckan.net/api/rest/revision/9c2545b8-ebf1-42bf-ac92-633ff5ceba3f> ; opmv:wasControlledBy [

a foaf:Agent ; foaf:name "GoLD CKAN 1.0"

] ; opmv:wasPerformedAt [

a time:Instant ; time:inXSDDateTime "2012-01-31T05:34:32Z"^^xsd:dateTime

] ; a opmv:Process

] ; a dcat:CatalogRecord ; dcat:dataset <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> ;

1337684531000000 1338193633000000
#1654 task amercader amercader ckan-v1.7 closed fixed [super] Update Publicdata.eu to the latest CKAN stable version

Tasks include:

  • #1813 Update ckanext-pdeu (4d)
  • #1814 Update harvesters (2d)
  • #1649 Verify ckanext-rdf works with latest CKAN (3d)
  • #1815 Reenable Sparql endpoint (?)
  • #1816 Update ckanext-apps (2-3d)
1326710647000000 1338203166000000
#1813 task amercader amercader ckan-sprint-2012-03-05 closed fixed Upadate ckanext-pdeu to work with CKAN 1.6

Tasks include:

Estimate 4d

  • Update requirements (git)
  • Search facets error
  • Error on dataset read (extras in context)
  • Url generator for rdf
  • Make templates work with latest theme
  • Use dicts in templates (isopen())
1329757128000000 1330102557000000
#1583 enhancement zephod johnglover ckan-sprint-2012-02-06 closed fixed Integrate QA into frontend - 1d


  • On dataset view page, add star rating after each resource view url. Probably similar to current format label after url.
  • Add star rating to resource view page, along with a way of displaying the reason for the rating and tips to improve. Should be unobtrusive.
  • Fix the table at http://thedatahub.org/qa/dataset/broken_resource_links - URLs should be truncated to a fixed not-too-long length and the other fields line up (eg by using a table)~
  • Handle 404s by showing that 'we think this resource is 404ing'.
  • Handle the absence of QA data on a resource (currently nothing is displayed. Maybe that's fine.
  • Do we need to integrate the 'reason for rating' string into the UI?

Next Steps:

  • Use db dump of thedatahub.org for final testing.
1324459327000000 1327582702000000
#99 enhancement rgrp rgrp closed wontfix Test DbIndex with sqla 0.5

Test DbIndex? with sqla 0.5 (and get it working if it does not work).

Cost: ??

Priority: low because DbIndex? is not currently that important to us (also wonder whether we should try using CKAN code here to stop us reinventing the wheel).

1249985537000000 1297081088000000
#856 defect wwaites dread closed invalid Tests for caching

Test coverage for ckan.lib.cache is only 40%. Can that be usefully improved?

1291724064000000 1311177085000000
#857 defect dread closed invalid Tests for CLI

Test coverage for ckan.lib.cli is 34%. Need to improve that.

1291724131000000 1311182581000000
#858 defect dread dread closed fixed Tests for diff

Test coverage for ckan.lib.diff is 24%. Need to improve

1291724171000000 1291734297000000
#859 defect pudo dread closed invalid Tests for User model

Test coverage of ckan.model.user is 53%. Notable exception - setting password. Room for improvement?

1291724348000000 1311177461000000
#1739 enhancement kindly kindly ckan-sprint-2012-02-06 closed fixed Fix issues where tests where not run.

Test failures in master due to tom and rufus not running tests. 0.3d.

1328047129000000 1328495826000000
#1055 defect dread dread ckan-v1.4-sprint-4 closed fixed @search_related tests not running

Tests marked decorated "@search_related" should only be run against postgresql, but in fact they don't get run at all.

1300985228000000 1300992395000000
#2723 defect aron.carroll shevski demo phase 2 closed fixed Search result summary badly displayed

Text goes over the order by text, with drop down floating on top meaning it's all impossible to read


1342949143000000 1343136190000000
#1183 defect johnglover nickstenning ckan-sprint-2011-10-28 closed fixed Downloads "Preview" button doesn't preview.

The "Preview" button is a nice idea, but it doesn't seem to actually "preview" anything if the file MIME type would ordinarily cause the browser to download the file. If so, the browser does indeed just download the file.

This is notable in the context of most hosted file services (including Google Storage) which will deliberately serve a MIME type of application/x-some-junk-here in order to force a download.

1307701669000000 1311771069000000
#2812 enhancement toby markw demo phase 3 closed invalid Groups description needs improving

The 'What are Groups?' text at <http://demo.ckan.org/group> is a bit naff. Here is a suggested replacement:

What are groups?

Groups allow you to group users and data together so that they are easier to manage. For example datasets in a group could be visible only to group members, or publicly visible but only editable by group members.

Groups can be used to collect together datasets from a single publisher, or related to a particular project, for example.

1344419953000000 1344524290000000
#1679 enhancement dread ckan-backlog new Default roles problem

The 'editor', 'anon_editor' and 'reader' roles are intended to have immutable actions. This was designed to prevent their names being subverted - e.g. an editor should always be able to edit! It also meant that when we add Actions (e.g. DELETE-PACKAGE) then it can be added sensibly to these roles in an upgrade just by changing the defaults table (ckan/model/authz.py).

The problem is that this immutability is only enforced on 'db upgrade'. So you can happily change the editor role using the paster command and it works, right up until you do an upgrade and realise permissions are different.

We should stop the paster commands being able to edit these roles. Or get rid of the immutability completely. Views?

1326823042000000 1339773923000000
#2411 enhancement toby markw ckan-future new .ini file is a mess

The .ini file needs cleaning up in assorted ways which Toby will expand upon.

1337686900000000 1338210872000000
#796 defect pudo ckan-v1.3 closed invalid AlphaPage for non-latin languages

The AlphaPage? implementation assumes that all packages start with a latin character. This is not true for Russian, Chinese, Arabian and Persian languages etc. We should either use a unicode solution (i.e. display a full class of characters based on the current locale) or use generic paging instead.

1289296662000000 1291640040000000
#1581 enhancement [email protected] johnglover ckan-future new Blog post about Google Analytics extension for CKAN

The CKAN Google Analytics extension has been updated to work with the latest version of CKAN, could make for a nice blog post.

Can ping John Glover in January for any details required.

Key link is: http://thedatahub.org/analytics/dataset/top though this should probably move to be under stats (e.g. http://thedatahub.org/stats/usage)

1324402800000000 1325474274000000
#318 defect johnglover wwaites ckan-sprint-2011-10-28 closed fixed Insufficient validation of resource URIs

The CKAN instance on data.gov.uk serves invalid URIs out of its API.

For example the following can be found,


In this URI, the : and / characters after the ? in the query part are invalid according to section 3.4 of RFC2396

Also URIs are not stripped of whitespace at the end.

This causes problems when other software with a more correct interpretation of what a valid URI is attempts to consume data from CKAN. In this instance the Talis triplestore complains about such URIs.

"Be liberal in what you accept and conservative in what you send" would seem apt.


  • Validation of urls as part of form entry or data loading
    • Need to consider situation where this should happen out-of-band (i.e. we allow load even with invalid data and then flag bad dates in separate validation process). In general doubtful that we should do this here because url invalidity is such a big deal
  • This code should support analysis of existing data so we can go through existing database and find invalid urls
    • Also useful to have this so we can do out of band validation
1274302723000000 1349778662000000
#849 defect pudo closed duplicate Import extra CSS from main template

The CSS @import directive at the bottom of the main CKAN css file is not understood in all browsers, move this to a seperate import.

1291714923000000 1291715179000000
#1152 enhancement amercader amercader ckan-backlog new True support for generic CSW servers

The CSW harvesters implemented at the moment were developed with the DGU project in mind, and they assume all remote CSW servers to implement the Gemini 2 specification. Gemini 2 is the profile defined in the UK for INSPIRE complying metadata, so obviously catalogs from other countries or non-INSPIRE complying ones won't be able to be harvested.

The changes needed to support generic CSW servers (i.e. those implementing the ISO 19139 profile) are:

  • Handling the validators (right now are hardcoded in the harvester

code). This probably involves issues discussed in the CREP 3 (ticket #1134)

  • Changes in the model to adapt the specification to ISO 19139
  • Renaming objects and classes which are now Gemini-centric

List of CSW servers tested:


1306141334000000 1313411822000000
#2902 enhancement ross ckan 2.0 new genshi is used in Group controller

The Group controller uses the genshi module where it may be better for it to use Markup() in the template.

See https://github.com/okfn/ckan/pull/117/files#r1524863

1346782230000000 1346782230000000
#1639 enhancement seanh seanh ckan-sprint-2012-02-06 closed fixed Move activity streams CSS into external file

The HTML and CSS for the activity streams templates needs to be tidied up. Currently the CSS is inlined in the templates themselves, and there is much unnecessary repetition of CSS.

Perhaps put this one in hold until there has been some design discussion about how the activity streams and the user, dataset and group pages with activity streams should look?

1326305019000000 1328526779000000
#1784 enhancement seanh seanh ckan-v1.7 closed wontfix Links to renamed objects break

The HTML templates for some activity stream activities link to objects (packages, resources, etc.) by their names, e.g. /dataset/foo, but if the object has since been renamed then the link will be broken.

Possible solutions are to use the object ID to find its current name at rendering time and link to that, or link to the object by ID rather than by name.

Linking by ID is both simpler to implement and better. Links by ID are ugly, but they are permanent and don't break when an object is renamed. We should be using links by ID everywhere on CKAN, so that when third-party sites copy-paste our URLs and link to us, their links don't break later.

Fixing this means fixing the dataset_link(), resource_link(), group_link() etc. functions in ckan/lib/helpers.py which will change the links everywhere in CKAN where those functions are used to link to objects (which should be everywhere where we link to an object).

If we want to have nice looking by-name URLs and what them to be permanent, then we need to change CKAN so that objects cannot be renamed.

1328613555000000 1338204797000000
#1145 enhancement timmcnamara ckan-backlog new Support the Handle System

The Handle System is an initiative to provide persistent references for resources. That is, it's basically a proxy system for preventing link rot.

Its documentation is here: http://www.handle.net/. Servers running CKAN could host a "Local Handle Service", which redirects a hash of a resource to an actual URL.

Some suggested use cases:

  • Researcher would like to cite where data came from
  • Agencies would like to have a way to prevent vendor lock-in from CKAN if they decide to move to another platform
1305764775000000 1339774502000000
#370 enhancement [email protected] ckan-backlog new Use better email encryption for author_email and maintainer_email

The JavaScript? email encryption used is not very reassuring. Google's MailHide? is a much better solution that is easily implemented.


Check on the Mailhide API where there are even some Python libraries already built.

1279821819000000 1339774649000000
#2850 enhancement aron.carroll demo phase 4 new Add JSON support for IE7

The JavaScript? just isn't going to work without it. Look into either the newer JSON3:


or the old faithful:


1344940692000000 1344940692000000
#1795 enhancement ross ross closed wontfix Add approval_status to Package (as for Group)

The Package model should have a approval_status as the Group model does. See migration 049.

1329234060000000 1338206587000000
#2879 enhancement ross ross ckan 2.0 closed wontfix Datasets (Package) should have a 'public' field

The Package model should have a boolean field added to it describing its visibility beyond that supplied via auth.

The related permission for viewing packages where public is set to False would be package.view (show, whatever), and *never* visible for unauthenticated users.

1345466389000000 1350562096000000
#1708 defect dread dread ckan-sprint-2012-02-06 closed fixed SOLR configuration lost

The SOLR url, user and password defined in the CKAN config file are ignored and the default SOLR url is used.

This causes:

  • "0 datasets" displayed on the home page
  • Dataset searches result in 0 results and a small message "There was an error while searching". (Nothing about it in the logs)

To reproduce

This bug is only visible if your SOLR instance is not at the default place. To quickly reproduce this problem, setup your machine as a SOLR multicore instance and run: "paster db clean && paster create-test-data && paster serve development.ini". It quits with the error: "solr.core.SolrException?: HTTP code=400, reason=Missing solr core name in path"

Code affected

1327493428000000 1327580995000000
#1013 defect sebbacon sebbacon ckan-v1.4-sprint-5 closed fixed Refactor & write tests for Wordpresser extension

The Wordpresser extension seems to be working well for DataGM, but needs caching and tests.

1298887392000000 1301909717000000
#2957 task dominik new New datastore on postgres prior to 9.0

The GRANT SELECT ON ALL and ALTER DEFAULT PRIVILEGES are not available in Postgres prior to version 9.0.

1349292214000000 1349292214000000
#2773 enhancement markw markw ckan-v1.9 new About page needs improving

The about page for the DataHub? (thedatahub.org/about) could be improved. More importantly the default about page for a generic CKAN instance should be completely different - focus more on Open Data rather than the community hub idea (as this is more relevant for most installations) and remove specific references to tdh.

1343646795000000 1343646795000000
#1694 enhancement seanh seanh ckan-sprint-2012-02-06 closed fixed Add quick functional test(s) for activity streams HTML rendering

The activity streams feature is already thoroughly tested via the logic layer and API calls, but there needs to be a quick test of the rendering code, e.g. fetch a user's page (thus testing that ckan doesn't crash while rendering it) and then assert that the expected activity streams HTML is in the fetched HTML.

1327322248000000 1328465254000000
#1689 enhancement kindly dread assigned List deleted datasets in API

The admin extension allows deleted datasets to be viewed, but there is no equivalent in the API. The package_list API call filters to just 'state=active' datasets.

'state' could be a parameter on the package_list call

  • Easy to call and implement
  • But getting permissions right for the different state values may not be easy - maybe a separate logic function is better?
1327062214000000 1340190040000000
#1792 enhancement toby kindly ckan-sprint-2012-03-05 closed fixed [super] api to use same validation and logic as wui forms.

The api currently uses the default validation schemas regardless of the form you specified on the front end. There should be a way to post through the api using the same validation rules.

1329146487000000 1333037710000000
#572 story johnbywater johnbywater ckan-v1.3 closed Write CKAN package from metadata document

The attributes we need to read in are being defined here: http://okfnpad.org/uklii

The design advice is to prepare, test, and maintain a dictionary of XPaths for each of the attributes.

Work effort is to be directed to the maintainability of the XPath statements. We need to know which are needed for which documents, and to have a way of preventing cruft.

1284040784000000 1288038237000000
#1371 enhancement johnglover johnglover ckan-sprint-2011-10-10 closed fixed Make logic layer functions for working with task_status table

The background tasks will use the logic API to access the task_status table so new logic layer functions are needed.

1317808776000000 1324037905000000
#941 enhancement thejimmyg rgrp ckan-v1.5 closed wontfix Submit apps or ideas for apps related to datasets (extension)

The basic purpose of this extension is provide a way to record 'apps' (applications) that relate to datasets in CKAN. Both existing apps and ideas for apps should be permitted.

1296403099000000 1310134339000000
#2968 defect seanh ckan 2.0 new Anyone can access organization members page

The button will not show if you are not authorized but browse to /organization/members/foo and you can edit the members, it does stop you when you try to save your changes, but you shouldn't be able to get to the page at all

1350296355000000 1350297070000000
#2949 defect amercader amercader new Reenable Data API button on the new theme

The checks to show or not the button need to be updated for the latest datastore version

1349107464000000 1349107464000000
#890 enhancement kindly pudo ckan-v1.6 closed invalid Introduce timed actions into ckanext-queue

The ckan queuing system should provide the option to subscribe to timed re-submissions of specific resources. This could look as follows:

  • routing_key: Package
  • operation: daily
  • payload: pkg.as_dict()

Where operation is one of daily, weekly, monthly or any other interval.

1294050769000000 1318599247000000
#2384 enhancement dread dread ckan-sprint-2012-05-15 closed fixed Rights tool factored out

The command line tool 'rights' is quite handy but it is glued to the CLI. I'm going to factor out the bit which searches for objects etc so it can be used by CreateTestData? etc and will be used by DGU.

1337080794000000 1337100810000000
#854 defect johnlawrenceaspden dread ckan-v1.4-sprint-6 closed fixed Tests for authorization_group

The coverage tool (run by buildbot in the ckan build) reports that only 24% of lines of ckan.controllers.authorization_group are run in tests and 38% of ckan.forms.authorization_group. This suggests a need for more tests.

1291723815000000 1304351843000000
#855 defect pudo dread closed invalid Authenticator tests

The coverage tool (run by buildbot in the ckan build) reports that only 38% of lines of ckan.lib.authenticator are run in tests. This suggests a need for more tests.

1291723967000000 1311176988000000
#1663 enhancement seanh seanh ckan-sprint-2012-02-06 closed fixed API call for creating an activity streams event

The create_activity() logic function already exists, but validation and authorization need to be added before it's hooked up to an API call.

1326794844000000 1328013153000000
#2679 enhancement icmurray icmurray ckan-v1.9 new Change default behaviour of TemplateController.view to 404.

The current behaviour of TemplateController?.view() (which is the fallback controller should all others fail) is to attempt to render (as a genshi template) the requested file.

Although this may be a feature that some instances want. In general, it leads to:

  • 500s when attempting to access a normal template (eg - http://datahub.io/importer/preview)
  • A way of inadvertantly serving things you may not want to serve. (Small risk, as it needs to be renderable as a genshi template).


  • Change the controller to 404
  • Ensure there's a way for existing ckan instances to override that behaviour should they need it.
1342436133000000 1342436133000000
#1172 defect dread thejimmyg ckan-sprint-2011-10-28 closed fixed Remove all try: except: blocks that don't re-raise the original exception

The current codebase has one or two try: except; blocks that don't catch specific exceptions. Under no circumstances should any broad try: except: blocks be allowed unless the exceptions they catch are immediately re-raised. Uncaught exceptions are wasting us quite a lot of time when trying to track down problems.

1306926049000000 1314118883000000
#926 enhancement sebbacon sebbacon ckan-v1.4-sprint-1 closed fixed Pick a simpler form framework

The current formalchemy setup conflates view, controller and model code in a way that makes it hard to debug and customise.

Review existing (and potentially non-existing) frameworks with a view to porting forms over to something more explicit and lightweight.

Implement the current Package forms as an example of how this would work.

Document and circulate.

Sub-ticket of #961 (form, validation, model sync meta-ticket)

1295869056000000 1298541597000000
#1464 enhancement thejimmyg ckan-v1.6 closed wontfix Replace RabbitMQ with Celeryd to support running multiple instances

The current harvesting implementation can only have one instance per server.

We could either:

  • Accept this
  • Change the RabbitMQ code to support multiple instances
  • Switch the entire harvesting to use Celery
  • Wait for James's redis based live feedback based system
1321375709000000 1328529042000000
#1064 defect amercader closed duplicate Remove Workers from ckanext-queue

The current implementation of Workers in ckanext-queue is broken. Basically the various consume / callback functions expect three arguments (routing_key, operation, payload) when they are in fact receiving only two of them (message_data, message). This is fairly easy to fix, but the question is if Workers add an extra complexity to use the messaging library directly.

1301417891000000 1323169787000000
#1014 defect sebbacon sebbacon ckan-v1.4-sprint-3 closed fixed Decouple oAuth repoze extension from DataNO, write tests

The current oAuth extension tightly couples repoze.who, ckan, and data.no. These components should be decoupled; they also need more tests

1298887493000000 1299245293000000
#1352 enhancement amercader ckan-backlog new Use logic functions instead of as_dict when indexing entities

The current search implementation uses the output of the the as_dict method of the domain Package object to update the index


It also uses package_to_api1 in the SynchronousSearch? plugin:


This prevents extensions from being able to index custom properties (e.g. faceting by custom extras not included in the model).

The search should use the logic function to get the package properties:

1316615397000000 1339774086000000
#1623 enhancement dread dread ckan-sprint-2012-01-23 closed fixed Dump to exclude deleted objects

The database dump currently contains all Packages and their associated objects, even those that have been set to state=deleted. We should exclude these from the dump now.

Dump = paster db simple-dump-csv/json


The dumps are designed for end-users to be able to run scripts on the mass of data. Since end-users don't see state=deleted packages then they shouldn't need them in the dump. In fact their presence in the dump probably confuses them.

Admins get the full database anyway in the backup pg_dump.

We only included them in the user dump because it was designed before use of state=deleted was established.

Time estimate: 2h

1326118987000000 1326892264000000
#1125 enhancement dread nils.toedtmann closed fixed Debian package "ckan" should not depend on "postgresql"

The debian package "ckan" with the two scripts "ckan-create-instance" and "ckan-instance-maintenance" depends against "postgresql". But "ckan-create-instance" is quite handy even when the DB is remote: it creates all the data dirs with the correct permissions, and the ckan and apache configs.

Please add a flag "--without-local-db" to "ckan-create-instance" and remove the postgres dependancy from the debain package.

1304538095000000 1310134813000000
#1610 enhancement ross ross closed duplicate Move webstore to Postgres instead of Sqlite

The default backing store for webstore should be Postgres and not sqlite. This was agreed as part of the #1574 storage changes and on http://ckan.okfnpad.org/newstorage

1325587341000000 1325852472000000
#2370 enhancement seanh seanh ckan-v1.9 assigned Add examples of all config settings in default config file

The default config file should contains examples (maybe commented out) for all config variables. Makes it easier to edit the config. For example openid_enabled is not in there.

1336042237000000 1340624662000000
#1259 enhancement johnglover pudo ckan-backlog new "Add a row" for Extras on Package form

The default package form offers 4 empty extras fields. Like the resource section, it should have an "add more" button to add another row.

1312302693000000 1312907056000000
#1270 defect dread dread ckan-sprint-2011-10-28 closed fixed Default site www.ckan.net is a redirect and loses POSTs

The default site was www.ckan.net, which had an Apache 301 redirect to ckan.net, but the problem was urllib was not rePOSTing stuff. This affected search parameters:

In [1]: import ckanclient

In [2]: c = ckanclient.CkanClient()

In [3]: c2 = ckanclient.CkanClient('http://ckan.net/api')

In [4]: c.package_search('', search_options={'groups':'openspending'})
{u'count': 2102,
 u'results': <generator object _result_generator at 0x10168b1e0>}

In [5]: c2.package_search('', search_options={'groups':'openspending'})
Out[5]: {u'count': 29, u'results': <generator object _result_generator at 0x10168b410>}

Thanks to borior for finding this and raising it.

1312911847000000 1312912188000000
#1066 enhancement dread dread ckan-v1.4-sprint-5 closed fixed Default reader role too permissive

The definition of the 'reader' role includes creating packages, which is too permissive for some CKAN instances (e.g. DGU). 'Reader' suggests only reading, so I think this role should avoid creating and editing.

All projects so far want all roles to be able to create users, so this stays as a Reader action for now, as a convenience.


  • Action.PACKAGE_CREATE removed from reader's default_role_actions
  • Visitor has a new default role, called 'anon_editor' which can edit packages, but not groups / auth groups - you have to log in for that.
  • Migration script not needed?
  • Code comments written, to make clear the suggested policy
1301645250000000 1301932136000000
#2578 defect toby toby demo phase 4 closed fixed demo theme breaks the tests

The demo branch fails huge numbers of tests.

many are going to be template based - these need to be sorted - ideally we need to get them to work with legacy/new templates but see how we go on that one

anyhow this is not a priority whilst we are branched but needs resolving before we merge into master

1340276924000000 1347901892000000
#1592 enhancement amercader amercader ckan-sprint-2012-01-09 closed fixed Add metadata_modified and metadata_created to package_dictize output

The dict returned by package_dictize does not include metadata_modified and metadata_created. These are really useful properties, so it's worth having them on the standard package dict representation, which is used in several places, like at the template level.

1324488909000000 1326304321000000
#822 enhancement Javier de la Cueva <[email protected]…> closed fixed Docs: Configuration as site-available instead of site-enabled for apache

The documentation for setting up Apache uses sites-enabled instead of sites-available and a2ensite demo.ckan.net

1290107103000000 1290506354000000
#1641 enhancement amercader amercader ckan-sprint-2012-01-23 closed fixed ckanext-archiver: Content-length header not reliable to check if resource has been modified

The download task in ckanext-archiver performs a HEAD request on the resource URL and checks if the "Content-Type" and "Content-Length" headers differ from the values stored to see if the resource needs to be updated [1].

The "Content-Length" header, although widely used, is not mandatory and some servers don't provide it, e.g.:

$ curl -I http://portfolio.theglobalfund.org/en/IATI/Activities?countryCode=AFG
HTTP/1.1 200 OK
Cache-Control: private
Transfer-Encoding: chunked
Content-Type: text/xml
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
Set-Cookie: ASP.NET_SessionId=3qhqekddgmre0kmk5cynq0sy; path=/; HttpOnly
X-AspNetMvc-Version: 3.0
content-disposition: attachment; filename=AFG_IATI_12012012.xml
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Thu, 12 Jan 2012 12:36:43 GMT

Also worth noting that requests, the python library that uses ckanext-archiver, sets an "Accept-Encoding: gzip" header by default, which depending on the configuration of the remote web server, may prevent the "Content-Length" server from being sent, e.g.:

$ curl -H "Accept-Encoding: gzip" -I http://iatistandard.org/published-temp/adb-activities.xml
HTTP/1.1 200 OK
Date: Thu, 12 Jan 2012 12:12:46 GMT
Server: Apache
Last-Modified: Mon, 28 Nov 2011 15:55:35 GMT
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: application/xml

curl -I http://iatistandard.org/published-temp/adb-activities.xml
HTTP/1.1 200 OK
Date: Thu, 12 Jan 2012 11:56:23 GMT
Server: Apache
Last-Modified: Mon, 28 Nov 2011 15:55:35 GMT
Accept-Ranges: bytes
Content-Length: 2686720
Vary: Accept-Encoding
Content-Type: application/xml

All this can lead to some resources never getting updated, and of course the size property of the resource not being set.

As we need to download the resource anyway, it would be better to check if the real length of the data has been modified (and store it).

[1] https://github.com/okfn/ckanext-archiver/blob/0a189262dca4ab5b286fb6a02b4ab8a201f639f3/ckanext/archiver/tasks.py#L72

1326376420000000 1326376777000000
#1305 defect nils.toedtmann amercader ckan-backlog closed fixed SMTP config for thedatahub.org and IATI

The email sending functionality (e.g for password reset) does not work on thedatahub.org and IATI (and probably some other instances) when using an address which is not a okfn.org one.

Could not send reset link: SMTPRecipientsRefused({u'[email protected]…': (550, 'relay not permitted')},)

As I said, [email protected]… works fine. The SMTP server used mail.okfn.org

1314956657000000 1315317033000000
#1483 defect johnglover johnglover ckan-sprint-2011-12-19 closed fixed Tasks sometimes fail to update task_status table when trying to report a failure

The error reports do not have a "value". The original task in the task status table is not made and it tries to make a new task, but can't.

Issue: why is the original task_status entry not made?

kindly: there is a chance of course that the task is quicker than the plugin to get to the point where it needs to update the table.

kindly: so we may have to find a way to update the task status before we run the task.

kindly: I think you can supply a task_id to celery. i.e we make our own guid as the task id. So we can use that one to populate the task status.

1322050605000000 1323279151000000
#1463 defect thejimmyg ckan-sprint-2011-11-21 closed wontfix QA extension no-longer works with packaged CKAN 1.5

The extension needs upgrading:

No module named webstore.database

The other option is to delay this functionality and merge with the publisher dashboards after the group refactor.

John, David Raznick, David Read, Ira, what do you think?

1321375614000000 1321874259000000
#1355 defect amercader ckan-backlog new Package extras property does not include the newly created ones

The extras in the package object sent to the extensions after editing (https://bitbucket.org/okfn/ckan/src/01efd5649c10/ckan/logic/action/update.py#cl-226) do not include the newly added.

1317034126000000 1339774056000000
#2310 enhancement icmurray icmurray ckan-backlog new Refactor the search-query construction in feeds.py

The feeds controller, used to construct atom feeds, duplicates code found in the package controller's search action (in order to construct the custom feed).

Refactor this to remove duplication.

1334592091000000 1337159386000000
#816 enhancement johnglover cygri ckan-sprint-2011-10-28 closed fixed Autocomplete for the resource format field

The field for entering the format of a resource should have autocomplete, populated from all the values that have already been entered in this field for any resource.

This would help improve data quality and consistency.

1289996574000000 1319812324000000
#2603 refactor icmurray icmurray ckan-v1.9 new Remove deprecated 'fields' parameter from resource_search

The fields parameter of resource_search was deprecated when fixing #2438. It can be removed in release 1.9, and the action tidied up as a result.

1340730601000000 1340730601000000
#2906 defect dominik demo phase 5 new Wrong flags in language dropdown

The flags in the dropdown on details pages are not right.

You can see the error on http://s031.okserver.org:2375/dataset/malawi-aid-projects/resource/b717c20e-2006-4ad4-82d2-59b57ebc1ab0.

1346861144000000 1346862889000000
#2491 enhancement toby aron.carroll ckan-sprint-2012-06-25 closed fixed Demo related page is missing breadcrumb

The following pages are missing complete breadcrumbs.

Add related page:


Needs to look like.

Dataset > {dataset_title} > Related > Create Related

Template here: https://github.com/okfn/ckan/blob/feature-2375-demo-theme/ckan/templates/related/pages/form_page.html#L6

Same deal with Edit related.


Needs to look like.

Dataset > {dataset_title} > Related > Edit Related


1338977634000000 1338979614000000
#2923 defect seanh ckan 2.0 new Change regularise -> regularize

The function is called regularise_html(), can't remember what file it's in.

1347530582000000 1347530582000000
Note: See TracReports for help on using and creating reports.