{22} Trac tickets (2647 matches)

Results (1401 - 1500 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1503 task dread markbrough ckan-sprint-2011-12-05 closed fixed Move from Bitbucket to Github

Reasons: more people familiar with Git, bigger Open Source community on Github.

Keeping track of the move here: https://docs.google.com/spreadsheet/ccc?key=0AvCDqUH8jVN8dGNVNEpXLWk3UTlJZXhFMjBPbXVtRmc&hl=en_GB#gid=0

1322745084000000 1323793662000000
#1518 defect rgrp markbrough ckan-sprint-2011-12-19 closed fixed CKAN Upload fails if filename has spaces in it

E.g. uploading a file with spaces in it: OECD Monthly Exchange Rates.zip

Gives a 404 Not Found response to the following file: http://test.ckan.org/api/storage/metadata/2011-12-05T193046/OECD%20Monthly%20Exchange%20Rates.zip

The upload wheel keeps spinning and the user is not informed that the upload has failed.

Analysis: turns out that google storage (possibly s3 but not checked) replaces with ' ' in keys with '+' on upload. This breaks things because we try and look up metadata about upload using the filename/key we put in but of course that does not exist because google has changed name.

Fix is trivial: replace ' ' in keys / filenames with '-'.

1323114236000000 1330020742000000
#2406 defect kindly markw ckan-v1.9 new Errors in source install doc

There are a few errors or unclarities in the source install doc - see

http://lists.okfn.org/pipermail/ckan-dev/2012-May/002203.html http://lists.okfn.org/pipermail/ckan-dev/2012-May/002204.html

1337595093000000 1340636804000000
#2407 enhancement amercader markw ckan-v1.9 new 'Access denied' message is unhelpful

This chap tried to use the API to add a dataset to a group and got the uninformative message 'Access denied':


The problem was he didn't have access to the group. A message like 'You don't have access to the group "LODcloud"' would have been clearer.

1337612500000000 1340633440000000
#2408 enhancement aron.carroll markw demo phase 3 closed fixed Only 4 'extras' can be added at a time

The settings > extras page for a dataset allows extra fields to be added, but only four can be added a time in a single atomic update. It would be helpful to have an 'add another' button on the form that can be used any number of times.

1337683639000000 1343657014000000
#2409 enhancement ross markw ckan-sprint-2012-05-29 closed fixed Accept headers not handled correctly

CKAN returns HTML when an Accept header should make it return RDF/XML:


1337683898000000 1337856979000000
#2411 enhancement toby markw ckan-future new .ini file is a mess

The .ini file needs cleaning up in assorted ways which Toby will expand upon.

1337686900000000 1338210872000000
#2415 defect kindly markw ckan-v1.9 new Confusing message on attempting to register new account with upper case

On registering a new account, if you try to use a capital letter in your 'Login', you get the unhelpful error message:

"The form contains invalid entries: Name: Url must be purely lowercase alphanumeric (ascii) characters and these symbols: -_"

What does 'Name' or 'Url' have to do with anything? It means 'Login must be ...'

But even better would be to make the login case-insensitive and allow capitals to be entered.

1337779189000000 1341268304000000
#2479 enhancement markw ckan-v1.9 new Meanings of Author and Maintainer fields are unclear

CKAN's default schema has fields for Author ('The name of the main contact, for enquiries about this particular dataset') and 'Maintainer ('another important contact person'). The distinction is not clear. Also the fields are often left blank.

The roles that seems most important are those of the original owner/publisher of the data, and the person maintaining the CKAN record/copy of it. So I suggest:

(1) Rename the fields 'Owner' and 'Maintainer'.

(2) Change the explanatory text for the relevant fields:

Owner: 'The person or organisation who create/collect/publish the data in this dataset.' Owner e-mail: 'E-mail address for enquiries to the Owner named above.' Maintainer: 'The person maintaining this dataset on [name of CKAN instance], if different from the above.' Maintainer e-mail: 'E-mail address for enquiries to the Maintainer named above.'

(3) When a logged-in user creates a new dataset, the main form should have a checkbox, checked by default, marked 'I am the maintainer of this dataset'. If checked, the Maintainer name and e-mail fields are populated from the user's profile.

1338452898000000 1339771115000000
#2480 enhancement markw ckan-v1.9 new Better message when dataset has no resources

If a dataset has no resources the resources list currently says '(none)'.

Here is a suggested improvement, provided that a maintainer is named: 'There are no data resources here yet. For information about this data, contact the dataset maintainer.'

1338453093000000 1339771086000000
#2483 enhancement markw ckan-v1.9 new Non-local resources should not have Download links

At present, a resource which is just a URL link to an external resource has a 'Download' button on the resource page. This gives the misleading impression that the resource is stored locally.

This is related to another small UI issue: I think the URL of a resource should be much more prominent, not buried in the 'Additional Information' table.

Suggested fix:

  • Put the URL prominently at the top of the resource page (above the preview) (and make it link to the resource)
  • Disable the Download button unless the resource is stored locally.
1338468734000000 1339771043000000
#2709 enhancement icmurray markw new Atom feeds are undocumented

There doesn't seem to be any documentation yet for Atom feeds.

1342624310000000 1342626212000000
#2731 enhancement markw new Some sites permanently 'down for maintenance'

A large number of XXX.ckan.net sites give the following message:

"This Site is Down for Maintenance We apologize for the inconvenience. ~ The Open Knowledge Foundation sysadmins."

The message is unhelpful and patently false - the sites do not exist. Some of them were supposed to have been redirected to a relevant group at thedatahub.org in this ticket (now closed):


However, the redirection only seems to have worked in one case, http://si.ckan.net.

The problem still affects the following sites - the first 4 of which have supposedly been merged:

Please sort this out by redirecting, removing the sites, giving a more helpful (and accurate) failure message, etc, as appropriate.

1343045168000000 1343051608000000
#2773 enhancement markw markw ckan-v1.9 new About page needs improving

The about page for the DataHub? (thedatahub.org/about) could be improved. More importantly the default about page for a generic CKAN instance should be completely different - focus more on Open Data rather than the community hub idea (as this is more relevant for most installations) and remove specific references to tdh.

1343646795000000 1343646795000000
#2812 enhancement toby markw demo phase 3 closed invalid Groups description needs improving

The 'What are Groups?' text at <http://demo.ckan.org/group> is a bit naff. Here is a suggested replacement:

What are groups?

Groups allow you to group users and data together so that they are easier to manage. For example datasets in a group could be visible only to group members, or publicly visible but only editable by group members.

Groups can be used to collect together datasets from a single publisher, or related to a particular project, for example.

1344419953000000 1344524290000000
#2813 enhancement toby markw demo phase 5 new Confusing sidebar on demo dataset page

On a dataset page on demo.ckan.org, the left sidebar is confusing.

  • It starts with some random links. Actually they are links to groups which the dataset is in, but this isn't clear.
  • The sidebar elements that are actually part of the dataset are 'Datset extent' and 'License', so these should be right at the top (if they belong in the sidebar at all), instead of which they are right at the bottom in the junk part of the page (i.e. probably lower than the bottom of the main page, and hence lower than anyone will scroll).
1344420206000000 1344445419000000
#2814 enhancement shevski markw assigned Demo: upload file behaves oddly
  1. Uploading a file behaves counter-intuitively (I would suggest wrongly).

When adding a new resource by uploading a file, I select a file called say create-group.png. I expect the following to happen:

  • the pathname of that file is filled in the box;
  • nothing is actually uploaded till I hit 'add' (confirming that I've got the right file etc).

Instead of this,

1344420360000000 1346670381000000
#2862 enhancement toby markw demo phase 4 new Revised revised groups description

Revised text for 'What are groups?' box at demo.ckan.org/group (after discussion with IB re #2812):

What are groups?

Groups allow you to group users and data together so that they are easier to manage. Group owners can assign roles and authorisations, giving each project or department control of its own data publishing.

Users can browse or facet by groups, which could be an organisation (for example, the Department of Health) or topic (e.g. Transport, Health), making it easier to find the data they are looking for.

1345114322000000 1345115072000000
#838 enhancement memespring memespring ckan-v1.3-sprint-1 closed fixed UI improvements December 2010

This is a meta ticket for a bunch of UI changes:

  • #839 Package page redesign
  • #846 Merge css files
  • #850 Show welcome message
  • #863 Prompt users to enter missing info
  • #864 Search results changes
  • #866 Add download formats to search results
1291296811000000 1295259773000000
#846 enhancement memespring memespring ckan-v1.3-sprint-1 closed fixed merge style.css and ckan.css

no need for separate files

1291635478000000 1291719074000000
#850 enhancement memespring memespring ckan-v1.3-sprint-1 closed fixed Show welcome message on first visit to package page

Control using cookies.

1291721923000000 1291721964000000
#863 enhancement memespring memespring closed wontfix prompt users for missing package information

e.g. license, owner etc

1291729787000000 1338206455000000
#864 enhancement memespring memespring ckan-v1.3-sprint-1 closed fixed Search results UI changes

as per http://ckan.org/wiki/UIRedesignSearch

1291736441000000 1291741028000000
#866 enhancement memespring memespring ckan-v1.3-sprint-1 closed fixed Include download formats in search results

as above

1291812213000000 1292240191000000
#870 defect memespring ckan-v1.3-sprint-1 closed fixed Menu items not selecting correctly

Since the main menu items have changed (part of the UI redesign) the selected items are not functioning correctly.

Specifically if Home / About or Add Package / Search are selected both menu items are highlighted.

This is because the code assumes only one top-level menu item per controller.

1292243583000000 1294914243000000
#895 defect memespring ckan-backlog new Add version number (or simular) to css/js includes query string

Updates to css after a new deploy don't come through without a hard refresh. Adding the version number to the include urls will solve this e.g.


1294343382000000 1339774593000000
#1218 enhancement dread [email protected] ckan-sprint-2011-10-28 closed fixed Colour the History tab icon

Change the color of the "History" tab icon to yellowish, like the rest of the icons in the other tabs ("View" and "Edit", "Authorization").


When you view a data package, for instance http://ckan.net/package/thesaurus-w, you will see "View", "Edit", "History" tabs on the top. "History" tab has a black-and-white icon which makes it look like an inactive/disabled tab (since the text is also grayed out when the tab is not selected). This is not a major issue, but it is a little bit confusing for the users. This icon exists in v1.3.2 and also in v1.4.1a (that runs on ckan.net).

The simple solution would be to change the color of the "History" tab icon and give it the same yellowish color like the rest of the icons in the other tabs (e.g. "View" and "Edit").

1310375768000000 1310389390000000
#1328 defect [email protected] assigned Unicode & paster commands

A possible bug in CKAN when I tried deleting users using "paster --plugin=ckan user delete" command.

To reproduce the bug do the following:

  1. Create a user with an ID (which in my case was a user's full name)

that contains non-unicode caracters like Norwegian "æ", "ø", or "å".

  1. Make sure that you can see something like the example below:

(pyenv) [email protected]:$ paster --plugin=ckan user Users: name=Rustæm

  1. Then try deleting the user with following command:

(pyenv) [email protected]:$ paster --plugin=ckan user delete "Rustæm"

You should now get a python encoding error. I know that this is quite rare case, but in our case it caused some trouble. Could you guys have a look at this bug?

CKAN ver. 1.3.3.

1315823110000000 1340191065000000
#1786 enhancement dread [email protected] ckan-sprint-2012-02-20 closed fixed Relationships lost when editing package

If I have a relationship between two packages, and then I edit any property of one of them (Author email, for example), the relationship will automatically be lost.

1328635226000000 1328640597000000
#206 enhancement nickstenning nickstenning v1.0 closed fixed UI review

Investigate and list (on the wiki at wiki:UiReviewNotes) CKAN's UI issues, in particular:

  • Overview pages: reduction in visual noise and unnecessary elements.
    • best example: package overview page
    • consider openness/downloadability icons
    • display common metadata fields in a more structured fashion?

Est cost: 2 hrs.

1260199941000000 1260881123000000
#207 enhancement dread nickstenning v1.0 closed fixed Reimplement tag selection in jQuery

Remove the current dependency on Yahoo YUI toolkit for doing tag autocompletion on package edit forms.

The (KForge-derived) behaviour files for CKAN can easily be switched to the new jQuery-based ones: see ticket:212. Once this is done, it would make sense to implement the tag-selection code with jQuery.

Relevant pages include:

1260200340000000 1271250740000000
#1183 defect johnglover nickstenning ckan-sprint-2011-10-28 closed fixed Downloads "Preview" button doesn't preview.

The "Preview" button is a nice idea, but it doesn't seem to actually "preview" anything if the file MIME type would ordinarily cause the browser to download the file. If so, the browser does indeed just download the file.

This is notable in the context of most hosted file services (including Google Storage) which will deliberately serve a MIME type of application/x-some-junk-here in order to force a download.

1307701669000000 1311771069000000
#1188 enhancement nickstenning ckan-backlog new Allow diffing against initial (blank) package version

Currently the history page only allows diffing between different versions of a package, but there doesn't appear to be any easy way to see the changes introduced by the first version of a package.

I'm requesting the ability to diff against a "blank slate" initial state of a project, so I can see the content of the first project commit.

Not sure if this is a vdm feature, so I'm putting this ticket in against ckan.

1308153160000000 1339774275000000
#1353 defect nickstenning ckan-v1.5 closed fixed No UI to remove resources

I have no idea whether this was a deliberate decision or not, but there is a total absence of any UI with which to delete resources from the currently deployed version of thedatahub.org.

1316729765000000 1317075904000000
#871 defect nils.toedtmann closed invalid Check whether localhost-only exim installtions need upgrading too

The infamous exim bug only needs one mail with prepared headers to travel through a exim system infect it. All local processes could do that, and some services (e.g. cron, webapps) send messages and might be convinced by malicious remote users to produce evil headers.

We should either rule out that this could happen on our systems, or upgrade all exims regardless of whether they are localhost-only or not.

BTW did we already run a rootkit checker like Rootkit hunter on eu1? If not we should maybe do it now - there was already an exploit out in the wild. ByteMark? has (a) already observed infections and (b) notified us because they remotely fingerprinted our mailer to be exim<4.70 (our EHLO banner contains the exim version), just as anyone could.

1292264117000000 1296340558000000
#1105 defect nils.toedtmann closed invalid test ticket, please ignore


1303508261000000 1303508330000000
#1117 defect thejimmyg nils.toedtmann closed invalid Depend deb package "ckan" against ubuntu package "python-pastescript"

... otherwise the scripts fails.

1304089619000000 1304277240000000
#1123 requirement dread nils.toedtmann closed fixed Please re-package CKAN packages as "noarch"

... or, if the CKAN packages do contain architecture-specific binary code, build packages for i386 too.

Currently, http://apt-alpha.ckan.org/debian only offers packages for amd64, but e.g. "m1.small" EC2 instances are i386.

We would need this in order to migrate the community instances to a packaged based CKAN.

Rufus, pls prioritise.

1304530050000000 1311863806000000
#1124 enhancement thejimmyg nils.toedtmann ckan-sprint-2011-12-05 closed fixed push apt package python-ckanext-solr into our debian repository

python-ckanext-solr is already available in http://apt-alpha.ckan.org/datanl-dev, but not yet in http://apt-alpha.ckan.org/debian (that is why we had to [pip-install it for DataGM). Please push into main repo.

1304537793000000 1323168156000000
#1125 enhancement dread nils.toedtmann closed fixed Debian package "ckan" should not depend on "postgresql"

The debian package "ckan" with the two scripts "ckan-create-instance" and "ckan-instance-maintenance" depends against "postgresql". But "ckan-create-instance" is quite handy even when the DB is remote: it creates all the data dirs with the correct permissions, and the ckan and apache configs.

Please add a flag "--without-local-db" to "ckan-create-instance" and remove the postgres dependancy from the debain package.

1304538095000000 1310134813000000
#1154 enhancement johnglover nils.toedtmann ckan-sprint-2011-10-28 closed fixed Make ckan robust against solr failure

According to pudo, a ckan with activated solr extension throws a 5xx when solr is unreachable. Instead, it should behave more like a ckan without ckanext-solr when this happens.

1306254472000000 1314287519000000
#1165 enhancement nils.toedtmann ckan-future new Add multi-site support to ckan

Currently, each ckan site needs its own ckan wsgi process. That eats a lot of resources where many ckan sites are served from one machine (e.g. eu3).

That would dramatically change if a ckan process could behave like multiple ckans (e.g. like Apache's "<VirtualHost?>", or tracd). Depending on the "Host:" header in the HTTP1.1 request, it would choose which local ckan ini file to obey.

I see two ways to constitute the map hostname-to-ini-file map:

  • ckan reads a set of ini files, and each ini file declares which servers names it is responsible for
  • In a global ini file, there are directives mapping servernames to ini files.

In either case there should be a global ckan ini having the default settings for all local ckan sites. Each site ini could be very short then, just having e.g. title, name, database credentials, active plugins etc.

1306413667000000 1339774466000000
#1167 task nils.toedtmann closed fixed Create a standard CKAN system image for Amazon EC2 (AMI)

We should create a public AMI with CKAN pre-installed and configured such that users can easily create their own EC2 machine with a running CKAN to play with.

There are three phases:

  1. [nils] Deploy an empty EC2 instance to become the CKAN image master instance
  2. Install a CKAN and give it a standard configuration.
  3. [nils] Create a AMI from the CKAN image master instance and publish it.

I am happy to do first and last. Who is installing and configuring CKAN?

Unfortunately AMIs are specific to region, architecture and storage type. We cannot maintain too many images, so a number of choices have to be made:

  • Which distribution/version? Ubuntu 10.04 LTS
  • Which architecture/instance-type? I suggest 64-bit/t1.micro
  • Which region? I suggest us-east-1 and maybe eu-west-1
  • Which storage type? EBS (way easier to make an AMI from than instance-store)
  • Install CKAN from deb packages via mercurial/virtualenv? I assume the latter because the AMI is targeted to developers?

1 2

1306423016000000 1311178516000000
#1345 enhancement kindly nils.toedtmann closed fixed Investigate possible memory leak

There is some evidence pointing to CKAN handling memory inefficiently or even leaking under certain conditions:

When we migrated ckan.net/thedatahub.org from eu7.okfn.org (32bit) to s053.okserver.org (64bit) (ticket) we experienced extraordinary memory usage peaks (ticket). Here are the observed value with Apache default settings:

  • eu7, mpm-prefork: base level ~0.6GB, peaks up to 2GB
  • s055, mpm-prefork: base level ~1GB, peaks up to 4GB
  • s055, mpm-worker: base level ~1.5GB, peaks up to 6GB

William reduced the life-time of a WSGI CKAN process from 500 requests down to 25 requests (changeset). This (together with two other tweaks) changed the situation drastically:

  • s055, mpm-event: base level ~1.4GB, no peaks

This suggests that the more requests a CKAN processes serves over time, the more memory it consumes, aka bad memory management or a leak.

To prove this theory, one could reduce the total number of WSGI CKAN processes as much as possible without killing the performance (e.g. down to processes=3), and then observing the relation between maximum-requests=25...500 and memory consumption.

On 14/09/11 17:49, David Read wrote:

Someone to do a bit of top-down memory-use profiling would be very useful. Also useful would be something in the tests that reported what test cases use lots of memory - this could be in the nose plugin.


1316041450000000 1343215489000000
#1415 enhancement thejimmyg nils.toedtmann ckan-sprint-2011-12-05 closed fixed Comments on current status of ckan deb packages

This is a scratch pad ticket with some comments on the current status of our ckan deb packages. I know that some of it is the deb packaging roadmap anyway, please forgive me if i mention them here again.

Rufus and me re-deployed some community ckan instances onto s022 (see http://trac.okfn.org/ticket/926). We followed the documentation http://docs.ckan.org/en/latest/install-from-package.html

  • Deb package version number: the version of the deb package is "python-ckan 1309471251~149be76faabc+lucid-1", and it's hard to guess from there that it contains a ckan 1.4.2a
  • When is 1.4.3/1.5.x expected as deb package?
  • There was a bug in the DB upgrade script /usr/share/pyshared/ckan/migration/versions/029_version_groups.py (line 150) which looks like it was fixed 1.4.1==>1.4.2 but was nevertheless present in this deb package.
  • The current script /usr/bin/ckan-std-install
    • does not set the Apache ServerName? according to the $INSTANCE variable
    • automatically configures a ckan extension named after $INSTANCE
    • depends on local postgres
    • could be replaced with "/usr/bin/ckan-deploy --name=ckan-std --domain=ckan-std.localhost (see next point)
  • (i think this is exactly James' plan): have more generic deployment script /usr/bin/ckan-deploy as part of python-ckan which takes arguments like
    • --domain=cc.ckan.net
    • --aliases=$list-of-domains
    • --name=cc (defaults to "domain")
    • --no-db (does not configure a DB)
    • --sql-alchemy=$DB_CONFIG_STRING (also runs "paster --plugin ckan db upgrade --config")
    • --extension $list-of-extesions
    • ...
1319457069000000 1323167941000000
#2485 defect icmurray nils.toedtmann ckan-v1.9 new Encourage leak containment by limiting the number of requests a CKAN process serves

CKAN has memory leaks. They can be contained by limiting the time-to-live of a ckan process. An easy way to achieve this is to limit the number of requests a ckan server process can serve before it gets killed and replaced.

One should ...

  • research ways to limit requests-per-process with the different web servers (see below for a start);
  • explain these safeguards in the CKAN documentation and encourage users to apply them;
  • consider the helper script ckan-create-instance to generate Apache configs with WSGIDaemonProcess ... maximum-requests=XY being active instead of commented out. Or at least let it warn the user to use MaxRequestsPerChild;
  • Investigate current CKAN deployments whether they suffer from mem leaks, and if so contain them.

How to limit requests-per-process

  • Apache:
    • Use WSGIDaemonProcess with maximum-requests=50 or whatever limit is appropriate. We did this sucessfully on datahub.io and the ckan farm
    • (This need to be verified) Without WSGIDaemonProcess, MaxRequestsPerChild 50 should achieve the same.
  • nginx/supervisord: to be researched.
1338551229000000 1340636028000000
#741 enhancement ollyc ckan-v1.2 closed fixed Extension / Plugin system for CKAN

Create plugin system using <http://pypi.python.org/pypi/pyutilib.component.core/>

Specification in CEP-003 (browser:doc/cep/cep-003.txt) and <http://okfnpad.org/ckan-plugins>

1287996890000000 1291989194000000
#327 defect pudo pudo closed wontfix Create a web hook worker for CKAN

This will be useful as an example consumer of the new queue notifications (#325).

  • A user can register any URL to be notified upon an event.
  • Possible extensions: GET with ID only vs. POST with serialized object
  • E-Mail notifications are also hooks, essentially.

We need to have a UI module that allows for the hook CRUD and some feedback (e.g. non 200 status codes)

1274807361000000 1296467361000000
#395 task pudo ckan-v1.3 closed duplicate Set up profiling to analyze performance issues

At the moment, some pages within CKAN tend to load slowly. We should create a profiling setup in which we can measure response times for complete requests and individual methods calls.

This could be used to identify bottlenecks and find an appropriate caching or tuning strategy to improve CKAN performance.

NB: We should also agree on a maximum request latency.

TODO: Read up on all those QoS tickets to avoid overlapping efforts.

1280824739000000 1294417538000000
#401 task pudo pudo closed fixed Provide base worker as queue consumer

We should create and document a basic queue consumer and task processor for CKAN. It will connect to the CKAN update queue, read messages and dispatch them to a consumer function (such as a downloader or an analysis app)

1281018724000000 1286189243000000
#402 task pudo pudo ckan-v1.3 closed duplicate Archiving worker to back up package resources from a CKAN instance

Write a worker that scans all packages in a ckan instance and uploads the data to storage.ckan.net or another suitable storage system.

  • Naming scheme?
    • Bucket: {ckan-instance-id}-{package-name}? {ckan-instance-id}-{package-id}?
      • What happens if names change
    • File: filename? hash?
  • Store hash back on ckan instance?

The caching worker will consumer update notifications and fetch packages.

Extra points for:

  • Properly checking for source file modification (Last-modified, Etag)
  • Using PIP VCS Backends for retrieval
  • OFS/S3 Storage
1281018912000000 1296467635000000
#403 requirement pudo closed fixed Create web service for OFS storage

Create a thin web service that will allow users to upload files to OFS on S3 without knowing S3 credentials. This could also provide static urls for all stored resources.

1281346110000000 1287392451000000
#404 requirement rgrp pudo datapkg-0.8 closed fixed Make registries pluggable

Allow using different indexes, including ckan, file, dcat, ...

1281346690000000 1297072955000000
#408 requirement pudo pudo ckan-v1.2 closed fixed Integrate CMS via proxying
  • As sysadmin/user of CKAN I want to integrate a CMS to support easy addition of additional content ...
  • I want to integrate with navigation
  • As a minimum I want to be able to add content pages (more stuff would be e.g. comments, sidebars, main menu ...)

Why: CMS are great at creating content and have very good editors and theme support ...

Previous work


  • deliverance: retheming on the fly


  1. Rewrite urls for proxying
  2. You need a theme to substitute into
  3. pylons

Extreme version:

  • 2 sources -> pylons/ckan app, cms
  • 1 theme source: ckan or cms

1 frontend/proxy app which re-themes everything

Owned by CKAN:

  • package/
  • tag/
  • group/
  • api/
  • /

Everything else proxied.

1281348581000000 1281609572000000
#410 requirement pudo pudo ckan-v1.2 closed fixed Latest comments widget on home page

This would be an extension to the disqus plugin that would patch itself into home/index.

1281348733000000 1281619720000000
#411 requirement pudo pudo ckan-v1.2 closed fixed Deploy and test Varnish on eu3
  • Need to figure out cache invalidation for CKAN
  • Evaluate against an internal cache (i.e. memcached)
1281349275000000 1287393033000000
#430 task thejimmyg pudo closed wontfix Upgrade CKAN to Pylons 1.0

Pylons 1.0 has been out for a while and appears to be running stable on a few production sites. It also introduces a few new features, among them the ability to specifiy controllers using the entrypoint mechanism.


This might be a bit ugly since the default behaviour of some pylons objects, such as tmpl_context, app_globals and redirect has changed.

1281962485000000 1298283075000000
#431 task pudo pudo closed wontfix Solr index testing tool

There seem to be a few conditions under which either queue processing or the indexer fail in their current state. To get a more systematic picture of these failures, we should have a small testing tool to compare the index to the database of a live CKAN instance.

1281963532000000 1295260262000000
#501 requirement pudo ckan-v1.2 closed duplicate Read-only maintenance mode

CKAN should have a read-only maintenance mode with a nice little banner on all pages, appropriate REST messages etc. Bonus points if this is triggered via an environment variable and thus can be triggered by the surrounding apache.

1282554617000000 1282724566000000
#517 requirement pudo pudo iati-1 closed fixed Standard domain model for entries

cf. Tariqs mail

Updated version: https://spreadsheets5.google.com/ccc?key=tuOtQjD0Psoqr1pWTS8EXZQ&hl=en#gid=0

Original Version

Publishing Entity:
Publishing Entity Type: (Donor, Recipient, Community Data..)
Activity period:
Verification status: enumeration of statuses (checked, not checked etc)
Resource links: to the actual IATI record
Number of activities: ...
Date record updated:
Date data updated:
License: Need this field even if it may be a standard license

So naively mashing these together, we get something like:

Publisher Type
Verification Status
Contact e-mail
Resource format
Resource URL
Resource ID
Activity period
Number of activities
Date record updated
Date data updated
1282893081000000 1284041561000000
#518 requirement pudo pudo iati-1 closed fixed IATI WUI Editing
  • Create custom form
1282893167000000 1283896718000000
#519 requirement pudo pudo iati-1 closed fixed IATI look and feel
  • Use what is on iatiregistry.org at the moment.
1282893224000000 1283536828000000
#520 requirement pudo pudo iati-1 closed fixed Browseable web interface onto the data
  • e.g. find/browse by country and by publishing entity and by donor
1282893270000000 1283538080000000
#521 requirement pudo pudo iati-1 closed fixed IATI Customized search 1282893401000000 1283897124000000
#522 requirement pudo pudo iati-2 closed fixed Create publishing entity account
  • Publishing entity emails system editors
  • System editor creates publishing entity account and emails publishing entity with details
  • Priority: 4/5
1282893668000000 1287392999000000
#523 requirement pudo pudo iati-1 closed fixed Read/write APIs for users and publishing entities

Read/write APIs for users and publishing entities to access and register data

  • Users have read access
  • Publishing entities have both (though can only write to 'their' records)
  • Priority: 4 for write (read access is 3)
1282893743000000 1283897688000000
#524 requirement pudo pudo iati-2 closed fixed Limit registering data to authenticated users either via API or manually.

Limit registering data to authenticated users either via API or manually.

  • Edits may only be made by authenticated entities
  • Priority: 5
1282893904000000 1285594971000000
#525 requirement pudo pudo iati-2 closed fixed Confirm and approve publishing entity account

  • So some kind of confirmation or approval by someone more senior?
  • Alternative: just have an account request.
  • Priority: 3
1282894040000000 1285595152000000
#526 requirement pudo pudo closed wontfix Integrate validation tool into the registry

Assumes 8 is done:

  1. Tool for validation an IATI XML record (referenced by a URL) for compliance with the IATI specification
  • Outside of scope of registry
  • Multiple validators with different levels of rigour
1282894300000000 1340626152000000
#530 task pudo pudo closed fixed Map IATI attributes to CKAN metadata schema

From lopad:

Publishing Entity (often/always? equal to the logged in user)
    * Domain logic: only can add entries for "yourself"
Activity period: period to which this record relates
Verification status: enumeration of statuses (checked, not checked etc)
Resource links: to the actual IATI record
Number of activities: ...
Date record updated:
Date data updated: 
License: Need this field even if it may be a standard license
1282899187000000 1287392210000000
#544 requirement pudo ckan-v1.3 closed duplicate Backport facet browsing to CKAN 1.2

This is in IATI, would be nice to have in generic CKAN.

1283267292000000 1291638966000000
#557 defect pudo ckan-v1.3 closed invalid Remove siblings from link_to/from relationship display

Displaying siblings is not needed for link_to/from.

1283542595000000 1297075159000000
#558 defect pudo ckan-v1.2 closed fixed Sort extras, relationships by key/object alphabetically 1283542680000000 1283781883000000
#559 defect pudo pudo ckan-v1.5 closed invalid Fix package lookup on group edit form

Lookup is needed to add a package but won't show up at some times.

Originally reported by Anja Jentzsch, re questions.

1283542776000000 1310127694000000
#561 defect pudo ckan-v1.2 closed duplicate Deleted packages are returned in the API

Anja is reporting this, severe bug, I think.

1283775578000000 1283775711000000
#562 defect pudo ckan-v1.2 closed fixed OpenID Login fails using Yahoo and Google

reported by LOD folks.

subsidary bug: error messages from OpenID providers are not shown on the form return page.

1283941119000000 1284063574000000
#647 requirement pudo pudo ckan-v1.3 closed fixed Authorization Groups for group-based access control

allow group-level authorization on all system entities

1284889418000000 1287391981000000
#648 requirement pudo ckan-v1.3 closed fixed Enable lock-down of package creation
  • copy exisiting tests, modify authz in setUp, adapt and extend tests
    • Problem: default_role_actions is read by init_db
    • Solution: nuke db after monkey-patching
    • role = model.Role('Reader'), del role.actions[...]

self.PRE_MAUTHZ_RULES = copy(mauthz.default_role_actions) mauthz.default_role_actions.remove((Role.READER, Action.CREATE)) #raise Exception(mauthz.default_role_actions) model.Session.remove() model.repo.rebuild_db()

  1. Start from the functional

is_authorized(user, Action.Create, model.Package)

-> Doing this will put 'Package' in the context field of the user_object_role table. This will trigger SQLAlchemy to attempt a join towards PackageRole? in all queries. Since for class-level role assignments there never is a PackageRole? join table entry, this will never return any results.

  • have a ckan install that would not allow visitors to either list packages or list groups
    • two paths: create the listing, but for each group/pkg decide that you cannot show this
    • lock down the whole page (/package/list)
      • this is class-based, not object-based

is_authorized(user, Action.Package_Create, model.System()) is_authorized(user, Action.Group_Create, model.System())

[Separating Package and Group roles may be useful going forward: PackageEditor?, GroupEditor? etc]

  1. Find a standard way to lock down classes
    • possibly add default rows in user_object_role
    • introduce lock-down into controllers: group new, package new, REST equivalents,
    • confirm tests
1284889519000000 1284889656000000
#649 story pudo closed fixed Introduce Authorization Groups in model

Authorization Groups:

(Doing this first seems more stable since we first want to "re-gain" exisiting functionality)

  • Create authz.py and model tests for authorization_group-based auth
    • tests: functional: create fixtures, do actual requests

lib-tests: test for membership addition, removal, group-right additions affecting all members, group-right removal, group rights affecting non-members.

# * Adapt model to get these passing

  • Integration into existing controllers should be negligible but may want ot have tests
    • do not want that. would mean design is broken
  • Need to create a new controller for editing authz groups (will have same permissions as any other object -- Reader, Editor, Administrator)
1284889884000000 1284889983000000
#650 story pudo pudo ckan-v1.2 closed fixed WUI editing for Authorization Groups
  • Copy & adapt Group controller tests to suit AuthorizationGroups?
  • Copy & adapt Group controllers and templates for Authorization Groups
1284890073000000 1287391932000000
#651 requirement pudo closed fixed Improve javascript helper for package/user addition on groups/authzgroups

This needs to be abstracted and we still need to be able to remove items before saving.

1284972211000000 1311171104000000
#652 requirement pudo ckan-v1.2 closed fixed Allow for addition and removal of groups on package/edit

Much more natural place to do this, even at the risk of overloading the package edit form.

1284998134000000 1284998201000000
#664 defect kindly pudo ckan-v1.4 closed fixed Sibling relationships shown for deleted packages 1285576600000000 1300371645000000
#669 task pudo pudo closed fixed Refactor Solr to become a generic worker 1286189594000000 1293057161000000
#670 task pudo pudo closed fixed HowTo: Creating a sysadmin 1286190347000000 1293056844000000
#671 task pudo pudo closed fixed Implement and show user names in WUI 1286190454000000 1293056852000000
#672 enhancement pudo pudo ckan-v1.3-sprint-1 closed fixed Facets in CKAN search results

This has been done in IATI ckan but should be ported over to CKAN main and put into the UI in a nice fashion.

IATI: http://bitbucket.org/okfn/iati/changeset/378431974c76 (related to #520)

Requires SOLR search backend (for postgres should fall back gracefully -- just do not show them).

1286190736000000 1291715338000000
#716 requirement pudo pudo iati-3 closed fixed Visual improvments to IATI CKAN 1287583741000000 1290000673000000
#717 bug pudo pudo closed fixed Fix and validate "setup_default_user_roles" in IATI 1287583892000000 1290005099000000
#718 story pudo pudo iati-3 closed fixed Show recent activities on home page 1287583989000000 1288459823000000
#719 story pudo pudo iati-3 closed fixed Improve overall layout of IATI

... probably by adapting aidtransparency.net

1287584092000000 1289910497000000
#720 story pudo pudo iati-3 closed wontfix Include CMS pages in IATI registry

Perhaps using deliverance?

  • Move About to the right of the nav tabs and make it an about IATI page
  • Make a Help / FAQ section linked of main menu
  • Front page more dynamic - latest data registered and news (microblog thing)
  • make the front page blurb noticable / shorter etc.
  • link to iatistandard.org from the front page blurb
1287584189000000 1288459344000000
#721 story pudo pudo iati-3 closed fixed Make the table preview a link to a new page. 1287584244000000 1288459534000000
#722 story pudo pudo iati-3 closed fixed Further adapt l10n for IATI
  • Change "Publishing Entity" to "Publisher"
1287584314000000 1288459703000000
#723 story pudo pudo iati-3 closed fixed Hide action links pointing to unavailable actions (package create) 1287584448000000 1288459610000000
#724 requirement pudo pudo iati-3 closed fixed Improve publisher workflow 1287584548000000 1289835202000000
#725 story pudo pudo iati-3 closed fixed Change to allow anyone (logged in) to create a publisher

With a pending state set ("unapproved")

1287584630000000 1289296038000000
#726 story pudo pudo iati-3 closed fixed Send message to sysadmins to approve new publishers 1287584670000000 1289835192000000
#727 story pudo pudo iati-3 closed fixed Sysadmin can approve a publisher 1287584754000000 1289835196000000
#729 bug pudo closed fixed Assets to be loaded from assets.okfn.org, not m.okfn.org

Move from hetzner box to s3.

1287736519000000 1288012854000000
Note: See TracReports for help on using and creating reports.