{22} Trac tickets (2647 matches)

Results (1001 - 1100 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1666 enhancement seanh seanh ckan-sprint-2012-02-06 closed fixed Add activity stream events for tags

...and show them on the tag pages.

This means adding tag_activity_list() and tag_activity_list_html() logic functions, adding the templating to render tags as HTML, and adding the tag HTML into the tag pages.

What exactly should go into a tag's activity stream? As with datasets we may want a more complex SQL query, where events from objects related to the tag (datasets, users, etc.) appear in the tag's stream.

1326795985000000 1328527113000000
#1665 task seanh seanh closed fixed Begin doing research into eurovoc

How big is it? How are we going to store it? etc.

1326795828000000 1329742600000000
#1664 enhancement seanh seanh ckan-future closed fixed Smarter group activity streams

We already have simple activity streams for groups that have new group or group updated events (see #1631). Smarter/more useful group activity streams would also contain the events for things related to the group: users, datasets, other groups, etc.

This should just mean using a more complex SQL query in the group_activity_list() logic function.

Things might get tricky when a user or dataset joins a group, this may show up as two events (the user or dataset was modified, and the group was modified). We may want to have two separate events for this (one to go in the user or dataset activity stream and one for the group), or maybe we just want one 'user/dataset joined group' event.

1326795654000000 1351863437000000
#1663 enhancement seanh seanh ckan-sprint-2012-02-06 closed fixed API call for creating an activity streams event

The create_activity() logic function already exists, but validation and authorization need to be added before it's hooked up to an API call.

1326794844000000 1328013153000000
#1662 defect dread closed wontfix OpenID not compatible with mounting CKAN at non-root URL

Mounting CKAN at a non-root URL was made to work properly here: #1659

Unfortunately OpenID doesn't play nicely and would require some work to get working.

1326730366000000 1326730414000000
#1661 defect dread ckan-backlog assigned Wrong Routes version installed by CKAN package

Jaakko Louhio reported that the wrong Routes version got installed during CKAN's package install.

He is using Ubuntu 10.04 and he believes it install Routes 1.12.3 instead of what we use which is 1.11.

1326718061000000 1339773949000000
#1660 defect rgrp lucychambers ckan-sprint-2012-02-06 closed wontfix CSV preview broken - OpenSpending

This CSV resource used to preview but now the format appears to be unsupported: "We are unable to preview this type of resource: x-osdata-csv"


This is used as an example by the OpenSpending? team all the time.

1326717846000000 1328013627000000
#1659 defect dread dread ckan-sprint-2012-01-23 closed fixed Cannot logout if CKAN mounted at non-root url

If you set WSGIScriptAlias to mount CKAN at a URL other than / then you cannot logout without adjusting the OpenID logged_out_url to match in who.ini config. e.g.

[plugin:openid] ... logged_out_url = /sub/dir/user/logged_out

Note: all the other URLs in who.ini should not have the /sub/dir/ - it is just this one that doesn't take account of the mounting point.

The solution is to fix-up the repoze.who OpenID plugin to take account of the mounting point.

1326716302000000 1326747205000000
#1658 enhancement ross ross closed wontfix Implement proxy caching

Implement proxy caching for performance gains

  • Analysis of best solution and what we can deliver in a reasonable time [2d]
  • Checks for presence of cookie 'auth_tkt' and appropriate setup for media [2d]
  • Remove session requirements for user who isn't logged in [3d]

  • Document the configuration and setup so that it can be easily modified and replicated [1d]
1326712354000000 1338206622000000
#1657 enhancement ross ross closed wontfix Configuration for reverse proxying

Provide configuration for reverse proxying that will correctly handle the mapping of a URL to a sub-folder (using X-SCRIPT-NAME)

  • Analysis of the best solution [1d]
  • Implement reverse proxying in tandem with #1653 [2d]
  • Document and store the configuration files. [1d]
1326711580000000 1338206640000000
#1656 enhancement ross closed duplicate Configuration for reverse proxying

Provide configuration for reverse proxying that will correctly handle the mapping of a URL to a sub-folder (using X-SCRIPT-NAME)

  • Analysis of the best solution [1d]
  • Implement reverse proxying in tandem with #1653 [2d]
  • Document and store the configuration files. [1d]
1326711575000000 1326711659000000
#1655 task amercader amercader ckan-sprint-2012-01-23 closed fixed Setup issues on s025 (Publicdata.eu)

Time estimate: 2d

  • Fix logs (apache, ckan, harvest): rotate, set suitable levels
  • Fix harvesting jobs: supervisord for gather consumer, cron job
  • Fix backups

Also it may be worth setting up a test instance ( on s023 ?)

1326710844000000 1327312857000000
#1654 task amercader amercader ckan-v1.7 closed fixed [super] Update Publicdata.eu to the latest CKAN stable version

Tasks include:

  • #1813 Update ckanext-pdeu (4d)
  • #1814 Update harvesters (2d)
  • #1649 Verify ckanext-rdf works with latest CKAN (3d)
  • #1815 Reenable Sparql endpoint (?)
  • #1816 Update ckanext-apps (2-3d)
1326710647000000 1338203166000000
#1653 enhancement toby ross ckan-sprint-2012-03-05 closed fixed Change URLs for multilingual site

To support multiple languages we should have an easy way to specify the language as part of the URL, so that URLs are both specific and we also reduce the dependency on the session.

  • Analysis [1d] - Find the best way of implementing this and how everyone else does their language URLs.
  • Write Middleware + update url_for to take account of the language. [2d]

  • Document the language setup, and how to replicate it. [1d]
1326710590000000 1329845387000000
#1652 enhancement kindly ross ckan-backlog assigned How we intergrate with Drupal Multiligual?
  • Analysis of how/where we can integrate with Drupal Multilingual
  • Is it likely to access CKAN via /en/data/... or /data/en/... (not sure, probably former) and we need to make sure that it ends up at /en/...
  • Check with Ian for other issues.
1326709894000000 1340187535000000
#1651 enhancement johnglover dread closed fixed Explicit link mapper

In this commit https://github.com/okfn/ckan/commit/1772a5c John Glover set map.explicit=True in ckan/config/routing.py.

The reason this was done was to avoid links collecting parameters. e.g. if you were on page /dataset/{id}/resource/{resource_id} then by default all the links on that page generated by url_for (Routes) would include the id and resource_id parameters as well. To avoid this, you had to go through all the links and add id=None and resource_id=None to the url_for parameters.

When map.explicit was changed to True, the value of the controller, action, id and any parameters were no longer automatically carried over into the generated links for the page. So previously links within the same controller didn't need to specify the controller (for example), but now they did. So when we did this we also had to fix up links that weren't explicit:

John made the config change on 5/11/2011 which was merged to master https://github.com/okfn/ckan/commit/5a01e67 21/11/2011. The related fixes mentioned were in within the same week. This all went into release 1.5.1. The requirement

1326709852000000 1326711005000000
#1650 enhancement ross ross ckan-future closed wontfix Add new language/translation of fields for datasets.

Datasets should have a field that specifies the language used in the dataset, this should be a two character ISO 639-1 code (see http://en.wikipedia.org/wiki/List_of_ISO_639-1_codes for list) . Also make sure all tests and migrations are resolved. [2d]

Determine the best approach to having 'translation_of' references to show which dataset the current one is a translation of. This may be a translation of field, or a 'copied from' field or another form of package relationship. [2d]

1326709392000000 1346670010000000
#1649 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Verify RDF Extension (ckanext-rdf) works
  • Make sure that the extension works as it should and isn't broken on current releases [3d]
  • Make it work with the EC schema, including extras [2d]
1326709096000000 1338205274000000
#1648 enhancement shevski ckan-backlog closed fixed Clarify that additional info = extra fields + add guidance

Super ticket: #1506

Need to decide which term to use and then have the same for editing as well as viewing a dataset.

In creating/editing a dataset, want more explanation about adding extra fields (probably as a tooltip or similar).. i.e. that this let's you add extra custom metadata such as 'location: uk' which is then searchable etc

1326674843000000 1330632344000000
#1647 enhancement shevski ckan-backlog new add links to ckan discuss & dev to thedatahub

In the footer as well as more clearly & directly on the About page

1326673852000000 1326707383000000
#1646 defect zephod dread ckan-sprint-2012-01-23 closed worksforme Resource navigator options display spuriously

When viewing a dataset, the "Resources" navigation button contained the Resource titles on the Resource navigator button, instead of in a drop-down mouse-hover menu.



1326452700000000 1327407044000000
#1645 enhancement icmurray icmurray ckan-sprint-2012-01-23 closed fixed Update and test existing DGU package form : Apply a simple theme

Theme the DGU form.

Doesn't need to be an exact replica of DGU, but just enough to show it's possible.

1326394622000000 1327311679000000
#1644 enhancement shevski ckan-backlog new Order default dataset page by most downloaded resources on thedatahub

Instead of alphabetically as we do currently, alternatively by most viewed datasets

for http://thedatahub.org/dataset

1326393542000000 1326393542000000
#1643 enhancement shevski ckan-backlog new Add fixed tags to thedatahub for better browsing

Similar to publicdata.eu, want to have themed areas such as finance, environment, census, etc and country tags

1326393293000000 1326393293000000
#1642 defect pudo ckan-backlog new Extra link generators generate garbled HTML

I had a package descriptions with URLs that contain "group:foo". This produces garbled output as the system tries to generate two sets of links: the outer link and an inner link.

Need to fix the parser.


Webdienst basierende Bereitstellung von Geobasisdaten der Freien und Hansestadt Hamburg. Folgende Geobasisdaten werden als WebMapTileService? (WMT-S) für die Dauer des Wettbewerbs netzbasiert unter der Creative Commons Lizenz zur Verfügung gestellt: Digitale Orthophotos 40 cm Auflösung (Layer: apps4d_DOP40), Digitale Stadtkarte (Layer: apps4d_DISK), Digitale Regionalkarte (Layer: apps4d_DIRK), Digitale Karte 1:5000 (Layer: apps4d_DK5).

Metadateneinträge zu den Daten im PortalU:

One fix is quoting the URLs

1326382171000000 1339773967000000
#1641 enhancement amercader amercader ckan-sprint-2012-01-23 closed fixed ckanext-archiver: Content-length header not reliable to check if resource has been modified

The download task in ckanext-archiver performs a HEAD request on the resource URL and checks if the "Content-Type" and "Content-Length" headers differ from the values stored to see if the resource needs to be updated [1].

The "Content-Length" header, although widely used, is not mandatory and some servers don't provide it, e.g.:

$ curl -I http://portfolio.theglobalfund.org/en/IATI/Activities?countryCode=AFG
HTTP/1.1 200 OK
Cache-Control: private
Transfer-Encoding: chunked
Content-Type: text/xml
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
Set-Cookie: ASP.NET_SessionId=3qhqekddgmre0kmk5cynq0sy; path=/; HttpOnly
X-AspNetMvc-Version: 3.0
content-disposition: attachment; filename=AFG_IATI_12012012.xml
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Thu, 12 Jan 2012 12:36:43 GMT

Also worth noting that requests, the python library that uses ckanext-archiver, sets an "Accept-Encoding: gzip" header by default, which depending on the configuration of the remote web server, may prevent the "Content-Length" server from being sent, e.g.:

$ curl -H "Accept-Encoding: gzip" -I http://iatistandard.org/published-temp/adb-activities.xml
HTTP/1.1 200 OK
Date: Thu, 12 Jan 2012 12:12:46 GMT
Server: Apache
Last-Modified: Mon, 28 Nov 2011 15:55:35 GMT
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: application/xml

curl -I http://iatistandard.org/published-temp/adb-activities.xml
HTTP/1.1 200 OK
Date: Thu, 12 Jan 2012 11:56:23 GMT
Server: Apache
Last-Modified: Mon, 28 Nov 2011 15:55:35 GMT
Accept-Ranges: bytes
Content-Length: 2686720
Vary: Accept-Encoding
Content-Type: application/xml

All this can lead to some resources never getting updated, and of course the size property of the resource not being set.

As we need to download the resource anyway, it would be better to check if the real length of the data has been modified (and store it).

[1] https://github.com/okfn/ckanext-archiver/blob/0a189262dca4ab5b286fb6a02b4ab8a201f639f3/ckanext/archiver/tasks.py#L72

1326376420000000 1326376777000000
#1640 enhancement amercader dread closed fixed Setup publicdata.eu harvester for Serbian CKAN datasets

Set-up publicdata.eu to harvest datasets at rs.ckan.net (Serbian community CKAN).

1326370425000000 1327340939000000
#1639 enhancement seanh seanh ckan-sprint-2012-02-06 closed fixed Move activity streams CSS into external file

The HTML and CSS for the activity streams templates needs to be tidied up. Currently the CSS is inlined in the templates themselves, and there is much unnecessary repetition of CSS.

Perhaps put this one in hold until there has been some design discussion about how the activity streams and the user, dataset and group pages with activity streams should look?

1326305019000000 1328526779000000
#1638 enhancement seanh seanh ckan-sprint-2012-01-23 closed wontfix Don't use JsonType in activity streams

Dump and load JSON explicitly instead.

1326304935000000 1326305570000000
#1637 enhancement seanh seanh ckan-v1.6 closed fixed API call for getting the list of activity detail items for a given activty stream item

(and add test cases for it)

1326304817000000 1326737169000000
#1636 enhancement seanh seanh ckan-v1.7 closed wontfix Purge items from activity streams when object (package etc.) purged from CKAN 1326304754000000 1338204258000000
#1635 enhancement seanh seanh ckan-backlog new Email notifications (e.g. for activity streams)

CKAN should be able to send email notifications to users.

Maybe have a notifications table in the db, and a server-side job that runs periodically and consumes rows from this table, mailing them to the users.

One thing that we may want to send users notifications of is activity stream events. So the activity streams code would have to add rows to the notifications table for the mailer job to consume. But remember that email notifications feature is separate from activity streams - we may want to send notifications of other things as well.

Need to implement (at least some of) #1634 before this can be implemented, in order to have something to send notifications about.

Analysis here: http://ckan.okfnpad.org/27

1326304587000000 1355141157000000
#1634 enhancement seanh seanh ckan-v1.7 closed duplicate Allow users to follow/unfollow activity streams of other users, datasets and groups

This is a fairly big feature to add. Analysis: http://ckan.okfnpad.org/27

This ticket is related to #1635 (email notifications for activity streams) which is about managing a list of activity streams for which the user receives email notifications. There are decisions to be made about how the two will work together.

1326304542000000 1334587390000000
#1633 enhancement seanh seanh ckan-v1.7 closed wontfix Render a dataset's activity stream on its' page

Add logic functions for getting the activity stream for a dataset in JSON and rendered HTML formats, add HTML activity stream into dataset page.

1326304298000000 1335877927000000
#1632 enhancement seanh seanh ckan-v1.7 closed wontfix Render a group's activity stream on its' page

Add logic functions for getting the activity stream for a group in JSON and rendered HTML formats, add HTML activity stream into group page.

1326304199000000 1335877871000000
#1631 enhancement seanh seanh ckan-sprint-2012-01-23 closed fixed Add activity stream events for new/changed groups 1326304020000000 1326736381000000
#1630 enhancement rgrp rgrp ckan-sprint-2012-02-06 closed fixed Integrate Data Explorer v2 into Data Viewer

Super ticket: #1602


  • Include (or git submodule) data explorer into vendor
  • Update existing js integration code
  • Fix css and any bugs

Est: 1d

Actual: 1d

1326279098000000 1330085600000000
#1629 defect dread dread ckan-sprint-2012-01-23 closed fixed permissions changed during upgrade to 1.5.1

This was seen on datacatalogs. When we upgraded it 1.5 to 1.5.1 we saw some permissions being reset so that it could be spammed. Anonymous and logged in users were given anon_editor and editor permissions. I don't know what the previous config was. Permissions for sysadmins remain unaffected.

1326215162000000 1326823222000000
#1628 defect dread ckan-sprint-2012-02-06 closed wontfix get ckanext-dgu working with ckan 1.5.1

johnglover said: I can confirm that even with the mapping fix, the ckanext-dgu dgu_form plugin template does not work properly with 1.5.1, so should probably not be installed at present. The edit page is ugly but should work (eg: http://dgu-os.okfn.org/dataset/edit/abandoned-vehicles), but the 'add a dataset' page is broken (eg: http://dgu-os.okfn.org/dataset/new)

1326214762000000 1328526967000000
#1627 defect dread dread ckan-sprint-2012-01-23 closed fixed favicon broken

On thedatahub.org the favicon doesn't display. i.e. the CKAN logo should appear in the browser's tab.

Original ticket #48

1326207102000000 1326890614000000
#1626 enhancement dread dread ckan-sprint-2012-01-23 closed fixed 'About CKAN' page update

thedatahub.org/about contains info that is very general to the CKAN and really quite technical. The text should be changed to be both specific to thedatahub.org and provide the context in a non-technical way. It should be easy to customise the About page to be appropriate for say new-york.ckan.net - a bit of info about who runs it, plus the general stuff about CKAN powering it and it was written by OKF to further open data.

1326205236000000 1326215877000000
#1625 enhancement seanh seanh ckan-sprint-2012-01-23 closed fixed Add activity stream events for new/changed users

This requires adding a logic function for emitting an activity stream event, and then editing the logic functions for creating or updating users and making them call the new emit event function. This same emit event function can later be used to emit activity stream events for other types of object as well.

1326187794000000 1326736328000000
#1624 defect dread pudo ckan-sprint-2012-01-23 closed fixed Typo in dataset edit mode

Futher Information -> Further Information

1326121197000000 1326216362000000
#1623 enhancement dread dread ckan-sprint-2012-01-23 closed fixed Dump to exclude deleted objects

The database dump currently contains all Packages and their associated objects, even those that have been set to state=deleted. We should exclude these from the dump now.

Dump = paster db simple-dump-csv/json


The dumps are designed for end-users to be able to run scripts on the mass of data. Since end-users don't see state=deleted packages then they shouldn't need them in the dump. In fact their presence in the dump probably confuses them.

Admins get the full database anyway in the backup pg_dump.

We only included them in the user dump because it was designed before use of state=deleted was established.

Time estimate: 2h

1326118987000000 1326892264000000
#1622 enhancement johnglover johnglover ckan-sprint-2012-01-23 closed fixed Deploy QA on DGU UAT test server - 0.5d
  • Update CKAN on DGU UAT to 1.5.1
  • Deploy Celery
  • Deploy QA extension
1326116380000000 1326127702000000
#1621 defect seanh seanh closed fixed UnicodeDecodeError when validating user password

A test case is currently failing for me on master:

ERROR: ckan.tests.functional.test_user.TestUserController.test_user_create_unicode
Traceback (most recent call last):
  File "/home/seanh/pyenv/lib/python2.6/site-packages/nose/case.py", line 197, in runTest
  File "/home/seanh/pyenv/src/ckan/ckan/tests/functional/test_user.py", line 342, in test_user_create_unicode
    res = res.follow()
  File "/usr/lib/pymodules/python2.6/paste/fixture.py", line 603, in follow
    return self.test_app.get(location, **kw)
  File "/usr/lib/pymodules/python2.6/paste/fixture.py", line 208, in get
    return self.do_request(req, status=status)
  File "/usr/lib/pymodules/python2.6/paste/fixture.py", line 389, in do_request
  File "/usr/lib/pymodules/python2.6/paste/wsgilib.py", line 343, in raw_interactive
    app_iter = application(basic_environ, start_response)
  File "/usr/lib/pymodules/python2.6/paste/lint.py", line 170, in lint_app
    iterator = application(environ, start_response_wrapper)
  File "/usr/lib/pymodules/python2.6/paste/cascade.py", line 130, in __call__
    return self.apps[-1](environ, start_response)
  File "/usr/lib/pymodules/python2.6/paste/registry.py", line 350, in __call__
    app_iter = self.application(environ, start_response)
  File "/usr/lib/pymodules/python2.6/repoze/who/middleware.py", line 69, in __call__
    auth_ids = self.authenticate(environ, classification, ids)
  File "/usr/lib/pymodules/python2.6/repoze/who/middleware.py", line 201, in authenticate
    userid = plugin.authenticate(environ, identity)
  File "/home/seanh/pyenv/src/ckan/ckan/lib/authenticator.py", line 29, in authenticate
    if user.validate_password(identity.get('password')):
  File "/home/seanh/pyenv/src/ckan/ckan/model/user.py", line 113, in validate_password
    hashed_pass = sha1(password_8bit + self.password[:40])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 12: ordinal not in range(128)

1326105806000000 1335878161000000
#1620 enhancement icmurray icmurray ckan-sprint-2012-02-06 closed fixed Update and test existing DGU package form : don't allow package-type selection when editing a package

[Child of #1526]

Can't change a dataset from containing time-series resources to one containing individual resources or visa-versa.

1326103837000000 1327589576000000
#1619 enhancement icmurray icmurray ckan-v1.7 closed fixed Update and test existing DGU package form : provide stubs for the publisher hierarchy

[Child of #1526]

Create the interface used to retrieve publisher and authz information from the publisher hierarchy.

1326103672000000 1332151351000000
#1618 enhancement icmurray icmurray ckan-v1.7 closed fixed Update and test existing DGU package form : provide stubs for the theme and sub-themes

[Child of #1526]

Provide the interface for the themes and subthemes hierarchy.

1326103557000000 1332151341000000
#1617 enhancement icmurray icmurray ckan-v1.7 closed fixed Update and test existing DGU package form : publisher / foi information

[Child of #1526]

Contact/FOI information is derived from Publisher/User?, but can be altered for each dataset.

Original estimate: 4 days

Time spent: 0 days

Time remaining: 4 days

1326103384000000 1332151332000000
#1616 defect amercader amercader ckan-sprint-2012-04-02 closed fixed Catch exceptions when rebuilding the search index

Right now if an exception is found while reindexing, the whole process stops and the remaining datasets are left out of the index. The process should continue after logging the exception. If more than a certain number of exceptions occur in a row, the process should stop.

1325844669000000 1332327635000000
#1615 enhancement thejimmyg ckan-v1.6 closed worksforme CKAN Should work behind a proxy server

This would allow deployment via Nginx or Apache using proxy to Paster, uWSGI. At the moment CKAN isn't aware of the proxy's IP address so when you perform an action which does a redirect (such as adding a package), CKAN redirects you to the *internal IP* not the external *proxy IP*.

We would like this work to facilitate testing within VMs as part of our new build infrastructure.

It would also be nice if CKAN worked when mounted at a path other than /. That could be dealt with in another ticket because it isn't a problem at the moment.

1325687841000000 1328888870000000
#1614 enhancement kindly kindly ckan-sprint-2012-01-09 closed fixed remove po files from git diff

Its a pain to see the difference between branches as there are normally a lot of po file transaction. Make the default be see that they have changed without actually show the diffs themselves.

1325686639000000 1325689136000000
#1613 defect dread dread ckan-sprint-2012-01-23 closed wontfix Post-dataset-edit URL has #section
  1. Viewing a dataset, hit 'Edit'
  2. Click on the "Basic Information" tab (note: URL has suffix #section-basic-information
  3. Click 'Save'
  4. URL still has suffix #section-basic-information

Affects 1.5, 1.5.1, 1.5.2a

1325685555000000 1326813924000000
#1612 enhancement kindly kindly ckan-sprint-2012-01-09 closed fixed Group view page slow

Group show that lists packages is slow due to not using query in pagination.

1325633737000000 1325688886000000
#1611 enhancement ross ross closed duplicate Implement auth API calls for webstore/external use

As part of #1574 we decided that it would be better for webstore (and future external services) to be able to authenticate simply with CKAN-Core.

Currently webstore access the CKAN database to obtain the key for the user but it would be better if this connection was not so tightly bound and that webstore used an API as any other external service might.

Need to discuss further with dread

1325590191000000 1325846987000000
#1610 enhancement ross ross closed duplicate Move webstore to Postgres instead of Sqlite

The default backing store for webstore should be Postgres and not sqlite. This was agreed as part of the #1574 storage changes and on http://ckan.okfnpad.org/newstorage

1325587341000000 1325852472000000
#1609 enhancement ross ross ckan-sprint-2012-01-23 closed fixed Celery task for ckanext-archiver to write to webstore.

From super Storage changes - #1574 - and http://ckan.okfnpad.org/newstorage we determined that ckanext-archiver should have a celery task for grabbing local file uploads and writing to webstore


When I upload a file to CKAN:

  • End up with file in permanent storage
  • IF file is ot type ... csv,xls,xlsx,sqlite,.sql
    • End up with new db in webstore
      • Where? {username}/{resource-id}/...
        • If a single table: name it after the file name (appropriately slugified)
      • A resource *always* corresponds to a 'database' in webstore ...
      • In Data Explorer have "Sheets" tab ...
  • Resource url = /dataset/{x}/resource/{y}/link -> cached_url ...
1325582253000000 1327057030000000
#1608 enhancement ross ross ckan-sprint-2012-01-23 closed fixed Merge storage into Core

As part of Storage Changes and http://ckan.okfnpad.org/newstorage it was decided to merge ckanext-storage back into core.

Other changes:

  • Remove /storage/upload page (we have frontend integration elsewhere now and we should also ensure that upload items relate to a resource or other domain object).
  • Provide a local path as well as remote path for form upload requests in API so that frontend can see url to CKAN path rather than cloud storage path ...
1325582044000000 1330036982000000
#1607 enhancement icmurray dread ckan-v1.7 closed fixed [super] Data.gov.uk Maintenance Refactor and UKLP Development Support

A ticket to collect all of the changes needed for CKAN to become the 'data' tab on DGU.

Design doc: https://docs.google.com/document/d/19h9bA1G4cQkv031m8jNCu6FEB3a8qpXUmdPAguM-Ofs/edit?hl=en_GB The design doc is the authoritative source of tasks. This ticket acts as a synopsis.

Current sprint (2012-02-20):

  • 7a. Public Publisher Dashboard (including QA Work and notifications) MUST PHASE 1
  • 6. Publisher Registration Improvements MUST PHASE 1
  • 3. Themes/Taxonomy? as well as tags WONT PHASE 2
  • 32. Browse by Tags PHASE 1 MUST
  • 12. Edit/Delete? for Harvested Datasets MUST URGENT PHASE 1
  • 13. Provider labelling MUST PHASE 1 [UKLP #14 MUST]
  • 17. Filter “UK Location Records” MUST PHASE 1 [UKLP #21 SHOULD]
  • 22. Mixed Licenses MUST PHASE 2

Plus, brought forward from last sprint:

  • Publisher hierarchy support
  • Improved dataset creation tools (form) - #1525
  • Edit/delete for harvested datasets

Overview of tasks: see the design doc.

1325503348000000 1337159969000000
#1606 enhancement dread ckan-backlog new metadata license config option

Add a config option to choose the metadata licence. Set default to Open Database License.

Currently the dataset edit form says "Important: By submitting content, you agree to release your contributions under the Open Database License." This is hard-coded, but not suitable for when DGU uses the CKAN form - they use the OGL.

1325501130000000 1339773981000000
#1605 enhancement kindly rgrp ckan-v1.7 closed fixed [super] Multilingual support in CKAN


  • dataset and resource metadata (and other objects such as groups?)
  • Field values in taxonomy (e.g. country names - Eurovoc)
    • #1665 Research into Eurovoc
    • Display Taxonomy in different languages
  • Field values not in taxonomy (e.g. title & description)
    • Use extra fields e.g. _i18n_title_fr = Le data.
    • (If lots of fields would need translating then would consider having a new package for each language, linked together with PackageRelationships?. But I think it is just title and description (resource description etc. are so minor, not worth translating?), so using extra field better.)
  • EC extension (templates, form)
    • Currently the pot file is just for CKAN core. New pot file for CKAN core and this extension? Or separate ones for extensions?
1325473367000000 1338193641000000
#1604 enhancement dread ckan-backlog new Get ckanext-moderatededits working with CKAN 1.5+ templates

ckanext-moderatededits requires an old and possibly development version of CKAN. It would be good to update it for later CKAN versions.

According to the README, you need CKAN from branch feature-1141-moderated-edits-ajax but the changelog suggests this branch went into version 1.4.2. So it possibly works with 1.4.2 and 1.4.3(.1). But CKAN 1.5 has revamped templates, so the genshi stream filters definitely don't work.

BTW history_ajax/read_ajax calls have been deprecated in CKAN since 1.5.2a and will need fixing up to use the Action API too as part of this.

1325352429000000 1325352429000000
#1603 enhancement zephod rgrp ckan-v1.7 closed duplicate Search query builder

Super ticket: #1745

Ability to build up search query using a nice javascript-y interface.

  • Add facets by selecting attribute and adding -> search facet options in dropdown -> added to search (with 'x' to remove -- as we currently do).
    • (a bit like the data.hri.fi)
  • Some improvements to css
  • Improvements to faceting
    • Ability to configure faceting and number of items to show (?)
  • Pure JS search implementation to make it easy to reuse across site
1325268364000000 1338202654000000
#1602 enhancement rgrp rgrp ckan-v1.6 closed fixed [super] Data Previewer / Viewer v2

We already have first pass of Data Preview that was done as part of #1357. This is about v2 of this. Two main components:

  • A new version of the separate Data Explorer library (now called Recline Data Explorer) - #1446 Est: 10-15d
    • Why:
      • Cleaner, backbone based solution.
      • Provides editing support if wanted.
      • Collaboration with Max Ogden
  • Integrate this into CKAN (into resource view page) - see #1630 Est: 1d
  • Improvements to Data Viewer / Previewer system in CKAN itself - Est: 2-3d
    • (Update Data Preview-er (View-er) to support images ...)
    • (WONTFIX here - (?) Update Data Preview-er to support geodata - #1151)

1325267544000000 1328175719000000
#1601 enhancement amercader rgrp ckan-v1.7 closed fixed [super] Well integrated Geo-spatial support including search

Need thoughts from Adria and James. RP thoughts

  • Integration into WUI search: #1469
  • JS Editor for adding/editing spatial attribute: #1818
  • Theme/template hook for adding the map e.g. on dataset page (or elsewhere)
  • Clear Documentation
  • Deployment
  • Announce post
  • (?) Map page for stats section showing location of datasets

Extras (probably separate):

  • csw support ...
1325259821000000 1338204029000000
#1600 enhancement rgrp rgrp ckan-v1.7 closed wontfix [super] User guide with integrated help in the WUI

Provide help to users in the WUI, with this linking to a detailed user guide.

  • User guide (suggest at help.thedatahub.org (or possibly help.ckan.org)
  • Snippets in the WUI with links to full user guide.
    • Provide helper methods for creating these links with config options for setting path to user guide so that the link location can vary on install by install basis
  • General info about publishing/working with/etc data (where we don't duplicate elsewhere)

Est: (?) 5-10d


See http://ckan.okfnpad.org/datahub-help

  • Step-by-step for adding a dataset including tips
  • API (perhaps link to main docs)
  • Command line interface and dpm
  • Using groups

Why do this

We already have wiki.ckan.org and docs.ckan.org. Why do this?

Answer for docs:

  • This is administrative guide and focused on general CKAN software.
  • Hard to provide completely generic help.
  • Integratable with the DataHub? site

Answer for wiki:

  • No spam
  • General feeling that wikis are a little bit loose and untidy. We want this to be smart, clean and useful.
  • Integratable with the DataHub?
1325257824000000 1333362051000000
#1599 enhancement rgrp rgrp ckan-sprint-2012-01-09 closed fixed [contrib] Simple embeddable dataset count widget (esp for group count)

Simple embeddable widget for use on 3rd party sites showing dataset counts for a given search query. Have a specific version just for groups.

  • Simple group count widget in JS for embedding in wordpress and elsewhere
    • Requested by several people (e.g Guo Xu from Econ working group). Already have something like this in CKAN JS for doing an embeddable search box.
  • All you need to do is do a dataset query over the API e.g. http://thedatahub.org/api/search/dataset?groups=economics and then embed in some html!

Estimate: 30m (for someone who knows their jquery).

1325246358000000 1325555201000000
#1598 enhancement rgrp ckan-backlog new Reinstate Ratings

Ratings were disabled approximately a year ago because:

  • Unclear purpose and UX. What did ratings tell you? How useful were they?
  • Spamming (esp by bots: you could submit an anonymous rating via a GET request which caused problems)

Both problems are solvable and it would be nice to have this feature reinstated.

  • Purpose: can make this more purposable by limiting to logged in users (or at least distinguishing logged in from non-logged in users)
    • Even better we could allow ratings to be made public (I'm interested in what someone else I respect finds important)
  • Spamming: limit to logged in users and / or use AJAX over an API to submit ...
1325177524000000 1325474818000000
#1597 enhancement dread ckan-sprint-2012-01-23 closed fixed Tag search - filter by group

I want to browse tags, but filtered for a particular group. Currently our tag API doesn't allow for filtering by group.

This is important for improving groups as communities within a site #1521. It would be easy to do this by adding an option to filter by a group. BUT are there any other use cases that would warrant a more complete faceted tag search?


BTW I can currently draw a tag cloud for a group - I can get the top tags used in a group like this:

curl http://thedatahub.org/api/action/package_search -d '{"q":"groups:country-ca", "facet.field":"tags", "rows":"0"}'

but it only contains the top 20 tags.

1324550492000000 1326821156000000
#1596 enhancement dread ckan-future new Refactor authz roles

Suggestions from rgrp:

  • Get rid of Roles, and replace them with direct assignment of actions, even though there are many actions, and extensions can add arbitrary ones.
  • Debatable whether we should cut the number of actions to correspond to the three roles defined by the base system.
  • Have a method of finding roles (or, in future, actions) relevant to a given protection object (e.g. FILE-UPLOAD(ER) not relevant to Packages)

(This ticket is split off from #1065)

1324549888000000 1338205019000000
#1595 defect kindly rgrp ckan-v1.7 closed wontfix Editing of a Group leads to recording a change against all associated datasets

Or at least it appears that way from revision log which displays all Group datasets against the changelog, see http://thedatahub.org/revision/4cdeeb42-3281-4f53-a29d-c694e1fd9217 -- not apparent from that page but see my user page: http://thedatahub.org/user/rufuspollock

This may turn out to be a UX bug rather than a real bug ...

Assigning to kindly for review and comment.

1324504875000000 1338203959000000
#1594 enhancement johnglover johnglover ckan-v1.7 closed wontfix [super] QA Improvements

Now that QA has been updated for Celery and CKAN 1.5.1, there are a number of improvements that we should probably implement.

  • Integrate QA into frontend - 1d - #1583
  • QA report improvements - 2.5d - #1584
  • QA - Check license as part of 5 star rating - #1587
  • QA - Give SPARQL endpoints a 4 star rating - #1588
  • QA - Give 5 star rating to datasets with link metadata - #1589

Main pad: http://ckan.okfnpad.org/qa

Some QA discussion (relating to Celery in particular) at http://ckan.okfnpad.org/queue

1324489620000000 1338203215000000
#1593 enhancement icmurray amercader ckan-sprint-2012-04-16 closed fixed [super] Create feeds from search results

Recent changes (see #191 and #1498) have made really easy to get search results ordered chronologically, and creating RSS or Atom feeds from them is relatively easy.

Feeds are extremely popular and there are several tools for consuming them, so it would be a great way to allow users to follow datasets related with their topics of interest.

Apart from the general feed with changes on the whole instance, other different "pre-set" feed endpoints could be implemented. The most obvious choices for custom feeds could be groups and tags:


These could be shown on the group/tag page, both on the <head> section, so browsers can autodiscover them, and on the UI, with a message like "Subscribe to this group/tag".

Another powerful option is to have a custom feed builder that accepts the same parameters as the search page, so users can subscribe to whatever query they are interested on


On this case a message "Subscribe to these search results" would appear.

Note that a lot of work on this front needs to be done for the IATI registry project (see #1590)

Other tickets related:

  • #1592: metadata_modified and metadata_created are not included in the search results, so an extra query for each package is needed right now.
1324489115000000 1334566668000000
#1592 enhancement amercader amercader ckan-sprint-2012-01-09 closed fixed Add metadata_modified and metadata_created to package_dictize output

The dict returned by package_dictize does not include metadata_modified and metadata_created. These are really useful properties, so it's worth having them on the standard package dict representation, which is used in several places, like at the template level.

1324488909000000 1326304321000000
#1591 enhancement icmurray dread ckan-sprint-2012-01-23 closed fixed Upgrade CKAN 1.5 instances

CKAN 1.5 installations need to be upgrade to 1.5.1.

Working methodically through the CKANs on http://datacatalogs.org/group/ckan :

At 1.5:

Other versions:

Now deleted from datacatalogs.org:

1324487658000000 1328789470000000
#1590 enhancement amercader amercader closed fixed Create customized feeds for the IATI Registry

We need a way to track changes on the registry (datasets edited or updated), globally and on a per country/publisher/etc. basis. RSS and Atom feeds are really popular, and after closing #191 and #1498 crating them from the search results should be fairly easy.

The following URLs are pretty self-explanatory:



As we need to implement custom wrappers for countries, publishers, etc, we might as well offer a fully customizable feed, e.g.:

http://localhost:5000/feed/custom.rss?q=activity_count:[* TO 100]

Apart from the actual feeds, there will be a small amount of work at the template level to add the links to the suitable pages (and maybe a generic page showing all available feeds)

1324486965000000 1326711608000000
#1589 enhancement johnglover johnglover ckan-backlog new QA - Give 5 star rating to datasets with link metadata

Super: #1594

From Richard Cyganiak on the CKAN Discuss list:

Regarding the fifth star (is the dataset linked to others?). This cannot be automatically determined just by looking at the format. It either requires inspection of the actual data, or information about links in the metadata. As you're probably aware, we've established conventions for recording information on data links in CKAN [1], as part of the work of the lodcloud group on the Data Hub. Link information is captured for hundreds of datasets. I would claim that we have the majority of four-star datasets covered there, and hence you can determine if they should get the fifth star by checking for the presence of a links:xxx field.

1324480600000000 1325475095000000
#1588 enhancement johnglover johnglover ckan-backlog new QA - Give SPARQL endpoints a 4 star rating

Super: #1594

From Richard Cyganiak on the CKAN Discuss list:

Besides considering the media type of resources, it would also make sense to check for the presence of a SPARQL endpoint. SPARQL endpoints are recorded for more than 300 datasets on the Data Hub using the pseudo-type "api/sparql". A few more are recorded with the format "SPARQL". I suggest that datasets with such resources should also be considered for the fourth star.

1324480405000000 1325475178000000
#1587 enhancement johnglover johnglover ckan-sprint-2012-06-25 closed fixed QA - Check license as part of 5 star rating

Super: #1594

From Richard Cyganiak on the CKAN discuss list:

According to (most of) the definitions of the five-star scheme, the first star already requires an open license, so a dataset shouldn't get any stars at all if it's not under an open license.

1324480208000000 1339433296000000
#1586 enhancement toby dread ckan-v1.7 closed duplicate 'Sort by' field for package search

We default to sort packages search results by "search rank", but we can change it to be "name", "title", "metadata_modified" date, "metadata_created" date. It works in the API now, so let's have (say) a drop-down in the Web UI for this.

Follows on from #191.

1324479588000000 1337079419000000
#1585 enhancement dread closed fixed Security fix

(details embargoed until 31/1/2012)

1324473465000000 1340633128000000
#1584 enhancement johnglover johnglover ckan-backlog new QA report improvements - 2.5d

Super: #1594

  • qa/{username}
  • qa/{groupname}
  • paginate QA results
  • search / filter QA results
  • list organisation report by default, but can disable via config option (done)
  • UX tidy up of report pages - hide border if no sidebar, etc
1324459433000000 1338981975000000
#1583 enhancement zephod johnglover ckan-sprint-2012-02-06 closed fixed Integrate QA into frontend - 1d


  • On dataset view page, add star rating after each resource view url. Probably similar to current format label after url.
  • Add star rating to resource view page, along with a way of displaying the reason for the rating and tips to improve. Should be unobtrusive.
  • Fix the table at http://thedatahub.org/qa/dataset/broken_resource_links - URLs should be truncated to a fixed not-too-long length and the other fields line up (eg by using a table)~
  • Handle 404s by showing that 'we think this resource is 404ing'.
  • Handle the absence of QA data on a resource (currently nothing is displayed. Maybe that's fine.
  • Do we need to integrate the 'reason for rating' string into the UI?

Next Steps:

  • Use db dump of thedatahub.org for final testing.
1324459327000000 1327582702000000
#1582 enhancement johnglover johnglover ckan-sprint-2012-01-09 closed fixed Deploy QA for thedatahub - 0.5d
  • deploy celery
  • deploy QA and archiver tasks
  • write up a blog post announcing QA on thedatahub
1324458494000000 1326110801000000
#1581 enhancement [email protected] johnglover ckan-future new Blog post about Google Analytics extension for CKAN

The CKAN Google Analytics extension has been updated to work with the latest version of CKAN, could make for a nice blog post.

Can ping John Glover in January for any details required.

Key link is: http://thedatahub.org/analytics/dataset/top though this should probably move to be under stats (e.g. http://thedatahub.org/stats/usage)

1324402800000000 1325474274000000
#1580 enhancement johnglover johnglover ckan-sprint-2012-01-09 closed fixed Documenting TaskStatus table and QA changes - 0.5d 1324399664000000 1324478635000000
#1579 enhancement ross ross ckan-backlog closed fixed Full text search in Webstore

We should allow full text search within databases for individual databases, although we may want to wait for any pending ports to postgres.

1324385698000000 1338212496000000
#1578 enhancement rgrp ckan-backlog new [super] Re-enable and refactor ratings 1324322443000000 1325473015000000
#1577 defect rgrp dread ckan-backlog new Can't upload file with foreign chars in filename

Looks like uploading a file with foreign characters fails due to encoding reasons.

URL: http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf
Module weberror.errormiddleware:162 in __call__
<<              __traceback_supplement__ = Supplement, self, environ
                   sr_checker = ResponseStartChecker(start_response)
                   app_iter = self.application(environ, sr_checker)
                   return self.make_catching_iter(app_iter, environ, sr_checker)
>>  app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<<                                                     self.cache_manager)
               environ[self.environ_key] = self.cache_manager
               return self.app(environ, start_response)
>>  return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<<                          headers.append(('Set-cookie', cookie))
                   return start_response(status, headers, exc_info)
               return self.wrap_app(environ, session_start_response)
           def _get_session(self):
>>  return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<<                  environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
               response = self.app(environ, start_response)
               # Wrapped in try as in rare cases the attribute will be gone already
>>  response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
               controller = self.resolve(environ, start_response)
               response = self.dispatch(controller, environ, start_response)
               if 'paste.testing_variables' in environ and hasattr(response,
>>  response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<<          if log_debug:
                   log.debug("Calling controller class with WSGI interface")
               return controller(environ, start_response)
           def load_test_env(self, environ):
>>  return controller(environ, start_response)
Module ckan.lib.base:123 in __call__
<<          # available in environ['pylons.routes_dict']    
                   return WSGIController.__call__(self, environ, start_response)
>>  return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<<                  return response(environ, self.start_response)
               response = self._dispatch_call()
               if not start_response_called:
                   self.start_response = start_response
>>  response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<<              req.environ['pylons.action_method'] = func
                   response = self._inspect_call(func)
                   if log_debug:
>>  response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<<                        func.__name__, args)
                   result = self._perform_call(func, args)
               except HTTPException, httpe:
                   if log_debug:
>>  result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
           def _inspect_call(self, func):
>>  return func(**args)
Module ckanext.storage.controller:2 in auth_form
Module ckan.lib.jsonp:26 in jsonpify
<<      Very much modelled after pylons.decorators.jsonify .
           data = func(*args, **kwargs)
           return to_jsonp(data)
>>  data = func(*args, **kwargs)
Module ckanext.storage.controller:301 in auth_form
<<          method = 'POST'
               authorize(method, bucket, label, c.userobj, self.ofs)
               data = self._get_form_data(label)
               return data
>>  authorize(method, bucket, label, c.userobj, self.ofs)
Module ckanext.storage.controller:79 in authorize
<<      if method != 'GET':
               # do not allow overwriting
               if ofs.exists(bucket, key):
               # now check user stuff
>>  if ofs.exists(bucket, key):
Module ofs.remote.botostore:53 in exists
<<          if bucket is None: 
                   return False
               return (label is None) or (label in bucket)
           def claim_bucket(self, bucket):
>>  return (label is None) or (label in bucket)
Module boto.s3.bucket:87 in __contains__
<<      def __contains__(self, key_name):
              return not (self.get_key(key_name) is None)
           def startElement(self, name, attrs, connection):
>>  return not (self.get_key(key_name) is None)
Module boto.s3.bucket:144 in get_key
<<          response = self.connection.make_request('HEAD', self.name, key_name,
               # Allow any success status (2xx) - for example this lets us
               # support Range gets, which return status 206:
>>  query_args=query_args)
Module boto.s3.connection:388 in make_request
<<          if isinstance(key, Key):
                   key = key.name
               path = self.calling_format.build_path_base(bucket, key)
               boto.log.debug('path=%s' % path)
               auth_path = self.calling_format.build_auth_path(bucket, key)
>>  path = self.calling_format.build_path_base(bucket, key)
Module boto.s3.connection:88 in build_path_base
<<      def build_path_base(self, bucket, key=''):
               return '/%s' % urllib.quote(key)
       class SubdomainCallingFormat(_CallingFormat):
>>  return '/%s' % urllib.quote(key)
Module urllib:1222 in quote
<<              safe_map[c] = (c in safe) and c or ('%%%02X' % i)
               _safemaps[cachekey] = safe_map
           res = map(safe_map.__getitem__, s)
           return ''.join(res)
>>  res = map(safe_map.__getitem__, s)
KeyError: u'\xed'
CGI Variables
AUTH_TYPE	'cookie'
CONTENT_TYPE	'; charset=utf-8'
HTTP_ACCEPT_CHARSET	'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
HTTP_ACCEPT_ENCODING	'gzip,deflate,sdch'
HTTP_CACHE_CONTROL	'max-age=259200'
HTTP_CONNECTION	'keep-alive'
HTTP_COOKIE	'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; ckan_user=elenaibp; ckan_display_name="Elena Mondo"; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; __utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10.1324298316; __utmc=119670349; __utmz=119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)'
HTTP_HOST	'thedatahub.org'
HTTP_REFERER	'http://thedatahub.org/dataset/edit/budget-library-czeck-republic'
HTTP_USER_AGENT	'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7'
HTTP_VIA	'1.1 localhost (squid/3.0.STABLE19)'
PATH	'/usr/local/bin:/usr/bin:/bin'
PATH_INFO	'/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
PATH_TRANSLATED	'/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
REMOTE_USER	u'elenaibp'
REMOTE_USER_DATA	'userid_type:unicode'
REQUEST_URI	'/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf'
SCRIPT_FILENAME	'/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py'
SCRIPT_URI	'http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
SCRIPT_URL	'/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
SERVER_ADMIN	'[no address given]'
SERVER_NAME	'thedatahub.org'
SERVER_SIGNATURE	'<address>Apache/2.2.14 (Ubuntu) Server at thedatahub.org Port 80</address>\n'
SERVER_SOFTWARE	'Apache/2.2.14 (Ubuntu)'
WSGI Variables
application	<beaker.middleware.CacheMiddleware object at 0x7f22601c7dd0>
beaker.cache	<beaker.cache.CacheManager object at 0x7f22601c7b50>
beaker.get_session	<bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x7f22601c7a90>>
beaker.session	{'_accessed_time': 1324298703.071357, '_creation_time': 1324293077.4139669}
mod_wsgi.application_group	'ckan.net|'
mod_wsgi.callable_object	'application'
mod_wsgi.listener_host	''
mod_wsgi.listener_port	'80'
mod_wsgi.process_group	'ckan.net'
mod_wsgi.reload_mechanism	'1'
mod_wsgi.script_reloading	'1'
mod_wsgi.version	(2, 8)
paste.cookies	(<SimpleCookie: __utma='119670349.1809834699.1323782464.1324293066.1324298316.4' __utmb='119670349.3.10.1324298316' __utmc='119670349' __utmz='119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' auth_tkt='f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode' ckan_apikey='decd48b1-49ee-4250-bff4-98ccca9c02a5' ckan_display_name='Elena Mondo' ckan_user='elenaibp' hide_welcome_message='1' thedatahub_net='27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e'>, 'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; ckan_user=elenaibp; ckan_display_name="Elena Mondo"; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; _ _utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10...)|utmcmd=(none)')
paste.registry	<paste.registry.Registry object at 0x7f226194df50>
paste.throw_errors	True
pylons.action_method	<bound method StorageAPIController.auth_form of <ckanext.storage.controller.StorageAPIController object at 0x7f2261dad990>>
pylons.controller	<ckanext.storage.controller.StorageAPIController object at 0x7f2261dad990>
pylons.environ_config	{'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons	<pylons.util.PylonsContext object at 0x7f2261daddd0>
pylons.routes_dict	{'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'}
repoze.who.identity	<repoze.who identity (hidden, dict-like) at 139785645747120>
repoze.who.logger	<logging.Logger instance at 0x7f225e23c098>
repoze.who.plugins	{'openid': <OpenIdIdentificationPlugin 139785625065680>, 'friendlyform': <FriendlyFormPlugin 139785618095248>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator object at 0x7f2260874c10>, 'auth_tkt': <AuthTktCookiePlugin 139785625065808>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f2260874c90>}
routes.route	<routes.route.Route object at 0x7f22601a1090>
routes.url	<routes.util.URLGenerator object at 0x7f2261dadf50>
webob._parsed_query_vars	(GET([]), '')
webob.adhoc_attrs	{'language': 'en-us'}
wsgi process	'Multiprocess'
wsgi.file_wrapper	<built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f2261da9af8>
wsgiorg.routing_args	(<routes.util.URLGenerator object at 0x7f2261dadf50>, {'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'})
1324317659000000 1325473564000000
#1576 enhancement rgrp rgrp ckan-sprint-2012-01-09 closed fixed Move stats extension back into core - 0.5d

Est: 0.5d.


  • Why do this?
    • tiny extension with few dependencies - and really nice to have out of the box
    • trial for doing this on larger scale
  • Do we keep as extension (even if in core)?
    • Ans: Yes, keep as extension because:
      • Already set up that way
      • Cleaner
      • Easier to disable / enable
  • How do we integrate with main them (e.g. have stats link)
    • Ans: not sure (this is part of more general issue of how we update theme for varying changes elsewhere). Best answer is to have some widgetization in theme.
  • Hide ratings section (at least until we reinstate ratings #1598)
    • Ans: no, let's not bother (and having ratings there encourages us to do #1598 and/or find out whether people are interested in ratings)
1324317313000000 1327056070000000
#1575 enhancement dread dread ckan-sprint-2012-01-23 closed fixed tag punctuation lost in ca.ckan.net import

Last week I imported ca.ckan.net datasets into thedatahub.org, but the tags seem to have lost their dashes, underscores and dots.

1324316860000000 1326808657000000
#1574 enhancement ross ross ckan-v1.7 closed fixed [super] Storage changes including automated save to Webstore

Allow uploading of files to push the data into webstore. Initially we were going to suggest changes to ckanext-storage but after further analysis we arrived at the conclusion that this should be implemented by ckanext-archiver as it already handles archiving of data from various sources and would be the best place to 'archive' to webstore.

  1. A user wants to upload a file to CKAN, and so chooses the file upload option as they do currently with ckanext-storage. The file upload itself should be handled by this code internal to core (see #1608) but still using ofs [5d]
  1. The user is provided the link to the file as currently.
  1. The archiver's update task will check configuration to determine if it should upload some mime-types (tabular formats) to webstore. Configuration should also determine where webstore is. (see #1609) [5d] - took 6
  • It should be stored in webstore using {username}/{resource-id} where {username} is the name of the user uploading the original file.
  1. The archiver will continue to move the file into permanent storage as per current practice.


  • #1608 Merge storage into Core [5d]
  • #1609 Celery task for ckanext-archiver to write to webstore. [5d] (took 6)
  • #1687: Deploy to datahub [1d]
  • #1681: Core storage documentation [1d?]
    • Internal (perhaps in README somewhere or wiki): how does process work, where does data end up and in what order etc
    • External: here's how to configure it and here's what you get ... (goes in main sphinx docs)
1324315633000000 1338202955000000
#1573 enhancement David Raznik jilly mathews ckan-future new Apps and Ideas

Estimate 2 weeks for someone to finish and test.

1324294593000000 1324294593000000
#1572 enhancement David Raznik jilly mathews ckan-future new Meta data Harvester

Need to write custom harvesters for each client. Is it worth having one for data hub?

1324294509000000 1324294509000000
#1571 enhancement rgrp jilly mathews ckan-backlog assigned [super] Issues Extension

"Proper" issues extension.


  • This would be an extension and improvement of existing todo extension - #842 (https://github.com/okfn/ckanext-issues)
    • Current extension does work but lacks polish
    • Should we polish or do significant improvements?
  • Functionality - see etherpad


  • (Rename todo(s) -> issue(s) in existing extension)
1324294225000000 1330863183000000
#1570 enhancement David Raznik jilly mathews ckan-future closed invalid Integrated file Storage

Is this ready for release? What needs to be done?

1324294142000000 1324314741000000
#1569 enhancement David Raznik jilly mathews ckan-future new Wordpressser

How much effort will this be to be ready to use?

1324294056000000 1324294056000000
#1568 enhancement David Raznik jilly mathews ckan-future closed duplicate Moderated Edits

Can this be released as a standard CKAN feature?

1324293776000000 1325267998000000
#1567 enhancement David Raznik jilly mathews ckan-future new Finish QA extension

Requires change to celeryd. Estimated 4 weeks.

1324293599000000 1324293599000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Note: See TracReports for help on using and creating reports.