{22} Trac tickets (2647 matches)

Results (1901 - 2000 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#2294 defect kindly rgrp ckan-v1.9 assigned Unauthorized action on Data API gets 302 rather than 401 or similar

Well known issue that 401 *in API* end up with 302 redirect to login page (which makes no sense and is very confusing for clients).

Fix this by ensuring that error messsages on API get passed through correctly.

Minor because affect is annoying but not terminal

1334432104000000 1340624003000000
#2317 enhancement rgrp ckan-v1.7 closed wontfix Corrections to dataset creation form for v1.7
  • Add home page url
  • Remove group option (seen it confuse several users) - perhaps replace with tags (?)
  • Check form looks ok in portugese (seems to be a problem with css)
1335022061000000 1340033433000000
#2318 enhancement seanh rgrp ckan-sprint-2012-04-30 closed fixed Bug with Portugese translation and Javascript

If you switch to Portugese and try to add a resource on dataset edit it will fail. This is because there is a string translation in js_strings.html that has quotes in it. When this is inserted into the file this causes a js exception which then prevents any further js processing.


CKAN.Strings.youHaveUnsavedChanges = "Você tem alterações não salvas. Certifique-se de ter clicado "Salvar Alterações" abaixo antes de sair desta página.";
Uncaught SyntaxError: Unexpected identifier

To fix is simple i imagine: need to escape " correctly in js_strings.html (or translations used).

I have temporarily patched this live in order to allow the hackday here to continue.

1335033889000000 1335774178000000
#2319 enhancement rgrp ckan-v1.8 closed wontfix Better auto-complete for groups on dataset edit page

Use jquery chosen?

1335211353000000 1340624083000000
#2320 enhancement rgrp ckan-v1.9 new Introduce new state: archived

Some datasets become obsolete: e.g. http://datahub.io/dataset/uncdb Do not want to delete them but want them marked as archived or deprecated or something.

1335211949000000 1340624119000000
#2321 enhancement rgrp ckan-v1.9 new Use resource names/titles in URLs

Would make them much nicer but requires:

  • Ensure uniqueness (or deal with ambiguity)
  • (?) Slug/names versus titles
    • Aside: what is currently called name should probably be title
1335236412000000 1340624174000000
#2322 enhancement ross rgrp ckan-sprint-2012-05-29 closed fixed Convenience offset on end of resource urls: /raw, /api, /viewer


/raw or /download (latter may be better)


=> redirects to resource url (download)



=> either redirects or directly serves /api/data/{resource-id}

Viewer (Explorer?)

Do data viewer but without anything else showing (full-screen?). this would not be just the data explorer but other cases too (including iframe etc ...)


Resource (pre)-viewer without anything else (perhaps include resource title)

Assigning for kindly for review.

These would take about 20m to implement and would be *very* convenient from my experience of using the data API.

Implementation at enhancement-2322-convenience-urls

1335236581000000 1344086640000000
#2328 enhancement rgrp ckan-v1.9 new Link to Catalog API for each Dataset, Group and Tag

At the moment no way for a normal user to find JSON versions of this material. Would it be worth adding back a link somewhere on the page?

1335442501000000 1340624331000000
#2329 enhancement rgrp ckan 2.0 assigned Add back in RSS/Atom links on relevant pages

E.g. on dataset page, on revision page, on user page and on search results.

1335442540000000 1346664271000000
#2331 defect kindly rgrp ckan-sprint-2012-05-29 reopened Search should AND terms not OR terms

Appears current default search in CKAN ORs terms rather than ANDing them (i.e. adding more terms increasing number of items found rather than reducing it).

Not sure when this crept in or if it has been there for a long time.

1335637485000000 1356474344000000
#2332 enhancement amercader rgrp ckan-v1.7 closed fixed Fixes for v1.7 release

A place to list crucial fixes for v1.7 release:

Related extension:

All not complete now moved to #2347

  • Related Item has comments on index page (they should not show up here -- why is this happening!)
  • Related item should be to left of history (history could in fact move to RHS)
    • What about losing icons since we duplicate for related item
  • Related item plus sign is very poorly aligned within box
    • Add related items should be in drop down like for resources (?) to make it easy to add items
    • Hover should show a title that gives some information about what related are (not at all clear)
  • Related item only includes App, Idea, Visualization but not API | Post | Paper | News Article options as in original ticket #2204

Search results:

  • #2331 - search should add not or
  • Search results should be similar across group pages and normal search results (suggest adopting group page style)
    • Search results resource link should link to resource page rather than url for that resource
    • Suggest we could tidy this a bit (e.g. have resources in mini-sidebar of search result ...?)

Data viewer:

a) Data Viewer checks if datastore is empty nad falls back to ther options if emtpy b) DataStore? enabled is off by default and only turned on either explicitly by user or by DataStorer? on successful store ...

  • (? not really a fix) Give out a link as well as embed instructions (I often want to link to the explorer in a given state)
    • Should this link to the resource page or to a dedicated viewer page - cf # 2322 (resource sub-urls like /download, /api ...)
    • Do #2322 ...
  • Decent powered by icon for the Embed for the DataHub? (make this text customizable and allow html ...)
  • Embed link uses uuid rather than dataset name (this goes to perma-url question ...)
  • Data viewer embed url includes all the resource info

Dataset page:

  • Put home page link somewhere more prominent (e.g. the top of the sidebar)
  • Put author more prominent (at top right)
  • (? - new work) List CKAN owners at top right in sidebar
  • Make resources stand out more
  • Rename resources tab and heading to Data & Resources

Dataset edit and create:

  • Corrections to dataset creation form for v1.7 - #2317
  • Better auto-complete for groups on dataset edit/new page - #2319
    • Remove groups box and replace with tags on the new page (or is this needed for organizations stuff?)
  • Description / notes text area is not wide enough.
    • Should also be larger on edit page
  • Problem with date instructions on add/edit resource: "Dates are in ISO Format — eg. 2012-12-25 or %c2010-05-31T14:30%d."
  • Add/edit resource: new code to auto-check urls being added is great but from UX point of view has draw back that (esp on low speed connections) page just seems to hang for a long time with nothing happening. I wonder if either
    • A) this could happen after resource created on form (e.g. it creates, and then an overlay comes up saying: checking url and gathering url information or similar)
    • Or B) we still do this before creating new resource form on page but we still should show something like a spinner and some info like "checking url and gathering url info ...)

Resource view:

  • Do not show no description in no description (just show dataset info)


  • Fix spacing above page title (i.e. dataset title etc)
  • Bootstrap 2.0.2 (@zephod was *supposed* to have done this) - should be trivial and gives important bugfixes for data viewer
  • Fix subnav width on resources pages (does not extend all the way to the right like it should)
  • Fix spacing of top menu and search box (do not look right)
  • Group pages:


  • Ensure that we out-of-the-box config for CKAN has (#2388):
    • Only logged in users able to create
    • Creation permissions have only creators able to update a dataset

Deployments (without deployment we cannot know these are working):

  • Social extension
  • Analytics
1335644116000000 1340033281000000
#2333 enhancement rgrp ckan-v1.9 new Introduce new resource type datastore

This is a resource that is *just* in the datastore. At the moment there are hacky ways to do this (create resource with meaningless URL etc) but we should do this properly:

  • New Option: DataStore? Table in add a resource form
  • URL field is disabled (or points at /api/data or something rather pointless)
  • Download button is disabled
    • Better would be to find a way to serialize an ES table or stream it ...
1335651360000000 1340624473000000
#2334 enhancement kindly rgrp ckan-v1.9 assigned Improved data import

Placeholder for thinking about this ...

  • Allow users to configure info for CSV import (e.g. separators, quotes etc)
1335651662000000 1340624538000000
#2404 enhancement rgrp ckan-future assigned Auto-screenshot for related items

As a User I want an image generated for my related item automatically so that I do not have to take picture myself and upload it somewhere

Implementation: (?) Use phantomjs or something like http://snapito.com/ Investigate for 1.8, will be needed for TDH Oct.

1337583347000000 1346669526000000
#2405 defect icmurray rgrp ckan-v1.9 assigned Improvements to data viewer embed
  • Decent powered by icon for the Embed for the DataHub?? (make this text customizable and allow html ...)
    • Current icon and text looks really bad
  • Embed link uses uuid rather than dataset name (this goes to perma-url question ...)
  • Data viewer embed url includes all the resource info (not needed -- just url and backend info ...)
  • Embed extension urls do not respect original domain but instead use site_url (I'm guessing). E.g. datahub.io domain ends up with links to embed on thedatahub.org
1337583565000000 1340635981000000
#2495 enhancement rgrp ckan-v1.8 closed duplicate Stats page has lost styling ...


1339056932000000 1340624905000000
#2529 enhancement rgrp ckan-v1.9 new DataHub (or CKAN) widgets

Simple widgets in pure JS. For example:

  • Count of datasets in a group (could generalise to a query but not sure how useful that is ...)
  • Embeddable list of top X (5) datasets for a given query
  • Embeddable list of *my* datasets

Either these live at: {site}/widgets and we have some kind of generator (form where I choose my group, or my query).

Or: we have this attached to areas of site where relevant.

Can combine the 2 so that the latter links to the former. Think first will be easier to do and possibly more useful long-term (e.g. can just link people to that page).

Cf. http://okfnlabs.org/ckanjs/

1339750049000000 1340624917000000
#2530 enhancement kindly rgrp ckan-v1.9 new DataHub purge fails on some revisions

See http://datahub.io/ckan-admin/trash and try to purge revisions (*not* datasets). It will fail on some of the revisions with errors like:

Problem purging revision 391db9e8-df57-4e0e-8fe6-d4e0c2318344: (IntegrityError?) update or delete on table "revision" violates foreign key constraint "group_extra_revision_revision_id_fkey" on table "group_extra_revision" DETAIL: Key (id)=(391db9e8-df57-4e0e-8fe6-d4e0c2318344) is still referenced from table "group_extra_revision". 'DELETE FROM revision WHERE revision.id = %(id)s' {'id': u'391db9e8-df57-4e0e-8fe6-d4e0c2318344'}

1339750498000000 1341268280000000
#2531 enhancement rgrp ckan-backlog new New state option: archived / deprecated

Deleted means things will get purged at some point.

Archived means they stay around but get hidden from search results and a big warning notice gets displayed saying this is archived / deprecated.

@richard cyganiak ...

1339750787000000 1339770649000000
#2535 enhancement rgrp assigned SSL certificate for DataHub + https by default

DataHub? is increasingly used and we should ensure it uses ssl as part of general security.

See also #1446 (Need to support https login for multiple instances as part of the CKAN package install)

1339758027000000 1346662082000000
#2582 enhancement rgrp ckan-v1.9 new Do not hide notes / readme on dataset pages

Current we hide most of readme and then let users reveal it. Stop doing this and if necessary add a quick link down to resources section. (Maybe also rename resources to Data and Resources ...?)

Aside: believe I have mentioned this somewhere a month + ago but could not find the ticket.

1340312340000000 1340625009000000
#2673 enhancement rgrp new simplify set of options for resources

Far too many resource options. Lets restrict back to data file and API. Visualizations etc can either get linked in description or in the Related items.

1342300559000000 1342300559000000
#2764 enhancement rgrp closed duplicate Simplify filestore

Definitely do:

  • Local file filestore: Remove pairtree (and OFS) and do something very simple


  1. Local file only. Allow for uploaders to GS / S3 in the background
    • Advantages: simpler, if local upload can do progress bars etc etc
    • Disadvantages: strain for web-app (upload a 2GB file what happens)
      • This is probably fixable ...
  2. Stick with how we are

New filestore without pairtree

Each uploaded file is located on disk at:




Need the uuid to avoid collisions.

Metadata: Store no metadata.

1343303928000000 1343320440000000
#2844 enhancement rgrp new SQL-only (no solr) version of CKAN
  • Search needs to run of local DB (again)
  • paster db clean attemps to connect to SOLR (still works as does db first but then excepts which is not nice UX)
1344859168000000 1345454527000000
#2874 enhancement rgrp rgrp assigned Clean up bin directory

Full of obsolete material

1345190508000000 1345190515000000
#2877 enhancement kindly rgrp assigned Bugs with datastore v2

In progress

  1. [major] q does not seem to work reliably. e.g. using the setup from this gist https://gist.github.com/1930806 and doing a ?q=DE yields no results (does not work with "q=de" either)
    • q=second does work ...
  2. [major] q does not work with 2 values (see below)
  3. Query on search with limit 0 results in total of 0 (should either be null or correct total). Queries with other limits yield correct total AFAICT
    • Also weird fact that limit is returned but as as as string - should it not be an integer
  4. Types on fields: could these not be canonical and as per recline (or is it important to allow exact sql types ...)

Multiple query values

Try a query such as: "second UK" and you will get 500 error:


ProgrammingError: (ProgrammingError) syntax error in tsquery: "second UK" 'select "_id", "id", "date", "x", "y", "z", "country", "title", "lat", "lon", count(*) over() as "_full_count"\n from "4f1299ab-a100-4e5f-ba81-e6d234a2f3bd" where _full_text @@ to_tsquery(%s) limit 100 offset 0' (u'second UK',)


Filter support: should think in more detail about this (may want to follow recline style)

Simple filters in query parameters would be nice too ...

1345250002000000 1346320395000000
#2895 enhancement rgrp new QA pages (e.g. broken resource links) are not paginated

And as a result timeout on e.g. the datahub ...

1346175485000000 1346175485000000
#1202 enhancement rgrp rolf closed invalid Links to datapkg utility don't lead to info about it

In the presentation of data packages, the link in "The information on this page and the downloads / resources are also available using the datapkg command line utility." (http://knowledgeforge.net/ckan/doc/datapkg/install.html) ends up at the home page of ckan.org

Perhaps link it to https://bitbucket.org/okfn/datapkg/overview for now, to save some searching for people who want to find out more?

1309263613000000 1315821804000000
#1203 defect johnglover rolf ckan-backlog new Moderated edits: html code shows as "changed" although it is not

I've installed the Moderated Edits extension (ckanext-moderatededits) and am editing a package imported from IATIregistry.org, with an extra field which contains a bit of HTML.

The editor indicates the field has changed, although the content hasn't (see screenshot). All I can find so far is a minor difference: in the field content, there is a code &#8212 and in the rendered table that is an —

1309274970000000 1313401579000000
#1549 enhancement ross ckan-backlog closed wontfix [super] Short link tool

It would be great to have a CKAN extension that allowed users (or CKAN itself) to generate short links to other URIs (both internal and external). Once created, shortlinks made by CKAN should be changeable. This would allow uploaded content to be moved without the user's link changing at all. The tool itself might also be of use as a general link-shortener to users other than the CKAN system itself.

Another useful feature would be for this to also collect some simple analytics such as the referrer and client IP for future reference. I'm not yet sure what we would do with the analytics other than some sort of popularity metric.


  • Core, or Extension, or Self-hosted?
1324036998000000 1325474219000000
#1550 enhancement ross ckan-backlog assigned Allow simple auth via the API

It should be possible to pass userid/username and api key and obtain a response from CKAN for external services that use CKAN auth. Those services shouldn't be talking to the DB directly.

1324049369000000 1346670055000000
#1551 enhancement ross ross ckan-backlog closed fixed CKAN auth for webstore changes

Webstore should use auth api ( #1550 ) for authenticating users accessing webstore rather than talking directly to the CKAN database. We also need it to suppose /user/ urls and /userid/ for accessing databases.

1324049966000000 1346662048000000
#1574 enhancement ross ross ckan-v1.7 closed fixed [super] Storage changes including automated save to Webstore

Allow uploading of files to push the data into webstore. Initially we were going to suggest changes to ckanext-storage but after further analysis we arrived at the conclusion that this should be implemented by ckanext-archiver as it already handles archiving of data from various sources and would be the best place to 'archive' to webstore.

  1. A user wants to upload a file to CKAN, and so chooses the file upload option as they do currently with ckanext-storage. The file upload itself should be handled by this code internal to core (see #1608) but still using ofs [5d]
  1. The user is provided the link to the file as currently.
  1. The archiver's update task will check configuration to determine if it should upload some mime-types (tabular formats) to webstore. Configuration should also determine where webstore is. (see #1609) [5d] - took 6
  • It should be stored in webstore using {username}/{resource-id} where {username} is the name of the user uploading the original file.
  1. The archiver will continue to move the file into permanent storage as per current practice.


  • #1608 Merge storage into Core [5d]
  • #1609 Celery task for ckanext-archiver to write to webstore. [5d] (took 6)
  • #1687: Deploy to datahub [1d]
  • #1681: Core storage documentation [1d?]
    • Internal (perhaps in README somewhere or wiki): how does process work, where does data end up and in what order etc
    • External: here's how to configure it and here's what you get ... (goes in main sphinx docs)
1324315633000000 1338202955000000
#1579 enhancement ross ross ckan-backlog closed fixed Full text search in Webstore

We should allow full text search within databases for individual databases, although we may want to wait for any pending ports to postgres.

1324385698000000 1338212496000000
#1608 enhancement ross ross ckan-sprint-2012-01-23 closed fixed Merge storage into Core

As part of Storage Changes and http://ckan.okfnpad.org/newstorage it was decided to merge ckanext-storage back into core.

Other changes:

  • Remove /storage/upload page (we have frontend integration elsewhere now and we should also ensure that upload items relate to a resource or other domain object).
  • Provide a local path as well as remote path for form upload requests in API so that frontend can see url to CKAN path rather than cloud storage path ...
1325582044000000 1330036982000000
#1609 enhancement ross ross ckan-sprint-2012-01-23 closed fixed Celery task for ckanext-archiver to write to webstore.

From super Storage changes - #1574 - and http://ckan.okfnpad.org/newstorage we determined that ckanext-archiver should have a celery task for grabbing local file uploads and writing to webstore


When I upload a file to CKAN:

  • End up with file in permanent storage
  • IF file is ot type ... csv,xls,xlsx,sqlite,.sql
    • End up with new db in webstore
      • Where? {username}/{resource-id}/...
        • If a single table: name it after the file name (appropriately slugified)
      • A resource *always* corresponds to a 'database' in webstore ...
      • In Data Explorer have "Sheets" tab ...
  • Resource url = /dataset/{x}/resource/{y}/link -> cached_url ...
1325582253000000 1327057030000000
#1610 enhancement ross ross closed duplicate Move webstore to Postgres instead of Sqlite

The default backing store for webstore should be Postgres and not sqlite. This was agreed as part of the #1574 storage changes and on http://ckan.okfnpad.org/newstorage

1325587341000000 1325852472000000
#1611 enhancement ross ross closed duplicate Implement auth API calls for webstore/external use

As part of #1574 we decided that it would be better for webstore (and future external services) to be able to authenticate simply with CKAN-Core.

Currently webstore access the CKAN database to obtain the key for the user but it would be better if this connection was not so tightly bound and that webstore used an API as any other external service might.

Need to discuss further with dread

1325590191000000 1325846987000000
#1649 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Verify RDF Extension (ckanext-rdf) works
  • Make sure that the extension works as it should and isn't broken on current releases [3d]
  • Make it work with the EC schema, including extras [2d]
1326709096000000 1338205274000000
#1650 enhancement ross ross ckan-future closed wontfix Add new language/translation of fields for datasets.

Datasets should have a field that specifies the language used in the dataset, this should be a two character ISO 639-1 code (see http://en.wikipedia.org/wiki/List_of_ISO_639-1_codes for list) . Also make sure all tests and migrations are resolved. [2d]

Determine the best approach to having 'translation_of' references to show which dataset the current one is a translation of. This may be a translation of field, or a 'copied from' field or another form of package relationship. [2d]

1326709392000000 1346670010000000
#1652 enhancement kindly ross ckan-backlog assigned How we intergrate with Drupal Multiligual?
  • Analysis of how/where we can integrate with Drupal Multilingual
  • Is it likely to access CKAN via /en/data/... or /data/en/... (not sure, probably former) and we need to make sure that it ends up at /en/...
  • Check with Ian for other issues.
1326709894000000 1340187535000000
#1653 enhancement toby ross ckan-sprint-2012-03-05 closed fixed Change URLs for multilingual site

To support multiple languages we should have an easy way to specify the language as part of the URL, so that URLs are both specific and we also reduce the dependency on the session.

  • Analysis [1d] - Find the best way of implementing this and how everyone else does their language URLs.
  • Write Middleware + update url_for to take account of the language. [2d]

  • Document the language setup, and how to replicate it. [1d]
1326710590000000 1329845387000000
#1656 enhancement ross closed duplicate Configuration for reverse proxying

Provide configuration for reverse proxying that will correctly handle the mapping of a URL to a sub-folder (using X-SCRIPT-NAME)

  • Analysis of the best solution [1d]
  • Implement reverse proxying in tandem with #1653 [2d]
  • Document and store the configuration files. [1d]
1326711575000000 1326711659000000
#1657 enhancement ross ross closed wontfix Configuration for reverse proxying

Provide configuration for reverse proxying that will correctly handle the mapping of a URL to a sub-folder (using X-SCRIPT-NAME)

  • Analysis of the best solution [1d]
  • Implement reverse proxying in tandem with #1653 [2d]
  • Document and store the configuration files. [1d]
1326711580000000 1338206640000000
#1658 enhancement ross ross closed wontfix Implement proxy caching

Implement proxy caching for performance gains

  • Analysis of best solution and what we can deliver in a reasonable time [2d]
  • Checks for presence of cookie 'auth_tkt' and appropriate setup for media [2d]
  • Remove session requirements for user who isn't logged in [3d]

  • Document the configuration and setup so that it can be easily modified and replicated [1d]
1326712354000000 1338206622000000
#1669 enhancement ross ross ckan-v1.7 closed fixed [super] Publisher profile

Implement the necessary backend changes to build profile for publishers using the group refactor.

See http://ckan.okfnpad.org/publisherprofiles

Subtickets are at:

  • Publisher profile analysis #1670 [2d]
  • Publisher profile sysadmin authz #1671 [1d]
  • Publisher profile user authz #1672 [1d]
  • Test auth logic hooks for publisher profile #1674 [3d]
  • Publisher admin authz #1675 [3d]
  • Publisher editor authz #1676 [2d]
  • Add approval status field to groups #1673 [1d]
1326802473000000 1338204071000000
#1670 enhancement ross ross ckan-sprint-2012-03-05 closed fixed Publisher profile documentation

Super #1669

Analysis of all permissions for users given their publishers to ensure that we have covered all of the bases and haven't missed any use-cases.

1326802549000000 1338205180000000
#1671 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Publisher profile sysadmin authz

Super #1669

Implement authz for sysadmins in publisher profile

1326802607000000 1328526614000000
#1672 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Publisher profile user authz

Super #1669

Authz for normal users in publisher profile as per sysadmin

1326802680000000 1328526625000000
#1673 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Add approval status field to groups

Super #1669

As groups need to be authorised in some cases, particularly publisher profiles, we need a field to denote how this is managed.

We should add an 'approval_status' field that will be used to denote its status. By default this will be "approved" but this will allow us to override it with a "pending" or "denied" status. The latter should also change the main status of the Group.

1326802753000000 1328526644000000
#1674 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Test auth logic hooks for publisher profile

Super #1669

Test all of the new auth logic hooks for publisher profile to make sure they work as expected. Also make sure that we have a test to compare the naming of the core auth funcs with the profile hooks (so we can see what may have changed in one but not the other).

1326802841000000 1328526677000000
#1675 enhancement ross ross ckan-sprint-2012-02-20 closed fixed Publisher admin authz

Super #1669

Authz for the publisher admin (who has permission to administer the publisher group themselves).

1326802923000000 1328709915000000
#1676 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Publisher editor authz

Super #1669

Editor authz for publisher groups need to be implemented and tested.

1326802971000000 1328526670000000
#1680 enhancement ross ross closed wontfix Group refactor top level element

The new group refactor allows for a hierarchy of groups where each group has a type - to be able to implement a tree of groups.

It will need a flag within the group to denote that it is a top level group, to circumvent the need to determine whether the group is a child of a parent node (for a specific use case - a publisher representing the department that only contains publishers).

Suggest 'is_top_level'

1326896362000000 1326900832000000
#1681 enhancement ross ross closed fixed Core storage document

Document all of the changes and usage for #1574 so that we can mark it 'done done'.

Make sure we document any setup issues/processes and the settings.py setup required for CKAN so that Mark can announce next week and it should be easily deployable/testable.

Internal (perhaps in README somewhere or wiki): how does process work, where does data end up and in what order etc

External: here's how to configure it and here's what you get ... (goes in main sphinx docs)

1326904939000000 1338202930000000
#1684 enhancement ross ckan-backlog assigned Remove all config from ckanext-archiver

ckanext-archiver currently has a settings file (and a default) and it should be passed in all relevant information from the context.

Remove all settings (ARCHIVE_DIR and MAX_CONTENT_LENGTH and others) and pass them in from CKAN.

1326983821000000 1346670037000000
#1685 enhancement ross ross ckan-backlog closed wontfix Move archiver download function to core

Archiver is included in some places just for access to download(), we should move this to lib in core.

1326984170000000 1346670530000000
#1687 enhancement ross ross closed invalid Deploy new storage changes from master to datahub

We need to deploy the new storage changes (and webstore) to thedatahub for further testing

1327056976000000 1329338671000000
#1690 enhancement ross ross closed fixed Rename storage settings with the ckan prefix

Missed the ckan prefix on the storage settings names so this needs to be fixed.

1327064844000000 1327066713000000
#1693 defect ross ross closed fixed register_pluggable_behaviour should not be in routing

We should try and move register_pluggable_behaviour into environment and out of routing as it is causing import issues when using simple search.

These functions already take a map and so they should be fine any time after the call to make_map

1327321474000000 1338206601000000
#1742 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Naming of a new package/group can clash with a route

It is possible to create a group or a dataset called new, or edit, which will then be inaccessible as it is blocked by the existing routes with those names.

We should maintain a list of invalid names in the verification that happens behind the scenes in the UI.

1328116317000000 1331555193000000
#1755 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Alpha paged view of publishers

The publisher index page should show an alpha paged list of publishers.

1328530634000000 1328631064000000
#1756 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Treeview of hierarchical publishers

The publisher index page should show a treeview of the publishers within the system. Probably behind a tab.

1328530689000000 1328709846000000
#1757 enhancement ross ross ckan-sprint-2012-03-05 closed fixed DGU Hide administrators from public view

On publisher view page - hide administrators from public view - only logged in admins of group should see them.

1328530775000000 1338205187000000
#1758 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Check user management auth

Ensure publisher edit page only allows admins to manage users. #1675 is related.

1328530915000000 1328709897000000
#1759 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Create publisher form access

Only sysadmin can access create publisher form.

1328530956000000 1329732744000000
#1760 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Extract current publisher hierarchy

Get access to live Drupal MySQL database to get latest publisher list for migration.

Write a one-off script for import of the data.

1328531001000000 1328709560000000
#1761 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU History visibility

History on user profile should only be visible to a user who is a member of at least one group.

1328532889000000 1328716686000000
#1762 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Join publisher form

A new form for create publisher -> Add publisher access to your account' New wizard Autocomplete publisher name if linked from publisher view. Email address used to publish should be activated/validated on first use. Original email still used to log in.

1328532992000000 1329729305000000
#1793 enhancement ross ross closed duplicate Publisher changes

Changes to publishers for customer

1329214299000000 1329214522000000
#1794 enhancement ross ross closed fixed Publisher changes

Changes to publishers for customer that would otherwise clutter up the master branch.

1329214301000000 1332148972000000
#1795 enhancement ross ross closed wontfix Add approval_status to Package (as for Group)

The Package model should have a approval_status as the Group model does. See migration 049.

1329234060000000 1338206587000000
#1831 enhancement ross ckan-backlog assigned Login with email address

Users should be able to log in using either their username, or their email address, both of which are unique.

Will require a change to UsernamePasswordAuthenticator? in ckan.lib.authenticator.py and possible a useful User.by_email in the user model if it doesn't already exist.

1330073906000000 1346670504000000
#2209 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Object->RDF mapping

Super is [1649]

We need to provide customisable mappings between CKAN models and RDF. This needs to be easily configurable for different clients and should take into account objects related to the model being mapped.

Current contenders for impl are:

  1. ordf.org's Object Description Mapper (which uses OWL)
  1. RDFAlchemy, although this looks like it may result in fixed mappings
  1. A custom scheme with a simple json file
  1. A template based approach using Genshi templates?

Approach is to get a mapping complete of the attributes directly in the package class, and then provide mappings for related objects (tags/resources etc).

We'll then need to be able to publish the RDF to a triplestore.

See http://ckan.okfnpad.org/rdfmapping for further analysis.

1331027277000000 1336566051000000
#2210 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Add method to group for adding members

It is necessary to add a helper method to Group for adding members to that group so that

  • nobody needs to work with the Member model
  • It handles existing membership and existing membership with a different capacity
1331028470000000 1331549369000000
#2211 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Logic layer access to publisher functionality

Most of the get_group/add_member style calls should be handled, or at least provided via logic layer.

Don't need to break existing code, but do need to provide access via logic layer

1331028751000000 1331549406000000
#2228 enhancement ross ross ckan-sprint-2012-03-19 closed fixed IGroupForm should allow overriding of non-edit templates

IGroupForm should allow overriding of non-edit templates so that it can specify the index +/or search +/or view.

1331637803000000 1332149030000000
#2239 enhancement ross ross ckan-sprint-2012-04-30 closed fixed EC ODP RDF/N3 templates

Implement the EC/ODP template for RDF to match the requested schemas.

Using the previous work on allowing for templatised RDF representations we should implement the proposed schema for EC/ODP.

We also agreed to allow upload of RDF as an extra (raised to top level in package schema) which will instead be used for the RDF template. On insertion extra CKAN triples will be added... initially this is just owl:sameAs although we're awaiting EC/ODP agreement on which triples we should add to show the CKAN link.

1332169703000000 1335516499000000
#2240 enhancement ross ross ckan-sprint-2012-04-02 closed fixed Document CityData ideas

Document the ideas raised on 14/03 for CityData? and distribute to team for discussion

1332169761000000 1332408769000000
#2255 enhancement ross ross ckan-sprint-2012-04-30 closed fixed Organizations (Privacy + Ownership)

[x] The publisher_form extension and the 'publisher' component of the system will be renamed to organizations to both make its functionality clear, and to differentiate it from the publish auth profile.

Ownership of datasets is important and this ticket will enforce membership for a dataset to a single individual organization. The workflow for adding a dataset to a publisher/organization will stay the same but there will be no mechanism for adding existing datasets to an organization from within the organization forms.

[x] New datasets can be created from the organization form, in which case the organization will be set in the dataset form dropdown (with privacy set to private - see below)

[x] Datasets created from direct access to /dataset/new will have a drop down showing all of the organizations available to the current user.

[x] When editing a dataset the organization can only be changed if the current user is the admin of the current organization, and only then to one of the organizations available to the current user.

Only Organization admin can see the edit Organization form.

[x] Privacy will be enforced by changing the dataset form to have a public/private switch next to the organization drop-down which will be set as the capacity with which the dataset is added to an Organization. Private means visible only within the Organization, and should be the default.

[x] Search will be affected as the public/private status of a dataset will need to be added to the schema

[x] Only public datasets showing in site search, and private showing in organization dataset search (if user is within the organization).

[x] Make sure resources are restricted in visibility if the parent dataset is private

[x] Documentation should be updated to reflect these changes.

[x] Verify, then add, then re-add to documentation.

  • A User joins or creates an Organization
  • If creator of the Organization User becomes administrator of the Organization otherwise they become a Member.
  • Creation of new Organization must be approved by a System Administrator
  • New Members must be approved by the Organization Administrator
1332417218000000 1334663670000000
#2284 enhancement ross ckan-backlog assigned Local queries/views/viz in Related stuff.

From #2204

not sure how much we have thought through using this for storing queries / views / visualizations coming from our data viewer. In particular, wonder if this necessitates some kind of support for arbitrary json data ...

### Comments

In the simplest form, right now all I need to store is store the url to the resource plus hashbang config for recline. That would be super easy ...

1334218700000000 1346669513000000
#2302 enhancement toby ross demo phase 3 closed fixed Simple theming

Needs re-testing after the ODS re-theme. ====

Simple image upload, text changes and css updates for a single instance. All done on a single configuration/admin page

See http://ckan.okfnpad.org/simple-themes for user stories

[x] Initial version will only take an image_url rather than allowing an upload.

[x] New page with form for title, tagline, image loader etc.

[x] Allow the user to paste in values for the CSS, do we really want a colour swatch?

[x] Generate local CSS override which should *somehow* override colors in the original CSS.

[x] Allow user to provide text for an about page.

1334582628000000 1344012491000000
#2303 enhancement ross ross ckan-sprint-2012-04-30 closed fixed Move ECPortal rdf dump paster

Move ECPortal rdf dump paster command into core so that it can be run on thedatahub for virtuoso install.

1334582692000000 1334664098000000
#2309 enhancement ross ckan-v1.8 closed wontfix Organizations review with SH

Review organizations code with Sean and explain the key parts.

1334591679000000 1340623966000000
#2312 enhancement ross ross ckan-future closed duplicate Analysis of how datasets could belong to users instead of Groups


Currently datasets can only be part of a group but that is quite heavyweight when a single user wants to upload a single dataset. To resolve this it would be great if a dataset could be attached to a user directly - find out how.

1334663770000000 1340188765000000
#2323 enhancement ross ross ckan-backlog closed fixed Auto-complete in organizations

Apparently auto-complete in Organizations may not work. Check and fix.

See https://github.com/okfn/ckan/commit/5eca7d5e37c0ef392b768b8b3768b2c3f93448b5

1335274354000000 1340188400000000
#2347 enhancement icmurray ross ckan-v1.8 closed fixed Related stuff dashboard and changes

Some additions to the related stuff functionality to make it more useful post the 1.7 release.

[x] Edit item (after checking auth)

[x] My related things on my user page

[x] Show user created related things on the users' page with links direct to the related page of that dataset. {0.5d}

[x] Related stuff usage counts { 1d }

Record the number of times the link for a related item is clicked. This is likely to mean that the link fires to call CKAN and then redirects to the actual URL.

[x] Un-attached related items {0.25d}

Allow related items to be created without being attached to datasets. This may mean that the dataset field is optional in the API.

[ ] Dashboard for related

A dashboard for related items, site-wide (not package specific) and orderable by date, views, {rating}, popularity of the dataset. Would be useful (possibly) for site-admin to mark one or more as 'featured'

[x] Dashboard + Controller (Related) {1d}

[x] Ordering at logic layer (date, views, rating, popularity of package, featured) {1d}

[x] date [x] views [x] featured

[x] 'Featured' related item with mechanism to fetch it. {0.5d}

[ ] UI within dashboard for sysadmins to check/uncheck feature status {0.5d}

[x] Changing auth for related.

Currently auth is based on deletion being possible by dataset owner and the related owner but it may be that a lot of the auth for related items should be configured to have more restrictive auth.

[x] Implement auth changes for publisher auth profile. { 0.5d }

[x] General issues

[x] Serverside validation of URLs

[x] Activity updates

[x] Make optional

[x] Related item only includes App, Idea, Visualization but not API | Post | Paper | News Article options as in original ticket #2204

[ ] UI Changes

[ ] General tidy up, fix form etc (icon for menu bar)

[ ] Highlight featured items py:if="related.featured"

[ ] Show either view count or popularity ${ related.view_count }

[ ] A user's related items should be on their home page

[ ] Related item plus sign is very poorly aligned within box

[ ] Add related items should be in drop down like for resources (?) to make it easy to add items

[ ] Hover should show a title that gives some information about what related are (not at all clear)

"Apps, Ideas, Visualization and other material related to this dataset"

[x] Related is a pretty terrible name in the UI - much better to have it called Apps, Ideas etc (perhaps title tag could even give more details e.g. "Apps, Ideas, Visualization and other material related to this dataset") -- btw i thought this was in #2332 but realize it wasn't (apologies for that!)

  • Went with Apps & Ideas (for now)

[x] Documentation, documentation, documentation - AFAICT I can't see anything in v1.7 or master docs. I imagine this would be a short section like http://docs.ckan.org/en/ckan-1.7/commenting.html (and probably coming right after that in the ToC) which says

  • What it does
  • How to enable (and perhaps includes a nice-screenshot!)
  • How to customize (e.g. can one customize the list of options of things one can create (e.g. can I set it to just be app and idea or ...)
  • Also liase with Mark W re a blog post about this from the user perspective (a screenshot walkthrough ...)

[ ] Clearly mark the old extension as deprecated from v1.7 forward: https://github.com/okfn/ckanext-apps (I just met someone last week who was working on integrating this and had no idea it was replaced by something better in v1.7)

1335879539000000 1343226149000000
#2348 enhancement ross ckan-future assigned Java client library for CKAN

Start a Java library for interacting with the CKAN Action API.

Currently requires support for Groups, Tags, Resources, and Search.

An example app would be very useful.

1335879980000000 1346670024000000
#2349 enhancement ross ross ckan-sprint-2012-06-25 closed fixed Make sure semantic.datahub.io gets created

[x] Redeploy Sparql endpoint

Done at linkeddata.openlinksw.com/sparql

[x] Make sure it is available at datahub.io/sparql or semantic.datahub.io/sparql

[x] Generate daily dumps

[x] Make dumps available via web

[x] Notify Hugh once running

[x] Announce to LD guys

[x] Think about how we can apply this to publicdata.eu (can we do the same?) ... (being done elsewhere)

1335880412000000 1340356629000000
#2353 enhancement ross ross ckan-sprint-2012-05-15 closed fixed Provisional user stories for CMS + Blogging

Provide some provisional user stories for the CMS/Blogging feature. Should include some of the functionality from wordpresser but be more generic.

1335884700000000 1336566004000000
#2364 enhancement ross ross ckan-sprint-2012-05-15 closed fixed Investigation of payment gateways

Need to investigate alternative payment gateways

1335889757000000 1337077531000000
#2365 enhancement ross ross ckan-sprint-2012-06-25 closed fixed Investigation of multisite

What would as a multisite CKAN look like?

This is really part of the work around turnkey/hosted CKAN

1335889911000000 1340266964000000
#2378 enhancement ross ckan-future assigned Extract metadata directly from resources that contain it.

Some files that might be uploaded as resources on a dataset, such as some image formats, already contain metadata. For example some jpeg files might contain Exif ( http://en.wikipedia.org/wiki/Exchangeable_image_file_format ) data.

[ ] Obtain list of embedded metadata 'standards'

[ ] Where the metadata is available it should be added as extra fields on the resource that is uploaded.

[ ] HTML?

[ ] Image formats (exif/gif etc)

[ ] PDF

[ ] Eventually consider ...

[ ] MS Office documents

1336651088000000 1346669985000000
#2379 enhancement ross ross ckan-sprint-2012-06-25 closed fixed RDF output, api/sparql

From: Pierre-Yves Vandenbussche

In my use case I need to fetch every sparql endpoint associated to a dataset. With the previous version of your endpoint, I was doing this query: SELECT DISTINCT ?dataset ?endpoint ?title ?identifier WHERE {

?dataset <http://www.w3.org/ns/dcat#distribution> ?distribution. ?distribution <http://purl.org/dc/terms/format> [ <http://moat-project.org/ns#taggedWithTag> [ <http://moat-project.org/ns#name> "api/sparql" ] ]. ?distribution <http://www.w3.org/ns/dcat#accessURL> ?endpoint. ?dataset <http://purl.org/dc/terms/title> ?title. ?dataset <http://purl.org/dc/terms/identifier> ?identifier.

} ORDER BY ?title

Using your new version,dcterms:title of a dataset is now a rdfs:label ... OK

Unfortunately, I don't have the information of "api/sparql" anymore. So I can not differentiate between a dump file and a SPARQL endpoint...

Add the required information to the RDF template.

1336735947000000 1340033026000000
#2380 enhancement ross ross opendatasuite 1 closed duplicate DataGM Upgrade

Provide a new test install of DataGM based on the ODS demo site being completed through June.

1337002144000000 1340627558000000
#2381 enhancement ross ross ckan-v1.8 closed fixed Related item count on ALL dataset pages

Make sure the related item count is available on all dataset pages (inc. follow) when it is enabled.

1337009133000000 1337011327000000
#2385 enhancement ross ross ckan-sprint-2012-05-29 closed fixed Specifying capacity in search api call

Currently you can specify the capacity in the API search, this should be stripped and default to public for now.

1337092899000000 1337102816000000
#2410 enhancement ross ross ckan-sprint-2012-05-29 closed wontfix New RDF serialization required VOID vocab for related datasets

Taken from http://lists.okfn.org/pipermail/ckan-dev/2012-May/002218.html

A few months ago, the RDF provided when dereferencing a dataset's URI was very useful. It described the datasets using many of the "right" RDF vocabularies (dcterms, void, etc).

The RDF that is returned now no longer has these nice descriptions. Instead, it seems to be a mindless transcription of the CKAN record.

For example, the older descriptions gave:


void:subset [

void:target <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>, <http://thedatahub.org/dataset/geonames-semantic-web> ; void:triples 50 ; a void:Linkset

], [

void:target <http://thedatahub.org/dataset/dbpedia>, <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> ; void:triples 52 ; a void:Linkset

], [

void:target <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>, <http://thedatahub.org/dataset/govtrack> ; void:triples 52 ; a void:Linkset

] ; void:triples 130005 ;

but now it's simply:


dct:relation [

rdf:value "130005" ; rdfs:label "triples"

], [

rdf:value "52" ; rdfs:label "links:dbpedia"

], [

rdf:value "50" ; rdfs:label "links:geonames-semantic-web"

], [

rdf:value "52" ; rdfs:label "links:govtrack"

] ;

My old copies of the dataset descriptions mentioned a "GoLD CKAN 1.0" that was credited for the nicer modeling.

Where did that go? Can we get it back?

Thanks, Tim Lebo


dc:source <http://ckan.net/> ; opmv:wasGeneratedBy [

opmv:used <http://ckan.net/api/rest/package/farmers-markets-geographic-data-united-states>, <http://ckan.net/api/rest/revision/9c2545b8-ebf1-42bf-ac92-633ff5ceba3f> ; opmv:wasControlledBy [

a foaf:Agent ; foaf:name "GoLD CKAN 1.0"

] ; opmv:wasPerformedAt [

a time:Instant ; time:inXSDDateTime "2012-01-31T05:34:32Z"^^xsd:dateTime

] ; a opmv:Process

] ; a dcat:CatalogRecord ; dcat:dataset <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> ;

1337684531000000 1338193633000000
#2413 enhancement ross ross ckan-sprint-2012-05-29 closed fixed Replace/Fork autoneg

Take revelvant parts of autoneg ( https://github.com/wwaites/autoneg.git ) so that we can remove the dependency (just for two functions) and thereby remove flup as well.

1337761125000000 1337766546000000
#2414 enhancement ross ross ckan-sprint-2012-06-25 closed fixed Remove LXML as a dependency on core CKAN

Core CKAN currently has a dependency on LXML and it would be nice if we could remove it (although this *may* be indirectly through genshi).

1337761192000000 1338471374000000
Note: See TracReports for help on using and creating reports.