{22} Trac tickets (2647 matches)

Results (1701 - 1800 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1720 enhancement johnglover johnglover ckan-sprint-2012-02-20 closed fixed Add converter functions to convert tags (with a given taxonomy) to/from tags field

Part of #1698

1327943471000000 1328625423000000
#1721 enhancement johnglover seanh ckan-sprint-2012-02-20 closed fixed Tag taxonomies table migration 1327944237000000 1329133348000000
#1722 enhancement seanh seanh ckan-sprint-2012-02-20 closed fixed Update the package model for vocabularies

See the etherpad for notes: http://ckan.okfnpad.org/22

1327950373000000 1329303197000000
#1723 enhancement seanh seanh ckan-sprint-2012-02-20 closed wontfix Update tests for taxonomies

In the tag-taxonomies branch tags now (optionally) belong to vocabularies and where tags previously had unique names, a CKAN instance can now have two tags with the same name as long as they belong to different vocabularies. The tests for the tags model should probably be updated to reflect this.

Other tests might also need to be updated.

1327950494000000 1329131103000000
#1724 enhancement seanh seanh ckan-sprint-2012-02-20 closed fixed Update logic action functions for vocabularies

ckan/logic/action/get.py:tag_list() Currently (I think) just returns all tags in the CKAN instance that the user has access to. Needs an optional argument to get only tags from a particular vocabulary? When called with no argument, should only return free tags? Also add an optional dataset argument, so you can get tags for a given dataset and/or vocabulary?

ckan/logic/action/get.py:tag_autocomplete() Add optional argument to get only tags for a given vocab? When called with no argument, should get only free tags?

ckan/logic/action/get.py:tag_search() Again, optional arguments for restricting by vocabulary and/or dataset.

1327950761000000 1329302302000000
#1725 defect dread dread ckan-sprint-2012-02-06 closed fixed Tag autocompletion and commas don't mix

If you use tag autocompletion, it automatically adds a comma after the tag. This is fine if you are going to add another tag, but not fine if it is the last one. If you don't manually remove the final comma than you get form validation error "Tag string: Tag "" length is less than minimum 2"

Affects only version 1.5.2a.

1328006709000000 1328204489000000
#1726 enhancement amercader amercader ckan-sprint-2012-02-06 closed fixed Update harvesting model

We need:

  • "current" column in harvest_object
  • "harvest_source_id" column in harvest_object, foreign key to harvest_source
  • "title" column in harvest_source

The extension should check on startup if the db needs updating and run the necessary scripts.

1328007127000000 1328523253000000
#1727 enhancement amercader amercader ckan-sprint-2012-02-06 closed fixed Add "current" property to harvest objects

Right now is quite difficult to get the most current version of a package via the harvest objects, and the queries involved are prone to errors. A new "current" field will be added which will identify unequivocally the most recent harvested object. Unfortunately this will involve some major changes:

  • Model update (See #1726)
  • Update queries for getting harvest sources' datasets
  • Update ckanext-csw queries for getting the objects
  • Update inspire and ckan harvesters
  • Document new field
1328007254000000 1328523641000000
#1728 enhancement amercader amercader ckan-sprint-2012-02-20 closed fixed Update UKLP harvesters

Three small changes:

  • Set Resource type to None instead of 'Undefined'
  • Set extra 'UKLP' = True instead of 'INSPIRE' = True

These two changes will require manual update of the database.

  • Allow datasets to be updated if the old source is inactive, even if the metadata date hasn't changed:
    if last_harvested_object.metadata_modified_date is None \
       or last_harvested_object.metadata_modified_date < self.obj.metadata_modified_date \
       or self.force_import \
       or last_harvested_object.metadata_modified_date == self.obj.metadata_modified_date and last_harvested_object.source.active is False:

1328007610000000 1329498181000000
#1729 enhancement johnglover seanh ckan-sprint-2012-02-20 closed duplicate Helper function for extensions to add vocabularies to a ckan instance

Add a helper function to make it easy for extensions to add new vocabularies to default_package_schema(). The helper function should take the name or ID of a vocabulary (which should already exist in the db) and add the necessary key: [schema] entry to default_package_schema(), with the necessary validation, authorisation, transformation.

1328007723000000 1329131023000000
#1730 enhancement johnglover seanh ckan-sprint-2012-02-20 closed fixed Form field for vocabularies

A function that takes a vocabulary name or ID as argument and returns a nice select box for selecting items from that given vocabulary. Meant to be used by form templates, to make it easy for them to integrate custom vocabularies. Could use http://harvesthq.github.com/chosen/

1328007897000000 1328714937000000
#1731 enhancement amercader amercader ckan-sprint-2012-03-19 closed fixed Publisher support for harvesting

Estimate: 4-5d

Only sysadmins can manage harvest sources right now, we need to allow publishers to handle their own sources.

We need to decide if there will be two different profiles on ckanext-harvest, the current one with only sysadmins allowed to manage the harvesting and a new one supporting publishers (ckan core can be configured to not use the publisher model)

Tasks include:

  • Auth checks on the different controller actions (and the command line interface?)
  • Modify the index page to just show sources the logged user has permissions on.

(Talk with Ross about the specific functions to use)

  • Create/edit form to show a dropdown with the user available publishers.
1328007919000000 1332152540000000
#1732 enhancement johnglover seanh ckan-v1.8 closed fixed Update CKAN's search for taxonomies

Initially we'll just add all tags from all vocabularies into the existing tags search facet (this should happen automatically). Later we want to enable templates to use separate search facets for different vocabularies.

Also need to make sure that search terms are matched against the texts of the tag names themselves (should happen already) and maybe also against the texts of the vocabulary names?

1328008169000000 1341224357000000
#1733 enhancement amercader amercader ckan-sprint-2012-02-20 closed fixed Delete (inactivate) harvest sources

Right now harvest sources can be inactivated (only via the cli), but we need to make sure:

  • Expose this on the web interface (only for sysadmins, see #1731)
  • Inactive sources are not shown in the web interface

Also for deleted packages:

a) Deleted packages get state="deleted" (not actually purged) so they are hidden on front end ACTION: need a button on the front end for "Delete" for sysamdins

Done in #1425. Not sure if the button is only shown to sysadmins.

b) No undelete: If you reharvest a deleted package, it remains deleted

  • Publisher should create a new record with new identifier and same/different content as applicable.
  • If publisher reharvests, a deleted document they get a error and are told to create new identifier for their updated.
  • You won't get the state where a package is updated in its deleted state
  • If they change the ID and reharvest, a new record appears on DGU.

c) Update CSW interface so that export doesn't export deleted packages d) Check geospatial search doesn't search deleted pacakges e) Check deleted packages don't appear in harvest source details, or in the counts or anywhere in the harvest interface or API.

1328008287000000 1329754689000000
#1734 enhancement amercader amercader ckan-v1.7 closed fixed [super] Updates on harvesting for UKLP

Some changes are needed in ckanext-harvest to support the new piece of work in the UKLP harvesters, mainly related to the deletion of existing sources.

  • #1726 Update harvesting model
  • #1727 Add "current" field to harvesting objects
  • #1728 Update UKLP harvesters
  • #1733 Delete (inactivate) harvest sources

At some point we also want to support a publisher model for the harvesting:

  • #1731 Publisher support for harvesting
1328008474000000 1332167315000000
#1735 enhancement zephod dread ckan-sprint-2012-02-20 closed wontfix "Unsaved changes" warning briefly when you click "Save"

Super ticket #1744

If you edit a dataset, simply add a tag and click on "Save" then you get the message "You have unsaved changed. Make sure to click 'Save Changes' below before leaving this page." inserted whilst the submission takes place. (This is because the click moves focus away from the tag field). This (reasonably common) edge-case could usefully be squashed.

Also note typo: "unsaved changed"

1328009420000000 1328609804000000
#1736 enhancement kindly kindly ckan-v1.7 closed fixed [Super] Multilingual Metadata

Translate all areas of metatdata to selected language. This includes finding datasets in all languages.

Multilingal Metadata: Tasks:

  • Add translation table. 2d
  • Api to add translation 3d
  • Add extention point to dataset view. 3d
  • Add extension point to search index to add translations. 2d.
  • Add all eurovoc data to translations. 1d.
  • Make multilingual solr schema 2d #1715
1328012697000000 1338205016000000
#1737 enhancement icmurray rgrp ckan-v1.8 closed fixed Expose solr-based search API

Super ticket: #1745

Required for some improvements to UX (such as autocomplete and better search).

1328014626000000 1340112732000000
#1738 enhancement kindly kindly ckan-sprint-2012-02-06 closed fixed Add extension point to search index, so can modify dataset dict before index.

It is useful for extensions to add or change items that go into the search index. Add an extension point for this.

1328031572000000 1330086105000000
#1739 enhancement kindly kindly ckan-sprint-2012-02-06 closed fixed Fix issues where tests where not run.

Test failures in master due to tom and rufus not running tests. 0.3d.

1328047129000000 1328495826000000
#1740 enhancement seanh ckan-future new Get rid of `from module import ...`

It's really bad to do from module import * and CKAN has a lot of them. I suggest a three-pronged approach:

  1. Don't add any more of them.
  1. When you're programming if you see an easy opportunity to remove one then do so.
  1. At some point we should task someone to go through the code and remove them all (which is what this ticket is for), but this will be a big job and may break things.

We should also get rid of most or all of the from module import foo and from module import foo, bar statements.

I think the right thing to do is just import module and then use module.foo in your code. But if you find yourself doing module.foo.bar then you may have a code smell.

See: http://docs.python.org/dev/howto/doanddont.html

1328094369000000 1328094884000000
#1741 enhancement kindly kindly ckan-sprint-2012-02-20 closed fixed Add extention point to dataset view.

Need to add extension point to change the data_dict that passed to the dataset view template. This extensions modify what data gets shown.

Tasks include:

  • Fix up data_dict and templates so that the pkg objects do not get used in the templates. This is to assure that all data passed is modifiable. (2d) (complete)
  • Add extension point to Ipackage controller (1d)
  • Test (1d)
1328105709000000 1329750838000000
#1742 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Naming of a new package/group can clash with a route

It is possible to create a group or a dataset called new, or edit, which will then be inaccessible as it is blocked by the existing routes with those names.

We should maintain a list of invalid names in the verification that happens behind the scenes in the UI.

1328116317000000 1331555193000000
#1743 defect seanh seanh ckan-v1.6 closed wontfix Activity streams still link to deleted things

For example, after deleting a dataset old activity stream events will still hyperlink to that dataset.

1328212016000000 1328527086000000
#1744 enhancement zephod rgrp ckan-sprint-2012-03-05 closed fixed UX improvements to dataset edit as of Dec 2011

Super ticket: #1506

  • Show edit summary form again, at least on edit pages (and prefill it for create to something sensible like: Creating dataset {title} {name}) [Done #1506]
  • Fix drop down for resource types in resource add/edit to limit to fixed set of options (as per http://wiki.ckan.org/Domain_Model/Resource)
  • Allow reordering of resources (or: and this is somewhat more complex but possibly simpler: allow editing of a resource_order field and we use that to sort resources (this is more like wordpress page_order -- these two options could become the same if we just hijack the existing position field on the join table and get rid of ordereredlist extension)) [DONE #1506]
  • Radically simplify set of licenses [DONE #1359]
  • "Unsaved changes" warning briefly when you click "Save" #1735
  • "Unsaved changed" typo on dataset edit page
1328224628000000 1330349472000000
#1745 enhancement rgrp ckan-v1.9 new Dataset search UX improvements as of Jan 2012

Changes to make search both more exploratory and more satisfying to use

  • Search query build - #1603
    • Ability to add new facet fields "live"
    • That is add fields which then contain faceted options (a bit like data.hri.fi)
  • (??) Autocomplete / drop down on search (i.e. search while you type)
    • Dubious about value / cost ratio here

Probably would involve to pure JS and HTML implementation.


Probably require

  • API changes to expose solr style API directly #1737
1328224941000000 1340033358000000
#1746 enhancement seanh ckan-backlog closed wontfix Activity streams pagination

Currently user, package and group activity streams only return the most recent 15 activities, even though all activities are kept in the db. Do we want to add pagination - to both the API and the HTML pages - to support retrieving older activities?

1328446488000000 1355141062000000
#1747 enhancement seanh ckan-backlog new Expire old activities

Currently the activity streams database tables just get longer and longer over time. Do we want to eventually delete the oldest activities, to keep the length of the table within limits?

1328446589000000 1339773859000000
#1748 enhancement seanh seanh ckan-future new Make activity streams conform to http://activitystrea.ms/ standard

At the very least we should make our JSON output conform to their spec:


1328450784000000 1338204337000000
#1749 enhancement seanh ckan-future assigned Allow creating activity details through API

Currently the activity_create() logic action function only lets you create top-level activity stream items, and not their related activity details. It should handle activity details via nested dicts.

1328465406000000 1338213933000000
#1750 enhancement seanh ckan-backlog new Move ckan/lib/activity.py into the model

Move ckan/lib/activity.py moved to into the model - say ckan/model/activity_extension.py, because it's so tightly knit with the model code, whereas most of the lib code is used in the controllers.

1328465888000000 1339773840000000
#1751 enhancement icmurray icmurray ckan-sprint-2012-02-20 closed fixed DGU dataset form: collection of updates from feedback

A collection of improvements to the dataset creation/edition form (feedback form DGU).

  • [x] validation error formatting
  • [x] geograohic coverage: add local authority
  • [x] resource format field
  • [x] resource format auto-complete
  • [x] renamed data tab -> files tab
  • [x] integrate Adris'a INSPIRE changeset, and migration scripts as part of build
  • [x] remove "discontinued" from the update frequency
  • [x] disable the disabling of save
  • [x] Remove ability to edit url on the edit-form (and remove "url not available")
1328526612000000 1329733458000000
#1752 enhancement icmurray icmurray ckan-sprint-2012-02-20 closed fixed DGU: collection of updates from feedback
  • [x] hide black stars on the data-view page
  • [x] Edit on view form (if you have the right permission)
  • [x] RES_FORMAT ~> Resource Format
  • [x] Map Based Search -> UK Location Map Based Search
  • [x] Note at the top right expalining that these are seach filters (if they are) -> this requires "add a dataset" and "other access" to be moved to a different place (just some whitespace is fine)
  • [x] Display publisher and provider (UKLP records only).
  • [x] analysis of filter by OGL / non-OGL
  • [x] provide /dump on the build server
1328526750000000 1329733515000000
#1753 enhancement kindly rgrp ckan-sprint-2012-03-05 closed fixed Last fixes of Webstorer + Deploy on the DataHub 1328527932000000 1330990061000000
#1754 enhancement seanh seanh ckan-sprint-2012-02-06 closed fixed Add activities for extras

When an extra is added to or removed from a dataset or is updated, an activity should go in the dataset's and user's activity streams.

1328528318000000 1328528330000000
#1755 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Alpha paged view of publishers

The publisher index page should show an alpha paged list of publishers.

1328530634000000 1328631064000000
#1756 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Treeview of hierarchical publishers

The publisher index page should show a treeview of the publishers within the system. Probably behind a tab.

1328530689000000 1328709846000000
#1757 enhancement ross ross ckan-sprint-2012-03-05 closed fixed DGU Hide administrators from public view

On publisher view page - hide administrators from public view - only logged in admins of group should see them.

1328530775000000 1338205187000000
#1758 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Check user management auth

Ensure publisher edit page only allows admins to manage users. #1675 is related.

1328530915000000 1328709897000000
#1759 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Create publisher form access

Only sysadmin can access create publisher form.

1328530956000000 1329732744000000
#1760 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Extract current publisher hierarchy

Get access to live Drupal MySQL database to get latest publisher list for migration.

Write a one-off script for import of the data.

1328531001000000 1328709560000000
#1761 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU History visibility

History on user profile should only be visible to a user who is a member of at least one group.

1328532889000000 1328716686000000
#1762 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Join publisher form

A new form for create publisher -> Add publisher access to your account' New wizard Autocomplete publisher name if linked from publisher view. Email address used to publish should be activated/validated on first use. Original email still used to log in.

1328532992000000 1329729305000000
#1763 enhancement seanh seanh ckan-v1.7 closed wontfix Add some simple model-level tests for activity streams 1328536285000000 1338204708000000
#1764 enhancement seanh seanh ckan-v1.7 closed wontfix Move activity streams API tests into tests/logic

Which is where the other API tests have now been moved.

1328536499000000 1335878051000000
#1765 enhancement seanh seanh ckan-sprint-2012-02-20 closed fixed Enhance Tag and Package models with vocabularies

Add a vocabulary column to the tags database table, change tags to have unique (tag_name, vocabulary_id) instrad of unique tag name, update methods in the Tag and Package classes to deal with the fact that tags may belong to vocabularies and that tag names are no longer unique.

1328537061000000 1329131124000000
#1766 enhancement johnglover johnglover ckan-sprint-2012-02-20 closed fixed Update and deploy EC Portal theme 1328537446000000 1328541227000000
#1767 enhancement johnglover johnglover ckan-sprint-2012-02-20 closed fixed Write end-to-end functional tests for tag vocabularies

Need to test:

  • API call to create new vocab
  • API call to add tags with a given vocab
  • API call to update tags
  • Dummy plugin to test that updating tags works via WUI
1328537586000000 1329326678000000
#1768 enhancement icmurray icmurray ckan-v1.7 closed fixed DGU: 7a. Public Publisher Dashboard (including QA Work and notifications) MUST PHASE 1 1328539740000000 1337159873000000
#1769 enhancement icmurray icmurray ckan-sprint-2012-02-06 closed fixed 6. Publisher Registration Improvements MUST PHASE 1 1328539789000000 1329737597000000
#1770 enhancement icmurray icmurray ckan-sprint-2012-02-06 closed wontfix 3. Themes/Taxonomy as well as tags WONT PHASE 2 1328539834000000 1329733935000000
#1771 enhancement icmurray icmurray ckan-sprint-2012-02-20 closed fixed 32. Browse by Tags PHASE 1 MUST 1328539862000000 1329733648000000
#1772 enhancement icmurray icmurray ckan-sprint-2012-02-06 closed fixed 13. Provider labelling MUST PHASE 1 [UKLP #14 MUST] 1328539902000000 1329733962000000
#1773 enhancement icmurray icmurray ckan-sprint-2012-02-20 closed fixed 17. Filter “UK Location Records” MUST PHASE 1 [UKLP #21 SHOULD] 1328540130000000 1329733534000000
#1774 enhancement icmurray icmurray ckan-sprint-2012-02-20 closed fixed 22. Mixed Licenses MUST PHASE 2 1328540153000000 1329733550000000
#1775 enhancement seanh seanh ckan-sprint-2012-03-05 closed fixed Enable adding many tags at once to controlled vocabularies via API

We already have API calls for adding one tag or removing one tag at a time to/from a vocab. Want more convenient ways to add or remove many in one call, or just pass in a list of tags to completely replace a vocab's current list.

  • Add tags_create() function that calls the existing tag_create() many times?
  • Also tags_delete()?
  • Or just let tag_create() and tag_delete() accept a list?
  • Add tags argument to vocabulary_update() to pass in a list of tags to replace the vocab's current tags? (But it should still be possible to simply rename a vocab without changing its tags and without having to pass in the tag list.)
1328541056000000 1329754459000000
#1776 enhancement seanh seanh ckan-sprint-2012-02-20 closed fixed Granular editing of vocabulary tags

Add API calls for adding one or more tags to and removing one or more tags from a vocabulary, without affecting the other tags in that vocabulary and without having to pass the full list of the vocabulary's tags.

1328541213000000 1329302364000000
#1777 enhancement kindly kindly ckan-sprint-2012-02-20 closed fixed Write about multilingual features for ec.

Basic description of multilingual features intended to be supported.

1328542835000000 1329393295000000
#1778 enhancement seanh seanh ckan-future assigned Replace classmethod's with module-level functions

In many places we have @classmethods where simple module-level functions would do (and would be more idiomatic in Python), e.g. the get() classmethods in the model/ classes.

Doing it with module functions lets us type module.function() instead of module.Class.method().

In many cases we're importing classes directly with from module import Class and then doing Class.method(), but from module import foo is bad style: http://docs.python.org/dev/howto/doanddont.html

1328543614000000 1340623743000000
#1779 enhancement kindly kindly ckan-sprint-2012-02-20 closed fixed Add multilingual translation table.

A new table with 3 columns should be added. term, term_tranlastion, language_code. This table will be used for all translations, including tags. The table should have indexs on both the term and (term, language_code) combination.

1328548631000000 1329393759000000
#1780 enhancement kindly kindly ckan-backlog closed duplicate Api to add translations.

Add api to translation to the term_translation table.

  • add to logic 2d
  • test 2d
1328548908000000 1328578479000000
#1781 enhancement kindly kindly ckan-sprint-2012-02-20 closed fixed Api to add translations.

Add api to translation to the term_translation table.

  • add to logic 2d
  • test 1d
1328549031000000 1329393822000000
#1782 enhancement zephod rgrp ckan-future new Improvements to datasets page UX (Feb 2012)
  • Relationships in sidebar [WONTFIX yet without some link to documentation or datastructure, something for me to render?]
  • Icons for *types* of resources in resource list
  • List of 'Owners' (administrators) at top right of sidebar (a la group page)
    • Much prefer owner to administrator ...
1328609441000000 1338205232000000
#1783 defect dread dread ckan-sprint-2012-02-20 closed fixed Locale change exception when unicode in URL

If the current page URL has unicode in it (e.g. u'/tag/biocombust\xedveis') then I'm not sure how why it isn't more encoded, but we are seeing links such as http://beta.dados.gov.br/dados/locale?locale=hu&return_to=/dados/tag/biocombust%C3%ADveis&hash=7fd941b5ba1a28bc84406c4191bf07b45a42a251 which cause an exception when the foreign character is passed to ckan.lib.hash.get_message_hash().

1328611485000000 1328612744000000
#1784 enhancement seanh seanh ckan-v1.7 closed wontfix Links to renamed objects break

The HTML templates for some activity stream activities link to objects (packages, resources, etc.) by their names, e.g. /dataset/foo, but if the object has since been renamed then the link will be broken.

Possible solutions are to use the object ID to find its current name at rendering time and link to that, or link to the object by ID rather than by name.

Linking by ID is both simpler to implement and better. Links by ID are ugly, but they are permanent and don't break when an object is renamed. We should be using links by ID everywhere on CKAN, so that when third-party sites copy-paste our URLs and link to us, their links don't break later.

Fixing this means fixing the dataset_link(), resource_link(), group_link() etc. functions in ckan/lib/helpers.py which will change the links everywhere in CKAN where those functions are used to link to objects (which should be everywhere where we link to an object).

If we want to have nice looking by-name URLs and what them to be permanent, then we need to change CKAN so that objects cannot be renamed.

1328613555000000 1338204797000000
#1785 enhancement seanh seanh ckan-v1.9 accepted Replace 'Revisions' page with site-wide activity stream

Replace the /revision page with a /activity page showing a site-wide activity stream of all activities in the site. Or perhaps the site-wide activity stream can go somewhere on the front page of the site instead?

1328632458000000 1340724312000000
#1786 enhancement dread [email protected] ckan-sprint-2012-02-20 closed fixed Relationships lost when editing package

If I have a relationship between two packages, and then I edit any property of one of them (Author email, for example), the relationship will automatically be lost.

1328635226000000 1328640597000000
#1787 enhancement dread ckan-future new [super] Improve RESTful API
  • Lists of entities should be full URLs, rather than just the names
  • Discoverability - /api/v3/rest should list the entity types that can be listed

This could be v3 of the RESTful interface.

1328702082000000 1328702082000000
#1788 defect zephod zephod ckan-sprint-2012-03-05 closed fixed IE7 compatibility problems

From Antonia Acuno, via Ira:

It appears that the current layout for the datahub does not work in IE7, which would mean a problem for us as most gov uses it and is one of those we have to support...any views?

Tested in several gov departments computers, so a common enough issue.


From Rufus: We haven't officially supported IE7 so far but can put engineering effort into this if absolutely needed. I note that IE 7 is now 3y out of date and is likely a security risk. Obviously this is something to ask Antonio about.

1328799250000000 1333133365000000
#1789 enhancement seanh ckan-backlog new Implement a tag_update() logic action function

So users can rename a tag and/or move it between vocabularies using the API.

Currently we have create_tag() and delete_tag(), but if you were to 'update' a tag by deleting it and then recreating it all the datasets that had that tag will have lost it and you'll have to re-add it to them all.

What should happen to datasets that have the tag, if the tag gets moved between vocabularies? All the datasets just keep the tag with the new vocabulary? This will become a problem if/when we support 'radio button'-style vocabularies (where each dataset can only have one tag from the vocabulary).

1328805413000000 1339773812000000
#1790 enhancement dread ckan-future new Click to delete tags, rather than have all existing tags in the tag text box

From Pablo:

Editing the tags field is clumsy when there are too many tags. Could show existing effectively as tags (like delicious), then allow clicks to delete. New tags added via text box.

1328888674000000 1328888674000000
#1791 defect dread dread ckan-sprint-2012-02-20 closed fixed Changing locale on /dataset/new causes exception

When you are on the /dataset/new page and you try and change locale then you get a 500 error.

This is because it adds the 'cache' parameter, to ensure any proxy cache in the chain does not just send the cached page.


1329134556000000 1329138315000000
#1792 enhancement toby kindly ckan-sprint-2012-03-05 closed fixed [super] api to use same validation and logic as wui forms.

The api currently uses the default validation schemas regardless of the form you specified on the front end. There should be a way to post through the api using the same validation rules.

1329146487000000 1333037710000000
#1793 enhancement ross ross closed duplicate Publisher changes

Changes to publishers for customer

1329214299000000 1329214522000000
#1794 enhancement ross ross closed fixed Publisher changes

Changes to publishers for customer that would otherwise clutter up the master branch.

1329214301000000 1332148972000000
#1795 enhancement ross ross closed wontfix Add approval_status to Package (as for Group)

The Package model should have a approval_status as the Group model does. See migration 049.

1329234060000000 1338206587000000
#1796 enhancement ross dread ckan-future closed fixed Get rid of lxml dependency

lxml is used in CKAN core in two places:

  • ckan/lib/helpers.py:239 Parsing Markdown and ensuring it is valid XML (i.e. tags close nicely so the HTML of the rest of the CKAN page is not disruptable).
  • reading the SOLR config XML to determine what version it is.

Both of these could be done with xml.dom built into Python and would save us a sizeable and slightly problematic dependency.

lxml is needed by these extensions: ckanext-inspire, ckanext-harvest, ckanext-csw & ckanext-wordpresser, so we'd have to add this to their requirements.

1329244440000000 1340635557000000
#1797 enhancement rgrp rgrp ckan-sprint-2012-03-05 closed fixed [super] Webstore v2 - Integrated and based on ElasticSearch

Main tasks:

  • Integrated seamlessly into CKAN (authorization etc)
  • Based off ElasticSearch?

Etherpad: http://ckan.okfnpad.org/webstore


  • New API point (config + controller) [0.75d]
    • /api/data/{resource-id}
  • Authorization [0.5d]
  • Documentation [0.25d]
  • Upgrade data viewer to use ES [0.5d]
  • Convert webstorer to new setup - http://github.com/okfn/ckanext-webstorer [1d] - #1753
    • See below
  • "Create DataStore? Resource" in Resource creation (??)
    • Just creates a resource with blank url and datastore enabled (webstore_url non null)
  • Tick box to enable DataStore? in Resource edit form (for time being would just set webstore_url to any non-null/non-empty value)
    • Also need to use this info in DataStore? extension?
  • Create nice instructions on Resource page about using the DataStore?
  • Rename webstore -> datastore
  • Deploy [0.5d]


Layout on ES


index = ckan-{ckan-id}
type-id = resource-id (1)

Request path

Req => Nginx => CKAN App (authz) => req to elasticsearch and returns results

See as an example: https://github.com/okfn/elastic-proxy


  1. How do we decide what to use in the viewer e.g. Webstore data versus DataProxy?
    • Check for existence of webstore_url on Resource (should really be webstore_enabled boolean or similar, however we stick with url for the time being because it exists and we avoid migration). If exists use Webstore o/w don't.
  2. How do we decide when webstore is activated
    • Ans: see previous.
  3. Do we (and how do we) allow for resources which are just for webstores
    • Ans: we have to set webstore_url = True ... on save. Or add a big button saying: "Create Blank Resource" ...
  4. How does webstorer work ...
    • CSV import - NS: make sure we add e.g. rownum or id = rownum ...
    • Separate mini-service (webstorer.thedatahub.org ...)
  5. How do we selectively disable recline where not appropriate (e.g. query support makes no sense for DataProxy?)
    • Ans: do not bother at present. Focus on Webstore route.
  6. Where does geodata support come in (in Recline or separately)
    • This is a separate issue

User Story 1

User -> CKAN instance -> Add Dataset -> Adds a Resource via uploading a CSV file -> Save

=> Result = New Dataset + Resource + Data API to resource and nice Preview of the data


  1. Allow normal upload process
  2. Webstorer queue task which should get triggered by new resource. Check the resource and if it can convert into the associated resource webstore database.
  3. Update the resource in some way (set webstore_url)

User Stories

User Story 2

User -> CKAN instance -> Add Dataset -> Adds a Resource via linking to some remote file (CSV or otherwise) or uploading some data (CSV or otherwise)

=> Result = New Dataset + Resource + nice Preview of the data (wherever possible)

User Story 3

User -> CKAN instance -> Add Dataset -> Adds a Resource which will be webstore database in CKAN -> User saves data to this data store.

=> Result = New Dataset + Resource (Webstore resource) which can now be written to or read from by the user

Solution: just create a resource with blank url (or better, have a new option to create webstore resource ...)

1329306185000000 1331412644000000
#1798 defect dread closed fixed API search in non-q fields has exception for unicode characters

You get an exception if you use the API to search packages and specify a non-ascii character in a field other than q.

For example:


This "N-dash" (Unicode character 2013) causes this exception:

Module ckan.controllers.api:460 in search
<<                      if ver in u'12':
                               # Otherwise, put all unrecognised ones into the q parameter
                               params = convert_legacy_parameters_to_solr(params)
                           query = query_for(model.Package)
                           results = query.run(params)
>>  params = convert_legacy_parameters_to_solr(params)
Module ckan.lib.search.query:38 in convert_legacy_parameters_to_solr
<<      for search_key in non_solr_params:
               value_obj = legacy_params[search_key]
               value = str(value_obj).replace('+', ' ')
               if search_key == 'all_fields':
                   if value:
>>  value = str(value_obj).replace('+', ' ')
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 30: ordinal not in range(128)

This problem affects CKAN 1.5 and 1.5.1 only.

1329395420000000 1329395560000000
#1799 defect dread dread ckan-sprint-2012-03-05 closed fixed To login as a different user or register, you need to logout first

When you are logged in, you cannot login as a different user before first logging out. (This is a restricting in repoze.who, which gives priority to an existing recognised identity over another one you supply.)

Yet, the CKAN login form appears when you try to do something you're not allowed to, even if you are logged in already. You login with a different identity and nothing happens - you remain as the first identity - this is incorrect.

(It is also a little confusing to be sent to the log-in page, although we do have a flash message explaining, and you might want to relogin as a superuser if you can.)

Whilst in this area, it would be worth fixing the problem for developers where you are logged in, do "paster db clean" and then try to log in - AuthTkt? recognises your old cookie but the User object doesn't exist, so you get an error logging in. In this instance we should tell people to log-out.

1329402820000000 1330002176000000
#1800 refactor seanh ckan-future new Tidy up *_list() and *_search() functions in ckan/logic/action/get.py

For consistency all the *_list() functions should list objects only and not accept an optional search query. There should be *_search() functions whenever search is needed.

Currently it is inconsistent, e.g. package_list() and resource_list() do not accept a search query option and there are package_search() and resource_search(), but user_list() does accept a query and there is no user_search(). tag_list() also accepts a query, and there is also a tag_search() that apparently duplicates the search functionality.

1329405129000000 1338204886000000
#1801 enhancement dread ckan-backlog closed fixed No links to password reset

You can reset your password (#1186) but you have to know the URI (/user/reset) - there is no link to this page!

1329405290000000 1343144718000000
#1802 enhancement dread dread ckan-sprint-2012-03-05 closed fixed Investigate CKAN deployment on Windows

See how possible it is to run CKAN on a Windows server.


  • Get CKAN code and dependencies installed on Windows
    • ideally as similar to standard package install as possible
    • virtualenv/pip should work
    • try Cygwin to help
    • SOLR and Postgres could potentially run on other servers if necessary.
  • Try to run on Apache
  • See if there are any issues
1329409512000000 1329830097000000
#1803 enhancement toby toby ckan-sprint-2012-03-19 closed fixed clean up routes 1329483497000000 1332333091000000
#1804 defect toby dread ckan-sprint-2012-03-05 closed fixed CKAN mounted at URL - changing language problem

e.g. clicking on Deutsch link is and redirect goes to so /dados/dados/ when it should be /dados.

Affects CKAN 1.6b only (not yet released).

1329484956000000 1330347185000000
#1805 enhancement toby closed fixed Error pages do not translate

eg 404 page

create a 404 error via a bad url - translation links go to /document/error not the actual bad url

1329489216000000 1329760795000000
#1806 task toby toby ckan-sprint-2012-03-05 closed fixed make drop down language selector

language chooser should look like the ec one

1329741393000000 1331295310000000
#1807 enhancement seanh seanh ckan-sprint-2012-03-05 closed fixed Recently added/updated datasets for EC portal

For EC Portal need a page (or HTML snippet embedded in the front page?) listing recently added or updated datasets.

  • Could be generated from the activity streams
  • Should it list datasets only, or other activities as well?
  • May be related to #1785 (Replace 'Revisions' page with site-wide activity stream)
1329742326000000 1330809641000000
#1808 enhancement seanh seanh ckan-sprint-2012-03-19 closed fixed Translation of content on dataset view pages

All data is that is translatable should be translated when viewed.

This includes tag names will be ID codes. When viewing a dataset page, look up the current language of the interface (ckan_lang or something in the environs) and display the string for the tag in that language.

Related to the dataset view extension points for translation, recently added by kindly/=.

1329742530000000 1331146804000000
#1809 enhancement johnglover johnglover ckan-sprint-2012-03-05 closed fixed Catch request exceptions in archiver link_checker task

Some request exceptions are currently not being caught (see the celery log on thedatahub for examples)

1329746267000000 1330528828000000
#1810 enhancement johnglover johnglover ckan-backlog closed wontfix Store list of Eurovoc tag names and IDs
  • Tag list not be added to CKAN core, what is the best way to do this in an extension?
  • Does the list need to be updated via API/WUI?
1329746612000000 1335874745000000
#1811 enhancement johnglover johnglover ckan-backlog closed wontfix Translate Eurovoc tags from ID to dataset language 1329747684000000 1335874765000000
#1812 enhancement johnglover johnglover ckan-backlog closed fixed Restrict editing rights/permissions based on publisher
  • see how this currently works with DGU first
1329747889000000 1335874864000000
#1813 task amercader amercader ckan-sprint-2012-03-05 closed fixed Upadate ckanext-pdeu to work with CKAN 1.6

Tasks include:

Estimate 4d

  • Update requirements (git)
  • Search facets error
  • Error on dataset read (extras in context)
  • Url generator for rdf
  • Make templates work with latest theme
  • Use dicts in templates (isopen())
1329757128000000 1330102557000000
#1814 enhancement amercader amercader ckan-sprint-2012-03-19 closed fixed Check publicadata.eu harvesters

Estimate 2d

Once ckanext-pdeu is running on CKAN 1.6, upgrade ckanext-harvest to be able to update the CKAN harvesters (default tags and extras).

Also check non-CKAN harvesters (specially scrappers) to see if they are still working.

Make a list of current harvesters with status and potential ones.

1329757408000000 1332152596000000
#1815 task amercader amercader ckan-future new Reenable Sparql endpoint on publicdata.eu

Estimate: ?

The 4store Sparql endpoint has been down for a long time. It needs to be reenabled and we need to check that the necessary informtion is pushed when creating /editing a dataset.

1329757723000000 1338204641000000
#1816 enhancement amercader amercader ckan-sprint-2012-03-19 closed fixed Update ckanext-apps to work with CKAN 1.5.1 - 1.6

Estimate [2-3d]

If possible, it should work with both 1.5.1 and 1.6.

Needed for PDEU, also at some point for IATI

1329757864000000 1331302835000000
#1817 enhancement johnglover johnglover ckan-sprint-2012-03-05 closed fixed Importer for EC publisher info
  • write importer for publisher info (currently in JSON format), add to ckanext-ecportal
  • add group translations to translations table
  • run on test server
1329762290000000 1329922818000000
#1818 enhancement amercader amercader ckan-future new Spatial metadata editor

Estimate 2.5d

Right now users need to manually enter a geojson value in the 'spatial' extra field in order to define a geometry for a dataset.

We need a map widget on the edit form that allows users to draw the geometry on top of a map. This will be based on the draw functions offered by OpenLayers? [1], [2].

Things to take into account:

  • Integration with the form sections.
  • Allow some degree of configuration (default extent, limit geometry types...)

[1] http://openlayers.org/dev/examples/draw-feature.html [2] http://openlayers.org/dev/examples/modify-feature.html

1329763941000000 1338203195000000
#1819 enhancement kindly kindly ckan-sprint-2012-04-02 closed fixed change search index code to use package_dictize instead of obj.to_dict

Search index code needs to use package_dictize so that it can get the new vocablurary information and is more consistant with the rest of the system. This is in preperation for their translation being put in the search indexer.

1329770555000000 1338193886000000
Note: See TracReports for help on using and creating reports.