ticket summary component version milestone type owner status created _changetime _description _reporter 256 Package relationships - 3. Edit in WUI ckan ckan-backlog requirement assigned 2010-02-23T12:36:01Z 2012-06-15T15:38:34Z "WUI: * Editable as part of package or separately? (e.g. like authz) * Do we normalize to only one type name of the pair? * Do we allow create of relationship from both ends (e.g. only from dependency to dependent or either way?) " dread 728 CSW Harvesting shall be optimised in respect of reharvesting only records that have changed uklii ckan-backlog requirement amercader assigned 2010-10-21T15:35:40Z 2011-07-08T11:33:04Z Hi Will, this is important again because some CSW servers we use have over 300 documents in. Could you take a look at modifying the filter please? johnbywater 794 Investigate reconciling UKLP Publisher and Provider with DGU uklii ckan-backlog requirement amercader assigned 2010-11-08T14:50:11Z 2011-07-20T16:33:01Z "This needs more analysis, but the GEMINI2 attribute ""metadata point of contact"" must be reconciled with the registered publisher (or agent). This might also be used to filter records harvested from a CSW source, but filtering also needs more analysis, as does distinction between agent and provider." johnbywater 143 Most active users listed on homepage ckan ckan-backlog enhancement thejimmyg assigned 2009-10-08T13:59:33Z 2011-08-03T11:53:01Z Display league of users' recent activity on homepage. dread 84 Revert support on versioned objects vdm ckan-future enhancement kindly assigned 2009-07-23T08:59:03Z 2012-06-25T12:13:05Z "Basic revert in the classic wiki form is already support by purging a Revision. However may wish to support: 1. Cases where multiple objects changed in a revision but only want to revert 1 (low priority) 2. Want to revert but have reversion as a new revision of that object. Seems low priority at present. Cost: ?" rgrp 763 Read-only mode - Setup ckan ckan-future enhancement assigned 2010-10-26T11:11:46Z 2012-05-28T11:56:44Z "Admin configures entering read-only mode in one of two places: * CKAN config file (e.g. ckan.ini) * environment variable from Apache config Once enabled, no writes can occur to the database (including user ratings and other usage stats)." dread 285 Paginate list of packages on tag read page ckan enhancement assigned 2010-04-07T18:23:26Z 2012-06-25T13:45:23Z "Is this worth doing? On hmg.ckan.net start to have a lot of packages with a given tag ... " rgrp 301 Package discussion pages ckan enhancement assigned 2010-04-26T16:57:13Z 2012-06-25T13:47:35Z "A package discussion page is like a wikipedia discussion page: an editable free text page for people to have discussion/post comments about a given package. It provides a way for people to make suggestions about a package without needing access to main package." rgrp 811 Extra field editing form layout breaks when there are long field names ckan ckan-backlog defect new 2010-11-17T11:53:32Z 2011-12-06T11:18:09Z The layout of the editing section for extra fields breaks when a field name is slightly too long. Field names jump over to the right. See http://ckan.net/package/edit/dbpedia for examples. cygri 812 Package edit form only allows three extra fields ckan ckan-backlog defect new 2010-11-17T11:56:50Z 2011-07-20T15:48:37Z "= Rationale = The package edit form is restricted to three extra fields. To enter more than three fields, one has to save the package and hit edit again (or hit preview). = Implementation = A mechanism similar to the one for resources (where you can add lines as you go) would solve this. So, have a button that adds more extra field rows via JS. (Extra fields don't need up/down buttons that the Resource table has) Nice to have: a blank field is added when you tab from the last filled-in field in the table." cygri 895 Add version number (or simular) to css/js includes query string ckan ckan-backlog defect new 2011-01-06T19:49:42Z 2012-06-15T15:36:33Z "Updates to css after a new deploy don't come through without a hard refresh. Adding the version number to the include urls will solve this e.g. mycssfile.css?v=12345678" memespring 140 News section on front page ckan ckan-backlog enhancement new 2009-10-07T08:02:21Z 2010-02-08T10:32:39Z "Have a news section (suggest as a sidebar item). News section will link to latest 3/4 blog posts on CKAN from blog.okfn.org. Details: * Suggest pulling via rss or similar. * Will want to cache this ... Cost: 4h?" rgrp 253 Package relationships ckan ckan-backlog enhancement assigned 2010-02-22T16:05:21Z 2012-06-15T15:38:46Z "= Overview = Functionality to formally associate packages. We see a need for specific parent-child, inheriting or dependency relations. Not only should this help navigation between packages in the web interface, but it also provides a mechanism to automatically pull dependencies when downloading a data package, in a similar manner as we see in software package management. = Examples = 1. There are 27 packages in data.gov.uk to do with the Data4NR's Health Poverty Index. There is currently no common link between these, unless you search for 'HPI' (which also brings up House Price Index), or look under tag 'health' (which also has 600 other results). There should be a link on each HPI package page to navigate to the other 'sibling' HPI packages, and to a 'root' package that has info about the set. This could be partially achieved using the existing tag or group concepts, but a more explicit/official/obvious marking of their relationship could be beneficial. 2. In ckan.net is freedict, a collection of translation dictionaries. You could make each dictionary a child package and use this system. But it would probably be better to make each dictionary a different resource in the same package. (There are other ideas to denote a resource as the data making up a 'portion' of package, or a 'whole' of the package, to help people downloading datasets in the software package style.) 3. OSM has had some Naptan data imported (bus stops), with special permission - i.e. a more liberal license. It would be useful to show this link on both OSM and Naptan packages in CKAN: OSM 'derives from' Naptan with a comment about the license change. I'm not sure this is useful to an automatic download or use of these datasets, but may aid exploration on the CKAN website and understanding the provenance of the bus stop data on it. 4. IPCC collection of data linked / mirrored. Not sure if there are useful relationships here? 5. Dracos gets postbox locations from crowd sourcing and OSM. We could say Dracos 'derives from' OSM. See more examples discussed here: http://trac.ckan.org/ticket/253 = Implementation = This is split into four tickets: * Model: ticket:254 * Read in WUI: ticket:255 * Edit in WUI: ticket:256 * API: ticket:257 No need for write access to be provided API for the moment. This ticket also encompasses ticket:169 (Package derivations) and ticket:176 (Package dependencies)." dread 277 Set some config options / settings in WUI (extension) ckan ckan-backlog enhancement zephod assigned 2010-03-22T16:21:01Z 2011-10-10T11:45:21Z "== Use case == As a ckan administrator I want to easily change options about the CKAN install. == Implementation == === Settings to be in DB === Suggested: {{{ ## Title of site (using in several places including templates and tag ckan.site_title = CKAN ## Logo image to use (replaces site_title string on front page if defined) ckan.site_logo = http://assets.okfn.org/p/ckan/img/ckan_logo_box.png ## Site tagline / description (used on front page) ckan.site_description = ## Used in creating some absolute urls (such as rss feeds, css files) and ## dump filenames ckan.site_url = ## Favicon (default is the CKAN software favicon) ckan.favicon = http://assets.okfn.org/p/ckan/img/ckan.ico ## An 'id' for the site (using, for example, when creating entries in a common search index) ## If not specified derived from the site_url # ckan.site_id = ckan.net ## API url to use (e.g. in AJAX callbacks) ## Enable if the API is at a different domain # ckan.api_url = http://www.ckan.net ## html content to be inserted just before </body> tag (e.g. google analytics code) ## NB: can use html e.g. <strong>blah</strong> ## NB: can have multiline strings just indent following lines # ckan.template_footer_end = }}} NB: these will still need to be stored somewhere for loading on initialization. do this in db init function ... === Settings / Options / KeyValues Table === Columns: * [namespace]: ? only if KeyValues (for settings this would then always be settings) * key * label * value (json) * type (e.g. date and to specify in advance what type should be) * description * tags: ?? (for grouping ...) === Loading settings from DB === Do this in ckan/config/environment.py === WUI === * /ckan-admin/settings * Show label, plus description plus text field == Depends == * Would be part of ckan-admin section and hence build on ticket:833 (Administrative dashboard) " dread 331 Timezone of CKAN timestamps should be configurable ckan ckan-backlog enhancement new 2010-05-31T10:40:40Z 2012-06-15T15:38:21Z "Revisions are timestamped using the server's clock, which may not relate to the expected timezone for the site. e.g. the Norway site has a server on GMT. No timezone info is displayed either. Would like to set timezone for a CKAN instance to use in rendering revision timestamps. For example, use CET or EST timezone." rgrp 350 Search engine optimisation ckan ckan-backlog enhancement reopened 2010-06-15T09:35:41Z 2012-06-15T15:38:10Z "Need to research what can easily be done to improve CKAN packages in the search rankings. Comments from Glen Barnes: We've been pretty successful at SEO without even really trying (see http://www.google.co.nz/search?client=safari&rls=en&q=auckland+google+transit+feed&ie=UTF-8&oe=UTF-8&redir_esc=&ei=dsYSTOzJLs2eceuZiI8I as an example). This to me is key. If we are to make data available it has to be findable which is the main reason for a catalogue. There are probably things we should be doing on CKAN like using slugged urls (http://www.ckan.net/package/ascoe -> http://www.ckan.net/package/ascoe/atmospheric-chemistry-studies-in-the-oceanic-environment), setting the H1 tag correctly (""Atmospheric Chemistry Studies in the Oceanic Environment"" on the example above). Some basic SEO 101 on page optimisations. " dread 351 Homepage: list new, updated and 'hot' packages ckan ckan-backlog enhancement new 2010-06-15T09:56:56Z 2012-06-15T15:37:57Z "Have a simpler list of exciting data, as opposed to the big revision list. For example: {{{ Hot data =========== New packages: package1, package2, package3 Updated resources: package1, package2, package3 Popular packages: }}}" dread 369 """Package Listing Key"" should appear on Tag results" ckan ckan-backlog enhancement new 2010-07-22T18:00:34Z 2012-06-15T15:37:46Z "Currently there's a nice legend titled ""Package Listing Key"" that appears in right side of ""Browse Packages"" results. The same key should show on other search results like when searching for a tag." shudson@… 370 Use better email encryption for author_email and maintainer_email ckan ckan-backlog enhancement new 2010-07-22T18:03:39Z 2012-06-15T15:37:29Z "The JavaScript email encryption used is not very reassuring. Google's MailHide is a much better solution that is easily implemented. http://www.google.com/recaptcha/mailhide/ Check on the Mailhide API where there are even some Python libraries already built." shudson@… 765 Read-only mode - API usage ckan ckan-backlog enhancement assigned 2010-10-26T11:18:17Z 2012-05-28T11:55:23Z "All writes to the API are captured and you are returned an error explaining the reason. Possible errors: * 503 temporary maintenance * 403 forbidden (if server if permanently read-only)" dread 837 CKAN integration with freebase gridworks / google refine ckan ckan-backlog enhancement new 2010-11-30T18:10:09Z 2012-06-15T15:36:45Z "Thread: http://lists.okfn.org/pipermail/ckan-discuss/2010-November/000718.html === Scenario 1 === 1. User installs Refine and CKAN extension for refine 2. On booting refine and asked to load data they can choose from any data package on CKAN.net (or any other CKAN instance) 3. They edit the dataset on Refine 4. On save (or perhaps as a separate option) they are prompted as to whether they wish to sync the dataset back to CKAN (either as a new package or as a new resource on the existing package) NB: for the dataset sync back some form of ""CKAN"" storage would be required (we already have storage.ckan.net running but a closer integration would be required) === Scenario 2 === 1. User visits a package on CKAN.net (or another CKAN instance) 2. There is a button on the page ""View and edit this dataset in Google Refine"" 3. Click button -- ask them if they have Google refine installed * Yes: instructions for loading dataset into refine * No: load dataset in hosted version of google refine (we could run this) 4. User edits dataset and hits save. As in previous scenario they are prompted to sync the dataset." rgrp 235 Resource format normalization and detection ckan ckan-v1.9 enhancement tobes assigned 2010-01-18T15:13:24Z 2012-06-25T12:33:44Z "Try to gather proper MIME information for all package resources in CKAN. This is a shared ticket with dcat-tools (https://bitbucket.org/pudo/dcat-tools), i.e. opendatasearch.org. This can then also be used by ckanrdf, the CKAN RDF conversion service. Sub-tasks: * Create a Google Spreadsheet with two Worksheets: ""MIME-Mappings"", i.e. ""CSV"" -> ""text/csv"" and ""Name mappings"", i.e. ""text/csv"" -> ""Comma-Separated Spreadsheet"". * Collect and map surface forms from all CKANs * Access this via Swiss and apply, store as a PackageResource extra field pending #826 (Resource extras). * Add heuristics for format auto-detections: * Map well-known file extensions * Recognize obvious magic (Zip, Tar) * Peek into Zipfile/Tarfiles * Define a convention for generic data types (many CKAN packages have only ""Spreadsheet"" defined, either detect specific type or set MIME to */tabular-data or similar) * See also: #816 (Autocomplete for the resource format field)" dread 250 RDF link in Atom feed ckan ckan-v1.9 enhancement icmurray assigned 2010-02-18T15:41:35Z 2012-06-25T13:37:10Z Add link to RDF representation of a package in our Atom feed. dread 2859 Fix the build ckan ckan-v1.8 defect icmurray new 2012-08-16T09:26:57Z 2012-08-16T09:26:57Z "The requires files have been removed. These were required by buildkit. We've lost the information regarding which packages were already in ubuntu repos, which conflicted and which were missing. But perhaps we can just package everything up." icmurray 2582 Do not hide notes / readme on dataset pages ckan ckan-v1.9 enhancement new 2012-06-21T20:59:00Z 2012-06-25T11:50:09Z "Current we hide most of readme and then let users reveal it. Stop doing this and if necessary add a quick link down to resources section. (Maybe also rename resources to Data and Resources ...?) Aside: believe I have mentioned this somewhere a month + ago but could not find the ticket." rgrp 2766 prevent draft datasets making it to activity stream ckan demo phase 4 enhancement seanh assigned 2012-07-26T16:06:35Z 2012-08-09T20:13:13Z "The new ckan creates datasets as part of a 3 phase process. To allow for this partially created datasets can have a state that is 'draft' or 'draft-complete'. These datasets should not be seen as active by the activity stream. If we click 'add dataset' and then complete the first phase of adding a dataset then we end up having a activity stream created. When we add a resource in the next phase (add data) again an activity stream item xxx added resource to dataset is created We do not want these adding. Essentially if a dataset has a state.startswith('draft') then we want the activity stream to ignore all actions involving it. finally when the state is changed form state.startswith('draft') to state=='active' we want a xxx has created dataset ... to be added to the stream. The best way to do this would be to branch from 2375-demo-theme-stable and get it to work there. Let me know if you need any help with this ticket or a better explanation of the problem." shevski 1136 Move to SessionExtension in vdm vdm enhancement kindly assigned 2011-05-12T14:34:15Z 2012-06-25T14:03:00Z When vdm was created there was no !SessionExtension so we use !MapperExtension for doing revisioning. Now that SessionExtension exists we should use it. We can also follow the existing SQLAlchemy recipe: <http://www.sqlalchemy.org/docs/orm/examples.html?highlight=versioning#versioned-objects> rgrp 2686 enabling datastore & data API breaks recline ckan ckan-backlog defect assigned 2012-07-17T09:06:51Z 2012-09-03T09:08:20Z "First I noticed that the gold prices dataset preview was not displaying & has data API enabled Secondly I tried enabling datastore for http://datahub.io/dataset/adur_district_spending/resource/281dffa6-ea9b-4446-be41-05dced06591f and after I saved the preview no longer worked. Unticking the datastore & data api checkbox brought it back Is this a known issue?" shevski 1198 Publisher hierarchy ckan ckan-backlog enhancement new 2011-06-23T09:16:32Z 2012-06-15T15:30:00Z "'Publisher' entities in the model. They are hierarchical. 'User-Publisher' connections with one or more roles (e.g. drafter, moderator). Authorization settings can control who can set what values in a 'published by' type field. Publishers and User-Publishers available to read in the API. Future tickets will provide: * API to write Publishers and User-Publishers * UI to edit Publishers and User-Publishers (This feature deprecates authorization groups)" dread 1466 Need to support https login for multiple instances as part of the CKAN package install ckan ckan-backlog enhancement new 2011-11-15T16:52:58Z 2012-02-06T11:51:02Z thejimmyg 1832 dataset purge API ckan ckan-backlog enhancement assigned 2012-02-24T10:02:25Z 2012-06-15T15:21:46Z "Purging datasets (deleting them fully, not just changing the state to 'deleted') is important for users testing dataset creation over the API on a test CKAN instance. Without this, they need to resort to more difficult methods such as: * cleaning and reloading the database * setting the test datasets to state 'deleted' and also appending a suffix '_00' and incrementing the number until there is no clash of names. Requested for NHSIC. == Implementation == * This could slot into the Action API. * Of course we would need to ensure the user's had been given the specific right to purge. * I suggest we log the full details of the dataset being purged." dread 2751 check translations for full demo site ckan demo phase 5 enhancement toby new 2012-07-25T11:40:43Z 2012-08-06T08:50:46Z need to check everything gets translated - sean did this before so will have info toby 2949 Reenable Data API button on the new theme ckan defect amercader new 2012-10-01T16:04:24Z 2012-10-01T16:04:24Z The checks to show or not the button need to be updated for the latest datastore version amercader 1135 Changeset model for vdm vdm enhancement kindly assigned 2011-05-12T14:19:46Z 2012-06-25T13:51:07Z "Move to Changeset model for vdm. A changeset model is like an Audit-Log model in which we just record Changesets with Change-Objects rather than have Revision-Objects for each Object that is revisioned. This change would also incorporate significant simplication of vdm." rgrp 1137 Remove need for statefulness in vdm vdm enhancement kindly assigned 2011-05-12T14:47:08Z 2012-06-25T13:46:14Z "Statefulness, especially statefulness for relation (esp m2m) is cause of most of the complexity in vdm. It is ''required'' because, atm, revision objects have FKs to continuity objects. This ticket proposes the following changes: NB: this could be limited just to case of join tables (leaving state stuff on other tables) * Remove FKs from revision to continuity (or allow for them to be nullable). * We could just limit this to m2m stuff * Delete of an object leads to: * Deletion of continuity object * Adding an entry in revision table with state set to deleted (we retain state on revision table) If this is done we will no longer need to worry about filtering on state on relationships as join table will only contain ""active"" relationships. If we do this on all tables we remove need for any state awareness in client (e.g. no need to filter tables on active state). The only disadvantage of this change is that undeletion becomes more problematic (we have to recreate some continuity objects)." rgrp 1244 Notes field carriage-returns converted to CRLF ckan enhancement assigned 2011-07-26T14:10:56Z 2012-06-20T11:20:53Z "When you edit a package in the web form, if the notes field had {{{\n}}} as the End Of Line symbol, it gets lost when you preview or save the package, and the notes field is displayed all on one line. This can be seen when editing annakarenina (as created by 'paster create-test-data'). The diff shows for example: {{{ - Some test notes + Some test notes ? + }}} but it would more clearly be shown as: {{{ - Some test notes\n + Some test notes ? ++ }}} This is a significant problem with DGU, since a lot comes in via the API. It's not clear what we should do about it. We could standardise on \n or \r\n when the form submission comes in. Do different browsers on different platforms do different things with EOLs? == Analysis == Displaying the package: the Markdown processor respects both EOLs when displaying the field, putting each line in a <p> tag. Creating the package edit form: placed into <textfield>. Browser displaying package edit form: <textfield> displays \n and \r\n as EOL. But \n\n gets compressed to one EOL. But on submission, both are returned as \r\n. Receiving the edited package: Somewhere along the line the EOL gets converted to \n\n. " dread 1322 Action API improvements ckan enhancement assigned 2011-09-08T09:39:09Z 2012-06-20T11:18:08Z "Focusing on improving Action API as the v3 API: * have an optional parameter of the data_dict called ""options"". Options would contain items that would get passed into the context. e.g. {{{{""options"" : {""ref_package_by"": ""id""}}. }}} * instead of using API version to change the way packages are referenced, use the ref_package_by. * All package_show, group_show etc. to accept an object 'name' as an alternative to object 'id'. * Action API is v3 of api, replacement for v1 & v2. Default for most urls is still v1, but if url is /api/action then default to v3. Next steps: * Add search API (package, resource, * Add Util API * Clarify JSONP still applies * Add doc strings, clarifying parameters" dread 1460 Improve extensions documentation ckan enhancement assigned 2011-11-12T16:15:23Z 2012-06-20T11:16:41Z "Current extensions documentation needs some work: http://docs.ckan.org/en/latest/plugins.html * Queue extension section may now be out of date (?) * Think about how it integrates with https://github.com/okfn/ckanext-example (especially tutorial and example extension) * Document all plugin points (auto-extract from CKAN source??)" rgrp 1689 List deleted datasets in API ckan enhancement kindly assigned 2012-01-20T12:23:34Z 2012-06-20T11:00:40Z "The admin extension allows deleted datasets to be viewed, but there is no equivalent in the API. The package_list API call filters to just 'state=active' datasets. 'state' could be a parameter on the package_list call * Easy to call and implement * But getting permissions right for the different state values may not be easy - maybe a separate logic function is better? " dread 2635 Non-destructive SOLR reindex ckan enhancement new 2012-07-09T10:23:14Z 2012-07-09T10:23:14Z "You can't run the search-index reindex on a live server because it will give us bad results for 2 to 3 hours while it runs. Can there be an option that doesn't delete the entire index at the start? Instead it could just delete any items that don't exist any more, then delete them and regenerate them one by one. So the total number of datasets doesn't change much." dread 3019 Cannot delete dataset extras ckan ckan 2.0 defect new 2012-11-14T18:44:38Z 2012-11-14T18:44:38Z Deleting extras in the web interface is broken seanh 3011 Recline fixes and updates for CKAN 2.0 ckan ckan 2.0 enhancement johnglover new 2012-11-07T11:40:55Z 2012-11-07T11:40:55Z Fixes and updates to Recline / Datapreview for CKAN 2.0. johnglover 3017 New stable branch for the demo server ckan ckan 2.0 enhancement johnglover new 2012-11-13T15:04:48Z 2012-11-13T15:04:48Z johnglover 1134 CREP0003: Description and Configuration of Harvesters ckan ckan-backlog CREP new 2011-05-11T10:14:28Z 2012-06-15T15:35:54Z "'''Proposer''': Adrià Mercader == Abstract == The new harvester interface allows to create harvesters for different sources, but right now harvesters don't have many ways to describe and configure themselves. We need a way of allowing them to: * Expose their type and other details so they can be used internally and on the UI. * Define configuration settings for particular harvester instances. == The Problem == === Harvester description === The current UI for adding and editing harvest sources is the same used in ckanext-dgu, and thus the 3 harvester types used in DGU to harvest various GEMINI realted sources are hardcoded in the form. The form will be migrated to a DGU-independent one, so we need the harvesters to provide all the necessary data. There is a current {{{get_type}}} method that returns the harvester type, but for make it compatible with the DGU forms, it returns a machine-readable string (e.g. ""CSW Server""), making it error prone. === Arbitrary configuration === In the current implementation, when the harvest process is started, ckanext-harvest looks for all the available plugins that implement the {{{IHarvester}}} interface and calls the appropiate methods for the current stage ({{{gather_stage}}},{{{fetch_stage}}},{{{import_stage}}}). At these stages, harvesters have no way of applying arbitrary configuration options, so all harvesters of the same type behave on the same way. For instance, the CKAN harvester needs a way to define the API version to use when harvesting remote instances (Right now, the version 2 is hardcoded on the code). == Specification == === Harvester description === Harvesters will need to provide the following information so the UI form can be built: * name: machine-readable name (e.g. ""waf""). This will be the value stored in the database, and the one used by ckanext-harvest to call the appropiate harvester. * title: human-readable name (e.g. ""Web Accessible Folder (WAF)""). This will appear in the form's select box. * description: a description of what the harvester does (e.g. ""A Web Accessible Folder (WAF) displaying a list of GEMINI 2.1 documents""). This will appear on the form as a guidance to the user. The way to provide it will be an {{{info}}} method that all harvesters must implement, which will return a dictionary with the previous elements: {{{ { 'name': 'csw', 'title': 'CSW Server', 'description': 'A server that implements OGC's Catalog Service for the Web (CSW) standard' } }}} === Arbitrary configuration === As different harvesters will have very different needs, we need to provide a way to persist arbitrary configuration flags for each harvest source. The more flexible way given the current architecture in my opinion would be to store the configuration options as a JSON encoded object as a property of the harvest source (There already is an unused DB field called {{{config}}} in the database) (Maybe using JsonType?). This will mean adding an extra field in the harvest source form to allow entering the configuration. This could be just a simple text field where users enter the JSON encoded object or a more clever mechanism (i.e an ""Add a configuration flag"" link that adds two new text fields for the key and value for each flag, and a mechanism to later build the JSON object). In any case, this should probably be hidden in an ""Advance options"" section. == Why do it this way == === Harvester description === The {{{info}}} method would provide a single point to get all the information related to the harvester, and future properties could be added to the dictionary returned without having to modify the interface. === Arbitrary configuration === There is an already existing {{{config}}} field in the database, so we won't need to change the model. Harvesters could access the config object at any of the stages. Of course they could provide default values in their implementations so users don't need to enter them everytime. == Implementation plan == === Deliverables === === Risks and mitigations === The highest risk on the harvesters {{{info}}} method side is that harvester implementation don't offer one of the necessary properties (namely name and title). This could fire a warning when showing the UI form or using the CLI. === Participants === Adrià Mercader to do it. === Progress === None yet." amercader 1447 disk space leakage ckan ckan-backlog defect kindly assigned 2011-11-07T11:54:03Z 2012-06-26T16:15:30Z "Periodically we see some CKAN servers fall over because they run out of disk space. We need to find out if there is a common cause and fix it. One problem in the past has been file handles running out when creating lots of tiny files in the data directory. Another problem has been several enourmous backups being created every day - pdeu on eu25." dread 1577 Can't upload file with foreign chars in filename ckan ckan-backlog defect rgrp new 2011-12-19T18:00:59Z 2012-01-02T03:06:04Z "Looks like uploading a file with foreign characters fails due to encoding reasons. {{{ URL: http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf Module weberror.errormiddleware:162 in __call__ << __traceback_supplement__ = Supplement, self, environ sr_checker = ResponseStartChecker(start_response) app_iter = self.application(environ, sr_checker) return self.make_catching_iter(app_iter, environ, sr_checker) except: >> app_iter = self.application(environ, sr_checker) Module beaker.middleware:73 in __call__ << self.cache_manager) environ[self.environ_key] = self.cache_manager return self.app(environ, start_response) >> return self.app(environ, start_response) Module beaker.middleware:152 in __call__ << headers.append(('Set-cookie', cookie)) return start_response(status, headers, exc_info) return self.wrap_app(environ, session_start_response) def _get_session(self): >> return self.wrap_app(environ, session_start_response) Module routes.middleware:130 in __call__ << environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1] response = self.app(environ, start_response) # Wrapped in try as in rare cases the attribute will be gone already >> response = self.app(environ, start_response) Module pylons.wsgiapp:125 in __call__ << controller = self.resolve(environ, start_response) response = self.dispatch(controller, environ, start_response) if 'paste.testing_variables' in environ and hasattr(response, >> response = self.dispatch(controller, environ, start_response) Module pylons.wsgiapp:324 in dispatch << if log_debug: log.debug(""Calling controller class with WSGI interface"") return controller(environ, start_response) def load_test_env(self, environ): >> return controller(environ, start_response) Module ckan.lib.base:123 in __call__ << # available in environ['pylons.routes_dict'] try: return WSGIController.__call__(self, environ, start_response) finally: model.Session.remove() >> return WSGIController.__call__(self, environ, start_response) Module pylons.controllers.core:221 in __call__ << return response(environ, self.start_response) response = self._dispatch_call() if not start_response_called: self.start_response = start_response >> response = self._dispatch_call() Module pylons.controllers.core:172 in _dispatch_call << req.environ['pylons.action_method'] = func response = self._inspect_call(func) else: if log_debug: >> response = self._inspect_call(func) Module pylons.controllers.core:107 in _inspect_call << func.__name__, args) try: result = self._perform_call(func, args) except HTTPException, httpe: if log_debug: >> result = self._perform_call(func, args) Module pylons.controllers.core:60 in _perform_call << """"""Hide the traceback for everything above this method"""""" __traceback_hide__ = 'before_and_this' return func(**args) def _inspect_call(self, func): >> return func(**args) Module ckanext.storage.controller:2 in auth_form Module ckan.lib.jsonp:26 in jsonpify << Very much modelled after pylons.decorators.jsonify . """""" data = func(*args, **kwargs) return to_jsonp(data) >> data = func(*args, **kwargs) Module ckanext.storage.controller:301 in auth_form << method = 'POST' authorize(method, bucket, label, c.userobj, self.ofs) data = self._get_form_data(label) return data >> authorize(method, bucket, label, c.userobj, self.ofs) Module ckanext.storage.controller:79 in authorize << if method != 'GET': # do not allow overwriting if ofs.exists(bucket, key): abort(409) # now check user stuff >> if ofs.exists(bucket, key): Module ofs.remote.botostore:53 in exists << if bucket is None: return False return (label is None) or (label in bucket) def claim_bucket(self, bucket): >> return (label is None) or (label in bucket) Module boto.s3.bucket:87 in __contains__ << def __contains__(self, key_name): return not (self.get_key(key_name) is None) def startElement(self, name, attrs, connection): >> return not (self.get_key(key_name) is None) Module boto.s3.bucket:144 in get_key << response = self.connection.make_request('HEAD', self.name, key_name, headers=headers, query_args=query_args) # Allow any success status (2xx) - for example this lets us # support Range gets, which return status 206: >> query_args=query_args) Module boto.s3.connection:388 in make_request << if isinstance(key, Key): key = key.name path = self.calling_format.build_path_base(bucket, key) boto.log.debug('path=%s' % path) auth_path = self.calling_format.build_auth_path(bucket, key) >> path = self.calling_format.build_path_base(bucket, key) Module boto.s3.connection:88 in build_path_base << def build_path_base(self, bucket, key=''): return '/%s' % urllib.quote(key) class SubdomainCallingFormat(_CallingFormat): >> return '/%s' % urllib.quote(key) Module urllib:1222 in quote << safe_map[c] = (c in safe) and c or ('%%%02X' % i) _safemaps[cachekey] = safe_map res = map(safe_map.__getitem__, s) return ''.join(res) >> res = map(safe_map.__getitem__, s) KeyError: u'\xed' CGI Variables AUTH_TYPE 'cookie' CONTENT_TYPE '; charset=utf-8' DOCUMENT_ROOT '/htdocs' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT '*/*' HTTP_ACCEPT_CHARSET 'ISO-8859-1,utf-8;q=0.7,*;q=0.3' HTTP_ACCEPT_ENCODING 'gzip,deflate,sdch' HTTP_ACCEPT_LANGUAGE 'en-US,en;q=0.8' HTTP_CACHE_CONTROL 'max-age=259200' HTTP_CONNECTION 'keep-alive' HTTP_COOKIE 'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt=""f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode""; auth_tkt=""f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode""; ckan_user=elenaibp; ckan_display_name=""Elena Mondo""; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; __utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10.1324298316; __utmc=119670349; __utmz=119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' HTTP_HOST 'thedatahub.org' HTTP_REFERER 'http://thedatahub.org/dataset/edit/budget-library-czeck-republic' HTTP_USER_AGENT 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7' HTTP_VIA '1.1 localhost (squid/3.0.STABLE19)' HTTP_X_FORWARDED_FOR '87.114.74.190' HTTP_X_REQUESTED_WITH 'XMLHttpRequest' PATH '/usr/local/bin:/usr/bin:/bin' PATH_INFO '/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf' PATH_TRANSLATED '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf' REMOTE_ADDR '193.34.146.142' REMOTE_PORT '55419' REMOTE_USER u'elenaibp' REMOTE_USER_DATA 'userid_type:unicode' REMOTE_USER_TOKENS [''] REQUEST_METHOD 'GET' REQUEST_URI '/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf' SCRIPT_FILENAME '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py' SCRIPT_URI 'http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf' SCRIPT_URL '/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf' SERVER_ADDR '193.34.146.146' SERVER_ADMIN '[no address given]' SERVER_NAME 'thedatahub.org' SERVER_PORT '80' SERVER_PROTOCOL 'HTTP/1.0' SERVER_SIGNATURE '<address>Apache/2.2.14 (Ubuntu) Server at thedatahub.org Port 80</address>\n' SERVER_SOFTWARE 'Apache/2.2.14 (Ubuntu)' WSGI Variables application <beaker.middleware.CacheMiddleware object at 0x7f22601c7dd0> beaker.cache <beaker.cache.CacheManager object at 0x7f22601c7b50> beaker.get_session <bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x7f22601c7a90>> beaker.session {'_accessed_time': 1324298703.071357, '_creation_time': 1324293077.4139669} mod_wsgi.application_group 'ckan.net|' mod_wsgi.callable_object 'application' mod_wsgi.listener_host '' mod_wsgi.listener_port '80' mod_wsgi.process_group 'ckan.net' mod_wsgi.reload_mechanism '1' mod_wsgi.script_reloading '1' mod_wsgi.version (2, 8) paste.cookies (<SimpleCookie: __utma='119670349.1809834699.1323782464.1324293066.1324298316.4' __utmb='119670349.3.10.1324298316' __utmc='119670349' __utmz='119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' auth_tkt='f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode' ckan_apikey='decd48b1-49ee-4250-bff4-98ccca9c02a5' ckan_display_name='Elena Mondo' ckan_user='elenaibp' hide_welcome_message='1' thedatahub_net='27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e'>, 'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt=""f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode""; auth_tkt=""f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode""; ckan_user=elenaibp; ckan_display_name=""Elena Mondo""; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; _ _utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10...)|utmcmd=(none)') paste.registry <paste.registry.Registry object at 0x7f226194df50> paste.throw_errors True pylons.action_method <bound method StorageAPIController.auth_form of <ckanext.storage.controller.StorageAPIController object at 0x7f2261dad990>> pylons.controller <ckanext.storage.controller.StorageAPIController object at 0x7f2261dad990> pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'} pylons.pylons <pylons.util.PylonsContext object at 0x7f2261daddd0> pylons.routes_dict {'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'} repoze.who.identity <repoze.who identity (hidden, dict-like) at 139785645747120> repoze.who.logger <logging.Logger instance at 0x7f225e23c098> repoze.who.plugins {'openid': <OpenIdIdentificationPlugin 139785625065680>, 'friendlyform': <FriendlyFormPlugin 139785618095248>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator object at 0x7f2260874c10>, 'auth_tkt': <AuthTktCookiePlugin 139785625065808>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f2260874c90>} routes.route <routes.route.Route object at 0x7f22601a1090> routes.url <routes.util.URLGenerator object at 0x7f2261dadf50> webob._parsed_query_vars (GET([]), '') webob.adhoc_attrs {'language': 'en-us'} wsgi process 'Multiprocess' wsgi.file_wrapper <built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f2261da9af8> wsgiorg.routing_args (<routes.util.URLGenerator object at 0x7f2261dadf50>, {'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'}) }}}" dread 1661 Wrong Routes version installed by CKAN package ckan ckan-backlog defect assigned 2012-01-16T12:47:41Z 2012-06-15T15:25:49Z "Jaakko Louhio reported that the wrong Routes version got installed during CKAN's package install. He is using Ubuntu 10.04 and he believes it install Routes 1.12.3 instead of what we use which is 1.11." dread 1041 Start Using the CKAN Wiki for Tutorial-style documentation ckan ckan-backlog enhancement thejimmyg assigned 2011-03-16T14:11:55Z 2011-08-03T11:52:47Z "For example, I will document the following: * A specific guide on using mercurial when working with our branching and merging policy * DONE - http://wiki.ckan.net/Becoming_a_CKAN_Developer * Policy document on how we use tickets, plan sprints and make releases * DONE - http://wiki.ckan.net/Becoming_a_CKAN_Developer * How to package software as .deb file for project deployment I'd love if someone else would write: * An authorisation tutorial covering the core model, the command line tools and examples of every possible way of using the system * A HOWTO guide with screenshots for adding a package " thejimmyg 1152 True support for generic CSW servers uklii ckan-backlog enhancement amercader new 2011-05-23T09:02:14Z 2011-08-15T12:37:02Z "The CSW harvesters implemented at the moment were developed with the DGU project in mind, and they assume all remote CSW servers to implement the Gemini 2 specification. Gemini 2 is the profile defined in the UK for INSPIRE complying metadata, so obviously catalogs from other countries or non-INSPIRE complying ones won't be able to be harvested. The changes needed to support generic CSW servers (i.e. those implementing the ISO 19139 profile) are: * Handling the validators (right now are hardcoded in the harvester code). This probably involves issues discussed in the CREP 3 (ticket #1134) * Changes in the model to adapt the specification to ISO 19139 * Renaming objects and classes which are now Gemini-centric List of CSW servers tested: https://spreadsheets.google.com/spreadsheet/ccc?key=0Atp3cZFjuIOAdDBVQWRINnlfN1d0b2lleHVEdjBSb2c&hl=en_US&authkey=CNu4hsEB#gid=0" amercader 1163 Improvements to Storage Extension ckan ckan-backlog enhancement rgrp new 2011-05-26T11:19:38Z 2011-07-08T14:03:28Z "Storage is now working but there are * Integrate with Resources (e.g. create a resource for each file upload and give option to associate with a package) * Should we introduce rule that files *not* associated with a Resource are periodically deleted? * Allow setting of a file name/path before upload * Allow for file overwriting/deleting etc (how should this work -- do we want to allow this sort of thing) * Integrate local file upload stuff in api/auth/* == Different Backend Issues == Local file store is rather different from 'remote' storage in various ways: * For remote you don't want to use many buckets as there are bucket limits while for local you want to. Should we there have a single path that users provide which we then partition differently for different backends. " rgrp 1168 Test system for deb packaging ckan ckan-backlog enhancement thejimmyg assigned 2011-05-26T20:33:14Z 2012-03-05T23:33:43Z "Get buildbot to: * build the deb packages * install them into a fresh virtual machine * run smoke tests on the installed ckan" dread 1257 Anti-Spam tools ckan ckan-backlog enhancement new 2011-08-02T11:12:45Z 2012-06-15T15:28:49Z "We are getting more and more spam on ckan.net and we need to improve our strategy of combating it. It is bad because google ranks who we link to (which we do want for legitimate links), and our front page contains the latest package edits, so spam is immediately very visible. Spam: * Package creation * Package edit * User creation * Group creation Systems to consider: * Automatic * captcha * bayesian scoring * Sysadmin * a tool to quickly analyse and remove packages, edits and users * Helpful users * button to click indicating spam, leading to any/all of: * add TODO item (which emails package admin and sysadmins) * quarantine the package/user so it doesn't show up on the front page and a warning is shown when you view it General thoughts: * We should learn from the wikipedia tools, as they are the experts * This is a serious problem that is only going to get worse and is a threat to the whole site." dread 1262 "Enforce ""create-user"" permission" ckan ckan-backlog enhancement new 2011-08-03T12:41:36Z 2011-12-05T13:01:52Z This does not seem to have any implications at the moment, it should lock down registration and remove all related links. pudo 1286 Remove remaining formalchemy stuff ckan ckan-backlog enhancement new 2011-08-23T16:29:56Z 2012-07-16T11:00:20Z "Stuff I've spotted: * forms/* * template/group/edit_form.html * template/package/edit_form.html This can go once new DGU form is in." rgrp 1311 Modal user register and login form ckan ckan-backlog enhancement rgrp new 2011-09-06T08:20:27Z 2011-09-06T08:20:27Z "Subticket of: #1294 Rather than having to visit a dedicated page it would be good if registration and login could be done from a modal form (separate or combined ...). == Why == * It could be used from dataset creation page in situations where user needs to be registered / logged in to create a dataset (so we could allow someone to start creating a dataset and only get them to login at the end ...) * It allows for quicker and easier logging in == Implementation == * See Friedrich's work on the datahub" rgrp 1343 [super] User related improvements (login, user pages etc) ckan ckan-backlog enhancement rgrp new 2011-09-14T16:18:18Z 2011-10-13T17:48:58Z " * Disallow account creation via openid - #1386 * Require email field - #1319 * Require email confirmation to be activated (?) * Improvements to user page (e.g. show activity and more info about user) - #1396 * Modal user login - #1311" rgrp 1352 Use logic functions instead of as_dict when indexing entities ckan ckan-backlog enhancement new 2011-09-21T14:29:57Z 2012-06-15T15:28:06Z "The current search implementation uses the output of the the as_dict method of the domain Package object to update the index https://bitbucket.org/okfn/ckan/src/56c79e3fc44c/ckan/lib/search/index.py#cl-48 It also uses package_to_api1 in the SynchronousSearch plugin: https://bitbucket.org/okfn/ckan/src/f9dfb0506594/ckan/lib/search/__init__.py#cl-93 This prevents extensions from being able to index custom properties (e.g. faceting by custom extras not included in the model). The search should use the logic function to get the package properties: {{{ get_action('package_show')(context,data_dict) }}} " amercader 1439 Action API discoverablility ckan ckan-backlog enhancement new 2011-11-01T15:39:30Z 2012-01-02T03:29:34Z "A good service API needs to be discoverable, so you are not always having to refer to the documentation html. Maybe /api/action should return a list of actions available? (Currently this returns a 404.) * It would be nice to sort these into get/create/update/delete. * #1438 Parameters for each of the actions must be discoverable too /api/action/{action_name} should also return the help text / parameters allowable. (Currently this returns 400 error)" dread 1489 Updating example theme/extension ckan ckan-backlog enhancement assigned 2011-11-24T12:32:00Z 2011-12-19T10:59:44Z "ckanext-example needs updating for CKAN 1.5: * theme changes * new forms About: 'ckanext-exampletheme' was created in Spring 2011 as an example CKAN extension that showed how to customise the look & operation of CKAN. This moved to github and renamed 'ckanext-example'. " dread 1534 Change revisions to record userid rather than username ckan ckan-backlog enhancement new 2011-12-07T17:26:30Z 2012-05-28T11:37:30Z "The use of username is problematic because username's can change. * Change all revision creation code to use user id (simplest is to change c.author field in lib/base.py (?)) * (?) Add a field ipaddr for ip address of anonymous users? (or just keep putting this in author field on Revision and then acception that those won't match when we do a look up against user table) * Change user view page to look up against user id rather than name * Perform migration on existing Revision objects * Match should probably be against both openid and username when searching Revisions' author field (especially true on CKAN where some people have already changed their username from being their openid)" rgrp 1542 Buttons to purge spam datasets and groups ckan ckan-backlog enhancement new 2011-12-08T17:22:10Z 2012-06-15T15:26:40Z "A sysadmin should be able to easily examine a suspect group or package, determine if it was created by a spammer (as opposed to being a legitimate object that has been graffitied by a spammer) and purge it. The existing two-stage revision delete is currently unreliable and perhaps too laborious. Olav and Richard have needs along this line." dread 1584 QA report improvements - 2.5d ckan ckan-backlog enhancement johnglover new 2011-12-21T09:23:53Z 2012-06-06T11:26:15Z "Super: #1594 * qa/{username} * qa/{groupname} * paginate QA results * search / filter QA results * ~~list organisation report by default, but can disable via config option~~ (done) * UX tidy up of report pages - hide border if no sidebar, etc" johnglover 1588 QA - Give SPARQL endpoints a 4 star rating ckan ckan-backlog enhancement johnglover new 2011-12-21T15:13:25Z 2012-01-02T03:32:58Z "Super: #1594 From Richard Cyganiak on the CKAN Discuss list: Besides considering the media type of resources, it would also make sense to check for the presence of a SPARQL endpoint. SPARQL endpoints are recorded for more than 300 datasets on the Data Hub using the pseudo-type ""api/sparql"". A few more are recorded with the format ""SPARQL"". I suggest that datasets with such resources should also be considered for the fourth star." johnglover 1589 QA - Give 5 star rating to datasets with link metadata ckan ckan-backlog enhancement johnglover new 2011-12-21T15:16:40Z 2012-01-02T03:31:35Z "Super: #1594 From Richard Cyganiak on the CKAN Discuss list: Regarding the fifth star (is the dataset linked to others?). This cannot be automatically determined just by looking at the format. It either requires inspection of the actual data, or information about links in the metadata. As you're probably aware, we've established conventions for recording information on data links in CKAN [1], as part of the work of the lodcloud group on the Data Hub. Link information is captured for hundreds of datasets. I would claim that we have the majority of four-star datasets covered there, and hence you can determine if they should get the fifth star by checking for the presence of a links:xxx field." johnglover 1827 'Register' link should be hidden if you not allowed to register ckan ckan-backlog enhancement new 2012-02-22T15:35:39Z 2012-06-15T15:22:10Z "{{{ I have just deny visitors the create-user permission: sudo -u ckanstd paster --plugin=ckan roles deny reader create-user -c /etc/ckan/std/std.ini sudo -u ckanstd paster --plugin=ckan roles deny anon_editor create-user -c /etc/ckan/std/std.ini and after restarting, the register link is *not* hidden, but now when you access the register page, it shows you this message ""Unauthorized to create a user"" (when not logged in). But anyway that is an improvement. }}}" dread 2268 UKLP Use same schema as form for harvested records uklii ckan-backlog enhancement thejimmyg new 2012-03-30T11:56:46Z 2012-03-30T11:56:46Z Datasets created via the harvesting (UKLP ones) and the ones created via the form use a different schema. That could lead to problems. We need to discuss the best approach. amercader 1366 Search inside extra fields ckan ckan-future defect assigned 2011-09-29T10:09:52Z 2012-05-28T12:05:07Z "SOLR search doesn't support searching for part of an extra field, but it does for other fields. i.e. title=""One Two Three"" matches q=one AND q=title:one and geographic_coverage=""England Scotland"" matches q=England BUT NOT q=geographic_coverage:England This problem emerged when we went to SOLR in #1275 (CKAN 1.5a). Tests were skipped. This is could be a problem for DGU and maybe elsewhere. " dread 989 Extending the model from plugins ckan ckan-future enhancement kindly new 2011-02-14T13:22:04Z 2012-06-18T15:45:11Z "We need to support extending the model from plugins. This could involve: * Adding a plugin hook to extend the mapper * Adding an upgrade hook for plugin schema migrations * Documenting how this is to be done * Find a way to avoid conflicts " pudo 1235 [super] Search Improvements ckan ckan-future enhancement new 2011-07-20T17:24:01Z 2011-07-20T17:24:01Z "Child tickets: * #234 UI Review - Autocomplete package names & tags in search * #193 Searching by time-related field * #191 Searching by modification date * #905 Unable to search with accented characters in package names * #906 Ability to search without accents for accented words * #924 Search box has no search button Broadly speaking though we need to choose PostgreSQL, Solr or something else. We don't want to invest our time maintaining two search backends with a limited abstraction layer between the two." thejimmyg 1438 Action API - parameter discovery/checking ckan ckan-future enhancement new 2011-11-01T15:38:10Z 2012-05-28T11:37:30Z "Many actions in the Action API require parameters. What params are needed should be listed and checked. Because currently, if you get them wrong you simply get a useless 500 error. Currently they are listed in the docs, extracted from the code manually. So you could GET /action/api/package_list to receive not only the help text, but a list of arguments. And if you send an extra or missing argument then an intelligent error message can be returned. = implementation = How about some sort of decorator on the action function: {{{ @logic_params(id, offset, limit) def get_package_list(context, data_dict): ... }}} This would do the param checking, and is there a way to extract these params from the function? Or do a registration of the logic function? I'd certainly like to keep the list of the list of params for the function with the function, for ease of reading the code. Another good thing would be to pass in the params named as themselves, rather than having them contained in the data_dict." dread 1818 Spatial metadata editor ckan ckan-future enhancement amercader new 2012-02-20T18:52:21Z 2012-05-28T11:06:35Z "Estimate 2.5d Right now users need to manually enter a geojson value in the 'spatial' extra field in order to define a geometry for a dataset. We need a map widget on the edit form that allows users to draw the geometry on top of a map. This will be based on the draw functions offered by OpenLayers [1], [2]. Things to take into account: * Integration with the form sections. * Allow some degree of configuration (default extent, limit geometry types...) [1] http://openlayers.org/dev/examples/draw-feature.html [2] http://openlayers.org/dev/examples/modify-feature.html " amercader 2331 Search should AND terms not OR terms ckan ckan-sprint-2012-05-29 defect kindly reopened 2012-04-28T18:24:45Z 2012-12-25T22:25:44Z "Appears current default search in CKAN ORs terms rather than ANDing them (i.e. adding more terms increasing number of items found rather than reducing it). Not sure when this crept in or if it has been there for a long time." rgrp 2412 More than one resource invalidatiing breaks dataset edit form ckan ckan-sprint-2012-06-25 defect zephod assigned 2012-05-22T20:05:48Z 2012-05-28T11:07:14Z "When attempting to add more than one resource at once, if more than one resource invalidates, this results in a js error, leaving the form in an inconsistent state. Repro: 1. Go to /dataset/new 2. Add a new resource. Fill in one of the fields with an invalid value. eg - last_modified, or size... 3. Add another resource, doing the same thing: make one of the fields invalid. 4. Try to save the dataset. 5. The entered resource information will be lost, and a js error ""Uncaught Error: Can't add the same model to a set twice,: backbone.js:586"" will be thrown." icmurray 2963 Timeout on tag pages with lots of datasets ckan ckan-v1.8 defect new 2012-10-15T10:13:19Z 2012-10-15T10:13:19Z "e.g. http://thedatahub.org/tag/lod Tags with less datasets work fine (e.g. http://thedatahub.org/tag/railways)" amercader 2279 Write tutorial for our DataStore API / Data Explorer ckan ckan-v1.8 enhancement rgrp new 2012-04-03T19:44:23Z 2012-05-28T11:43:46Z "Unfortunately ES is a little bit complex / opaque to newcomers. Need to write a simple tutorial walking through this. This is documentation / example only (no code!)" rgrp 2363 Documentation of best caching practice. ckan ckan-v1.8 enhancement toby new 2012-05-01T16:16:57Z 2012-06-19T09:56:34Z Need better documentation on best practices in making page loads faster for non logged in users. kindly 2887 """Welcome to CKAN!"" on front page is untranslatable" ckan ckan-v1.8.1 defect new 2012-08-22T09:17:59Z 2012-08-22T09:20:33Z "ckan/templates/home/index.html contains: <h1 class=""page_heading"">Welcome to ${g.site_title}!</h1> which means that just ""Welcome to"" ends up in the ckan.po files for translation. This makes the phrase untranslatable in languages that have a different word ordering than English where the site title needs to be somewhere other than at the end of the sentence." seanh 2405 Improvements to data viewer embed ckan ckan-v1.9 defect icmurray assigned 2012-05-21T06:59:25Z 2012-06-25T14:53:01Z " * Decent powered by icon for the Embed for the DataHub? (make this text customizable and allow html ...) * Current icon and text looks really bad * Embed link uses uuid rather than dataset name (this goes to perma-url question ...) * Data viewer embed url includes all the resource info (not needed -- just url and backend info ...) * Embed extension urls do not respect original domain but instead use site_url (I'm guessing). E.g. datahub.io domain ends up with links to embed on thedatahub.org " rgrp 2745 Password reset returns an exception if the key parameter is missing ckan ckan-v1.9 defect new 2012-07-24T16:05:31Z 2012-07-24T16:05:31Z "Instead of showing a notice, the password reset page throws an exception if the key parameter is missing: {{{ Module ckan.controllers.user:329 in perform_reset c.reset_key = request.params.get('key') if not mailer.verify_reset_link(user_obj, c.reset_key): h.flash_error(_('Invalid reset key. Please try again.')) abort(403) if not mailer.verify_reset_link(user_obj, c.reset_key): Module ckan.lib.mailer:100 in verify_reset_link if not user.reset_key or len(user.reset_key) < 5: return False return key.strip() == user.reset_key return key.strip() == user.reset_key AttributeError: 'NoneType' object has no attribute 'strip' }}} Apart from the obvious fix of checking for the 'key' parameter, it seems like is quite common to get these reset urls without the key parameter, so I suspect some email clients might strip the query params when building the links. We could avoid this problem by making the key part of the url instead of a param: http://thedatahub.org/en/user/reset/3086e91c-fe09-4a98-92e1-19de67a9ac9d/b4c2d03fa8 instead of: http://thedatahub.org/en/user/reset/3086e91c-fe09-4a98-92e1-19de67a9ac9d?key=b4c2d03fa8 " amercader 2430 Look into marking tickets as dependent on other tickets in trac ckan ckan-v1.9 enhancement seanh accepted 2012-05-25T11:58:06Z 2012-07-02T13:05:46Z seanh 2431 Trac git integration ckan ckan-v1.9 enhancement seanh accepted 2012-05-25T11:58:48Z 2012-07-02T13:06:02Z seanh 2733 Datastore logic functions ckan ckan-v1.9 enhancement johnglover new 2012-07-23T15:54:46Z 2012-07-30T13:48:25Z "== Where does the data go? == In a postgres database configured by the ckan.datastore_write_url config option which is a sqlalchemy url. The user should have rights to create tables. == Whats the api like? == We will just implement it as logic functions like the rest of CKAN and will part of core. After that we may add some nicer api functions that use these but that is a secondary concern. == What are the initial logic functions? == * datastore_create * datastore_delete * datastore_show == What is the JSON input format for datastore_create == To begin with it can have the following keys. It is fairly consistent with Max Ogdens' gut servers. Except adds resource_id. {{{ { resource_id: resource_id # the data is going to be stored against. fields: a list of dictionaries of fields/columns and their extra metadata. records: a list of dictionaries of the data eg [{""dob"": ""2005"", ""some_stuff"": ['a', b']}, ..] } }}} * The first row will be used to guess types not in the fields and the guessed types will be added to the headers permanently. Consecutive rows have to conform to the field definitions. * rows: can be empty so that you can just set the fields * fields are optional but needed if you want to do type hinting or add extra information for certain columns or to explicitly define ordering. eg: [{""id"": ""dob"", ""type"": ""timestamp"" }, {""id"": ""some_stuff"", ""type"": ""text""}, ...]. A header items values can not be changed after it has been defined nor can the ordering of them be changed. They can be extended though. * Any error results in total failure!! For now pass back the actual error. * Should be transactional == What json does datastore_delete take? == {{{ { resource_id: resource_id # the data is going to be deleted. filters: dictionary of matching conditions to delete e.g {'key1': 'a. 'key2': 'b'} this will be equivalent to ""delete from table where key1 = 'a' and key2 = 'b' "". No filters (either not present or not defined) then delete the table. If we want truncate then add truncate: true to truncate the table. } }}} == What json does datastore_search take? == {{{ { resource_id: resource_id # the data is going to be selected. filters : dictionary of matching conditions to select e.g {'key1': 'a. 'key2': 'b'} this will be equivalent to ""select * from table where key1 = 'a' and key2 = 'b' "" q: full text query limit: limit the amount of rows to size default 100 offset: offset the amount of rows fields: list of fields return in that order, defaults (empty or not present) to all fields in fields order. sort: comma separated field names with ordering e.g ""fieldname1, fieldname2 desc"" } }}} Some free code: https://gist.github.com/3163864 == What json does datastore_search return? == {{{ { fields: same type as datastore_create accepts (i.e. with metadata) offset: The same offset that was supplied in datastore_show limit: The original limit filters: The filters that were applied in data_show total: # total matching records without size or offset records: [same as data_create] # list of matching results } }}} On error will return: {{{ { __error__: … sql error … } }}} == What types are allowed? == Aim to support as many postgres/postgis types that have string representations. http://www.postgresql.org/docs/9.1/static/datatype.html http://www.postgresql.org/docs/9.1/static/sql-createdomain.html == IDs == Each row in a table will be given an _id column which has an id generated by us which you can use in queries. == Other Features == Each row will store the _full_text index of all the data in the row. At some later point there will most likely be a way to index fields add constraints etc." johnglover 2466 Fix spam handling on trac ckan ckan-v1.9 task seanh accepted 2012-05-28T13:51:43Z 2012-07-02T13:08:59Z seanh 2818 Improve related item schema ckan demo phase 4 defect seanh assigned 2012-08-09T09:22:56Z 2012-08-29T09:15:18Z "Problem: When creating a related item (e.g. a Visualisation), if you don't put in a URL it succeeds, but on the related items and apps pages it renders it as a link to the same page. Expected: Always require a URL and it should only submit if one is added" danieljohnlewis 2790 logic.action.user_show is slow ckan demo phase 4 enhancement kindly new 2012-08-01T20:21:23Z 2012-08-15T09:42:14Z "This is a very slow call it would benefit from the sort of speed-ups that package-search received for me locally this is taking 6 seconds for rufus using the datahub data I have. I think a lot of this is the dataset retrival/dictization can we just grab json blobs from solr? also is it possible to specify a sort order/paging? I've put this as a demo-theme ticket as it is an big issue on the demo we are at 25 second page loads - which i can get down to about 8.8 secs so this is the main pain point now" toby 2357 Create build script for front-end resources ckan demo phase 5 enhancement aron.carroll accepted 2012-05-01T15:29:41Z 2012-07-25T12:48:44Z Should minify and concatenate JavaScript and CSS as part of #2354 aron.carroll 2641 Adapt spatial widgets to new theme ckan demo phase 5 enhancement johnmartin assigned 2012-07-09T15:02:27Z 2012-11-11T18:34:14Z Dataset extent map and spatial filter need to adapted to the new theme, as they are not showing up now amercader 1328 Unicode & paster commands ckan defect assigned 2011-09-12T10:25:10Z 2012-06-20T11:17:45Z "A possible bug in CKAN when I tried deleting users using ""paster --plugin=ckan user delete"" command. To reproduce the bug do the following: 1. Create a user with an ID (which in my case was a user's full name) that contains non-unicode caracters like Norwegian ""æ"", ""ø"", or ""å"". 2. Make sure that you can see something like the example below: (pyenv) rm@mycomputer:$ paster --plugin=ckan user Users: name=Rustæm 3. Then try deleting the user with following command: (pyenv) rm@mycomputer:$ paster --plugin=ckan user delete ""Rustæm"" You should now get a python encoding error. I know that this is quite rare case, but in our case it caused some trouble. Could you guys have a look at this bug? CKAN ver. 1.3.3." minspamboks@… 1509 Mis-dated old revisions of datasets ckan defect assigned 2011-12-05T13:06:38Z 2012-06-20T11:13:34Z "e.g. http://thedatahub.org/dataset/osm%402011-07-12T12%3A16%3A47.590358 gives: {{{ This is the current revision of this dataset, as edited at 2011-07-12 12:16. }}} but it should say the date 2011-07-12 (which is the data being displayed). The problem is looking at PackageRevision, when it might be a tag or extra." dread 2719 Feeds controller does not catch NotAuthorized exception ckan defect new 2012-07-21T12:14:23Z 2012-07-21T12:14:23Z Results in bad user experience and WebApp errors emailed out. Seen in 1.7.1 dread 2673 simplify set of options for resources ckan enhancement new 2012-07-14T21:15:59Z 2012-07-14T21:15:59Z Far too many resource options. Lets restrict back to data file and API. Visualizations etc can either get linked in description or in the Related items. rgrp 2947 Redirect to the resource page from /resource/{res-id} ckan enhancement new 2012-10-01T13:15:25Z 2012-10-01T13:15:25Z "Use story: As a user of the datasore I want to be able to put the resource id without the dataset name in a url and see the resource page. Proposed redirect: /resource/{res-id} -> dataset/foo-bar/resource/{res-id}" dominik 3026 Support icons on nav_named_link ckan enhancement amercader new 2012-11-30T18:51:56Z 2012-11-30T18:51:56Z nav_named_link won't include the nice icon even when passing the icon keyword param. amercader 2959 Changing a Group's name through the action api disassociates it from its datasets in the index ckan ckan 2.0 defect icmurray new 2012-10-04T15:18:55Z 2012-10-04T15:18:55Z "Repro: - Create a new Group, named ""test-group"". - Add a dataset to it. - Verify the dataset belongs to the group by visiting the Group's read-page - Update the Group through the action api (group_update), using the uid in the ""id"" field, and a new name in the ""name"" field. - Visit the group's read-page. The list of datasets will be empty. This was an issue when editing a Group through the web interface, which was fixed in [1]. However it only fixes the issue in the group controller. [1] https://github.com/okfn/ckan/commit/dbe25d8b8d7fabfc40c5d794a920b91cec349335" icmurray 1171 Citation instructions on dataset and resource view pages ckan ckan 2.0 enhancement mark.wainwright assigned 2011-06-01T09:33:19Z 2012-09-11T10:18:25Z "Some sort of citation helper. Something small on the dataset and resource page that would show how to cite. wwaites: Some related thoughts on this from opb: http://homepages.inf.ed.ac.uk/opb/papers/ssdbm2006.pdf timclicks: I'm looking at Dataverse for the first time[0]. It seems very popular in the social sciences. I noticed that there is a recommended citation for each dataset. For example, [1] is has this one: ""Targeted Input Programme (TIP) 2000-01"", http://hdl.handle.net/1902.1/SSC-MWI-TIP2000-01-M1 V1 [Version]"" == Implementation == Add a small box at bottom of dataset / resource page (or in sidebar on dataset page) with title ""Cite this"" with contents like: %title. %author. Retrieved %date. %site_title. For resource: %title = %dataset_title. %resource_name. Could also add export to ref managers (e.g. to bibtex) but that is for later." dread 2831 Create a limited subset of markdown that's supported ckan ckan 2.0 enhancement new 2012-08-09T11:41:07Z 2012-08-09T11:41:07Z "Allowing people to use the full range of markdown results in extremely messy output across the site. I'd suggest limiting support to only a subset of common use cases. * Allow all inline elements, this allows bold, italic, code and links. * Allow lists. Disallow * Horizontal Rules * Headings * Block quote and code (this may turn out to be useful and so could be included) This way you get markdowns paragraph handling and a few inline styles without breaking the entire layout of the page. Here's the full syntax if anyone is interested http://daringfireball.net/projects/markdown/syntax" aron.carroll 2951 Paster command for building css from less ckan ckan 2.0 enhancement new 2012-10-01T19:12:20Z 2012-10-02T10:50:30Z "With the latest template, css and js changes in 2.0, there are a number of things that need preparation prior to a production deployment. One of these is: - the less files should be compiled to css (main.css, not just main.debug.css) This ticket is to provide a paster command which will compile the .less into the main.css file. The idea of the paster command is that it will be run by developers, and they will check in the resulting .css files. It will also provide a convenient mechanism for production scenarios with node.js installed on the production server, as they will be able to compile the .less there as well. ## Background The .css files that need to be served are built using `less`. The css files that are generated *are* checked-in to the repo; but they are a build artifact. So the general workflow is: - commit changes to the less files - build the main.css file and commit This paster command slots into the above workflow for convenience. ## Notes - doc/frontend-development.rst - bin/less to run this paster command, `Node` will be required. So that dependency should be checked. ## The paster command This is the proposed behaviour of the paster command: - ensure `custom.less` does not exist - for each colour in{fuchsia,green,maroon,red}: - generate a `custom.less` file for $colour. There's a paster command that does this already: `paster color <color name>. - generate the `css` from the `less` files. ie - the equivelant of running `bin/less --production`. - this will generate a `main.css` file, which should be renamed to `$color.css`. - ensure `custom.less` does not exist - generate the `css` from the `less` files. This will create a `main.css` with the default colour scheme. - call the paster command that minifies css and js files. (This command does not exist yet, see #2950)" icmurray 2952 incorporate javascriopt translations into translations workflow ckan ckan 2.0 enhancement seanh new 2012-10-01T19:15:45Z 2012-10-01T19:15:45Z There are now javascript translations. There's a paster command, `trans js`. This is a ticket to ensure that using `trans js` is documented in the normal ckan translation workflow (which is followed after a feature freeze, prior to release). And to see if there's any need for further incorporation into the current translation process. icmurray 2996 Fix behaviour of View Profile button in user popover when already on user's profile ckan ckan 2.0 enhancement new 2012-10-17T14:30:38Z 2012-10-17T14:30:38Z "The user popovers that appear when you hover the mouse over a user in an activity stream include a View Profile button that takes you to that user's profile page. If you are already on that user's profile page then it simply reloads the same page, but it sends you to the Datasets tab if you were on another tab. Possible solutions: Don't show button if already on user's page? Button just makes popover disappear, if already on user's page? Button does reload the page, but reloads the same tab (datasets, followers, activity stream) that you were on. Marking this low priority" seanh 1062 Data preview encoding error ckan ckan-backlog defect johnglover assigned 2011-03-29T10:55:43Z 2011-07-27T13:35:31Z "The preview of ""Species Misc Turtle Download"" at http://ckan.net/package/taxonconcept results in the following error: Unable to Preview - Had an error from dataproxy: Data Transformation Error (Data transformation failed. Reason: 'utf8' codec can't decode byte 0x8b in position 1: unexpected code byte " sebbacon 1288 Package edit/creation can't include 'relationships' field ckan ckan-backlog defect new 2011-08-24T16:34:55Z 2012-06-15T15:28:18Z "When you create or edit a package (via the API), you aren't able to specify the relationships it has. (If you do you get 409 {{{ {""__junk"": [""The input field __junk was not expected.""]} }}}) The normal way to create relationships is via /api/rest/relationships/ and this works. But when you GET a package, the dictionary lists all relationship details. So this bug creates a problem for editing a package that has relationships - you want to GET it, make any edits and then PUT it back. The work-around is to delete the 'relationships' key from the dict before you PUT it back. == Options == Ideally, CKAN would read the 'relationships' key and make the necessary changes. This is a chunk of work. Another good option is to allow an unchanged 'relationships' value, but barf it is edited. This is also a chunk of work. A bad option would be to just ignore the 'relationships' value, since users will get frustrated changing this value and wonder why it never saves, not understanding it is different to all the rest, without error message. A final option is to get rid of relationships altogether." dread 1336 License fudge ckan ckan-backlog defect johnglover new 2011-09-13T11:07:37Z 2011-12-06T12:04:33Z "cset:4b59ab34137d ckan/logic/action/get.py: {{{ - isopen = model.Package.get_license_register()[license_id].isopen() - result_dict['isopen'] = isopen + try: + isopen = model.Package.get_license_register()[license_id].isopen() + result_dict['isopen'] = isopen + except KeyError: + # TODO: create a log message this error? + result_dict['isopen'] = False }}} This change hides problems with the license server and returns potentially incorrect values for openness. This has been noted as 'temporary fix' but seems to be forgotten about, since it has been merged to default and gone into release 1.4.3. I suggest the licenses are cached (I thought this was already the case when CKAN first requests them after start-up?). I suggest failure would return 503." dread 1382 Deleted resources are present for harvested package uklii ckan-backlog defect thejimmyg new 2011-10-11T12:58:09Z 2011-10-11T12:58:09Z Perhaps the importer deletes them before re-importing. We shouldn't have deleted resources, so let's investigate. thejimmyg 2395 paster db clean/init don't work when spatial extension enabled ckan ckan-backlog defect new 2012-05-16T09:16:33Z 2012-06-15T14:41:53Z "If you have a spatial enabled database then if you don't disable the spatial extension in the CKAN config temporarily then you get errors when you run paster db clean and paster db init. Can't you just modify the clean and init functions to run without extensions enabled? The wider problem is that extensions do their own inits every time you do load_environment, which seems crazy and inefficient to me, since this occurs every time a request comes into CKAN. But that is another problem/ticket." dread 924 Search box has no search button ckan ckan-backlog enhancement new 2011-01-24T11:12:13Z 2011-12-06T11:20:36Z "The search box at the top-right of CKAN's page doesn't have a 'go' button. I feel that a larger percentage of users expect a 'go' or 'search' button on the right-hand side of the box to press to start searching. Techies tend to know the keyboard shortcut of pressing 'carriage-return' but it might be better to follow standard practise on this. Examples with 'search' button: Internet Explorer, Firefox, Google, Amazon, trac Examples without: ?" dread 1259 """Add a row"" for Extras on Package form" ckan ckan-backlog enhancement johnglover new 2011-08-02T16:31:33Z 2011-08-09T16:24:16Z "The default package form offers 4 empty extras fields. Like the resource section, it should have an ""add more"" button to add another row. " pudo 1314 ckanclient search - generator improvements ckan ckan-backlog enhancement assigned 2011-09-07T11:36:50Z 2012-06-20T11:20:33Z "Apparently the search generator always makes two requests, even if you don't want to see the search results, which might be slow. Can this be optimised? Maybe we should also provide a second search function that doesn't use the generator - the original simple search function (that leaves the user to deal with limit & offset)." dread 1326 Write a set of auth plugin functions to integrate with Druapl ckan ckan-backlog enhancement new 2011-09-12T09:51:24Z 2011-09-12T09:51:24Z Ticket #787 described join auth between CKAN and Drupal. The authentication part is live and implemented. This ticket is a placeholder for work that will be needed in the new auth system to link authorization functions to Drupal. It is dependent on the groups refactor. thejimmyg 1393 Don't skip search tests ckan ckan-backlog enhancement johnglover assigned 2011-10-13T11:30:53Z 2011-11-01T13:19:50Z Now we don't use postgres search, all the tests involving search now don't need to be skipped when running on sqlite. Should help coders spot earlier if these tests break. dread 1424 Openness notice should be clearer ckan ckan-backlog enhancement new 2011-10-26T16:54:49Z 2011-10-26T16:54:49Z "ckan-discuss discussion suggests changes to the 'openness' indicator ( http://lists.okfn.org/pipermail/ckan-discuss/2011-October/001786.html ) Dataset view page: * If there is an explicit but non-OKD compliant license, such as CC-BY-NC, then this should be stated explicitly, perhaps: “This dataset is Not Open. License: Creative Commons Attribution Noncommerical. This is not an open license as it does not meet the Open Knowledge Definition.” * If the license is marked as “Other::License Not Specified”, then this should be stated explicitly, perhaps: “This dataset is Not Open. It is published without an explicit license, the publisher reserves all rights to the dataset.” * 3. If the license field was left empty by the contributor of the Data Hub record, then again this should be stated explicitly, perhaps: “This dataset is Not Open. The license of this dataset is unknown or unspecified. Start an enquiry on IsItOpenData » * There is a bug so that non-open licenses doesn't have an openness notice. * If downloadable resources are not available, this should not affect 'openness' - check this has been removed." dread 2247 Resource preview glitch in some browsers ckan ckan-backlog enhancement new 2012-03-20T12:30:14Z 2012-03-20T12:30:14Z "From Ira: Preview for google spreadsheets are not displaying correctly for me in Firefox v.10.0.02, fine in Chrome. http://i.imgur.com/KJaqz.png" zephod 2310 Refactor the search-query construction in feeds.py ckan ckan-backlog enhancement icmurray new 2012-04-16T16:01:31Z 2012-05-16T09:09:46Z "The feeds controller, used to construct atom feeds, duplicates code found in the package controller's search action (in order to construct the custom feed). Refactor this to remove duplication." icmurray 2277 Use the new atom feeds in IATI ckan ckan-backlog refactor icmurray new 2012-04-02T17:10:28Z 2012-04-02T17:10:28Z Atom feeds have been implemented in core in #1593. The IATI-specific implementation can now be removed when IATI move to CKAN >= 1.6.1 icmurray 1544 delete old git branches ckan ckan-backlog task new 2011-12-12T15:10:10Z 2011-12-12T15:10:10Z "We have 150 odd branches ({{{git branch -a}}}) - most of them old - we should prune them. At very least, branches that have been merged in should be deleted. Look at old branches that haven't been merged in and wonder why. May be of some use: {{{ git branch --merged git branch --no-merged }}}" dread 948 Highlight (to a sysadmin) which packages are deleted ckan ckan-future enhancement assigned 2011-02-02T11:32:49Z 2012-05-28T11:58:00Z "When a customer logs in as a sysadmin then he/she see all packages, including deleted and pending ones. These are hidden to the average user, but the sysadmin has no idea of this until he clicks on the package and sees at the bottom 'state: deleted'. It should be more obvious than that on the search view - an icon, message or crossed-out name to packages are deleted." dread 1285 Errors cause emails ckan ckan-future enhancement assigned 2011-08-23T16:29:04Z 2012-05-28T11:55:51Z "Currently a sysadmin gets an email when an exception is not caught. But there are occasions when we DO want to catch an exception so we can fail nicely for the user, but the sysadmin STILL gets an email to know to fix something. e.g. if there is an exception when search indexes a package. You want to catch the exception and still run any other notify calls. " dread 1581 Blog post about Google Analytics extension for CKAN ckan ckan-future enhancement mark.wainwright@… new 2011-12-20T17:40:00Z 2012-01-02T03:17:54Z "The CKAN Google Analytics extension has been updated to work with the latest version of CKAN, could make for a nice blog post. Can ping John Glover in January for any details required. Key link is: http://thedatahub.org/analytics/dataset/top though this should probably move to be under stats (e.g. http://thedatahub.org/stats/usage)" johnglover 2301 Replace old-style string formatting with format() method (at least in strings marked for translation) ckan ckan-future enhancement seanh new 2012-04-16T12:27:44Z 2012-06-25T14:50:12Z "Some strings in CKAN are passed to gettext with more than one %s in them, this is no good as translators may need to change the order of substituted words. Strings should not use the old style % substitution, they should use the new format method in Python 2.6+, e.g.: 'This {food} is {adjective}.'.format(food='spam', adjective='absolutely horrible') http://docs.python.org/release/2.6.8/tutorial/inputoutput.html#fancier-output-formatting We should probably only change strings that are marked for translation and that contain multiple %s, don't want to make too much work or cause too many strings to change and need to be re-translated." seanh 2326 dataset permalinks ckan ckan-future enhancement toby new 2012-04-25T16:39:06Z 2012-05-28T13:46:39Z "create permalinks eg \dataset\~a5-DQ\nice-name ~XXXXX defines the permalink the nice-name is ignored * packages table needs a new field for this data * existing datasets need updating * new datasets need the identifier creating * routing needs updating * needs to work for all urls like edit Once this is implemented we can decide when we will use permalinks vs the normal urls." toby 2348 Java client library for CKAN ckan ckan-future enhancement assigned 2012-05-01T13:46:20Z 2012-09-03T11:00:24Z "Start a Java library for interacting with the CKAN Action API. Currently requires support for Groups, Tags, Resources, and Search. An example app would be very useful." ross 2433 API uses name not id for some version 3 calls ckan ckan-v1.8 defect toby new 2012-05-25T14:54:08Z 2012-06-19T09:57:00Z see https://github.com/okfn/ckan/pull/5 toby 3002 API v1/2 'legacy' search parameters must be escaped before they are put into a Solr query string ckan ckan-v1.8 defect new 2012-10-19T09:32:22Z 2012-10-19T09:32:22Z "Just to track @tauberer patch on Github. Would be nice to write a test for it. Probably going to 1.8.1" amercader 2294 Unauthorized action on Data API gets 302 rather than 401 or similar ckan ckan-v1.9 defect kindly assigned 2012-04-14T19:35:04Z 2012-06-25T11:33:23Z "Well known issue that 401 *in API* end up with 302 redirect to login page (which makes no sense and is very confusing for clients). Fix this by ensuring that error messsages on API get passed through correctly. Minor because affect is annoying but not terminal" rgrp 2203 Data Viewer support for PDF ckan ckan-v1.9 enhancement kindly assigned 2012-03-04T11:24:53Z 2012-09-25T10:29:21Z " * Boot viewer for format/mimetype: application/pdf, application/x-pdf, application/acrobat, applications/vnd.pdf, text/pdf, text/x-pdf * propose we use pdf.js - https://github.com/mozilla/pdf.js * We could also try iframing for browsers with native support but this seems more complex" rgrp 2243 Fix ckanext-example ckan ckan-v1.9 enhancement seanh reopened 2012-03-19T15:58:30Z 2012-06-25T14:49:28Z seanh 2298 Add sort-by controls to the search results page ckan ckan-v1.9 enhancement icmurray new 2012-04-16T09:39:22Z 2012-06-25T14:52:27Z "Sort-by functionality was exposed through the package controller in [1]. But no controls were added to the search-page. * What should the sortBy controls/widget look like? * Which fields should be exposed? The above commit is in the release-1.6.1 branch, so this work is dependant upon release-1.6.1 from being merged into master. [1] https://github.com/okfn/ckan/commit/8685c6000d1cb211928b4dbc63990fb72d884f8c" icmurray 2341 Fix Jenkins issue when testing branches that contain model changes ckan ckan-v1.9 enhancement seanh accepted 2012-05-01T12:46:36Z 2012-06-25T15:51:12Z seanh 2342 Fix Jenkins issue when testing branches with different solr schema versions ckan ckan-v1.9 enhancement seanh accepted 2012-05-01T12:47:25Z 2012-06-25T15:51:34Z seanh 2350 Hooks in package controller for validating form data ckan ckan-v1.9 enhancement icmurray new 2012-05-01T14:16:01Z 2012-06-25T14:52:46Z "When the validation schema was moved from the package controller into the logic layer, the schema's role was changed from that of processing a form and validating data; to just validating data. Whilst is makes sense to have a schema for validating data as it comes in and leaves the logic layer; there's no longer the hook available to PackageController subclasses to do form validation/processing. Add a hook into _save_new() and save_edit() prior to the logic-layer call which subclasses can then override. NB - form processing/validation *is* different to validating data. For example, many data can be inferred from a user's input; or there may be aspects of the form that need to be filled in, but do not make sense to pass as data into the logic-layer, eg. agreeing to T&Cs. - [ ] Add hooks - [ ] Use in ckanext-example" icmurray 2530 DataHub purge fails on some revisions ckan ckan-v1.9 enhancement kindly new 2012-06-15T08:54:58Z 2012-07-02T22:31:20Z "See http://datahub.io/ckan-admin/trash and try to purge revisions (*not* datasets). It will fail on some of the revisions with errors like: Problem purging revision 391db9e8-df57-4e0e-8fe6-d4e0c2318344: (IntegrityError) update or delete on table ""revision"" violates foreign key constraint ""group_extra_revision_revision_id_fkey"" on table ""group_extra_revision"" DETAIL: Key (id)=(391db9e8-df57-4e0e-8fe6-d4e0c2318344) is still referenced from table ""group_extra_revision"". 'DELETE FROM revision WHERE revision.id = %(id)s' {'id': u'391db9e8-df57-4e0e-8fe6-d4e0c2318344'}" rgrp 2679 Change default behaviour of TemplateController.view to 404. ckan ckan-v1.9 enhancement icmurray new 2012-07-16T10:55:33Z 2012-07-16T10:55:33Z "The current behaviour of TemplateController.view() (which is the fallback controller should all others fail) is to attempt to render (as a genshi template) the requested file. Although this may be a feature that some instances want. In general, it leads to: - 500s when attempting to access a normal template (eg - http://datahub.io/importer/preview) - A way of inadvertantly serving things you may not want to serve. (Small risk, as it needs to be renderable as a genshi template). Solution: - Change the controller to 404 - Ensure there's a way for existing ckan instances to override that behaviour should they need it." icmurray 2603 Remove deprecated 'fields' parameter from resource_search ckan ckan-v1.9 refactor icmurray new 2012-06-26T17:10:01Z 2012-06-26T17:10:01Z The fields parameter of resource_search was deprecated when fixing #2438. It can be removed in release 1.9, and the action tidied up as a result. icmurray 2621 Remove the deprecated 'fields' parameter from tag_search and tag_autocomplete ckan ckan-v1.9 refactor icmurray new 2012-06-28T16:22:49Z 2012-06-28T16:22:49Z "This was deprecated in 1.8 as it wasn't accessible via GET requests due to being a dict. See #2439 In a future release of CKAN (probably 1.9) it can be removed. Internal uses of it were removed in #2439, but there are tests that still use it." icmurray 2336 Move Jenkins' install script into ckan core so it can be versioned ckan ckanbuild enhancement seanh accepted 2012-04-30T13:35:14Z 2012-06-25T15:53:57Z seanh 2446 Create demo dataset history/comparison page ckan demo phase 4 enhancement johnmartin assigned 2012-05-28T13:12:18Z 2012-11-11T18:33:39Z "This is a low priority for the demo site. Discussion: https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62414120/comments Implementation: http://s031.okserver.org:2375/dataset/history/adur_district_spending" aron.carroll 2457 Create demo tags list page ckan demo phase 4 enhancement johnmartin assigned 2012-05-28T13:28:55Z 2012-11-11T18:34:38Z "This includes the tag page as well for now. Discussion: https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62998445/comments Implementation: http://s031.okserver.org:2375/en/tag" aron.carroll 2461 Create demo revisions page ckan demo phase 4 enhancement toby assigned 2012-05-28T13:34:15Z 2012-08-09T20:23:58Z "I believe this is to be merged in with site activity. Discussion: https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts Implementation: http://s031.okserver.org:2375/en/revision" aron.carroll 2698 markdown preview ckan demo phase 4 enhancement toby assigned 2012-07-18T14:01:25Z 2012-08-09T20:14:12Z for description / other fields with markdown support shevski 2775 Add bin/less to paster serve command ckan demo phase 4 enhancement toby new 2012-07-30T22:01:26Z 2012-08-09T20:26:02Z "Ideally the ./bin/less command would be run when the server is started. * Also it would be good to have a paster command to build the production CSS with {{{ ./bin/less --production }} * The command could also detect missing node binaries and redirect to the documentation." aron.carroll 2822 Resource additional info titles format/i18n ckan demo phase 4 enhancement toby new 2012-08-09T09:30:20Z 2012-08-09T20:26:25Z "the title for additional info should be translated capitalised etc" toby 2834 Defer all publish calls until all modules have been initialised ckan demo phase 4 enhancement new 2012-08-09T17:08:13Z 2012-08-13T11:18:12Z "Currently if a module uses sandbox.publish() in its initilaize() function then any modules initialised afterwards will miss the event. We should queue all calls to .publish() in the pubsub module in an array until all modules have loaded then iterate over and execute them." aron.carroll 2850 Add JSON support for IE7 ckan demo phase 4 enhancement new 2012-08-14T10:38:12Z 2012-08-14T10:38:12Z "The JavaScript just isn't going to work without it. Look into either the newer JSON3: http://bestiejs.github.com/json3/ or the old faithful: https://github.com/douglascrockford/JSON-js/" aron.carroll 2674 Data preview not loading on s031 ckan demo phase 5 defect kindly assigned 2012-07-16T10:38:22Z 2012-08-07T14:22:04Z "Not loading for all resources as far as I can tell; e.g. http://s031.okserver.org:2375/dataset/afghanistan-election-data/resource/f6331f99-51f6-44d9-95b9-b20f3b74f360 Fine on demo.ckan.org" shevski 2906 Wrong flags in language dropdown ckan demo phase 5 defect new 2012-09-05T16:05:44Z 2012-09-05T16:34:49Z "The flags in the dropdown on details pages are not right. You can see the error on http://s031.okserver.org:2375/dataset/malawi-aid-projects/resource/b717c20e-2006-4ad4-82d2-59b57ebc1ab0." dominik 2416 Normalise resource/data types ckan demo phase 5 enhancement toby accepted 2012-05-23T17:09:06Z 2012-07-18T13:23:22Z "Currently we have far too many types that are essentially the same format. The new demo theme is using icons for common types. So we need to normalise the common formats into pretty strings. e.g. application/json, JSON, .json and json all should be output as “json” See the following Basecamp thread for UI examples and discussion. https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61906351/comments#comment_179681056" aron.carroll 2451 Create demo user list page ckan demo phase 5 enhancement johnmartin assigned 2012-05-28T13:20:13Z 2012-11-11T18:35:02Z "Discussion: https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62268474/comments Implementation: http://s031.okserver.org:2375/user" aron.carroll 2493 Demo site 404 needs some love ckan demo phase 5 enhancement aron.carroll accepted 2012-06-06T12:34:41Z 2012-07-24T13:16:55Z "templates/error_document_template.html I've converted the 404 page to the new style but we probably want to serve this as a 'full page' ie content is 100% page width not sidebar and primary content just go to a non-existant url to see Discusion: https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/63374042/ Implementation: http://s031.okserver.org:2375/404" toby 2555 Demo site needs a breadcrumb helper ckan demo phase 5 enhancement toby new 2012-06-18T13:43:03Z 2012-07-18T13:33:04Z Something to make building breadcrumbs a bit nicer aron.carroll 2644 user dashboard for demo theme ckan demo phase 5 enhancement shevski assigned 2012-07-10T09:00:21Z 2012-08-06T12:23:56Z "we now have a user dashboard that needs theming not sure if we need sam to look at it http://localhost:5000/user/dashboard" toby 2785 Allow resources to be re-ordered ckan demo phase 5 enhancement johnmartin assigned 2012-08-01T10:22:03Z 2012-08-29T10:25:16Z "Not sure where this functionality should be added, possibly in one of the sidebar widgets when editing a resource? Ira, what are your thoughts?" aron.carroll 2823 resource additional info title order ckan demo phase 5 enhancement toby new 2012-08-09T09:32:53Z 2012-08-09T09:32:53Z "Order the items so that none user fields are first from ticket #2707 " toby 2833 Load module templates before calling .initialize() ckan demo phase 5 enhancement new 2012-08-09T17:05:39Z 2012-08-09T17:05:39Z "I think this would be a nice feature for remote loading templates if the options.template value ends in "".html"". {{{ ckan.module('my-module', { options: { template: 'my-template.html' }, initialize: function () { this.template // This is the loaded template. } }); }}}" aron.carroll 2835 Client module needs a template loading method ckan demo phase 5 enhancement new 2012-08-09T17:10:33Z 2012-08-09T17:10:33Z "{{{ Client#getTemplate(name, params, success, error); }}} Where params, success and error are optional arguments. test/index.html already has an implementation called loadFixture()." aron.carroll 2901 Language Dropdown bug in footer with IE7 ckan demo phase 5 enhancement new 2012-09-04T10:31:14Z 2012-09-04T10:31:14Z "The language dropdown in Internet Explorer 7 behaves oddly. Essentially I think the problem is with the way that the dropdown decides to go into ""drop-up mode"" (e.g there aint enough screen space below the initial state so it has to drop up)." johnmartin 2656 Feed with few results has bad paging link, causing exception ckan defect seanh new 2012-07-11T10:05:12Z 2012-07-11T10:05:12Z "This page http://thedatahub.org/feeds/custom.atom?q=wombat has 0 results and contains a link to http://thedatahub.org/feeds/custom.atom?q=wombat&page=0 which the page=0 causes this exception: {{{ ckan.lib.search.common.SearchError'>: SOLR returned an error running query Error: ""'start' parameter cannot be negative"" }}}" dread 1261 Investigate dots in extras search ckan ckan-backlog defect new 2011-08-03T10:19:28Z 2011-08-03T10:19:28Z It seems that searching for extras_foo:value works with solr, but extras_foo.bar:value doesn't. No theory on why. pudo 1317 password reset - improve user search ckan ckan-backlog defect assigned 2011-09-07T17:12:19Z 2012-06-20T11:20:21Z "In password reset, it gets confused if you have two similar users. This is because with the string the user provides, it searches several fields, not just name but also fullname and email address, allowing you to search for these. But only name is unique. Specific problem: Ira searches for ""Irina"" then it finds both: {{{<User name=irina fullname=Irina email=irina.bolychevsky@okfn.org ]}}} and {{{<User name=shevski fullname=Ira email=>}}} (I think) Maybe need to choose which field it searches?" dread 1260 Remove duplicate functions from _util.html ckan ckan-backlog enhancement new 2011-08-03T10:17:32Z 2011-08-15T09:44:59Z There seems to be both a list view for dictized and non dictized data structures for all entities in _util.html at the moment. Probably in the back of someone's mind already, but cleanup here would be nice. pudo 1647 add links to ckan discuss & dev to thedatahub ckan ckan-backlog enhancement new 2012-01-16T00:30:52Z 2012-01-16T09:49:43Z In the footer as well as more clearly & directly on the About page shevski 1823 Spring clean bin directory ckan ckan-backlog enhancement new 2012-02-20T21:28:51Z 2012-05-28T11:12:34Z Huge number of accumulated (and likely unnecessary) scripts in /bin directory. rgrp 2622 Login fails in Opera 12 ckan defect new 2012-06-28T16:56:42Z 2012-06-28T16:56:42Z "Try to login to CKAN using Opera 12, get ""Login failed. Bad username or password. (Or if using OpenID, it hasn't been associated with a user account.)""" seanh 2784 model dictize sensitive data ckan defect icmurray new 2012-08-01T09:51:25Z 2012-08-01T09:51:25Z The model dictize layer doesn't consistently remove sensitive data from the dictized models. It should use the current context to decide whether to include sensitive data or not. icmurray 3013 common-error-messages is unreadable ckan defect dominik new 2012-11-10T13:18:25Z 2012-11-10T13:18:25Z "Since the update of the doc theme, the page became unreadable. http://docs.ckan.org/en/latest/common-error-messages.html" dominik 3029 JSONP parameter scuppers Search in API ckan defect seanh assigned 2012-12-11T15:00:35Z 2012-12-11T16:37:04Z "{{{http://datahub.io/api/2/search/package?jsonp=jsonpcallback&q=canada}}} returns {{{ {""count"": 0, ""results"": []} }}} I believe this worked in CKAN 1.4 or 1.5, but it is broken on 1.7.1, 1.8 and whatever demo.ckan.org is running. I suspect the jsonpcallback parameter is getting sent to SOLR. This bug prevents using javascript on another site to search CKAN (although hopefully the action API would work)." dread 1069 Stub datasets (request for datasets) ckan enhancement tobes assigned 2011-04-01T14:08:39Z 2012-06-25T13:50:15Z "Idea is to have stubs for datasets that someone wants but don't yet exist (or haven't been discovered) in the way one has stub pages on a wiki. We could do this within the existing model by a slight 'abuse' - create a dataset and mark it with a special tag e.g. todo.does-not-yet-exist or similar ... (Just as we have datasets listed that exist but aren't available ...) Alternative would be to have a request for datasets subsystem. I prefer the stub dataset model because it's simpler, provides a simple workflow (as a dataset is found or comes into existence), and the package page provides a natural space in which to accumulate information about what is wanted and what exists. == Implementation == * Agree a new dedicated tag. e.g. todo.does-not-exist == Related == * This ticket has clear links to http://getthedata.org/" rgrp 1130 First time users ckan enhancement assigned 2011-05-09T10:59:21Z 2012-06-25T14:11:54Z Send users to FAQ first time on CKAN lucychambers 2535 SSL certificate for DataHub + https by default ckan enhancement assigned 2012-06-15T11:00:27Z 2012-09-03T08:48:02Z "DataHub is increasingly used and we should ensure it uses ssl as part of general security. See also #1446 (Need to support https login for multiple instances as part of the CKAN package install)" rgrp 2683 Add no-cache header to _tracking API call's response to make sure it doesn't get cached ckan enhancement new 2012-07-16T13:49:37Z 2012-07-16T13:49:37Z seanh 2688 Allow ordering of groups in WUI ckan enhancement new 2012-07-17T10:27:55Z 2012-07-17T10:27:55Z Currently the group_index page just shows the entire list of groups, forcing the ordering to be by name. It would be better if it could be sortable by name (or reversed) or by package_count (or reversed) ross 2709 Atom feeds are undocumented ckan enhancement icmurray new 2012-07-18T15:11:50Z 2012-07-18T15:43:32Z There doesn't seem to be any documentation yet for Atom feeds. markw 2731 Some sites permanently 'down for maintenance' ckan enhancement new 2012-07-23T12:06:08Z 2012-07-23T13:53:28Z "A large number of XXX.ckan.net sites give the following message: ""This Site is Down for Maintenance We apologize for the inconvenience. ~ The Open Knowledge Foundation sysadmins."" The message is unhelpful and patently false - the sites do not exist. Some of them were supposed to have been redirected to a relevant group at thedatahub.org in this ticket (now closed): http://trac.okfn.org/ticket/933 However, the redirection only seems to have worked in one case, http://si.ckan.net. The problem still affects the following sites - the first 4 of which have supposedly been merged: * http://lt.ckan.net * http://hu.ckan.net * http://be.ckan.net * http://gr.ckan.net * http://nz.ckan.net * http://bg.ckan.net * http://fi.ckan.net * http://katalogdanych.centrumcyfrowe.pl * http://pl.ckan.net * http://southampton.ckan.net * http://sl.ckan.net Please sort this out by redirecting, removing the sites, giving a more helpful (and accurate) failure message, etc, as appropriate." markw 2777 bug: user attributes ckan enhancement new 2012-07-31T09:19:23Z 2012-07-31T09:19:23Z icmurray 2796 Need a datahub one-pager ckan enhancement mark.wainwright new 2012-08-02T16:28:36Z 2012-08-16T15:04:55Z "A one-pager explaining what the datahub is and with howto/examples for new users. This would make it much easier to explain the value in using the datahub for storing data. " ross 2814 Demo: upload file behaves oddly ckan enhancement shevski assigned 2012-08-08T10:06:00Z 2012-09-03T11:06:21Z "3. Uploading a file behaves counter-intuitively (I would suggest wrongly). When adding a new resource by uploading a file, I select a file called say create-group.png. I expect the following to happen: * the pathname of that file is filled in the box; * nothing is actually uploaded till I hit 'add' (confirming that I've got the right file etc). Instead of this, * the file is immediately and, seemingly, irrevocably uploaded; * the box is filled with a mysterious URL (https://commondatastorage.googleapis.com/ckan-demo ...) * the mysterious URL is so long I can't even see the final element, which is my file name. " markw 2844 SQL-only (no solr) version of CKAN ckan enhancement new 2012-08-13T11:59:28Z 2012-08-20T09:22:07Z " * ~~Search needs to run of local DB (again)~~ * paster db clean attemps to connect to SOLR (still works as does db first but then excepts which is not nice UX)" rgrp 2851 Preview of PDF tries to connect to datastore ckan enhancement new 2012-08-15T12:12:46Z 2012-08-15T12:12:46Z "e.g. http://datahub.io/dataset/ccc-gistemp/resource/80ebdbd6-d91b-4fef-9db4-d3dfbd7e868e CKAN attempts to render the PDF as a datastore tabular date even though the mimetype of the PDF is correctly specified." ross 2870 1.8 tag_list not defined ckan enhancement seanh new 2012-08-16T10:58:52Z 2012-08-16T10:58:52Z "From: <thedatahub.org@s077.okserver.org> Date: Thu, Aug 16, 2012 at 3:20 AM Subject: WebApp Error: <class 'genshi.template.eval.UndefinedError'>: ""tag_list"" not defined To: ckan-sysadmin@okfn.org, kindly@gmail.com URL: http://ckan.net/dataset/uk-postboxes Module weberror.errormiddleware:162 in __call__ << __traceback_supplement__ = Supplement, self, environ sr_checker = ResponseStartChecker(start_response) app_iter = self.application(environ, sr_checker) return self.make_catching_iter(app_iter, environ, sr_checker) except: >> app_iter = self.application(environ, sr_checker) Module beaker.middleware:73 in __call__ << self.cache_manager) environ[self.environ_key] = self.cache_manager return self.app(environ, start_response) >> return self.app(environ, start_response) Module beaker.middleware:155 in __call__ << headers.append(('Set-cookie', cookie)) return start_response(status, headers, exc_info) return self.wrap_app(environ, session_start_response) def _get_session(self): >> return self.wrap_app(environ, session_start_response) Module routes.middleware:131 in __call__ << r'\1', oldpath) response = self.app(environ, start_response) # Wrapped in try as in rare cases the attribute will be gone already >> response = self.app(environ, start_response) Module pylons.wsgiapp:125 in __call__ << controller = self.resolve(environ, start_response) response = self.dispatch(controller, environ, start_response) if 'paste.testing_variables' in environ and hasattr(response, >> response = self.dispatch(controller, environ, start_response) Module pylons.wsgiapp:324 in dispatch << if log_debug: log.debug(""Calling controller class with WSGI interface"") return controller(environ, start_response) def load_test_env(self, environ): >> return controller(environ, start_response) Module ckan.lib.base:239 in __call__ << try: res = WSGIController.__call__(self, environ, start_response) finally: model.Session.remove() >> res = WSGIController.__call__(self, environ, start_response) Module pylons.controllers.core:221 in __call__ << return response(environ, self.start_response) response = self._dispatch_call() if not start_response_called: self.start_response = start_response >> response = self._dispatch_call() Module pylons.controllers.core:172 in _dispatch_call << req.environ['pylons.action_method'] = func response = self._inspect_call(func) else: if log_debug: >> response = self._inspect_call(func) Module pylons.controllers.core:107 in _inspect_call << func.__name__, args) try: result = self._perform_call(func, args) except HTTPException, httpe: if log_debug: >> result = self._perform_call(func, args) Module pylons.controllers.core:60 in _perform_call << """"""Hide the traceback for everything above this method"""""" __traceback_hide__ = 'before_and_this' return func(**args) def _inspect_call(self, func): >> return func(**args) Module ckan.controllers.package:322 in read << template = template[:template.index('.') + 1] + format return render(template, loader_class=loader) def comments(self, id): >> return render(template, loader_class=loader) Module ckan.lib.base:153 in render << try: return cached_template(template_name, render_template, loader_class=loader_class) except ckan.exceptions.CkanUrlException, e: raise >> loader_class=loader_class) Module pylons.templating:249 in cached_template << return content else: return render_func() >> return render_func() Module ckan.lib.base:102 in render_template << if loader_class == NewTextTemplate: return literal(stream.render(method=""text"", encoding=None)) return literal(stream.render(method=method, encoding=None, >> return literal(stream.render(method=""text"", encoding =None)) Module genshi.core:183 in render << method = self.serializer or 'xml' generator = self.serialize(method=method, **kwargs) return encode(generator, method=method, encoding=encoding, out=out) def select(self, path, namespaces=None, variables=None): >> return encode(generator, method=method, encoding=encoding, out =out) Module genshi.output:57 in encode << _encode = lambda string: string if out is None: return _encode(''.join(list(iterator))) for chunk in iterator: out.write(_encode(chunk)) >> return _encode(''.join(list(iterator))) Module genshi.output:569 in __call__ << def __call__(self, stream): strip_markup = self.strip_markup for event in stream: if event[0] is TEXT: data = event[1] >> for event in stream: Module genshi.core:288 in _ensure << # unchanged yield event for event in stream: yield event >> for event in stream: Module genshi.core:288 in _ensure << # unchanged yield event for event in stream: yield event >> for event in stream: Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream): for mark, event in stream: kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK): >> for mark, event in stream: Module genshi.filters.transform:1145 in __call__ << :param stream: The marked event stream to filter """""" for mark, event in stream: yield mark, event if mark is ENTER: >> for mark, event in stream: Module genshi.filters.transform:714 in __call__ << stream = iter(stream) next = stream.next for mark, event in stream: if mark is None: yield mark, event >> for mark, event in stream: Module genshi.filters.transform:682 in _mark << def _mark(self, stream): for event in stream: yield OUTSIDE, event >> for event in stream: Module genshi.core:288 in _ensure << # unchanged yield event for event in stream: yield event >> for event in stream: Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream): for mark, event in stream: kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK): >> for mark, event in stream: Module genshi.filters.transform:1145 in __call__ << :param stream: The marked event stream to filter """""" for mark, event in stream: yield mark, event if mark is ENTER: >> for mark, event in stream: Module genshi.filters.transform:714 in __call__ << stream = iter(stream) next = stream.next for mark, event in stream: if mark is None: yield mark, event >> for mark, event in stream: Module genshi.filters.transform:682 in _mark << def _mark(self, stream): for event in stream: yield OUTSIDE, event >> for event in stream: Module genshi.core:288 in _ensure << # unchanged yield event for event in stream: yield event >> for event in stream: Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream): for mark, event in stream: kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK): >> for mark, event in stream: Module genshi.filters.transform:1145 in __call__ << :param stream: The marked event stream to filter """""" for mark, event in stream: yield mark, event if mark is ENTER: >> for mark, event in stream: Module genshi.filters.transform:714 in __call__ << stream = iter(stream) next = stream.next for mark, event in stream: if mark is None: yield mark, event >> for mark, event in stream: Module genshi.filters.transform:682 in _mark << def _mark(self, stream): for event in stream: yield OUTSIDE, event >> for event in stream: Module genshi.core:288 in _ensure << # unchanged yield event for event in stream: yield event >> for event in stream: Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream): for mark, event in stream: kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK): >> for mark, event in stream: Module ckanext.googleanalytics.plugin:93 in download_adder << [downloaded %s times]</span>''' count = None for mark, (kind, data, pos) in stream: if mark and kind == START: href = data[1].get('href') >> for mark, (kind, data, pos) in stream: Module genshi.filters.transform:714 in __call__ << stream = iter(stream) next = stream.next for mark, event in stream: if mark is None: yield mark, event >> for mark, event in stream: Module genshi.filters.transform:682 in _mark << def _mark(self, stream): for event in stream: yield OUTSIDE, event >> for event in stream: Module genshi.core:288 in _ensure << # unchanged yield event for event in stream: yield event >> for event in stream: Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream): for mark, event in stream: kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK): >> for mark, event in stream: Module genshi.filters.transform:1175 in __call__ << """""" callable_value = hasattr(self.value, '__call__') for mark, (kind, data, pos) in stream: if mark is ENTER: if callable_value: >> for mark, (kind, data, pos) in stream: Module genshi.filters.transform:714 in __call__ << stream = iter(stream) next = stream.next for mark, event in stream: if mark is None: yield mark, event >> for mark, event in stream: Module genshi.filters.transform:682 in _mark << def _mark(self, stream): for event in stream: yield OUTSIDE, event >> for event in stream: Module genshi.core:288 in _ensure << # unchanged yield event for event in stream: yield event >> for event in stream: Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream): for mark, event in stream: kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK): >> for mark, event in stream: Module genshi.filters.transform:1145 in __call__ << :param stream: The marked event stream to filter """""" for mark, event in stream: yield mark, event if mark is ENTER: >> for mark, event in stream: Module genshi.filters.transform:714 in __call__ << stream = iter(stream) next = stream.next for mark, event in stream: if mark is None: yield mark, event >> for mark, event in stream: Module genshi.filters.transform:682 in _mark << def _mark(self, stream): for event in stream: yield OUTSIDE, event >> for event in stream: Module genshi.template.base:605 in _include << from genshi.template.loader import TemplateNotFound for event in stream: if event[0] is INCLUDE: href, cls, fallback = event[1] >> for event in stream: Module genshi.template.base:565 in _flatten << elif kind is EXPR: result = _eval_expr(data, ctxt, vars) if result is not None: # First check for a string, otherwise the iterable test >> result = _eval_expr(data, ctxt, vars) Module genshi.template.base:277 in _eval_expr << if vars: ctxt.push(vars) retval = expr.evaluate(ctxt) if vars: ctxt.pop() >> retval = expr.evaluate(ctxt) Module genshi.template.eval:178 in evaluate << __traceback_hide__ = 'before_and_this' _globals = self._globals(data) return eval(self.code, _globals, {'__data__': data}) >> return eval(self.code, _globals, {'__data__': data}) Module ?:51 in <Expression u""tag_list(c.pkg_dict.get('tags', ''))""> << <li py:if=""c.pkg_dict.get('tags')"" class=""sidebar-section""> <h3>Tags</h3> ${tag_list(c.pkg_dict.get('tags', ''))} </li> >> ${tag_list(c.pkg_dict.get('tags', ''))} Module genshi.template.eval:309 in lookup_name << val = BUILTINS.get(name, val) if val is UNDEFINED: val = cls.undefined(name) return val >> val = cls.undefined(name) Module genshi.template.eval:410 in undefined << """"""Raise an ``UndefinedError`` immediately."""""" __traceback_hide__ = True raise UndefinedError(key, owner=owner) >> raise UndefinedError(key, owner=owner) UndefinedError: ""tag_list"" not defined CGI Variables CKAN_CURRENT_URL '/dataset/uk-postboxes' CKAN_LANG 'en' CKAN_LANG_IS_DEFAULT True CKAN_PAGE_CACHABLE True CONTENT_TYPE '; charset=utf-8' DOCUMENT_ROOT '/etc/apache2/htdocs' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT 'text/html,text/plain,text/xml' HTTP_ACCEPT_CHARSET 'ISO-8859-1,utf-8;q=0.7,*;q=0.7' HTTP_ACCEPT_ENCODING 'gzip' HTTP_ACCEPT_LANGUAGE 'vi,en-us;q=0.7,en;q=0.3' HTTP_CONNECTION 'close' HTTP_HOST 'ckan.net' HTTP_USER_AGENT 'coccoc/1.0 ()' PATH_INFO '/dataset/uk-postboxes' PATH_TRANSLATED '/etc/ckan/datahub/apache.wsgi/dataset/uk-postboxes' REMOTE_ADDR '127.0.0.1' REMOTE_PORT '47065' REQUEST_METHOD 'GET' REQUEST_URI '/dataset/uk-postboxes' SCRIPT_FILENAME '/etc/ckan/datahub/apache.wsgi' SERVER_ADDR '127.0.0.1' SERVER_ADMIN '[no address given]' SERVER_NAME 'ckan.net' SERVER_PORT '80' SERVER_PROTOCOL 'HTTP/1.0' SERVER_SIGNATURE '<address>Apache/2.2.22 (Ubuntu) Server at ckan.net Port 80</address>\n' SERVER_SOFTWARE 'Apache/2.2.22 (Ubuntu)' WSGI Variables application <beaker.middleware.CacheMiddleware object at 0x7f5312a51650> beaker.cache <beaker.cache.CacheManager object at 0x7f5312a51790> beaker.get_session <bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x7f5312a516d0>> beaker.session {'_accessed_time': 1345083636.079552, '_creation_time': 1345083636.079552} mod_wsgi.application_group 'ckan.net|' mod_wsgi.callable_object 'application' mod_wsgi.handler_script '' mod_wsgi.input_chunked '0' mod_wsgi.listener_host '' mod_wsgi.listener_port '8080' mod_wsgi.process_group 'datahub' mod_wsgi.request_handler 'wsgi-script' mod_wsgi.script_reloading '1' mod_wsgi.version (3, 3) paste.cookies (<SimpleCookie: >, '') paste.registry <paste.registry.Registry object at 0x7f52e47477d0> paste.throw_errors True pylons.action_method <bound method PackageController.read of <ckan.controllers.package.PackageController object at 0x7f52e477b810>> pylons.controller <ckan.controllers.package.PackageController object at 0x7f52e477b810> pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'} pylons.pylons <pylons.util.PylonsContext object at 0x7f52e477be90> pylons.routes_dict {'action': u'read', 'controller': u'package', 'id': u'uk-postboxes'} repoze.who.logger <logging.Logger object at 0x7f53127b2b50> repoze.who.plugins {'openid': <OpenIdIdentificationPlugin 139994769488336>, 'friendlyform': <FriendlyFormPlugin 139994769488528>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator object at 0x7f5312d893d0>, 'auth_tkt': <AuthTktCookiePlugin 139994771858704>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f5312d89350>} routes.route <routes.route.Route object at 0x7f5312955a50> routes.url <routes.util.URLGenerator object at 0x7f52eb8c9090> webob._parsed_query_vars (GET([]), '') webob.adhoc_attrs {'language': 'en-us'} wsgi process 'Multi process AND threads (?)' wsgi.file_wrapper <built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f5308983a80> wsgi.version (1, 1) wsgiorg.routing_args (<routes.util.URLGenerator object at 0x7f52eb8c9090>, {'action': u'read', 'controller': u'package', 'id': u'uk-postboxes'}) " ross 2874 Clean up bin directory ckan enhancement rgrp assigned 2012-08-17T08:01:48Z 2012-08-17T08:01:55Z Full of obsolete material rgrp 2877 Bugs with datastore v2 ckan enhancement kindly assigned 2012-08-18T00:33:22Z 2012-08-30T09:53:15Z "In progress 1. ~~[major] q does not seem to work reliably. e.g. using the setup from this gist https://gist.github.com/1930806 and doing a ?q=DE yields no results (does not work with ""q=de"" either)~~ * ~~q=second does work ...~~ 1. [major] q does not work with 2 values (see below) 2. Query on search with limit 0 results in total of 0 (should either be null or correct total). Queries with other limits yield correct total AFAICT * Also weird fact that limit is returned but as as as string - should it not be an integer 3. Types on fields: could these not be canonical and as per recline (or is it important to allow exact sql types ...) === Multiple query values === Try a query such as: ""second UK"" and you will get 500 error: http://localhost:5000/api/3/action/datastore_search?resource_id=4f1299ab-a100-4e5f-ba81-e6d234a2f3bd&q=second%20UK {{{ ProgrammingError: (ProgrammingError) syntax error in tsquery: ""second UK"" 'select ""_id"", ""id"", ""date"", ""x"", ""y"", ""z"", ""country"", ""title"", ""lat"", ""lon"", count(*) over() as ""_full_count""\n from ""4f1299ab-a100-4e5f-ba81-e6d234a2f3bd"" where _full_text @@ to_tsquery(%s) limit 100 offset 0' (u'second UK',) }}} === Suggestions === Filter support: should think in more detail about this (may want to follow recline style) Simple filters in query parameters would be nice too ..." rgrp 2888 Datapreview in Iframe ckan enhancement new 2012-08-23T18:11:01Z 2012-08-23T18:11:01Z Try and put datapreview in Iframe. kindly 2895 QA pages (e.g. broken resource links) are not paginated ckan enhancement new 2012-08-28T17:38:05Z 2012-08-28T17:38:05Z And as a result timeout on e.g. the datahub ... rgrp 2911 Internal documentation of Organization Groups ckan enhancement new 2012-09-06T14:23:04Z 2012-09-06T14:23:04Z A summary/user story doc of how organizations and groups are expected to work. kindly 2934 Webtests for CKAN ckan enhancement new 2012-09-18T11:10:47Z 2012-09-19T13:10:01Z "The tests in CKAN only test single pages (functional tests) or single methods (unit tests). In order to make sure that the whole system still works as expected, we need tests that cover whole processes. An example user story to be tested would be: ""As a user with create permissions I want to be able to log in, create a new resource, save it and then make sure that it is in the system."" or: ""As a user I want to use the search functionality and open a resource from the search page."" These user stories require a different testing system with interaction and processes. " Dominik 2938 Explain bundling in doc/resources.rst ckan enhancement Toby new 2012-09-24T12:13:55Z 2012-09-24T12:13:55Z "Since it makes sense to bundle resources into one js file that are commonly used together, there should be some explanations on how separate js scripts are merged into one file for production. Explanations could include how groups are handled or different resource files." dominik 2939 Orgs are groups ckan enhancement toby new 2012-09-25T09:07:14Z 2012-09-25T09:07:14Z holder for ticket number toby 2943 Chrome does not resize preview ckan enhancement new 2012-10-01T11:08:06Z 2012-10-01T11:25:59Z Chrome does not resize iframe after a full refresh/ on first load dominik 2944 Recline preview does not work in Opera ckan enhancement new 2012-10-01T11:26:42Z 2012-10-01T11:26:42Z Opera cannot show a recline preview. I shows 0 records. dominik 2945 Pdf preview does not load in IE ckan enhancement new 2012-10-01T11:27:49Z 2012-10-01T11:27:49Z The pdf preview does not load in IE 9. dominik 2948 Negative range breaks datastore backend ckan enhancement new 2012-10-01T14:30:54Z 2012-10-01T14:31:05Z Recline allows negative ranges. The backend returns an error 500 for that which breaks the recline preview. dominik 2992 Delete resource should send me back to edit dataset page ckan enhancement new 2012-10-16T17:19:16Z 2012-10-16T17:19:16Z johnmartin 3012 data.gov auth ckan enhancement toby new 2012-11-08T16:50:09Z 2012-11-08T16:50:09Z toby 3020 Update CKAN coding standards ckan enhancement new 2012-11-16T20:11:51Z 2012-11-16T20:11:51Z seanh 3021 Logout doesn't work without JS ckan enhancement johnmartin accepted 2012-11-20T11:19:10Z 2012-11-20T11:46:08Z "Essentially, the functionality should be as follows: - Add logout link that has `.js-hide` attached to it within the header that isn't hidden within a dropdown See http://plus.google.com/ (when logged in) with and without JS to see an example of the actual sign-out working without JS" johnmartin 3023 New methods on IPackageController to provide access to the data_dict ckan enhancement amercader new 2012-11-22T17:00:57Z 2012-11-22T17:00:57Z Extension hooking into the edit and create methods of the IPackageController interface receive the package object. This may not include all the fields that came from the form. The new extension points will pass the validated data_dict so extensions can have access to it amercader 3025 Add requests to core requirements ckan enhancement amercader new 2012-11-27T21:35:24Z 2012-11-27T21:35:24Z Because yes please amercader 3027 solr for 2.0 ckan enhancement kindly new 2012-12-04T02:22:09Z 2012-12-04T02:22:09Z "change mm support solr 3 and 4 add *_date field" kindly 3030 clean up helper functions ckan enhancement new 2012-12-18T12:33:58Z 2012-12-18T12:33:58Z toby 2957 New datastore on postgres prior to 9.0 ckan task new 2012-10-03T19:23:34Z 2012-10-03T19:23:34Z The ``GRANT SELECT ON ALL`` and ``ALTER DEFAULT PRIVILEGES`` are not available in Postgres prior to version 9.0. dominik 2913 IGroupForm has package_form() method ckan ckan 2.0 defect new 2012-09-11T14:45:39Z 2012-10-15T12:22:59Z Should be group_form() I think seanh 2920 Genshi text template message extractor no longer needed? ckan ckan 2.0 defect new 2012-09-13T09:57:29Z 2012-09-13T09:57:29Z "In setup.py, do we still need: ('templates/**.txt', 'genshi', { 'template_class': 'genshi.template:TextTemplate' }), in message_extractors? Doesn't like there are any txt files in there anymore." seanh 2921 Add docstring to top of lib/extract.py file ckan ckan 2.0 defect new 2012-09-13T10:00:07Z 2012-09-13T10:00:07Z "I think it couldn't hurt for this module to have a docstring at the top of the file explaining what the module is for. I know from setup.py that it's there to provide the extract_ckan() string extractor function for Babel, but I think looking at the file on its own it's not so obvious. Also a couple of other small fixes: jinja2_cleaner looks like a helper function only meant to be used by extract_ckan(), might make things clearer if it was called _jinja2_cleaner(), just so it doesn't look like a public function the module wants to export. jinja_extensions seems to be a module-level variable that is only used in one function, could be moved into the function, unless you think there might be more functions that want it in future." seanh 2922 Better docstring for CKANInternationalizationExtension ckan ckan 2.0 defect new 2012-09-13T10:01:47Z 2012-09-13T10:01:47Z "I'm unsure about what's going on here. As I understand it, when we run `python setup.py extract_messages` it's going to use the extract_ckan() function from ckan/lib/extract.py to process the HTML files. For the HTML files that are jinja2 templates, extract_ckan() will call jinja2_cleaner() which will call regularise_html() on the strings. So the strings are regularised when they are extracted from the source files. But then we have the parse() method of CkanInternationalizationExtension also calling regularise_html(). I don't get what's happening here. Why do the strings need to be regularised twice? My guess is that CkanInternationalizationExtension is used when the strings are extracted from the templates at runtime, and they need to be regularised at this time in order to match them against the regularised strings in the mo files to find the translations to output? Maybe CkanInternationalizationExtension needs a better docstring saying what it does?" seanh 2923 Change regularise -> regularize ckan ckan 2.0 defect new 2012-09-13T10:03:02Z 2012-09-13T10:03:02Z The function is called regularise_html(), can't remember what file it's in. seanh 2924 Better docs for trans js command, and add to release process ckan ckan 2.0 defect new 2012-09-13T10:04:31Z 2012-09-13T10:04:31Z "Add a better docstring to for trans js command explaining what it does and why, and how to use it. Also add this command to the CKAN Release Process as it's needed there" seanh 2925 Remove trans mangle paster command? ckan ckan 2.0 defect new 2012-09-13T10:06:08Z 2012-09-13T10:06:08Z "- Is `trans mangle` really necessary? If you upload a pot file to Transifex, it can generate a po file for you with 100% strings translated into a fictional pseudo language where everything is really long strings of unicode characters. I found this worked well for coverage testing, and also tests handling of unicode and long strings all over the place." seanh 2964 Last organization admin can remove herself ckan ckan 2.0 defect new 2012-10-15T10:14:18Z 2012-10-15T10:14:18Z "If you are the only admin of an organization you can edit the organization's members and demote yourself, then the org has no admins and no one (except sysadmins maybe) can edit it. Last admin should not be able to remove or demote herself. Also applies to groups." seanh 2967 Organization members edit page reloads after demoting self ckan ckan 2.0 defect new 2012-10-15T10:17:07Z 2012-10-15T10:17:07Z Edit an organizations members page and demote yourself from organization admin, after saving the members edit page reloads even though you are no longer an admin and should no longer be able to access this page. seanh 2968 Anyone can access organization members page ckan ckan 2.0 defect new 2012-10-15T10:19:15Z 2012-10-15T10:31:10Z The button will not show if you are not authorized but browse to /organization/members/foo and you can edit the members, it does stop you when you try to save your changes, but you shouldn't be able to get to the page at all seanh 2969 Group members page 500s ckan ckan 2.0 defect new 2012-10-15T10:20:54Z 2012-10-15T10:20:54Z The group members page (e.g. /group/members/roger) seems broken, IDs instead of user names are shown for the members, and clicking on a member 500s seanh 2970 Organization and group member links use id not name ckan ckan 2.0 defect new 2012-10-15T10:22:11Z 2012-10-15T10:22:11Z e.g. it the 'Members' button links to /organization/members/0a44... instead of /organization/members/foobar seanh 2993 """logged_in"" and ""visitor"" show in user list at /users" ckan ckan 2.0 defect new 2012-10-17T09:42:02Z 2012-10-17T14:40:26Z seanh 3001 Multilingual plugin crashes CKAN on add dataset when some languages are default ckan ckan 2.0 defect new 2012-10-18T16:50:48Z 2012-10-18T16:50:48Z "Enable the multilingual plugins: ckan.plugins = stats synchronous_search multilingual_dataset multilingual_group multilingual_tag and set your default language to one not supported by the multilingual plugin, e.g. ckan.locale_default = cs_CZ now run CKAN and try to add a dataset: File '/home/seanh/Projects/ckan171/ckan/ckanext/multilingual/plugin.py', line 141 in before_index text_field_items['text_' + default_lang].extend(all_terms) KeyError: 'text_cs_CZ' It doesn't matter what language you are viewing the site in in your browser, the default language setting in the ini file determines whether it crashed or not. A number of supported languages are defined at the top of ckanext/multilingual/plugin.py. I think if the default language is not one of these it crashes. I think this affects all versions of CKAN since the multilingual plugin was added so at least 1.7, 1.8 and 2.0" seanh 3014 Crash when deleting a non-empty vocabulary ckan ckan 2.0 defect new 2012-11-13T10:50:08Z 2012-11-13T10:50:08Z "From Knud Möller: when I try to delete a non-empty tag vocabulary via the API (through HTTP), I get an internal server error. Checking the logs, this turns out to be a consistency error raised by sqlalchemy: Error - <class 'sqlalchemy.exc.IntegrityError'>: (IntegrityError) update or delete on table ""vocabulary"" violates foreign key constraint ""tag_vocabulary_id_fkey"" on table ""tag"" DETAIL: Key (id)=(21421955-7560-467c-af30-9f790b73e6ae) is still referenced from table ""tag"". 'DELETE FROM vocabulary WHERE vocabulary.id = %(id)s' {'id': u'21421955-7560-467c-af30-9f790b73e6ae'} URL: http://33.33.33.10:5000/api/action/vocabulary_delete The error makes sense, but I'm wondering if it would be useful to extend the API to also allow the deletion of non-empty vocabularies, possibly via a parameter (not sure what best practice in API design is). At the very least, it would be cool if the error message coming back in the response had more information in it." seanh 2329 Add back in RSS/Atom links on relevant pages ckan ckan 2.0 enhancement assigned 2012-04-26T12:15:40Z 2012-09-03T09:24:31Z E.g. on dataset page, on revision page, on user page and on search results. rgrp 2654 UI support for ordering groups on group_read page ckan ckan 2.0 enhancement assigned 2012-07-10T18:11:31Z 2012-09-03T08:49:16Z "The group_index page has no support in WUI for ordering the groups displayed. Should allow sorting by name Add support for this for datahub now, and discuss for new 1.9 UI" ross 2886 Configurable related items ckan ckan 2.0 enhancement new 2012-08-22T08:55:18Z 2012-08-22T08:55:18Z "Related items (Apps & Ideas) have a collection of types, which are currently fixed. Some of these types are ambiguous, such as ideas in that they may not have a link - and if they do it is likely to be to a blog post - another existing type. Whilst the URL is required, and we think this should stay required, we should also allow users to change the types found in Apps & Ideas to a shortened list that suits their requirements." ross 2902 genshi is used in Group controller ckan ckan 2.0 enhancement new 2012-09-04T18:10:30Z 2012-09-04T18:10:30Z "The Group controller uses the genshi module where it may be better for it to use Markup() in the template. See https://github.com/okfn/ckan/pull/117/files#r1524863" ross 2917 Organization admins can delete themselves ckan ckan 2.0 enhancement johnmartin new 2012-09-12T12:10:43Z 2012-09-12T13:13:56Z Organization administrators can delete themselves from the user management pages. We should disable this on the front-end as well as the back-end. ross 2926 I*Form tests for per-type templates ckan ckan 2.0 enhancement new 2012-09-13T16:01:21Z 2012-09-13T16:01:21Z "Finish the tests for the per-package-type custom template support in IPackageForm, IGroupForm and IOrganizationForm (i.e. the new_template(), read_template() etc. methods). The new way that template inheritance works in jinja looks like it should allow this to be tested nicely without duplicating entire templates in the example extensions." seanh 2927 Test new package/group/organization type URLs support ckan ckan 2.0 enhancement new 2012-09-13T16:02:33Z 2012-09-13T16:02:33Z IPackageForm, IGroupForm and IOrganizationForm allow you to define e.g. new package types with new top-level URLs, but the example extensions don't have tests for this yet. seanh 2928 Run CKAN tests with example_i*form extensions enabled ckan ckan 2.0 enhancement new 2012-09-13T16:05:34Z 2012-09-13T16:05:34Z Before releasing CKAN 2.0 we need to run all the CKAN tests with a modified test-core.ini with the example_idatasetform, example_igroupform and example_iorganizationform plugins enabled. If any tests fail, fix the bugs. This needs to be done for each release so add it to the release process. seanh 2929 Remove is_fallback() from I*Form extension interfaces? ckan ckan 2.0 enhancement new 2012-09-13T16:09:57Z 2012-09-13T16:09:57Z "If a plugin wants to take over the default package type for example, it can return ['dataset'] from its package_types() method. I don't see how there can be a package type that is not the default 'package' and is not returned by the package_types() method of a plugin, because the URLs for such a package type would not be mapped. Am I missing something or is is_fallback() not needed? Same for IGroupForm and IOrganizationForm" seanh 2931 Better docstring for app_globals.py ckan ckan 2.0 enhancement new 2012-09-17T14:13:14Z 2012-09-17T14:16:18Z "''' The application's Globals object ''' is not very informative. " seanh 2932 Add docstring to system_info.py ckan ckan 2.0 enhancement new 2012-09-17T14:50:29Z 2012-09-17T14:50:29Z What is the system_info table for? It looks like a way to override config file settings in the database, perhaps so that admins can use a web interface to change site title etc. Could do with a docstring explaining the intended purpose of the table. seanh 2933 Update theming docs ckan ckan 2.0 enhancement new 2012-09-17T15:21:12Z 2012-09-17T15:21:12Z "There's quite a lot of custom stuff going on, with all the custom jinja extensions, and the use of Fanstatic plus custom fanstatic extensions. People who are new to CKAN wanting to write CKAN themes are likely to be confused. I think the whole lot could do with documentation, maybe in the Theming and Customizing chapter in the sphinx docs. I suggest completely replacing the contents of doc/theming.rst, and putting a 'legacy theming' link at the bottom linking to a doc/legacy_theming.rst file with the old contents." seanh 2956 Allow for resource editing in CKAN 2.0 ckan ckan 2.0 enhancement shevski assigned 2012-10-03T15:11:40Z 2012-10-04T10:27:22Z On the edit dataset page... there needs to be a way for users to be able to see all the resources associated to the dataset and edit them individually. johnmartin 2961 Preview plugin endpoint ckan ckan 2.0 enhancement dominik assigned 2012-10-08T11:39:27Z 2012-10-18T16:33:10Z "Users could write ckan extensions that offer previews for a certain datatype. Advantages: - We can say that you can add your own previews - Every preview would have to define it's own dependencies - External development (community creates new extensions?) Questions to answer: - Specification of the interface - Precedence of previews for the same data types - Plugins only define the datatype that they are responsible for or do we call a function that returns whether the extension is responsible. As far as I can see, this should be fairly easy to implement by following these instructions: http://docs.ckan.org/en/latest/writing-extensions.html#writing-a-plugin-interface" dominik 2966 'Add' button text is wrong when editing organization members ckan ckan 2.0 enhancement new 2012-10-15T10:15:49Z 2012-10-15T10:15:49Z e.g. if I just changed a member's capacity I am not adding anything 'save' is better maybe seanh 2971 """Are ytou sure you want to delete this member?"" should say which member" ckan ckan 2.0 enhancement new 2012-10-15T10:27:20Z 2012-10-15T10:27:20Z when deleting members from groups and orgs seanh 2972 Remove any imports of authz.py and delete file ckan ckan 2.0 enhancement new 2012-10-15T10:37:22Z 2012-10-15T10:37:22Z it is no longer used seanh 2973 Move new_authz.py into logic/auth/__init__.py ckan ckan 2.0 enhancement new 2012-10-15T10:39:05Z 2012-10-15T10:39:05Z "Makes sense to keep all auth stuff under logic/auth. Also decide which functions from new_authz.pu should be marked private with _. Also decide which functions from new_authz.py are really helpers for the auth functions in get.py, update.py and delete.py, and should maybe moved to another file, e.g. logic/auth/helpers.py" seanh 2974 General of all auth functions ckan ckan 2.0 enhancement new 2012-10-15T10:40:12Z 2012-10-15T10:40:12Z "From Toby: A general cleanup of all auth functions (in logic/auth) to check their fitness, error messages, etc." seanh 2975 Tests for auth functions and new actions ckan ckan 2.0 enhancement new 2012-10-15T10:41:00Z 2012-10-15T10:41:00Z "From Toby: tests around the auth functions and new actions - probably needs an improved testing mechanism and things like test data creation on via actions " seanh 2976 Polish group and organization member pages ckan ckan 2.0 enhancement new 2012-10-15T10:41:49Z 2012-10-15T10:41:49Z "From Toby: A little polish to the member pages and probably a little hardening of the controller and logic actions" seanh 2977 Fix user autocomplete on group and organization member pages ckan ckan 2.0 enhancement new 2012-10-15T10:42:29Z 2012-10-15T10:42:29Z "From Toby: @johnmartin the user autocomplete on member add needs fixing it gets data but does not understand what to do with it " seanh 2978 Tests for permissions for organizations and groups ckan ckan 2.0 enhancement new 2012-10-15T10:43:26Z 2012-10-15T10:43:26Z "From Toby: checking the logic around the who can do what from the user stories - via some tests may be a good approach " seanh 2979 Requesting membership to groups and organizations ckan ckan 2.0 enhancement new 2012-10-15T10:44:04Z 2012-10-15T10:44:04Z "From Toby: The requesting membership user stories and implementation remain needed " seanh 2988 UI functional tests for CKAN 2.0 ckan ckan 2.0 enhancement new 2012-10-15T11:02:15Z 2012-10-15T11:02:15Z Afaik the new CKAN 2.0 frontend has no functional tests seanh 2989 """Add dataset to organization"" should auto-select the organization" ckan ckan 2.0 enhancement new 2012-10-15T11:04:12Z 2012-10-15T11:04:12Z "'Add dataset to organization' button, when you get through to the third stage of the new dataset form the organization you came from is not selected. I don't see any option to choose the group when adding or updating a dataset, but if I add a dataset via the ""Add dataset to group"" button on a group's page, then the dataset seems to get added to that group. (And I can also add/remove existing datasets by editing the group.) I wonder if organizations should work the same way, instead of having an Organization drop-down when creating or updating a dataset. The add dataset page needs to somehow indicate that you're adding a dataset to a certain group or organization though, doesn't currently." seanh 2990 Fix descriptions of groups and organizations on /groups and /organizations pages ckan ckan 2.0 enhancement new 2012-10-15T11:04:51Z 2012-10-15T11:04:51Z seanh 2991 Add blocks to header.html template ckan ckan 2.0 enhancement new 2012-10-16T11:10:20Z 2012-10-16T11:10:20Z Currently if you want to customise the header.html template you have to copy the whole thing, it isn't broken up into blocks like other templates are. seanh 2994 Add dataset and user popovers throughout the site ckan ckan 2.0 enhancement new 2012-10-17T14:26:04Z 2012-10-17T14:26:04Z "If you hover the mouse over a dataset or user in an activity stream, you now get a popover with some info about that dataset or user and a follow/unfollow button. These popovers should appear wherever datasets or users are listed, throughout the site." seanh 2995 Popovers for resources, groups, organizations, tags... ckan ckan 2.0 enhancement new 2012-10-17T14:27:21Z 2012-10-17T14:27:21Z "If you hover the mouse over a dataset or user in an activity stream, you now get a popover with some info about that dataset or user and a follow/unfollow button. It would be nice to add similar popovers for other types of object in CKAN, e.g. resources, groups, organizations, tags..." seanh 2997 Add activity streams to dataset pages ckan ckan 2.0 enhancement new 2012-10-17T14:31:46Z 2012-10-17T14:31:46Z Add an activity stream tab to dataset pages, like we have on user profile pages. Dataset activity streams are already implemented in the backend. seanh 2998 Add activity streams to group pages ckan ckan 2.0 enhancement new 2012-10-17T14:33:14Z 2012-10-17T14:33:14Z Before we do this we should implement smarter group activity streams (i.e. what activities should appear in a group's activity stream?) and following of groups. seanh 2999 Add activity streams to organization pages ckan ckan 2.0 enhancement new 2012-10-17T14:34:17Z 2012-10-17T14:34:17Z Before we do this we need to merge the organizations branch into master, we need to implement smarter activity streams for organizations, and we need to implement following of organizations. seanh 3000 Add number of followers to dataset pages ckan ckan 2.0 enhancement new 2012-10-17T14:36:00Z 2012-10-17T14:36:00Z User profile pages now show some interesting stats about the user: num. datasets, num. edits and num. followers. Would be nice to add some similar info to dataset pages. At least num. followers can be done (already implemented in backend). seanh 3005 Add following/unfollowing of groups: model, API, frontend, tests ckan ckan 2.0 enhancement seanh new 2012-10-23T17:35:44Z 2012-10-23T17:35:44Z seanh 3007 Adding a dataset creates multiple activities ckan ckan 2.0 enhancement new 2012-10-29T13:15:07Z 2012-10-29T13:23:39Z Adding a new dataset creates multiple activity stream activities, e.g. seanh created the dataset foo, seanh add the resource bar to the dataset foo, and seanh updated the dataset foo are all created when I add a new dataset. I wonder if these can be collapsed into a single activity. seanh 3010 Pin images don't appear in data explorer ckan ckan 2.0 enhancement new 2012-10-31T17:45:06Z 2012-10-31T17:45:06Z "When minified files are in use (set debug = false in ini file) the pin images for points don't show up in the data explorer map view when viewing a resource with geolocation data. set debug = true will fix the problem by using unminified files instead but also turns on a bunch of debug stuff you wouldn't want on a production site. middleware.py, around line 73, is where the decision to turn minified files on or off based on the debug setting is made. This can be hacked on a production site to run with unminified files but otherwise run on production mode thereby getting around the issue with the pins. But that doesn't fix the underlying bug." seanh 3016 CKAN 2.0 template tweaks ckan ckan 2.0 enhancement johnmartin new 2012-11-13T13:30:17Z 2012-11-13T13:30:17Z Just a ticket to keep track of a few suggested template changes. johnmartin 3018 Load more in activity streams ckan ckan 2.0 enhancement johnmartin new 2012-11-14T13:34:11Z 2012-12-10T12:02:30Z Activity streams should be able to load more than 15 items within them. Suggest the default amount of loading is around 30 and then click to load more. johnmartin 3028 Feature: dashboard activity stream filtering ckan ckan 2.0 enhancement seanh new 2012-12-10T12:07:24Z 2012-12-10T12:07:24Z https://github.com/okfn/ckanext-pdeu/issues/13 seanh 2915 Refactor form_to_db_schema_options() ckan ckan 2.0 refactor new 2012-09-12T10:37:49Z 2012-09-12T10:59:48Z "Having two methods form_to_db_schema(self) and form_to_db_schema_options(self, options) seems unnecessary, why not just have form_to_db_schema(self, options=None)? Fixing this might break existing extensions although I don't think any are using form_to_db_schema_options() so it should be okay. The same thing goes for db_to_form_schema_options(). Also why are we passing a dict 'options' that always contains the same three keys 'api', 'context' and 'type'? Why not three params api=None, contenxt=None, type=None?" seanh 2981 Remove config from jinja templates ckan ckan 2.0 refactor new 2012-10-15T10:46:00Z 2012-10-15T10:46:00Z "From Toby: remove the config from jinja2 templates get all such stuff via g. (before 2.0) " seanh 2982 Move functionality from controllers into template helpers ckan ckan 2.0 refactor new 2012-10-15T10:46:42Z 2012-10-15T10:46:42Z "From Toby: strip more functionality from controllers and add via template helpers eg activity streams " seanh 2983 Refactor lib/base.py to remove circular import issues ckan ckan 2.0 refactor new 2012-10-15T10:47:24Z 2012-10-15T10:47:24Z "From Toby: c) refactor lib/base.py to remove the circular import issues (render functions to lib/render.py - would fix much of that) " seanh 2984 Auto-populate context with user, model, session, etc. ckan ckan 2.0 refactor new 2012-10-15T10:48:21Z 2012-10-15T10:48:21Z "From Toby: make the context auto-populated with things like user/model/session if not user supplied - especially for extensions (pre 2.0) maybe via a helper or else a decorator on the action - helper seems nicer maybe done via get_action()" seanh 2985 Make lib/helpers.py more template-specific ckan ckan 2.0 refactor new 2012-10-15T10:49:48Z 2012-10-15T10:49:48Z "From Toby: make lib/helpers more template specific and less generally shared with none template uses (some functions are shared but many shouldn't be) - again help avoid circular imports " seanh 2986 Make lib/base.py more like it was originally intended ckan ckan 2.0 refactor new 2012-10-15T10:50:45Z 2012-10-15T10:50:45Z "From Toby: make lib/base.py be more like it was originally intended eg defines stuff that is shared eg _ but maybe try to include stuff like render/get_action if the circular import stuff can be solved - which should be doable if hard and likely needs an extra file or two for the externals _ etc " seanh 2987 Remove all direct calls to logic action and auth functions ckan ckan 2.0 refactor new 2012-10-15T10:51:27Z 2012-10-15T10:51:27Z From Toby seanh 1182 "Comments from deleted packages appear in ""Recent Comments"" feed" ckan ckan-backlog defect new 2011-06-09T22:24:11Z 2012-06-15T15:31:59Z "When a package has been deleted, say for spam moderation, comments still appear in the recent comments section. This is a problem because non-admin users will be shown a warning that they're not authorised to view the package if they click on the link. At CKAN.net currently, this affects the most recent comment." timmcnamara 1185 Administrators can't delete packages from web UI ckan ckan-backlog defect new 2011-06-15T04:23:38Z 2012-06-15T15:31:29Z "Administrators have ""View"", ""Edit"" and ""History"" tabs. However, I can't see a way to delete a package from the web UI. Version: CKAN.net as of today" timmcnamara 1203 "Moderated edits: html code shows as ""changed"" although it is not" ckan ckan-backlog defect johnglover new 2011-06-28T15:29:30Z 2011-08-15T09:46:19Z "I've installed the Moderated Edits extension (ckanext-moderatededits) and am editing a package imported from IATIregistry.org, with an extra field which contains a bit of HTML. The editor indicates the field has changed, although the content hasn't (see screenshot). All I can find so far is a minor difference: in the field content, there is a code — and in the rendered table that is an —" rolf 1355 Package extras property does not include the newly created ones ckan ckan-backlog defect new 2011-09-26T10:48:46Z 2012-06-15T15:27:36Z "The extras in the package object sent to the extensions after editing (https://bitbucket.org/okfn/ckan/src/01efd5649c10/ckan/logic/action/update.py#cl-226) do not include the newly added. " amercader 1642 Extra link generators generate garbled HTML ckan ckan-backlog defect new 2012-01-12T15:29:31Z 2012-06-15T15:26:07Z "I had a package descriptions with URLs that contain ""group:foo"". This produces garbled output as the system tries to generate two sets of links: the outer link and an inner link. Need to fix the parser. Text: Webdienst basierende Bereitstellung von Geobasisdaten der Freien und Hansestadt Hamburg. Folgende Geobasisdaten werden als WebMapTileService (WMT-S) für die Dauer des Wettbewerbs netzbasiert unter der Creative Commons Lizenz zur Verfügung gestellt: Digitale Orthophotos 40 cm Auflösung (Layer: apps4d_DOP40), Digitale Stadtkarte (Layer: apps4d_DISK), Digitale Regionalkarte (Layer: apps4d_DIRK), Digitale Karte 1:5000 (Layer: apps4d_DK5). Metadateneinträge zu den Daten im PortalU: * [apps4d_DOP40](http://www.portalu.de/trefferanzeige?docuuid=D6082285-6CAC-49C5-B7ED-B5A72BC17DEC&plugid=/kug-group:kug-iplug-udk-db_hh) * [apps4d_DISK](http://www.portalu.de/trefferanzeige?docuuid=D2DA3359-3DC9-4CA4-AE9B-7C63D53B220F&plugid=/kug-group:kug-iplug-udk-db_hh) * [apps4d_DIRK](http://www.portalu.de/trefferanzeige?docuuid=B7DDD599-F677-4C80-BED5-F11D82D5D1DD&plugid=/kug-group:kug-iplug-udk-db_hh) * [apps4d_DK5](http://www.portalu.de/trefferanzeige?docuuid=2AE6D23E-48A5-4D85-BC0A-160737E0C8D2&plugid=/kug-group:kug-iplug-udk-db_hh) One fix is quoting the URLs" pudo 1668 repoze version discrepency ckan ckan-backlog defect new 2012-01-17T12:02:26Z 2012-01-17T12:02:26Z "There's a discrepency in repoze.who versions between the source and package installs: * repoze.who - package 1.0.18 vs source 1.0.19 * repoze.who-friendlyform - package 1.0b3 vs source 1.0.8 We get a test failure [1] with the 1.0b3 version (from the ubuntu 10.04 python-repoze.who-plugins package). But we've not noticed any problems on s057 instances (br, no, ie etc) which have the package versions of repoze.who. The reason the package install uses the earlier packaged versions rather than the ones we'd like is that repoze uses all sorts of horrendous import hacks, making it too difficult to put into our 'ckan-conflict' source package. James suggests we 'do something horrible like dynamically patch repoze on CKAN import'. [1] http://buildbot.okfn.org/builders/builder-ckan/builds/1371/steps/shell/logs/stdio ERROR: ckan.tests.functional.test_user.TestUserController.test_user_create_unicode" dread 2197 Storage Metadata API: add/update not working with local file storage (Pairtree) ckan ckan-backlog defect assigned 2012-02-28T09:29:37Z 2012-09-03T08:48:48Z "If OFS is configured with Pairtree to use a local file storage, all POST requests to add/update metadata ( /api/storage/metadata/{label} ) will fail. This is due to the use of [https://github.com/okfn/ofs/blob/master/ofs/remote/botostore.py BotoOFS] specific ''private'' methods in [https://github.com/okfn/ckan/blob/master/ckan/controllers/storage.py StorageAPIController.set_metadata()], eg: self.ofs.'''_require_bucket'''(bucket), self.ofs.'''_get_key'''(b, label), self.ofs.'''_update_key_metadata'''(k, metadata) ... those methods can't be found in [https://github.com/okfn/ofs/blob/master/ofs/local/pairtreestore.py POTFS] and this causes errors. The API should use only [https://github.com/okfn/ofs/blob/master/ofs/base.py OFSInterface] methods, or should conditionally make calls based off the actual type of ''self.ofs''. ''PS: I did set ""ckan"" as ""Component"" in the ticket because storage has been integrated back into the core in CKAN 1.6''" zydio 2486 Should be able to use . in dataset names ckan ckan-backlog defect new 2012-06-02T16:46:23Z 2012-06-15T14:36:18Z seanh 2607 'Upload a file' appears on resource form when storage not enabled ckan ckan-backlog defect assigned 2012-06-27T13:30:08Z 2012-09-03T09:09:43Z "if the user tries to upload a file they will get ""Failed to get credentials for storage upload. Upload cannot proceed"" Maybe add a test for it this time, this bug has appeared and reappeared before" seanh 737 Markdown syntax summary page ckan ckan-backlog enhancement new 2010-10-22T16:59:09Z 2011-12-06T11:17:19Z "I suggest we produce a quick Markdown cheat-sheet page, showing the key runes: e.g. create a title and quote some text. This page can link to the full Markdown docs for advanced users. A user going to the Markdown docs that we link will have to read a couple of pages of the raison-d'etre of Markdown before he gets to the syntax. And it's not very easy to read, and being white on black it looks like proper geek stuff. " dread 909 DCat importer for CKAN lod2 ckan-backlog enhancement assigned 2011-01-17T12:05:58Z 2012-09-03T10:53:22Z "Write an importer that supports most well known variants of DCat in importing the data as CKAN packages. In particular, the following sources should be supported: * CKANrdf generated exports * opengov.se RDF (not really DCat) * Sunlight Nationdatacatalog as harvested by dcat-tools. " pudo 979 Edit Resource extras in the API ckan ckan-backlog enhancement kindly assigned 2011-02-11T13:09:37Z 2011-09-05T11:30:44Z Follows on from #826. We can now edit resource extras in the WUI (to some extent - see #978 for remaining issues) and we can view resource extras in the API, but we can't yet edit them in the API. dread 1009 Improvements to user accounts sytem ckan ckan-backlog enhancement pudo new 2011-02-25T12:13:11Z 2011-07-08T12:36:14Z " * Forgot password (email a new password) * Confirm email * Do not show register page if you are logged in (redirect to home page) * ticket:1010 - listing of users. * Do not use /user for general user account home page (either user normal user page /user/{id} or /user/myaccount) " rgrp 1077 Move to simpler vdm system ckan ckan-backlog enhancement kindly new 2011-04-08T23:14:24Z 2012-06-18T15:45:45Z "== Option 1: 'Changeset' Model == See ticket:1135 for vdm ticket. This would involve a) moving to changeset in vdm b) doing the migration in ckan to support this. Have developed a new ""changeset"" based model for revisioning in vdm. === Implementation === * The main challenge with this change is schema and data migration Every revisioned object has a revision_id and revision attribute. Approximate algorithm: {{{ Revision -> Changeset for revtype in [PackageRevision, ...]: for pkgrev in package_revision: changeset = lookupchangeset(package_revision) ChangeObject(cset, (table, id), dictize(pkgrev)) }}} Question: * does pkg include tags attributes or not? or we have to dictize, pkgrev, pkg2tagrev, and tag. Probably the latter. == Option 2: Simplify Revision Object Model == Just use a simpler vdm, see ticket:1136 (move to !SessionExtension) and ticket:1137 (remove need for statefulness in vdm). == Discussion == Advantage of Option 1 versus 2: * Easier support for pending state and similar behaviour * No need to introduce new tables (and hence migrations) when making something revisioned (or not). === Disadvantages === * Migration is required * More difficult to query revision history. * Could be addressed by having !ChangeObject have separate cols for table name and id but would likely be more difficult. * Performance (?) * Have one big !ChangeObject table to query when looking at changed objects rather than many revision tables. * Not sure this is a biggie as even with Revision model biggest revision object tables are probably on the order of the !ChangeObject table == Conclusion == Implement Option 2 and leave Option 1 for present. Option 1 includes Option 2 so it seems that that is required in either case (so we may as well with Option 2). Option 1 requires significant effort (esp migration) so leave for present and then review the situation at some later date." rgrp 1101 Integrate googlanalytics into site nav ckan ckan-backlog enhancement new 2011-04-21T13:52:06Z 2012-06-15T15:36:22Z "There's a stats plugin (e.g. at http://trac.ckan.org/ticket/832). Output from the googleanalytics plugin should append to that page, if the stats plugin is present. Possibly the stats plugin and the googleanalytics plugin should be merged? Finally, if the stats plugin is active, then a link to the stats page should be added to the main site footer." sebbacon 1120 Atom feeds of each tag ckan ckan-backlog enhancement new 2011-05-02T04:08:05Z 2012-06-15T15:36:08Z "Tags could/should have an Atom feed. This would mean that every edit to relevant packages could be easily monitored. See [1]. [1] http://lists.okfn.org/pipermail/ckan-discuss/2011-May/001162.html" tsm 1144 Support DSPL ckan ckan-backlog enhancement new 2011-05-19T00:06:49Z 2012-06-15T15:35:17Z "DSPL, the Dataset Publishing Language, is being promoted by Google for its ""Google Public Data Explorer"" system. It is an XML format with metadata. The format is described on the [http://code.google.com/apis/publicdata/docs/developer_guide.html developer docs ofthe Google Code site]. Google provides a [http://code.google.com/apis/publicdata/docs/dsplgen.html Python script which reads CSV data and generates DSPL] Sample from http://code.google.com/apis/publicdata/docs/dspl_sample.html: {{{ #!xml <?xml version=""1.0"" encoding=""UTF-8""?> <dspl xmlns=""http://schemas.google.com/dspl/2010"" xmlns:geo=""http://www.google.com/publicdata/dataset/google/geo"" xmlns:geo_usa=""http://www.google.com/publicdata/dataset/google/geo/us"" xmlns:time=""http://www.google.com/publicdata/dataset/google/time"" xmlns:quantity=""http://www.google.com/publicdata/dataset/google/quantity"" xmlns:entity=""http://www.google.com/publicdata/dataset/google/entity""> <import namespace=""http://www.google.com/publicdata/dataset/google/time""/> <import namespace=""http://www.google.com/publicdata/dataset/google/quantity""/> <import namespace=""http://www.google.com/publicdata/dataset/google/entity""/> <import namespace=""http://www.google.com/publicdata/dataset/google/geo""/> <info> <name> <value>My statistics</value> </name> <description> <value>Some very interesting statistics about countries</value> </description> <url> <value>http://www.stats-bureau.com/mystats/info.html</value> </url> </info> <provider> <name> <value>Bureau of Statistics</value> </name> <url> <value>http://www.stats-bureau.com</value> </url> </provider> <topics> <topic id=""geography""> <info> <name><value>Geography</value></name> </info> </topic> <topic id=""social_indicators""> <info> <name><value>Social indicators</value></name> </info> <topic id=""population_indicators""> <info> <name><value>Population indicators</value></name> </info> </topic> <topic id=""poverty_and_income""> <info> <name><value>Poverty & income</value></name> </info> </topic> <topic id=""health""> <info> <name><value>Health</value></name> </info> </topic> </topic> </topics> <concepts> <!-- As noted in the tutorial, this concept should extend quantity:amount.--> <concept id=""population""> <info> <name> <value>Population</value> </name> <description> <value>Size of the resident population.</value> </description> </info> <topic ref=""population_indicators""/> <type ref=""integer""/> </concept> <!-- This country concept is defined for educational purposes only. A country concept exists in the Google geo dataset. See: http://code.google.com/apis/publicdata/docs/canonical/geo.html --> <concept id=""country"" extends=""geo:location""> <info> <name> <value>Country</value> </name> <description> <value>My list of countries</value> </description> </info> <type ref=""string""/> <property id=""name""> <info> <name><value xml:lang=""en"">Country name</value></name> <description> <value xml:lang=""en"">The official name of the country</value> </description> </info> <type ref=""string""/> </property> <table ref=""countries_table""/> </concept> <!-- This US state concept is defined for educational purposes only. A US state concept exists in the Google geo US dataset. See: http://code.google.com/apis/publicdata/docs/canonical/geo.us.html --> <concept id=""state"" extends=""geo:location""> <info> <name> <value>State</value> </name> <description> <value>US states</value> </description> </info> <type ref=""string""/> <property concept=""country"" isParent=""true""/> <table ref=""states_table""/> </concept> <concept id=""gender"" extends=""entity:entity""> <info> <name> <value>Gender</value> </name> <description> <value>Gender, Male or Female</value> </description> <pluralName><value>Genders</value></pluralName> <totalName><value>Both genders</value></totalName> </info> <type ref=""string""/> <table ref=""genders_table""/> </concept> <concept id=""unemployment_rate"" extends=""quantity:rate""> <info> <name> <value>unemployment rate</value> </name> <description> <value>The percent of the labor force that is unemployed, not seasonally adjusted.</value> </description> <url><value>http://www.bls.gov/cps/cps_htgm.htm</value></url> </info> <topic ref=""social_indicators""/> <type ref=""float""/> <attribute id=""is_percentage""> <type ref=""boolean""/> <value>true</value> </attribute> </concept> </concepts> <slices> <slice id=""countries_slice""> <dimension concept=""country""/> <dimension concept=""time:year""/> <metric concept=""population""/> <table ref=""countries_slice_table""/> </slice> <slice id=""states_slice""> <dimension concept=""state""/> <dimension concept=""time:year""/> <metric concept=""population""/> <metric concept=""unemployment_rate""/> <table ref=""states_slice_table""/> </slice> <slice id=""countries_gender_slice""> <dimension concept=""country""/> <dimension concept=""gender""/> <dimension concept=""time:year""/> <metric concept=""population""/> <table ref=""countries_gender_slice_table""/> </slice> </slices> <tables> <table id=""countries_table""> <column id=""country"" type=""string""/> <column id=""name"" type=""string""/> <column id=""latitude"" type=""float""/> <column id=""longitude"" type=""float""/> <data> <file format=""csv"" encoding=""utf-8"">countries.csv</file> </data> </table> <table id=""countries_slice_table""> <column id=""country"" type=""string""/> <column id=""year"" type=""date"" format=""yyyy""/> <column id=""population"" type=""integer""/> <data> <file format=""csv"" encoding=""utf-8"">country_slice.csv</file> </data> </table> <table id=""states_table""> <column id=""state"" type=""string""/> <column id=""name"" type=""string""/> <column id=""country"" type=""string""> <value>US</value> </column> <column id=""latitude"" type=""float""/> <column id=""longitude"" type=""float""/> <data> <file format=""csv"" encoding=""utf-8"">states.csv</file> </data> </table> <table id=""states_slice_table""> <column id=""state"" type=""string""/> <column id=""year"" type=""date"" format=""yyyy""/> <column id=""population"" type=""integer""/> <column id=""unemployment_rate"" type=""float""/> <data> <file format=""csv"" encoding=""utf-8"">state_slice.csv</file> </data> </table> <table id=""genders_table""> <column id=""gender"" type=""string""/> <column id=""name"" type=""string""/> <data> <file format=""csv"" encoding=""utf-8"">genders.csv</file> </data> </table> <table id=""countries_gender_slice_table""> <column id=""country"" type=""string""/> <column id=""gender"" type=""string""/> <column id=""year"" type=""date"" format=""yyyy""/> <column id=""population"" type=""integer""/> <data> <file format=""csv"" encoding=""utf-8"">gender_country_slice.csv</file> </data> </table> </tables> </dspl> }}} " timmcnamara 1145 Support the Handle System ckan ckan-backlog enhancement new 2011-05-19T00:26:15Z 2012-06-15T15:35:02Z "The Handle System is an initiative to provide persistent references for resources. That is, it's basically a proxy system for preventing link rot. Its documentation is here: http://www.handle.net/. Servers running CKAN could host a ""Local Handle Service"", which redirects a hash of a resource to an actual URL. Some suggested use cases: - Researcher would like to cite where data came from - Agencies would like to have a way to prevent vendor lock-in from CKAN if they decide to move to another platform" timmcnamara 1179 Support tag aliases ckan ckan-backlog enhancement new 2011-06-07T06:47:01Z 2012-06-15T15:32:12Z "A small number of tags are near-duplicates of each other. Perhaps we could support word stemming from NLTK and/or manual tag aliases: statistics statistik ... survey surveying surveys" timmcnamara 1184 Support Wuala as CKAN storage option ckan ckan-backlog enhancement new 2011-06-14T06:59:11Z 2012-06-15T15:31:46Z "Most of CKANs storage options are tied to the USA. This brings concerns of data security for some organisations who may wish to adopt the system. [http://wua.la Wuala] is a distributed file system that stores data in a peer-to-peer manner. The company behind it, LaCie sells storage for a fee. However, they also enable clients to have 'free' storage space when machines act as a storage node. In order to be a storage node, a machine needs to be online for more than 14% of the time - roughly 4h per day. Most CKAN servers are likely to have a far greater uptime than this. Supporting Wuala would go some way to enabling CKAN to be used in a secure manner. That is, CKAN could be promoted for organisational use where there is lots of data to be stored and large geographic distances to be managed. There is a [http://www.wuala.com/Wuala%20API/Sample%20Clients/Python/wualaAPI/ Python client available] and a fairly [http://www.youtube.com/watch?v=3xKZ4KGkQY8 long Google Tech Talk] that overviews the system." timmcnamara 1188 Allow diffing against initial (blank) package version ckan ckan-backlog enhancement new 2011-06-15T15:52:40Z 2012-06-15T15:31:15Z "Currently the history page only allows diffing between different versions of a package, but there doesn't appear to be any easy way to see the changes introduced by the first version of a package. I'm requesting the ability to diff against a ""blank slate"" initial state of a project, so I can see the content of the first project commit. Not sure if this is a vdm feature, so I'm putting this ticket in against ckan." nickstenning 1201 seperate out logic in atom feeds to logic layer. ckan ckan-backlog enhancement new 2011-06-24T15:21:32Z 2011-07-08T11:24:57Z Simplify the logic in the atom feed an make all feeds use logic layer to return lists. kindly 1227 Display packages' tags in search results ckan ckan-backlog enhancement new 2011-07-19T00:11:02Z 2012-06-15T15:29:07Z "In when displaying search results, it would be useful to also display the tags of a package. Sometimes it's difficult to infer the scope of what the package does from the title and the first sentence of the description. Tags are quite concise way to display rich information. ENV=datacatalos.org, with CKAN 1.4.2a" timmcnamara 1232 [super] Interface improvements ckan ckan-backlog enhancement new 2011-07-20T16:11:36Z 2011-09-13T21:15:36Z "Child tickets: * #1194 ""Welcome back"" message for newly registered user * #1202 Links to datapkg utility don't lead to info about it * #925 Change the search box icon to remove the down arrow * #923 Search box doesn't work in leaderboard page in stats extension * #1034 Flash message cached * #737 Markdown syntax summary page * #811 Extra field editing form layout breaks when there are long field names " thejimmyg 1233 [super] Improve wiki-style functionality for history ckan ckan-backlog enhancement new 2011-07-20T16:29:52Z 2011-09-13T21:17:48Z "At the moment we have a good revisioning system but a poor history interface. We need to improve this in a number of areas: * #191 Searching by modification date * #193 Searching by time-related field * #301 Package discussion pages * #1236 Package history page should provide links to pages at particular revisions, similar to the wikipedia pages * #1236 Viewing old revisions or unmoderated changes should have a message at the top of the package page * Other improvements as per my word doc." thejimmyg 1240 [super] API v4 ckan ckan-backlog enhancement kindly assigned 2011-07-24T16:41:00Z 2012-01-02T03:01:52Z "(Just creating this ticket as somewhere to keep notes) * Decide on REST api versus action API * Do we want to support both? * Tidying * Unify on /api/v{version num}/... structure (do we want a default option that points to current default? e.g. /api/default/ ...) * extras merged into normal field list in package * Get rid of /rest/ so just have api/v1/package * Get rid of separation of search api from 'rest' api * Propose that GET on REST index is search e.g. /package/?q=... * This is also resolves issue whereby GET at root returns whole package set (a *bad* idea) as this would now become the matchall search query (with a default limit on items returned) * Resource read/write in API (separate from package) * Does this need authorization work? * user/account API - read/write * Remove autocomplete -- can just use search * Do not worry about backwards compat as should only be used in our js (if others using it too bad!) " rgrp 1255 Drupal consistancy checks. ckan ckan-backlog enhancement kindly assigned 2011-08-01T17:32:48Z 2011-08-15T09:20:54Z Make a robust way to make sure the drupal database is consistent with the ckan data. kindly 1278 Refactor authorized_query calls ckan ckan-backlog enhancement new 2011-08-15T13:32:57Z 2011-08-15T13:32:57Z "There are some functions that still use the Auhtorizer().authorized_query method: {{{ ./ckan/controllers/authorization_group.py:24: query = ckan.authz.Authorizer().authorized_query(c.user, model.AuthorizationGroup) ./ckan/lib/base.py:237: groups = ckan.authz.Authorizer.authorized_query(c.user, model.Group, ./ckan/lib/search/sql.py:55: q = authz.Authorizer().authorized_query(username, model.Group) ./ckan/lib/search/sql.py:118: q = authz.Authorizer().authorized_query(self.options.get('username'), model.Package) ./ckan/logic/action/get.py:154: query = Authorizer().authorized_query(user, model.Group, model.Action.EDIT) ./ckan/tests/test_authz.py:158: q = self.authorizer.authorized_query(self.notadmin.name, model.Package) ./ckan/tests/test_authz.py:353: q = self.authorizer.authorized_query(self.notmember.name, model.Package) ./ckan/tests/test_authz.py:357: q = self.authorizer.authorized_query(self.member.name, model.Package) ./ckan/tests/functional/test_authorization_group.py:44: group_count = Authorizer.authorized_query(u'russianfan', model.AuthorizationGroup).count() }}}" amercader 1287 NAVL validation errors - Junk fields should be listed explicitly ckan ckan-backlog enhancement thejimmyg assigned 2011-08-24T16:25:02Z 2012-03-05T23:34:19Z "When you create a package, but specify a key that is not allowed (e.g. 'relationships') then you get error message: {{{ {""__junk"": [""The input field __junk was not expected.""]} }}} It should mention the actual key which is not expected. e.g. {{{ {""relationships"": [""The input field 'relationships' was not expected.""]} }}} Kindly said that James' version of NAVL was better in this respect, so this might be best solved by moving to that." dread 1341 Delete spam users from ckan ckan ckan-backlog enhancement kindly reopened 2011-09-14T10:10:34Z 2011-11-01T09:59:00Z Spam users where added to thedatahub and we need to clean them. kindly 1358 Generate configuration documentation automatically from the deployment_ini_tmpl file ckan ckan-backlog enhancement zephod assigned 2011-09-26T22:32:30Z 2011-10-10T14:43:43Z "At the moment documentation of config options is duplicated between source (deployment_ini_tmpl in ckan/config which is used to generate user ini file) and the docs. Suggest we write a script that automatedly generates reference documentation for the config from the source. May be obsoleted by #277 (some config in db)" rgrp 1403 Refactor groups index page ckan ckan-backlog enhancement zephod new 2011-10-17T10:31:52Z 2011-10-17T10:32:46Z "Groups are listed alphabetically with paging - not an ideal user experience. We would like to list groups in order of 'popularity': The number of datasets they contain. Following this chain of thought, then, it would be nice to rearrange the groups table by clicking on column headers and having it sort by that column. Furthermore, then, we'd like to implement a full-fledged groups search feature (if this is at all feasible). The forthcoming groups refactor will probably have some bearing on this task. " zephod 1406 Re-enable RSS subscriptions ckan ckan-backlog enhancement new 2011-10-17T14:22:07Z 2011-11-10T13:01:28Z "RSS 'subscribe' buttons appeared in many places on the site but were not very helpful. They took (confused) users pointed to the raw feed code, and Google Reader could not understand the feed. Safari, however, could interpret it correctly. Their presentation needs to be clear and consistent; the RSS feed really needs testing in a variety of readers; and we need to decide exactly which items should get a feed. (Package updates? Groups?)" zephod 1411 Force resource format to be lower case (also mimetype) ckan ckan-backlog enhancement zephod new 2011-10-22T21:40:04Z 2011-10-22T21:40:04Z "Format should be lowercase. Automatically lower case (for extra points have a bit of javascript to force lower case when entering). * Even more points: do a update on thedatahub repo to make all format lower case (or script this as an update?)" rgrp 1414 track user log-ins on thedatahub.org ckan ckan-backlog enhancement new 2011-10-24T11:13:02Z 2011-10-24T11:13:02Z "Set up tracking for user logins so that we have stats about how many active users of thedatahub exist want to be able to see who logged in the the last x months" shevski 1423 Edit resources suggestions ckan ckan-backlog enhancement new 2011-10-26T15:11:46Z 2012-05-28T11:14:38Z " * Description vs Name - Edit Resources view is showing the name of the package rather than the description, and a lot (all?) of the packages before the upgrade don't have names, so might be good to swap this round again, e.g.: http://thedatahub.org/dataset/edit/iati-registry * Moving resources - Moving them up or down the list used to be quite useful if you had a lot of resources that you might want to leave on the resources page, but only one or two that were actually current and that you wanted to draw attention to. This doesn't exist any more on CKAN but I think it would be good to add it back in." markbrough 1429 Provide DOIs for datasets in a CKAN instance ckan ckan-backlog enhancement rgrp new 2011-10-30T12:21:45Z 2011-10-30T12:21:45Z "DOI = digital object identifier = http://www.doi.org/ As a Publisher I want a DOI for my dataset so that it can be cited by and linked to by others in a standard and easy way. == Details == * Probably implement as extension rather than core" rgrp 1432 [super] Data processing system for CKAN and Webstore ckan ckan-backlog enhancement new 2011-11-01T10:19:07Z 2012-06-15T15:27:21Z "Super ticket: #1190 A data processing system which utilizes the Webstore. One could get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules)" rgrp 1459 Featured Dataset feature ckan ckan-backlog enhancement rgrp new 2011-11-12T15:50:12Z 2011-11-12T15:50:12Z "Provide way to mark a dataset as featured. Featured database show up on the front page. TODO: detail this more." rgrp 1507 Minor fixes to dataset add on Group edit form - 0.5d ckan ckan-backlog enhancement zephod assigned 2011-12-05T12:33:49Z 2012-05-28T11:40:20Z "Group edit dataset add form needs some work * Dataset name is not cleared when you add * No way to remove item from list of datasets to be added if I make a mistake * (2nd Apr 2012) It now seems that option to add multiple datasets at once has disappeared (perhaps during the CSS/HTML refactor ...) " rgrp 1535 Plump for auth header of: X-CKAN-API-KEY ckan ckan-backlog enhancement new 2011-12-07T17:31:22Z 2012-06-15T15:26:59Z "When using the API, the apikey needs to be supplied in a header called 'Authorization'. Because some proxys / deployments use this header for other things, a configurable header was provided as an alternative, with default ""X-CKAN-API-KEY"". Rufus suggests having *one* way for this. a) making this not configurable any more b) making X-CKAN-API-KEY the default (keep Authorization allowed, but not documented, for backwards compatibility)" dread 1550 Allow simple auth via the API ckan ckan-backlog enhancement assigned 2011-12-16T15:29:29Z 2012-09-03T11:00:55Z It should be possible to pass userid/username and api key and obtain a response from CKAN for external services that use CKAN auth. Those services shouldn't be talking to the DB directly. ross 1571 [super] Issues Extension ckan ckan-backlog enhancement rgrp assigned 2011-12-19T11:30:25Z 2012-03-04T12:13:03Z """Proper"" issues extension. * Etherpad: http://ckan.okfnpad.org/issues * Code: https://github.com/okfn/ckanext-issues Overview * This would be an extension and improvement of existing todo extension - #842 (https://github.com/okfn/ckanext-issues) * Current extension does work but lacks polish * Should we polish or do significant improvements? * Functionality - see etherpad == Steps == * ~~(Rename todo(s) -> issue(s) in existing extension)~~" jilly mathews 1578 [super] Re-enable and refactor ratings ckan ckan-backlog enhancement new 2011-12-19T19:20:43Z 2012-01-02T02:56:55Z rgrp 1598 Reinstate Ratings ckan ckan-backlog enhancement new 2011-12-29T16:52:04Z 2012-01-02T03:26:58Z "Ratings were disabled approximately a year ago because: * Unclear purpose and UX. What did ratings tell you? How useful were they? * Spamming (esp by bots: you could submit an anonymous rating via a GET request which caused problems) Both problems are solvable and it would be nice to have this feature reinstated. * Purpose: can make this more purposable by limiting to logged in users (or at least distinguishing logged in from non-logged in users) * Even better we could allow ratings to be made public (I'm interested in what someone else I respect finds important) * Spamming: limit to logged in users and / or use AJAX over an API to submit ..." rgrp 1604 Get ckanext-moderatededits working with CKAN 1.5+ templates ckan ckan-backlog enhancement new 2011-12-31T17:27:09Z 2011-12-31T17:27:09Z "ckanext-moderatededits requires an old and possibly development version of CKAN. It would be good to update it for later CKAN versions. According to the README, you need CKAN from branch feature-1141-moderated-edits-ajax but the changelog suggests this branch went into version 1.4.2. So it possibly works with 1.4.2 and 1.4.3(.1). But CKAN 1.5 has revamped templates, so the genshi stream filters definitely don't work. BTW history_ajax/read_ajax calls have been deprecated in CKAN since 1.5.2a and will need fixing up to use the Action API too as part of this." dread 1606 metadata license config option ckan ckan-backlog enhancement new 2012-01-02T10:45:30Z 2012-06-15T15:26:21Z "Add a config option to choose the metadata licence. Set default to Open Database License. Currently the dataset edit form says ""Important: By submitting content, you agree to release your contributions under the Open Database License."" This is hard-coded, but not suitable for when DGU uses the CKAN form - they use the OGL." dread 1635 Email notifications (e.g. for activity streams) ckan ckan-backlog enhancement seanh new 2012-01-11T17:56:27Z 2012-12-10T12:05:57Z "CKAN should be able to send email notifications to users. Maybe have a notifications table in the db, and a server-side job that runs periodically and consumes rows from this table, mailing them to the users. One thing that we may want to send users notifications of is activity stream events. So the activity streams code would have to add rows to the notifications table for the mailer job to consume. But remember that email notifications feature is separate from activity streams - we may want to send notifications of other things as well. Need to implement (at least some of) #1634 before this can be implemented, in order to have something to send notifications about. Analysis here: http://ckan.okfnpad.org/27" seanh 1643 Add fixed tags to thedatahub for better browsing ckan ckan-backlog enhancement new 2012-01-12T18:34:53Z 2012-01-12T18:34:53Z Similar to publicdata.eu, want to have themed areas such as finance, environment, census, etc and country tags shevski 1644 Order default dataset page by most downloaded resources on thedatahub ckan ckan-backlog enhancement new 2012-01-12T18:39:02Z 2012-01-12T18:39:02Z "Instead of alphabetically as we do currently, alternatively by most viewed datasets for http://thedatahub.org/dataset" shevski 1652 How we intergrate with Drupal Multiligual? ckan ckan-backlog enhancement kindly assigned 2012-01-16T10:31:34Z 2012-06-20T10:18:55Z " * Analysis of how/where we can integrate with Drupal Multilingual * Is it likely to access CKAN via /en/data/... or /data/en/... (not sure, probably former) and we need to make sure that it ends up at /en/... * Check with Ian for other issues. " ross 1679 Default roles problem ckan ckan-backlog enhancement new 2012-01-17T17:57:22Z 2012-06-15T15:25:23Z "The 'editor', 'anon_editor' and 'reader' roles are intended to have immutable actions. This was designed to prevent their names being subverted - e.g. an editor should always be able to edit! It also meant that when we add Actions (e.g. DELETE-PACKAGE) then it can be added sensibly to these roles in an upgrade just by changing the defaults table (ckan/model/authz.py). The problem is that this immutability is only enforced on 'db upgrade'. So you can happily change the editor role using the paster command and it works, right up until you do an upgrade and realise permissions are different. We should stop the paster commands being able to edit these roles. Or get rid of the immutability completely. Views?" dread 1684 Remove all config from ckanext-archiver ckan ckan-backlog enhancement assigned 2012-01-19T14:37:01Z 2012-09-03T11:00:37Z "ckanext-archiver currently has a settings file (and a default) and it should be passed in all relevant information from the context. Remove all settings (ARCHIVE_DIR and MAX_CONTENT_LENGTH and others) and pass them in from CKAN." ross 1697 A Configurable list of states for a Dataset ckan ckan-backlog enhancement new 2012-01-24T10:23:50Z 2012-05-28T11:23:09Z "Currently have 'active' and 'deleted' suggest also: * 'draft' * 'hidden' (Do we need both). Also write out workflows related to these." rgrp 1717 [super] Search UX improvements ckan ckan-backlog enhancement new 2012-01-26T18:53:01Z 2012-02-24T13:02:19Z "1. Make it possible to search by tag (e.g. by typing tag:csv into the search bar and clicking enter, it should add the 'csv' tag facet to the search) 2. Rename and standardise the list of format tags, on search page this should also be called 'Format' instead of 'res_format' (in the right hand side bar on search page). 3. Make it possible to view full list of tags, formats and groups by clicking on the name. From here you should be table to click on a classification and go back to a search page faceted by that classification. E.g. from search page, click on 'tags', on tag page click on 'london' or whatever, and be navigated back to search page with search within 'london' tag only. Or y'know, a better way of doing it. 4. More standard classifications, such as 'Location' and 'Theme' - like on publicdata.eu 5. Blue search button should be displayed in line with the search bar, not underneath 6. Datasets should be displayed in order of most viewed or downloaded instead of alphabetically. For alphabetic search we could consider adding a way to facet by first letter of dataset name" shevski 1747 Expire old activities ckan ckan-backlog enhancement new 2012-02-05T12:56:29Z 2012-06-15T15:24:19Z Currently the activity streams database tables just get longer and longer over time. Do we want to eventually delete the oldest activities, to keep the length of the table within limits? seanh 1750 Move ckan/lib/activity.py into the model ckan ckan-backlog enhancement new 2012-02-05T18:18:08Z 2012-06-15T15:24:00Z Move ckan/lib/activity.py moved to into the model - say ckan/model/activity_extension.py, because it's so tightly knit with the model code, whereas most of the lib code is used in the controllers. seanh 1789 Implement a tag_update() logic action function ecportal ckan-backlog enhancement new 2012-02-09T16:36:53Z 2012-06-15T15:23:32Z "So users can rename a tag and/or move it between vocabularies using the API. Currently we have create_tag() and delete_tag(), but if you were to 'update' a tag by deleting it and then recreating it all the datasets that had that tag will have lost it and you'll have to re-add it to them all. What should happen to datasets that have the tag, if the tag gets moved between vocabularies? All the datasets just keep the tag with the new vocabulary? This will become a problem if/when we support 'radio button'-style vocabularies (where each dataset can only have one tag from the vocabulary)." seanh 1831 Login with email address ckan ckan-backlog enhancement assigned 2012-02-24T08:58:26Z 2012-09-03T11:08:24Z "Users should be able to log in using either their username, or their email address, both of which are unique. Will require a change to UsernamePasswordAuthenticator in ckan.lib.authenticator.py and possible a useful User.by_email in the user model if it doesn't already exist." ross 2284 Local queries/views/viz in Related stuff. ckan ckan-backlog enhancement assigned 2012-04-12T08:18:20Z 2012-09-03T10:51:53Z "From #2204 not sure how much we have thought through using this for storing queries / views / visualizations coming from our data viewer. In particular, wonder if this necessitates some kind of support for arbitrary json data ... ### Comments In the simplest form, right now all I need to store is store the url to the resource plus hashbang config for recline. That would be super easy ... " ross 2508 Make it possible to run CKAN tests for each language ckan ckan-backlog enhancement new 2012-06-11T10:42:15Z 2012-06-15T14:32:51Z "Mistakes in translated strings can cause CKAN to crash or otherwise not work, but it's not practical to manually test every page and function of CKAN in every language that we have new translations for before a CKAN release. It'd be great if the tests could automatically be run for each language. This is probably a big job, we would have to get the tests to respect a language setting in the ini file, check for any individual test cases that specify the language (e.g. in the URL), and also fix test cases that look for specific English words in HTML output, etc. In the meantime, a good stop-gap solution might be a script that tests for common mistakes in the po files." seanh 2513 Dataproxy should not default to utf8 ckan ckan-backlog enhancement assigned 2012-06-13T08:23:40Z 2012-09-03T10:54:06Z "Unless explicitly told by the source web server the dataproxy should not assume that the content it has can be encoded as UTF-8. Even though the chars from 128 - 255 overlap an attempt to decode some byte array as utf8 will fail whenever a latin1 char whose bitpattern has the MSB set. This will mean that the UTF8Recoder can be more rigid in its acceptance of data, Postel aside." ross 2531 New state option: archived / deprecated ckan ckan-backlog enhancement new 2012-06-15T08:59:47Z 2012-06-15T14:30:49Z "Deleted means things will get purged at some point. Archived means they stay around but get hidden from search results and a big warning notice gets displayed saying this is archived / deprecated. @richard cyganiak ..." rgrp 2554 Research Virtuoso cartridges ckan ckan-backlog enhancement assigned 2012-06-18T13:37:25Z 2012-09-03T11:07:13Z "Look into writing a cartridge for importing CKAN data into a Virtuoso quadstore http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSponger#How Does It Work?" ross 2590 Publisher dashboard ckan ckan-backlog enhancement shevski assigned 2012-06-25T10:03:37Z 2012-09-03T09:10:16Z "Need proper user stories but ... Publisher admins/editors may need a more useful group read page showing things like: * The current search * Recent activity * People within the group * Followers * Others? " ross 2729 searching for tags:[tag] works but tag:[tag] doesn't ckan ckan-backlog enhancement kindly new 2012-07-22T09:58:29Z 2012-07-22T09:59:36Z "which is confusing since you can only search for one tag like this at a time. I.e. tags:economics,cvs or tags:economics, csv or tags:economics+CSV doesn't work for example; therefore tag:economics, should also work! http://s031.okserver.org:2375/dataset?q=tags%3Aeconomics&sort=relevance+asc" shevski 2732 New file upload functionality ckan ckan-backlog enhancement assigned 2012-07-23T15:53:09Z 2012-09-03T09:07:50Z "We should simplify upload and storage of files, initially only to local storage with archiver eventually being fixed to archive data externally. WIP pad is http://ckan.okfnpad.org/uploads '''Simplifying uploads''' Currently uploads are too painful/difficult/fiddly to use and/or configure. We want to simplify uploads so that they are done directly to the CKAN server, without support for remote services (S3 etc) and/or the dependencies it introduces. We want to fix: * File uploads themselves * Storage of uploaded files * Notification of the upload to other components '''File uploads''' Things file upload should do: * Allow sysadmin to disable * Allow auth'ed users to upload * Store whatever they send on disk, and store DB entry linking the file to the person * When creating the resource, the user should be able to choose from all of the files they have uploaded but not yet associated with a resource. This will allow for bulk upload and then a delayed association. Whenver a user creates a resource they either upload a file now, or see previously uploaded files. {{{ Can we do the upload asynchronously and then associate the uploaded key with the resource before the save ? What happens if the user tries to submit before asymc upload finishes ? Should we delay them? }}} The upload workflow should look like... 1. File upload should be a straightforward file upload with normal auth checks and normal processing of the posted data. 2. a. When called via ajax then the ID of the newly created file should be returned, b. When called via WUI then it should also be given the url to redirect to after the file upload has been handled - the id will be passed as a query param. 3. The resource save should check whether it has a file id and in that case updates the file object to point to the resource. This should enable: * Separate file upload into a user's temporary store, either individually or as a batch. * Creating resources and simply choosing from previously uploaded, unassigned files * Adding files/data to a resource after the fact. '''File storage''' File storage should be local to the CKAN install, and not a remote service. Any archiving to remove storage providers should be outside of the main request. File storage should: * allow moving data, a sysadmin should be able to move the storage root and change configuration and have the system continue running (i.e. don't store absolute paths). * provide maintainability, it should be easy to determine which old files are not associated with resources and thus can be cleaned up. * allow for collection of information (i.e. estimate of storate space used) * check whether there is enough space and handling the conequences cleanly * ensure files to be written only underneath its own root folder, checks should be made after any path generation that the file begins with the location of the file storage. * Have a configurable maximum accepted blob size during upload. * Should store what meta-data was provided with the upload, such as mimetype. Somewhere in the DB we should store ... ||'''Column'''||'''Notes'''|| ||id||An identifier|| ||owner||The owning user, who uploaded the file|| ||path||The path (from the 'storage root') to the file|| ||size||The size in bytes of the file on disk|| ||mimetype||The mimetype of the file, as provided by the uploader|| ||upload_date||When the data was uploaded|| ||resource||The ID of the resource it belongs to. A unidirectional relationship.|| ||archived_url||The URL where this file has been archived|| Generating paths should try and separate the files, perhaps based on username of the owner, or some other mechanism to avoid a single folder full of files. '''Notifications''' We need to make sure that it is possible to notify other components within the system that an upload has taken place, or at least make it easy for them to be notified. The primary use case for this is to notify the component that will translate/upload certain formats to the data store. We could do this based on the post-upload update to the file model (i.e. when we record the total received size of the file). " ross 2962 Search across multiple ckan instances ckan ckan-backlog enhancement new 2012-10-08T22:50:22Z 2012-10-08T22:50:22Z "Could be done by: - using the solr distributed search - difficult set up - merging result sets from apis - make sure that results can be merged properly (score, facets, ...)" dominik 653 Trackback links for packages ckan ckan-backlog requirement new 2010-09-21T09:40:25Z 2012-06-15T15:37:16Z "When people link to a package, a track-back link is auto-created. (Similar system as for blogs). As suggested by Tim Davies: Allowing some form of ‘track back’ against datasets When a non-technical user comes to look at a dataset it would be really useful for them to be able to see if anyone has created an interface interpretation of it already. I found quite a few cases in research of end-users struggling to make sense of a dataset when good interfaces to that data had already been built and blogged about, but without there being any link from the dataset listing to those data uses. Accepting track backs could also make it easier for technical users to find blog posts / shared code etc. relating to a given dataset. " dread 818 Rethinking the author and maintainer fields ckan ckan-backlog requirement new 2010-11-17T14:18:44Z 2012-06-15T15:37:01Z "The semantics of the Author and Maintainer fields are really unclear at the moment. This leads to very inconsistent usage. Also, perhaps Name and Email are not the only fields that are needed for a contact. Here is a table that shows the current usage of these fields in CKAN: http://richard.cyganiak.de/2010/ckan/ckan-ppl.html We note several problems: * Author and Maintainer are often the same * Author and Maintainer are often used interchangeably * People really want to specify URLs for the contacts and stick them into random places because there is no field for it * Multiple comma-separated names in a single field I'm not sure what to do about this, but a redesign is necessary in my opinion. Some ideas: * Remove the maintainer field? * Make really clear that Author doesn't refer to the metadata on CKAN, but to the original data * Add an “author URL” field?" cygri 1273 Create docs for API v3 ckan ckan-backlog requirement new 2011-08-15T12:41:23Z 2011-08-15T12:41:23Z amercader 2546 ODS Managing homepage content ckan ckan-backlog requirement assigned 2012-06-18T10:54:02Z 2012-09-03T09:10:37Z " == Requirements == Require the ability for users to control some level of content that is visible on the home page of their ODS installation. This may be through RSS/Atom feeds (see #2234) or another mechanism but should result in admins being able to change blocks of text on their homepage. This should not be configuration, but accessible through WUI. == Interface == None == User Stories == * As a system administrator I want to have control over content displayed on the front page beyond featured/popular items. * As a system administrator I don't want to manage content through having to write an extension. == Tasks == [ ] Analysis == Estimates == " ross 1384 CKAN wiki needs updating to refer to thedatahub.org instead of ckan.net and datasets instead of packages ckan ckan-backlog task rgrp new 2011-10-12T10:07:57Z 2011-10-12T10:07:57Z Most articles still refer and link to ckan.net, wiki.ckan.net and to packages shevski 2960 Private datasets are not private ckan ckan-backlog task new 2012-10-08T11:04:15Z 2012-10-08T11:04:15Z At the moment everyone can read all resources. There is no support for hiding resources that are in private datasets. dominik 2524 If there are no translation files for selected language fall back to default lang. ckan ckan-ecportal enhancement kindly new 2012-06-13T17:37:28Z 2012-06-19T14:53:28Z If a user selects a language there are no mo files for then an error is raised. Revert to default language instead. kindly 1096 [super] CKAN Hosted ckan ckan-future defect rufuspollock new 2011-04-19T17:44:22Z 2012-06-15T15:34:44Z "Many users of CKAN want to have their own instance without much effort. Setting these up in separate places is a maintenance nightmare, we should much rather have some tenant separation in core CKAN. Some ideas: * introduce model.Site and c.site * site has: custom CSS, extra_template_path, title, languages list, package_form, group_form (all configured via web UI) * Subdomain detector to activate sites. * use site in Authorizer instead of System, have a NullSite for global things * allow cross-site search * packages are in a list of sites, m:n rather than 1:n * list of sites is string-based, can contain sites not in site table to express harvested external material which is not editable locally. " pudo 1457 Bug with DataNL instance ckan ckan-future defect new 2011-11-10T11:37:21Z 2011-11-10T11:37:21Z " ""when logging into http://register.data.overheid.nl/ with OpenID, the /user/me page gives a 404. CKAN version 1.3.4."" n the manual it says an API key kan be created via http://test.ckan.net/user/me /. However, when I try the corresponding http://register.data.overheid.nl/user/me, I get a 404 error (not found). " jilly.mathews 1165 Add multi-site support to ckan ckan ckan-future enhancement new 2011-05-26T12:41:07Z 2012-06-15T15:34:26Z "Currently, each ckan site needs its own ckan wsgi process. That eats a lot of resources where many ckan sites are served from one machine (e.g. eu3). That would dramatically change if a ckan process could behave like multiple ckans (e.g. like Apache's ""<VirtualHost>"", or tracd). Depending on the ""Host:"" header in the HTTP1.1 request, it would choose which local ckan ini file to obey. I see two ways to constitute the map hostname-to-ini-file map: * ckan reads a set of ini files, and each ini file declares which servers names it is responsible for * In a global ini file, there are directives mapping servernames to ini files. In either case there should be a global ckan ini having the default settings for all local ckan sites. Each site ini could be very short then, just having e.g. title, name, database credentials, active plugins etc." nils.toedtmann 1458 Support previewing kml files in data viewer ckan ckan-future enhancement rgrp assigned 2011-11-10T14:48:08Z 2012-06-25T14:02:12Z "Super ticket: #1151 (viewing geo data) E.g. preview of http://thedatahub.org/dataset/louisvillecrime should bring up a map" rgrp 1557 Complete Webstore Preview Extension ckan ckan-future enhancement David Rasnik new 2011-12-19T10:40:53Z 2011-12-19T10:40:53Z "Finish any work out standing on web store preview extension to be able to package and release. Ref James and I going through existing features and trying to mention any polishing that needed doing to get exiting features ready for release with projects such as CKAN hosted." jilly mathews 1558 Publisher Tools ckan ckan-future enhancement David Raznik new 2011-12-19T10:46:13Z 2011-12-19T10:46:13Z Summarise final set of requirements for this and finish development and test. Estimated 10 working days. jilly mathews 1560 Follow extension ckan ckan-future enhancement David Raznik new 2011-12-19T10:51:19Z 2011-12-19T10:51:19Z "Estimate 2 days to finish dev and test. David can you add any info needed here?" jilly mathews 1561 To do extension ckan ckan-future enhancement David Raznik new 2011-12-19T10:52:52Z 2011-12-19T10:52:52Z Can we finish this ready for release on data hub and CKAN Hosted. jilly mathews 1562 Finish Geo Spatial ckan ckan-future enhancement Adria new 2011-12-19T10:56:33Z 2011-12-19T10:56:33Z Estimated 4 weeks of Adria's time. I guess this will need to be broken down into more tickets. This feature is being requested by a number of potential customers and we have some ideas of requirements between Rufus and Jilly for this. This is the most popular new feature we talk about to new clients. jilly mathews 1564 Structured Data (Data API) ckan ckan-future enhancement David Raznik new 2011-12-19T11:07:14Z 2011-12-19T11:07:14Z "Basic websotre exists but this may be not what is described yet. CKAN provides a rich API for the data itself, allowing users to query retrieve and use data instantly from datasets in CKAN without needing to download or process it first." jilly mathews 1565 Admin dashboard finished? ckan ckan-future enhancement Rufus Pollock new 2011-12-19T11:11:32Z 2011-12-19T11:11:32Z Is testing complete and ready for release? jilly mathews 1567 Finish QA extension ckan ckan-future enhancement David Raznik new 2011-12-19T11:19:59Z 2011-12-19T11:19:59Z Requires change to celeryd. Estimated 4 weeks. jilly mathews 1569 Wordpressser ckan ckan-future enhancement David Raznik new 2011-12-19T11:27:36Z 2011-12-19T11:27:36Z How much effort will this be to be ready to use? jilly mathews 1572 Meta data Harvester ckan ckan-future enhancement David Raznik new 2011-12-19T11:35:09Z 2011-12-19T11:35:09Z Need to write custom harvesters for each client. Is it worth having one for data hub? jilly mathews 1573 Apps and Ideas ckan ckan-future enhancement David Raznik new 2011-12-19T11:36:33Z 2011-12-19T11:36:33Z Estimate 2 weeks for someone to finish and test. jilly mathews 1596 Refactor authz roles ckan ckan-future enhancement new 2011-12-22T10:31:28Z 2012-05-28T11:36:59Z "Suggestions from rgrp: * Get rid of Roles, and replace them with direct assignment of actions, even though there are many actions, and extensions can add arbitrary ones. * Debatable whether we should cut the number of actions to correspond to the three roles defined by the base system. * Have a method of finding roles (or, in future, actions) relevant to a given protection object (e.g. FILE-UPLOAD(ER) not relevant to Packages) (This ticket is split off from #1065)" dread 1667 Add an extension point for rendering activity streams ckan ckan-future enhancement seanh new 2012-01-17T10:29:11Z 2012-05-28T11:24:55Z "Currently rendering of activity stream events to HTML works by looking up a rendering function in a dictionary that maps activity types ('new package', 'changed group', etc.) to rendering functions that take an activity stream event and return the rendered HTML. There needs to be an extension point where extensions can register their own rendering functions for particular activity types." seanh 1740 Get rid of `from module import ...` ckan ckan-future enhancement new 2012-02-01T11:06:09Z 2012-02-01T11:14:44Z "It's really bad to do {{{from module import *}}} and CKAN has a lot of them. I suggest a three-pronged approach: 1. Don't add any more of them. 2. When you're programming if you see an easy opportunity to remove one then do so. 3. At some point we should task someone to go through the code and remove them all (which is what this ticket is for), but this will be a big job and may break things. We should also get rid of most or all of the {{{from module import foo}}} and {{{from module import foo, bar}}} statements. I think the right thing to do is just {{{import module}}} and then use {{{module.foo}}} in your code. But if you find yourself doing {{{module.foo.bar}}} then you may have a code smell. See: http://docs.python.org/dev/howto/doanddont.html" seanh 1748 Make activity streams conform to http://activitystrea.ms/ standard ckan ckan-future enhancement seanh new 2012-02-05T14:06:24Z 2012-05-28T11:25:37Z "At the very least we should make our JSON output conform to their spec: http://activitystrea.ms/" seanh 1749 Allow creating activity details through API ckan ckan-future enhancement assigned 2012-02-05T18:10:06Z 2012-05-28T14:05:33Z Currently the activity_create() logic action function only lets you create top-level activity stream items, and not their related activity details. It should handle activity details via nested dicts. seanh 1778 Replace classmethod's with module-level functions ckan ckan-future enhancement seanh assigned 2012-02-06T15:53:34Z 2012-06-25T11:29:03Z "In many places we have @classmethods where simple module-level functions would do (and would be more idiomatic in Python), e.g. the {{{get()}}} classmethods in the model/ classes. Doing it with module functions lets us type module.function() instead of module.Class.method(). In many cases we're importing classes directly with {{{from module import Class}}} and then doing {{{Class.method()}}}, but {{{from module import foo}}} is bad style: http://docs.python.org/dev/howto/doanddont.html" seanh 1782 Improvements to datasets page UX (Feb 2012) ckan ckan-future enhancement zephod new 2012-02-07T10:10:41Z 2012-05-28T11:40:32Z " * Relationships in sidebar [WONTFIX yet without some link to documentation or datastructure, something for me to render?] * ~~Icons for *types* of resources in resource list~~ * List of 'Owners' (administrators) at top right of sidebar (a la group page) * Much prefer owner to administrator ..." rgrp 1787 [super] Improve RESTful API ckan ckan-future enhancement new 2012-02-08T11:54:42Z 2012-02-08T11:54:42Z " * Lists of entities should be full URLs, rather than just the names * Discoverability - /api/v3/rest should list the entity types that can be listed This could be v3 of the RESTful interface." dread 1790 Click to delete tags, rather than have all existing tags in the tag text box ckan ckan-future enhancement new 2012-02-10T15:44:34Z 2012-02-10T15:44:34Z "From Pablo: Editing the tags field is clumsy when there are too many tags. Could show existing effectively as tags (like delicious), then allow clicks to delete. New tags added via text box." dread 1824 Add vocabulary pages ckan ckan-future enhancement new 2012-02-21T17:24:49Z 2012-05-28T11:35:58Z "For a free tag foo you can visit the page at /tag/foo and see a list of all the datasets that have the tag foo, and when the tag appears on dataset view pages etc. it's linked to this tag page. We should do the same thing for vocabulary tags. A tag bar in vocabulary baz should be hyperlinked to a page /tag/baz/bar, or perhaps /vocab/baz/bar." seanh 2200 Add vocabulary_id option to tag_show() logic action function ckan ckan-future enhancement seanh new 2012-02-29T16:41:18Z 2012-05-28T11:35:00Z Currently tag_show() only works with free tags, it's not possible to get a tag_show for a vocabulary tag. seanh 2202 Display page view count on dataset and resource pages ckan ckan-future enhancement reopened 2012-03-03T09:04:15Z 2012-05-28T11:35:29Z Just like we display download counts we should display view counts. rgrp 2234 Write a CKAN extension for pulling items from RSS/Atom feeds into CKAN templates ckan ckan-future enhancement assigned 2012-03-16T12:59:15Z 2012-09-03T10:52:47Z "You configure the extension with some RSS and/or Atom feeds, it automatically reads items from these feeds and makes them available in the template context, you write a custom template to e.g. display 'news' items from a Wordpress blog on your front page. This extension might be simpler and less fragile than ckanext-wordpresser, and also more generally useful. Details - Mark Pilgrim's Universal Feed Parser might be useful for reading the feeds - Feed items should probably be cached somewhere Suggested - The news item 'widget' should be wrapped in a known class so that it can be styled easily regardless of the format of any HTML entry. - For non-HTML formatted items (Atom should tell you the content type of the entry) maybe we should have a template for rendering each item along with any enclosures that it might reference - Caching is pretty crucial and should probably obey the ttl of the feed. " seanh 2235 Group drop down on dataset edit should use chosen and sort groups by name ckan ckan-future enhancement new 2012-03-16T14:15:57Z 2012-05-28T11:32:06Z rgrp 2244 clean up auth ckan ckan-future enhancement toby new 2012-03-19T17:22:51Z 2012-05-28T11:43:22Z "* cache admins etc * aim to move check_access so independent of logic function * general spruce up" toby 2258 Customizable contributor agreement ckan ckan-future enhancement new 2012-03-26T08:45:49Z 2012-05-28T11:32:27Z " * Customize text at bottom of forms * Also need to make clear that this does not apply to the data itself (that is covered by the license you choose on your dataset ...)" rgrp 2265 'More Like This' for a dataset ckan ckan-future enhancement new 2012-03-27T16:20:20Z 2012-06-15T14:42:30Z "When viewing a dataset, it would be nice to show a couple of 'Related Datasets'. i.e. ones that are similar. SOLR has a feature for finding documents similar to a particular document, called 'More Like This'. We would like this for DGU." dread 2282 log api calls for users ckan ckan-future enhancement toby new 2012-04-10T16:18:45Z 2012-05-28T13:13:49Z "This was part of #2251 but is now it's own ticket. we need clarity on this. The use cases so far have been possible limiting api calls for users. so we want to monitor the user or ip address what about the api calls? do we want to record all calls or just some? do we want to record the type of call and to what level of detail? recording the url just seems to specific to me." toby 2287 Consistent datetime and time period strings localization ckan ckan-future enhancement new 2012-04-12T18:08:43Z 2012-06-11T10:44:05Z "Find all places where date, times and time periods are rendered as strings in CKAN. Make them all go through the same code path (e.g. the helper function in helpers.py) with localization support. For example, on the user index page right now there are time period strings like ""less than 1 month"" that currently do not get translated (and the way they're implemented doesn't look i18n-friendly). ISO date format everywhere might be a good idea. Need to consider local timezone issues. Python standard library and Babel may have useful helpers for this. Should the local timezone displayed be a global setting per CKAN instance? Or should it change depending on the location of the user who is viewing the page?" seanh 2378 Extract metadata directly from resources that contain it. ckan ckan-future enhancement assigned 2012-05-10T11:58:08Z 2012-09-03T10:59:45Z "Some files that might be uploaded as resources on a dataset, such as some image formats, already contain metadata. For example some jpeg files might contain Exif ( http://en.wikipedia.org/wiki/Exchangeable_image_file_format ) data. [ ] Obtain list of embedded metadata 'standards' [ ] Where the metadata is available it should be added as extra fields on the resource that is uploaded. [ ] HTML? [ ] Image formats (exif/gif etc) [ ] PDF [ ] Eventually consider ... [ ] MS Office documents " ross 2404 Auto-screenshot for related items ckan ckan-future enhancement assigned 2012-05-21T06:55:47Z 2012-09-03T10:52:06Z "As a User I want an image generated for my related item automatically so that I do not have to take picture myself and upload it somewhere Implementation: (?) Use phantomjs or something like http://snapito.com/ Investigate for 1.8, will be needed for TDH Oct." rgrp 2411 .ini file is a mess ckan ckan-future enhancement toby new 2012-05-22T11:41:40Z 2012-05-28T13:14:32Z The .ini file needs cleaning up in assorted ways which Toby will expand upon. markw 2552 Controlling access to features ckan ckan-future enhancement assigned 2012-06-18T11:26:10Z 2012-09-03T10:52:24Z "== Requirements == To provide a freemium service it is necessary to be able to provide differing __levels__ of functionality based on the '''type''' of user (see #2550). These levels can be specific to the data hub but may require overriding functionality from core to provide these checks. Initial implementation should focus on limiting access to datastore disk space. == Interface == These changes are currently only for the data hub and should be kept as much as possible within the data hub extension. == User Stories == * As a system component I want to find out if the current user has access to a feature (i.e. storage) and if so to what extent (xMb, xGb or unlimited). * As a system administrator I don't expect to need to manage the levels of users '''or''' the features that this applies to. == Tasks == [ ] Clarification of requirements/analysis [ ] Tests [ ] Code [ ] Model [ ] API [ ] UI [ ] Documentation == Estimates == " ross 2573 package_search does not allow solr's per-field facet parameters ckan ckan-future enhancement new 2012-06-19T13:27:19Z 2012-06-25T14:05:01Z "Solr allows its facet parameters to be specified on a per-field basis, eg. `facet.limit` applies to ''all'' facet fields, but solr allows it to be overriden for a specific field, eg. `facet.tags.limit`. We don't support this at the moment because we have a whitelist of valid solr query parameters that we accept. See `ckan.lib.search.query.VALID_SOLR_PARAMETERS`." icmurray 2810 heroku ckan support ckan ckan-future enhancement new 2012-08-07T18:40:58Z 2012-08-07T18:40:58Z Get ckan working on heroku kindly 1800 Tidy up *_list() and *_search() functions in ckan/logic/action/get.py ckan ckan-future refactor new 2012-02-16T15:12:09Z 2012-05-28T11:34:46Z "For consistency all the *_list() functions should list objects only and not accept an optional search query. There should be *_search() functions whenever search is needed. Currently it is inconsistent, e.g. package_list() and resource_list() do not accept a search query option and there are package_search() and resource_search(), but user_list() does accept a query and there is no user_search(). tag_list() also accepts a query, and there is also a tag_search() that apparently duplicates the search functionality." seanh 1815 Reenable Sparql endpoint on publicdata.eu pdeu ckan-future task amercader new 2012-02-20T17:08:43Z 2012-05-28T11:30:41Z "Estimate: ? The 4store Sparql endpoint has been down for a long time. It needs to be reenabled and we need to check that the necessary informtion is pushed when creating /editing a dataset." amercader 2382 Investigate options for basic geocoding ckan ckan-future task amercader new 2012-05-14T17:39:20Z 2012-05-28T11:42:05Z A simple way for geocoding place names would be very useful, e.g in the spatial search, defining a geometry for a dataset (on the form or bulk) amercader 2762 test_related.py crashes ckan ckan-v1.8 defect new 2012-07-26T11:55:53Z 2012-07-26T11:55:53Z "/home/seanh/Projects/ckan/ckan/ckan/tests/functional/test_related.py ImportError (cannot import name assert_regexp_matches)" seanh 2919 Remove IController from docs ckan ckan-v1.8 defect new 2012-09-12T14:47:33Z 2012-09-12T14:47:33Z Still in docs but seems gone from code seanh 2936 Updating a group via the API clears its packages ckan ckan-v1.8 defect new 2012-09-19T09:47:46Z 2012-09-19T09:47:46Z "If the group dict that you post to the API does not have any 'packages' key then all the group's packages get removed. I think it would be better if you could just update e.g. the group's description without having to also post the list of packages, and apparently this is how other update API actions work. Might be worth checking all the update API actions for this behaviour, and making sure they're all consistent" seanh 2958 Uploading files with unicode characters in filename fails in CKAN 1.7 and 1.8 ckan ckan-v1.8 defect new 2012-10-04T09:29:36Z 2012-10-04T09:29:36Z "e.g. 2012_08(주요국가).xls I tested in CKAN 2.0 and it seemed to work fine there." seanh 2965 Stats extension broken on 1.8 ckan ckan-v1.8 defect new 2012-10-15T10:15:41Z 2012-10-15T10:15:57Z " * Graphs not showing (looks like a flot related file is missing) * Wrong groups counts (e.g. Data Explorer Examples show 1800 datasets when it onlu has 8)" amercader 2499 Documentation for the internal analytics ckan ckan-v1.8 enhancement toby new 2012-06-07T11:58:17Z 2012-06-07T11:58:17Z "There's no docs for setting up the internal analytics, nor what it provides. == Setup == Add: `ckan.tracking_enabled = true` to .ini file Run: `paster tracking -c dev.ini` == Related links == http://trac.ckan.org/ticket/2251" icmurray 2761 Document all the errors you can get when setting up filestore, and how to fix them ckan ckan-v1.8 enhancement new 2012-07-26T11:36:06Z 2012-07-26T11:36:06Z "Add it to a 'Troubleshooting' section on the filestore page: http://docs.ckan.org/en/ckan-1.7.1/filestore.html For the error messages and their solutions, see various threads on ckan-dev" seanh 2771 Documentation and examples for IDatasetForm and IGroupForm ckan ckan-v1.8 enhancement seanh new 2012-07-27T12:30:38Z 2012-10-15T12:19:24Z "Add minimal, working IDatasetForm and IGroupForm example extensions to core, with tests. The IDatasetForm example should use tag vocabularies (two birds with one stone) The IDatasetForm and IGroupForm docs are not very good (and are somewhat spread around different doc chapters), fix them up, and reference the new working examples. Tab Vocabularies docs should reference IDatasetForm example. When using convert_to/from_extras() you have to remove any free extras from the form or it won't work, this needs to be documented (in the docstring maybe) There have been recent changes to the schemas that IDatasetForm and IGroupForm use, make sure the docs are up to date." seanh 3003 New pg databases should be created with UTF8 encoding rather than system default ckan ckan-v1.8 enhancement new 2012-10-19T09:33:19Z 2012-10-19T09:33:19Z "Just to track @tauberer patch on Github. Probably going to 1.8.1" amercader 2763 Multilingual tests failing ckan ckan-v1.8.1 defect new 2012-07-26T11:56:59Z 2012-10-15T12:24:24Z test_multilingual_plugin.TestDatasetTermTranslation.test_dataset_index_translation, test_multilingual_plugin.TestDatasetTermTranslation.test_group_read_translation both failing for me on master seanh 2815 db_to_form_package_schema() strips tracking summary, isopen ckan ckan-v1.8.1 defect seanh new 2012-08-08T16:47:07Z 2012-10-15T12:23:41Z If an IDatasetForm plugin with a db_to_form_schema() based on db_to_form_package_schema() (which is in turn based on default_package_schema()) is in use then the 'tracking_summary' dict and the 'isopen' bool get stripped from package dicts during validation, e.g. during package_show(), and these values are then not available to templates. seanh 2846 Organizations allows you to set an organization as the parent organization of itself ckan ckan-v1.8.1 defect new 2012-08-13T12:22:16Z 2012-10-15T12:24:04Z that doesn't make any sense seanh 2881 Remove or improve 'Domain Model' docs page ckan ckan-v1.8.1 defect new 2012-08-21T09:05:17Z 2012-08-22T09:00:56Z "http://docs.ckan.org/en/ckan-1.8/domain-model.html Currently not very useful. Also these pages: http://docs.ckan.org/en/latest/domain-model-dataset.html http://docs.ckan.org/en/latest/domain-model-resource.html http://docs.ckan.org/en/latest/domain-model-task-status.html This just seems very incomplete to me (surely there is more to the domain model the dataset, resource and task status?) and very likely to not get updated as changes are mode to the code. Would anyone seriously refer to this rather than just looking at the source code? If we want to have something along these lines in the docs, I suggest adding docstrings to packages, modules, classes and methods and then using autodoc to pull them into sphinx." seanh 2882 Remove User Stories Overview from docs? ckan ckan-v1.8.1 defect new 2012-08-21T09:18:43Z 2012-08-21T09:18:43Z "Why is this here? Is it up to date? Is anyone keeping it up to date? Does this belong in some sort of CKAN planning or roadmap document rather than in the documentation? http://docs.ckan.org/en/latest/user-stories.html" seanh 2884 Lock icon shown next to unknown licenses ckan ckan-v1.8.1 defect new 2012-08-21T16:01:13Z 2012-08-21T16:01:13Z Set the license of a dataset to blank (the first option) or to 'License Not Specified' or 'Other (Non-Commercial)' and a lock icon appears next to the license on the dataset read page. Shouldn't this be a question mark or something? seanh 2891 Update or remove DataStore google presentation ckan ckan-v1.8.1 defect new 2012-08-28T10:23:43Z 2012-08-28T11:42:57Z "Currently this page: http://docs.ckan.org/en/ckan-1.8/datastore.html has an embedded Google docs presentation about data store which contains some outdated information, e.g. links to datastoreclient which was merged into ckanclient. Suggest moving the google presentation into the documentation proper, so it works with grep etc." seanh 2892 IDatasetForm's form_to_db_schema() breaks API? ckan ckan-v1.8.1 defect new 2012-08-28T11:42:05Z 2012-08-28T11:42:43Z "It looks like when an IDatasetForm plugin with a form_to_db_schema() method is active, then the posted data dict is validated against the schema even if the user is posting to an API endpoint like /api/action/package_update etc. This breaks the API, for example 'name' and 'title' are optional keys for package_update (you can supply only the 'id'), but the schema validation will fail if either are not present in the posted dict. It may be we need different schema for package create and package update, currently IDatasetForm has just the one form_to_db_schema() method." seanh 2893 Tags missing from form_to_db_package_schema() ckan ckan-v1.8.1 defect new 2012-08-28T12:11:04Z 2012-08-28T12:11:04Z "With an IDatasetForm plugin active with a form_to_db_package_schema() method that returns a schema based on ckan/logic/schema.py:form_to_db_package_schema() I'm getting validation errors like this: [ckan.controllers.api] Validation error: ""{'__type': 'Validation Error', '__junk': [u'The input field __junk was not expected.']}"" They seem to be caused when a data dict with key 'tags' is passed to e.g. the package_update API, for example in ckan/tests/functional/api/model/test_vocabulary.py there are various tests that get this error when they call package_update (if you run the tests with a suitable IDatasetForm plugin active). In ckan/logic/schema.py:form_to_db_package_schema() 'tags' is for some reason deliberately removed from the schema." seanh 2894 Empty lists get stripped from package dicst in package_show() ckan ckan-v1.8.1 defect new 2012-08-28T12:40:34Z 2012-08-28T12:40:34Z "If there's an IDatasetForm plugin active with a form_to_db_schema(), then packages with e.g. no extras get the 'extras' key entirely stripped by the validation in package_show, which causes some tests to crash, e.g. in ckan/tests/functional/api/test_activity.py (if the tests are run with a suitable IDatasetForm plugin active). If a package has no extras the dict returned by package_show should still have an 'extras' key with an empty list as the value. Suspect this also effects packages with no tags, no groups, etc." seanh 2896 resource_create crashes on ValidationErrors with KeyError: 'resources' ckan ckan-v1.8.1 defect new 2012-08-29T16:29:14Z 2012-08-29T16:29:14Z "ckan/logic/action/create.py:resource_create(): try: pkg_dict = _get_action('package_update')(context, pkg_dict) except ValidationError, e: errors = e.error_dict['resources'][-1] raise ValidationError(errors) This assumes that package_update always returns an error dict with 'resources' in it. This assumption doesn't hold." seanh 2897 db_to_form_package_schema() crashes historical dataset page ckan ckan-v1.8.1 defect new 2012-08-30T12:41:35Z 2012-08-30T12:41:35Z "If there's an IDatasetForm plugin active with a db_to_form_schema() method based on db_to_form_package_schema(), then viewing historical versions of datasets (e.g. http://127.0.0.1:5000/dataset/dasd%402012-08-30T12%3A26%3A18.524191) crashes. Reason is that revison_id and revision_timestamp need to be added to db_to_form_package_schema(), something like: schema.update({'revision_id': [validators.ignore_missing, unicode]}) schema.update({'revision_timestamp': [validators.ignore_missing, unicode]}) " seanh 2900 snippets/package_list.html template crashes on dataset that has no 'notes' ckan ckan-v1.8.1 defect new 2012-08-30T14:00:07Z 2012-08-30T14:09:48Z "Sometimes a dataset dict comes through package_show() with 'notes': None and then the 'notes' key gets stripped from the dict when validation happens (validation only happens if there's an IDatasetForm plugin with a db_to_form_schema() method) and then package_list.html template crashes, e.g. on user profile pages. Fix in the template? Or in the schema? The template also crashes on packages with no tags (empty list gets stripped from dict) and probably other empty fields as well" seanh 2912 ckanext/organizations: Neither public or private initially selected when adding dataset ckan ckan-v1.8.1 defect new 2012-09-10T09:57:56Z 2012-10-15T12:24:57Z When adding a dataset with the organizations extension on, neither the Public or the Private radio button is initially selected. Public should be selected. seanh 2914 Add form_to_db_schema_options() to IDatasetForm and IGroupForm ckan ckan-v1.8.1 defect new 2012-09-12T10:29:55Z 2012-10-15T12:25:28Z This optional method is supported when creating or updating groups and packages and is used in DefaultGroupForm and DefaultPackageForm, but seems to be missing from the interfaces. Should be added with docstring. seanh 2883 Add high level dev overview to 'For CKAN Developers' section of docs ckan ckan-v1.8.1 enhancement new 2012-08-21T10:52:35Z 2012-08-22T10:04:50Z Here's a draft: https://gist.github.com/3414107 seanh 2890 Collect data previews and data store docs in one chapter ckan ckan-v1.8.1 enhancement new 2012-08-28T10:20:36Z 2012-08-28T17:44:27Z "Currently there is this page: http://docs.ckan.org/en/latest/data-viewer.html which covers Recline Data Explorer and other kinds of data preview in CKAN. It is under the Publishing Datasets section in the documentation. I had to to a search for 'recline' to find it. Separately there is this page: http://docs.ckan.org/en/ckan-1.7.1/datastore.html which covers datastore, datastorer, and the data api. I suggest collecting this together in one chapter called 'Data Previews'. If I understand it right the general gist would be: CKAN has builtin previews of data resources on resource pages, enabled by default. Images, Google Documents, and web page resources will be loaded into embedded iframes for preview. Text-like files will be displayed raw. CSV or Excel files uploaded to CKAN will be previewed using Recline Data Explorer. Additionally, you can enable CKAN's DataStore, requires you to install ElasticSearch and nginx and put datastore.enabled=1 in your ini file. Lets you use the Data API to query data. Does having DataStore enabled mean you get preview of more types of resources? Any resource that's available via the Data API will be previewed using Recline, You can install ckanext-datastorer, and then CSV and Excel files _linked to_ as CKAN resources will be previewed using Recline also. Requires celeryd." seanh 3006 Update translations from Transifex ckan ckan-v1.8.1 enhancement new 2012-10-29T10:22:24Z 2012-10-29T10:22:24Z There are some new translations already on the CKAN 1.8 resource on Transifex already (e.g. 100% Norwegian translation I think). We should at least pull these from Transifex for 1.8.1, and may as well do a call for translations too. seanh 2335 Unicode characters don't work in CKAN ini files ckan ckan-v1.9 defect seanh assigned 2012-04-30T12:42:06Z 2012-06-25T11:43:55Z You get a UnicodeDecodeError, looks like it tries to decode the ini file as ascii. This is a problem because someone might try to set their site description to something with an umlaut in it, for example. seanh 2406 Errors in source install doc ckan ckan-v1.9 defect kindly new 2012-05-21T10:11:33Z 2012-06-25T15:06:44Z "There are a few errors or unclarities in the source install doc - see http://lists.okfn.org/pipermail/ckan-dev/2012-May/002203.html http://lists.okfn.org/pipermail/ckan-dev/2012-May/002204.html " markw 2415 Confusing message on attempting to register new account with upper case ckan ckan-v1.9 defect kindly new 2012-05-23T13:19:49Z 2012-07-02T22:31:44Z "On registering a new account, if you try to use a capital letter in your 'Login', you get the unhelpful error message: ""The form contains invalid entries: Name: Url must be purely lowercase alphanumeric (ascii) characters and these symbols: -_"" What does 'Name' or 'Url' have to do with anything? It means 'Login must be ...' But even better would be to make the login case-insensitive and allow capitals to be entered. " markw 2485 Encourage leak containment by limiting the number of requests a CKAN process serves ckan ckan-v1.9 defect icmurray new 2012-06-01T11:47:09Z 2012-06-25T14:53:48Z "CKAN has [ticket:1345 memory leaks]. They can be contained by limiting the time-to-live of a ckan process. An easy way to achieve this is to limit the number of requests a ckan server process can serve before it gets killed and replaced. One should ... * research ways to limit requests-per-process with the different web servers (see below for a start); * explain these safeguards in the CKAN documentation and encourage users to apply them; * consider the helper script '''[https://github.com/okfn/ckan/blob/master/ckan_deb/usr/bin/ckan-create-instance ckan-create-instance]''' to generate Apache configs with '''WSGIDaemonProcess ... maximum-requests=XY''' being active instead of [https://github.com/okfn/ckan/blob/master/ckan_deb/usr/lib/ckan/common.sh#L262 commented out]. Or at least let it warn the user to use '''!MaxRequestsPerChild'''; * Investigate current CKAN deployments whether they suffer from mem leaks, and if so contain them. ---- How to limit requests-per-process * Apache: * Use '''WSGIDaemonProcess''' with '''maximum-requests=50''' or whatever limit is appropriate. We did this sucessfully on [http://trac.okfn.org/ticket/904 datahub.io] and the [http://trac.okfn.org/ticket/1245 ckan farm] * (This need to be verified) Without WSGIDaemonProcess, '''!MaxRequestsPerChild 50''' should achieve the same. * nginx/supervisord: to be researched. " nils.toedtmann 2520 Document undocumented config options ckan ckan-v1.9 defect seanh assigned 2012-06-13T11:52:48Z 2012-06-25T11:48:28Z "There are 21 undocumented config options in CKAN, some of which are not mentioned in the config file template either: ckan.admin.name ckan.admin.email ckan.default.group_type ckan.page_cache_enabled ckan.cache_enabled ckan.cache_expires ckan.extra_resource_fields ckan.extra_resource_group_fields ckan.storage.key_prefix ckan.storage.max_content_length ckan.feeds.authority_name * ckan.feeds.date * ckan.feeds.author_name * ckan.feeds.author_link * ckan.mail_from ckan.gravatar_default * ckan.plugins ckan.api_url ckan.auth.profile ckan.datastore.enabled ckan.tracking_enabled There are also some options that are in the default deployment.ini even though they're deprecated: ckan.async_notifier carrot_messaging_library ckan.build_search_index_synchronously See email to ckan-dev from David Read: http://lists.okfn.org/pipermail/ckan-dev/2012-June/002447.html It'd be best if the docs could be automatically pulled from the source into sphinx using autodoc, see #1358" seanh 1745 Dataset search UX improvements as of Jan 2012 ckan ckan-v1.9 enhancement new 2012-02-02T23:22:21Z 2012-06-18T15:29:18Z "Changes to make search both more exploratory and more satisfying to use * Search query build - #1603 * Ability to add new facet fields ""live"" * That is add fields which then contain faceted options (a bit like data.hri.fi) * (??) Autocomplete / drop down on search (i.e. search while you type) * Dubious about value / cost ratio here Probably would involve to pure JS and HTML implementation. == Implementation == Probably require * API changes to expose solr style API directly #1737 " rgrp 1785 Replace 'Revisions' page with site-wide activity stream ckan ckan-v1.9 enhancement seanh accepted 2012-02-07T16:34:18Z 2012-06-26T15:25:12Z Replace the /revision page with a /activity page showing a site-wide activity stream of all activities in the site. Or perhaps the site-wide activity stream can go somewhere on the front page of the site instead? seanh 2257 cleanup template vars ckan ckan-v1.9 enhancement toby new 2012-03-23T14:35:07Z 2012-06-19T09:11:11Z "look at reducing what is pulled into templates eg ckan.lib.helpers ensure that these changes don't break existing extensions etc" toby 2291 DataStorer incorrectly attempts to store JSON in DataStore ckan ckan-v1.9 enhancement kindly new 2012-04-14T19:19:07Z 2012-06-25T15:05:37Z "See http://thedatahub.org/dataset/ecb13156-cf11-4e91-b51c-f287d154276e/resource/5db67915-40cb-467e-9d72-194ea4973670 This was marked as json format (and is JSON) but appears to have been imported as a quasi-CSV. Either this hsould not be imported at all or it should be imported as JSON ... (which is straightforward to do). Assigning to David for triage and review." rgrp 2320 Introduce new state: archived ckan ckan-v1.9 enhancement new 2012-04-23T20:12:29Z 2012-06-25T11:35:19Z Some datasets become obsolete: e.g. http://datahub.io/dataset/uncdb Do not want to delete them but want them marked as archived or deprecated or something. rgrp 2321 Use resource names/titles in URLs ckan ckan-v1.9 enhancement new 2012-04-24T03:00:12Z 2012-06-25T11:36:14Z "Would make them much nicer but requires: * Ensure uniqueness (or deal with ambiguity) * (?) Slug/names versus titles * Aside: what is currently called name should probably be title " rgrp 2324 Link API docs to relevant version not just latest ckan ckan-v1.9 enhancement new 2012-04-25T10:58:52Z 2012-08-17T10:30:11Z "The link to the API docs in the footer always points to the latests docs, which may not be correct for the particular CKAN version. Also, remove the 'API' link (to /api/1) as pointless and misleading" amercader 2328 Link to Catalog API for each Dataset, Group and Tag ckan ckan-v1.9 enhancement new 2012-04-26T12:15:01Z 2012-06-25T11:38:51Z At the moment no way for a normal user to find JSON versions of this material. Would it be worth adding back a link somewhere on the page? rgrp 2333 Introduce new resource type datastore ckan ckan-v1.9 enhancement new 2012-04-28T22:16:00Z 2012-06-25T11:41:13Z "This is a resource that is *just* in the datastore. At the moment there are hacky ways to do this (create resource with meaningless URL etc) but we should do this properly: * New Option: DataStore Table in add a resource form * URL field is disabled (or points at /api/data or something rather pointless) * Download button is disabled * Better would be to find a way to serialize an ES table or stream it ... " rgrp 2334 Improved data import ckan ckan-v1.9 enhancement kindly assigned 2012-04-28T22:21:02Z 2012-06-25T11:42:18Z "Placeholder for thinking about this ... * Allow users to configure info for CSV import (e.g. separators, quotes etc)" rgrp 2337 Sort out ini file confusion in docs ckan ckan-v1.9 enhancement seanh assigned 2012-04-30T14:57:56Z 2012-06-25T11:43:20Z "For example if you follow the source install instructions then you have development.ini, then when you go onto Post-Installation Setup it talks about std.ini. People don't realise that these files are the same. Either fix the docs or fix CKAN to use same ini file names in package and source installs/" seanh 2370 Add examples of all config settings in default config file ckan ckan-v1.9 enhancement seanh assigned 2012-05-03T10:50:37Z 2012-06-25T11:44:22Z The default config file should contains examples (maybe commented out) for all config variables. Makes it easier to edit the config. For example openid_enabled is not in there. seanh 2407 'Access denied' message is unhelpful ckan ckan-v1.9 enhancement amercader new 2012-05-21T15:01:40Z 2012-06-25T14:10:40Z "This chap tried to use the API to add a dataset to a group and got the uninformative message 'Access denied': http://lists.okfn.org/pipermail/ckan-dev/2012-May/002229.html The problem was he didn't have access to the group. A message like 'You don't have access to the group ""LODcloud""' would have been clearer." markw 2426 Get rid of CKAN's formalchemy dependency ckan ckan-v1.9 enhancement kindly new 2012-05-25T11:47:42Z 2012-06-25T09:21:35Z seanh 2427 Reduce cruft installed into CKAN's virtualenv due to Pylons ckan ckan-v1.9 enhancement kindly new 2012-05-25T11:48:27Z 2012-07-02T22:32:03Z seanh 2473 Make datstorer store field ordering in _meta field ckan ckan-v1.9 enhancement kindly assigned 2012-05-29T12:11:32Z 2012-06-25T15:07:51Z Recline views should have a default table order, being the same as the csv that was imported. kindly 2475 Upgrade trac.ckan.org to trac 0.12.3 ckan ckan-v1.9 enhancement seanh accepted 2012-05-29T18:46:16Z 2012-07-02T13:09:24Z "This is not trivial, requires a database upgrade. But 0.12.3 has some nice new features (multiple vcs repository support, ticket comment editing and nicer preview) and the latest spam filter plugin requires 0.12. http://trac.edgewall.org/wiki/ChangeLog" seanh 2477 Add extensions section to readthedocs ckan ckan-v1.9 enhancement seanh accepted 2012-05-30T15:54:04Z 2012-07-24T10:02:03Z Need a central up to date place for currently supported extensions (closest we have is http://wiki.ckan.org/List_of_Extensions) & their documentation. I suggest this gets added to a dedicated section of docs.ckan.org shevski 2478 Get NeverNotifyUpdaterPlugin installed on trac.ckan.org ckan ckan-v1.9 enhancement seanh accepted 2012-05-30T16:52:14Z 2012-07-02T13:09:51Z seanh 2479 Meanings of Author and Maintainer fields are unclear ckan ckan-v1.9 enhancement new 2012-05-31T08:28:18Z 2012-06-15T14:38:35Z "CKAN's default schema has fields for Author ('The name of the main contact, for enquiries about this particular dataset') and 'Maintainer ('another important contact person'). The distinction is not clear. Also the fields are often left blank. The roles that seems most important are those of the original owner/publisher of the data, and the person maintaining the CKAN record/copy of it. So I suggest: (1) Rename the fields 'Owner' and 'Maintainer'. (2) Change the explanatory text for the relevant fields: Owner: 'The person or organisation who create/collect/publish the data in this dataset.' Owner e-mail: 'E-mail address for enquiries to the Owner named above.' Maintainer: 'The person maintaining this dataset on [name of CKAN instance], if different from the above.' Maintainer e-mail: 'E-mail address for enquiries to the Maintainer named above.' (3) When a logged-in user creates a new dataset, the main form should have a checkbox, checked by default, marked 'I am the maintainer of this dataset'. If checked, the Maintainer name and e-mail fields are populated from the user's profile. " markw 2480 Better message when dataset has no resources ckan ckan-v1.9 enhancement new 2012-05-31T08:31:33Z 2012-06-15T14:38:06Z "If a dataset has no resources the resources list currently says '(none)'. Here is a suggested improvement, provided that a maintainer is named: 'There are no data resources here yet. For information about this data, contact the dataset maintainer.'" markw 2483 Non-local resources should not have Download links ckan ckan-v1.9 enhancement new 2012-05-31T12:52:14Z 2012-06-15T14:37:23Z "At present, a resource which is just a URL link to an external resource has a 'Download' button on the resource page. This gives the misleading impression that the resource is stored locally. This is related to another small UI issue: I think the URL of a resource should be much more prominent, not buried in the 'Additional Information' table. Suggested fix: * Put the URL prominently at the top of the resource page (above the preview) (and make it link to the resource) * Disable the Download button unless the resource is stored locally. " markw 2516 Make 'Assign to:' field on trac.ckan.org into a dropdown list ckan ckan-v1.9 enhancement seanh accepted 2012-06-13T09:07:22Z 2012-07-02T13:13:42Z there's a setting for this seanh 2529 DataHub (or CKAN) widgets ckan ckan-v1.9 enhancement new 2012-06-15T08:47:29Z 2012-06-25T11:48:37Z "Simple widgets in pure JS. For example: * Count of datasets in a group (could generalise to a query but not sure how useful that is ...) * Embeddable list of top X (5) datasets for a given query * Embeddable list of *my* datasets Either these live at: {site}/widgets and we have some kind of generator (form where I choose my group, or my query). Or: we have this attached to areas of site where relevant. Can combine the 2 so that the latter links to the former. Think first will be easier to do and possibly more useful long-term (e.g. can just link people to that page). Cf. http://okfnlabs.org/ckanjs/" rgrp 2543 facet.sort is not available in the package_search action ckan ckan-v1.9 enhancement new 2012-06-18T09:55:35Z 2012-06-25T14:04:51Z "Not all solr facet parameters are available through the pcakage_search action. In particular, `facet.sort` has been asked for; but this ticket should check to see if there are other parameters that would be easy to add too. See: http://wiki.apache.org/solr/SimpleFacetParameters#facet.sort" icmurray 2572 clean up stats plugin ckan ckan-v1.9 enhancement toby new 2012-06-19T11:24:14Z 2012-06-28T16:05:24Z attempt to disengage the stats plugin from core as much as possible toby 2579 move sort_by functions into lib.helpers ckan ckan-v1.9 enhancement toby new 2012-06-21T12:29:58Z 2012-06-28T16:02:17Z "make these more available but keep existing functionality so not to break any users remove_field() drill_down_url() etc" toby 2585 Escape solr control characters in search queries, add advanced search screen ckan ckan-v1.9 enhancement new 2012-06-22T10:26:13Z 2012-06-25T11:51:18Z "Suggestion from David Read: We noticed that some search queries produce unexpected search results in CKAN, due to them containing special characters. For example if you were to search for ""Spend over £25,000 - NHS Leeds"" then it would not come up with the dataset with that exact name. It was excluding datasets with the word ""NHS"" due to the dash/minus sign. It works fine if you escape the minus sign: ""Spend over £25,000 \- NHS Leeds"". So in data.gov.uk I've added escaping of such control characters in our plugin and this useful routine: http://fragmentsofcode.wordpress.com/2010/03/10/escape-special-characters-for-solrlucene-query/ Perhaps you would consider providing this in CKAN core in future? I think there is an occasional case when power users would want to use the special characters - brackets, +, -, boolean operators etc. but maybe these could be reserved for an 'advanced search' screen?" seanh 2619 Omit private datasets from public activity streams ckan ckan-v1.9 enhancement seanh assigned 2012-06-28T11:49:00Z 2012-10-29T17:18:57Z "Activities about private datasets should not appear in public activity streams. I don't think you want to actually purge the activities from the db, because you might still want them to appear in private activity streams. I do think that when a dataset goes private all its past activity should go private, because I imagine that users are going to want to hide everything about the dataset and not have any past activities 'leaking out' I don't think you want to consider whether the dataset was private when the activity happened, rather if a dataset is private now then all its past activities are private (and the simplest thing would be to say that if a dataset is public now then all its past activities become public as well, but is that a privacy concern?) The easiest way to implement this is going to be by modifying the *_activity_list() action functions in get.py, after they pull their activity lists out of the db they should pass them through a function that filters out stuff about private datasets. An activity about a private dataset is one whose object_type is 'dataset' and whose object_id matches the id of a private dataset. You should also check the object_type and object_id of all of the activity object's activity detail objects, if any of those match a private dataset then mark the whole activity as private. Currently all activity streams are public so should have all private datasets filtered out from them, except for the dashboard activity stream which is private to the individual user. In this case private datasets that the user has permission to see should not be filtered." seanh 2625 Add i18n strings from non-core but supported extensions to ckan.pot file ckan ckan-v1.9 enhancement new 2012-07-02T13:48:23Z 2012-07-02T13:48:23Z Have to decide which non-core extensions are going to be supported first. seanh 2663 h.resource_display_name needs love ckan ckan-v1.9 enhancement toby new 2012-07-11T14:42:26Z 2012-07-11T14:42:26Z "This function is shit and needs cleaning up and a doc string description is markdown and should be treated properly either we should truncate all or leave it to the templates but work universally url if no name / desc this is in demo-theme branch" toby 2708 limit extra data for package/group show ckan ckan-v1.9 enhancement kindly new 2012-07-18T14:40:20Z 2012-07-18T14:40:20Z " context['package_limits'] = { 'tags': 5, <- get first 5 'extras': 0, <- get all } only get what you ask for have to be explicit context['group_limits'] = {} only main item start with datasets/groups expand if we like it" toby 2758 file storage gives error if config not available but no useful user information ckan ckan-v1.9 enhancement toby new 2012-07-26T07:28:29Z 2012-07-26T07:28:29Z "We get an error which should be improved the actual problem is this but not passed to user ⇝ KeyError: 'ofs.impl' View as: Interactive (full) | Text (full) | XML (full) Module ckan.controllers.storage:2 in auth_form view Module ckan.lib.jsonp:26 in jsonpify view >> data = func(*args, **kwargs) Module ckan.controllers.storage:407 in auth_form view >> authorize(method, bucket, label, c.userobj, self.ofs) Module ckan.controllers.storage:200 in ofs view >> StorageAPIController._ofs_impl = get_ofs() Module ckan.controllers.storage:71 in get_ofs view >> storage_backend = config['ofs.impl'] Module paste.registry:146 in __getitem__ view >> return self._current_obj()[key] KeyError: 'ofs.impl'" toby 2773 About page needs improving ckan ckan-v1.9 enhancement markw new 2012-07-30T11:13:15Z 2012-07-30T11:13:15Z The about page for the DataHub (thedatahub.org/about) could be improved. More importantly the default about page for a generic CKAN instance should be completely different - focus more on Open Data rather than the community hub idea (as this is more relevant for most installations) and remove specific references to tdh. markw 2788 Speed improvements on creating/updating and indexing ckan ckan-v1.9 enhancement amercader new 2012-08-01T14:56:32Z 2012-08-01T14:56:32Z "Specially needed when importing large numbers of datasets. Profiling the import command from the harvesting extension has shown some areas where improvements could be made. " amercader 2829 Archiver fails on 403 http response ckan ckan-v1.9 enhancement johnglover new 2012-08-09T10:34:44Z 2012-08-09T10:34:44Z "Had this issue with the archiver on my local machine need to be logged in (I am admin) to see via web front end {{{ $ paster archiver update -c ../ckan/development.ini 2012-08-09 11:01:37,636 INFO [ckanext.archiver.commands] Archival of dataset resource data added to celery queue: opencontext-chogha-mish-fauna (1 resources) 2012-08-09 11:01:37,671 INFO [ckanext.archiver.commands] Getting dataset metadata: south-african-national-gov-budget-2012-13 2012-08-09 11:01:37,900 INFO [ckan.lib.base] /api/action/package_show render time 0.043 seconds Traceback (most recent call last): File ""/home/toby/okfn/pyenv/bin/paster"", line 8, in <module> load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')() File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py"", line 104, in run invoke(command, command_name, options, args[1:]) File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py"", line 143, in invoke exit_code = runner.run(args) File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py"", line 238, in run result = self.command() File ""/home/toby/okfn/pyenv/src/ckanext-archiver/ckanext/archiver/commands.py"", line 98, in command response = app.post(api_url + '/package_show', data) File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py"", line 262, in post expect_errors=expect_errors) File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py"", line 243, in _gen_request return self.do_request(req, status=status) File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py"", line 406, in do_request self._check_status(status, res) File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py"", line 439, in _check_status res.body)) paste.fixture.AppError: Bad response: 403 Forbidden (not 200 OK or 3xx redirect for /api/action/package_show) {""help"": ""Return the metadata of a dataset (package) and its resources.\n\n :param id: the id or name of the dataset\n :type id: string\n\n :rtype: dictionary\n\n "", ""success"": false, ""error"": {""message"": ""Access denied"", ""__type"": ""Authorization Error""}} }}}" toby 2838 Context variables accepted by action functions need to be documented ckan ckan-v1.9 enhancement new 2012-08-10T20:35:11Z 2012-08-10T20:35:11Z "I was doing this: {{{ context = {'model': base.model, 'session': base.model.Session, 'user': toolkit.c.user or toolkit.c.author, 'extras_as_string': True} group_dict = logic.get_action('group_show')(context, {'id': group_id}) }}} in an extension and one of the group_dicts fields, one that uses convert_to/from_extras, was coming out with the wrong value. It took me ages to realise that I had to pass {{{'extras_as_string': True}}} in the context. I don't think this or other context variables are documented anywhere." seanh 2885 Labels (editor, admin) not translated in authorization pages for datasets or groups ckan ckan-v1.9 enhancement new 2012-08-22T08:49:11Z 2012-08-22T08:52:04Z seanh 2889 Support JSON values in extras, when returning JSON from API ckan ckan-v1.9 enhancement new 2012-08-24T12:35:37Z 2012-08-24T12:35:37Z "For example, edit the warandpeace dataset and add an extra with key ""comments"" and value ""[{u'date': u'Tue Aug 21 13:01:48 GMT 2012', u'comment': u'testing ckan', u'userId': u'11702'}, {u'date': u'Wed Aug 22 13:01:48 GMT 2012', u'comment': u'testing ckan again', u'userId': u'11702'}]"", now visit API URLs such as: /api/search/dataset?sort=metadata_modified+desc&all_fields=1&limit=10 /api/rest/dataset/warandpeace /api/action/package_show?id=warandpeace The value of the extra comes out as a string in the dataset JSON, rather than having its contents unpacked into a list of JSON objects." seanh 2537 Test and document ckanbuild ckan ckanbuild enhancement seanh accepted 2012-06-15T15:48:48Z 2012-06-25T15:57:10Z "https://github.com/okfn/ckanbuild Verify that what's there so far still works, write a README explaining how it works" seanh 2538 Add multiple-instance support to ckanbuild ckan ckanbuild enhancement seanh accepted 2012-06-15T15:51:39Z 2012-06-25T15:57:16Z "Probably use ansible to do this. To create an instance, create a dir at /etc/ckan/MYSITE, and put MYSITE.wsgi, MYSITE.ini and who.ini files in it. Also put a MYSITE file in /etc/apache2/sites-available. See the example files already present in ckanbuild. Booting a new site should be a single command. May not handle the postgres/solr/elastic-search side of things yet, could just require the user to set these up herself first and then pass them as args to the create-instance command." seanh 2539 Investigate the existing ckan debian package for ckanbuild ckan ckanbuild enhancement seanh accepted 2012-06-15T15:54:21Z 2012-06-25T15:57:25Z Do we want to build on top of the existing debian packaging code? Or throw it away and start fresh? seanh 2540 Implement a way of upgrading ckan sites using ckanbuild ckan ckanbuild enhancement seanh accepted 2012-06-15T15:55:40Z 2012-06-25T15:57:30Z When there are multiple ckan sites installed on a single server via ckanbuild, there needs to be some way of upgrading them all to a new ckan version at once. seanh 2541 Add non-core extensions to ckanbuild ckan ckanbuild enhancement seanh accepted 2012-06-15T15:57:06Z 2012-06-25T15:57:36Z We want some extensions from outside of CKAN core to be included in ckanbuild. These would be pip installed into the virtualenv before packaging the debian package. Decide which extensions to include. seanh 2542 Create jenkins job to run ckanbuild, and run tests ckan ckanbuild enhancement seanh accepted 2012-06-15T15:58:08Z 2012-06-25T15:57:43Z It should run the script to create the debian package, boot a VM, install the debian package on the VM, boot a CKAN instance, then run the tests. seanh 2548 Object ownership for groups/package ckan datahub-july enhancement kindly assigned 2012-06-18T11:02:11Z 2012-06-26T10:28:59Z " == Requirements == We need to be able to easily determine who the owner of a dataset or group is. Datasets and Groups should have an Owner, who may change over time but is a specific user within the CKAN instance. It should be easy for CKAN components to determine the user and for the initial version we should ignore the can of worms labelled 'ownership transfer'. At this point migration is likely to be the biggest issue, and would suggest that it is acceptable that the last user to edit a dataset be set as the current owner. More tickets should arise as a result of this work where we may be able to optimise some queries to use the new feature. == Interface == None == User Stories == None == Tasks == [ ] Analysis/Clarification [ ] Tests [ ] Migration [ ] Code/Schema changes [ ] Documentation == Estimates == " ross 2721 deleted groups should not show on 'Add to Groups' dropdown ckan demo phase 4 defect toby new 2012-07-22T09:17:12Z 2012-08-09T20:30:14Z "Groups previously deleted still show up in the add dataset process in step 3 'Additional info' http://s031.okserver.org:2375/dataset/new_metadata/ff" shevski 2728 deleted group shows on search index - for admins ckan demo phase 4 defect toby new 2012-07-22T09:53:04Z 2012-08-15T09:45:44Z 'test-group', which has been deleted,shows up on main search page under groups - and can be filtered by - see http://s031.okserver.org:2375/dataset?groups=test-group shevski 2857 no UI to follow another user although option on dashboard ckan demo phase 4 defect toby new 2012-08-15T16:59:43Z 2012-08-16T08:51:37Z "e.g. second option in left hand sidebar is follow another user' http://s031.okserver.org:2375/user/dashboard but if you click on that and then any user, there's no follow button to me the dashboard user stuff is highly unfinished and probably should not be included unless someone is very keen for it" shevski 2863 non-admin users can edit other's draft datasets ckan demo phase 4 defect toby new 2012-08-16T10:54:39Z 2012-08-17T11:05:02Z edit button shows up for daniel lewis for http://s031.okserver.org:2375/dataset/ff shevski 2869 user profile not linked on disqus until page refresh ckan demo phase 4 defect new 2012-08-16T10:58:34Z 2012-08-16T10:58:34Z "Reported by Daniel: Problem: DISQUS comments don't appear with profile details (inc. username). How to reproduce: 1) Go to a dataset, e.g. I went to http://s031.okserver.org:2375/dataset/ff 2) I logged into DISQUS using the Google Authentication, e.g. in my case that is okfn.daniel.lewis@gmail.com 3) I made a comment and submitted 4) It comes up with the comment, but no linked profile until the page gets refreshed" shevski 2873 ckanext-qa: dataset summary incorrect ckan demo phase 4 defect toby assigned 2012-08-16T14:13:49Z 2012-08-17T12:58:03Z " For example see: http://s031.okserver.org:2375/qa/dataset/five_stars Which has: Example dataset as a score of: 4 After examination of the dataset ( http://s031.okserver.org:2375/dataset/example-dataset ) it is made up of a CSV and an HTML file. " danieljohnlewis 2718 can't add dataset to more than one group ckan demo phase 4 enhancement toby new 2012-07-20T10:35:50Z 2012-08-09T20:30:03Z "trying to add a dataset to another group means it's no longer part of the first group http://demo.ckan.org/dataset/edit/afterfibre" shevski 2780 way for admins to undelete datasets ckan demo phase 4 enhancement toby new 2012-07-31T12:20:48Z 2012-08-15T09:43:31Z "Since admins can see deleted datasets - there should be a way for them a) to know they are currently deleted & not viewable by normal users - ticket: #2779 b) way to undelete such datasets - this ticket I suggest a button on the edit form instead of the delete button i.e. remove normal delete button with 'deleted dataset, only admins can view' with undelete button next to the message?" shevski 2828 Draft datasets are confusing - tickets need creating ckan demo phase 4 enhancement toby new 2012-08-09T09:56:18Z 2012-08-09T21:22:04Z "reported as editing datasets incorrect e.g. clicking on edit here http://s031.okserver.org:2375/dataset/ff takes you to create dataset page http://s031.okserver.org:2375/dataset/edit/ff but this is the correct behaviour of a draft dataset We need to show draft datasets correctly proper tickets need making for the different issues after review of issues with them - who can see, where, admins and viewing, orgs too etc" shevski 2830 Need method to undelete groups ckan demo phase 4 enhancement toby new 2012-08-09T10:50:08Z 2012-08-09T21:22:21Z need controller action and front-end method toby 2861 Add qa breadcrumb ckan demo phase 4 enhancement toby new 2012-08-16T10:47:26Z 2012-08-20T17:21:10Z can we add in the breadcrumb navigation to qa pages so that you can go back to http://s031.okserver.org:2375/qa from sub pages such as http://s031.okserver.org:2375/qa/dataset/five_stars shevski 2862 Revised revised groups description ckan demo phase 4 enhancement toby new 2012-08-16T10:52:02Z 2012-08-16T11:04:32Z "Revised text for 'What are groups?' box at demo.ckan.org/group (after discussion with IB re #2812): What are groups? Groups allow you to group users and data together so that they are easier to manage. Group owners can assign roles and authorisations, giving each project or department control of its own data publishing. Users can browse or facet by groups, which could be an organisation (for example, the Department of Health) or topic (e.g. Transport, Health), making it easier to find the data they are looking for." markw 2820 English Language: Visualization -> Visualisation ckan demo phase 5 defect new 2012-08-09T09:27:35Z 2012-08-09T09:27:35Z "Problem: In the English version (which has a UK flag, indicating British English), the word ""Visualization"" is used. For an example see the ""Filter by type"" drop down on the /apps page. Expected: This should be ""Visualisation"" in British English. Any instances of ""Visualize"" should be changed to ""Visualise"" too. " danieljohnlewis 2898 Looses data entered on step 1 of create dataset after login ckan demo phase 5 defect new 2012-08-30T13:08:14Z 2012-08-30T13:08:40Z "To reproduce: 1. Enter details for new dataset without being logged in 2. Hit login page for just before step 2 3. Once logged in you end up back at the empty add dataset form (instead of expected 2nd step of dataset edit)" johnmartin 2583 make sure that we implement authentication where needed ckan demo phase 5 enhancement toby new 2012-06-22T10:04:38Z 2012-07-12T09:44:34Z in development many auth checks may have been lost we need to check they are still working etc toby 2697 create dataset validation ckan demo phase 5 enhancement johnmartin assigned 2012-07-18T14:00:35Z 2012-08-29T10:25:25Z Includes: missing fields, existing field checks (i.e. whether a name/dataset already exists with that name) during input (i.e. no need to submit form to check) shevski 2699 workflow for associating datasets with groups ckan demo phase 5 enhancement shevski assigned 2012-07-18T14:02:56Z 2012-08-09T10:12:13Z "needs review & speccing out e.g. datasets created by a user who belongs to a certain publisher (group) get auto added to this group" shevski 2702 Future Javascript wishlist for demo ckan demo phase 5 enhancement shevski assigned 2012-07-18T14:07:55Z 2012-08-06T12:26:24Z "tooltip on popular datasets with number of views facets to update automatically creating a dataset without reloading page between steps hover on licences information autocomplete on search terms group filtering social share buttons in lightboxes dataset counts on homepage" shevski 2725 Case sensitivity on tags ckan demo phase 5 enhancement toby new 2012-07-22T09:34:27Z 2012-07-23T08:06:13Z "My feeling is that 'country-US' and 'country-us' should be the same tag. However currently tags with caps are treated differently see http://s031.okserver.org:2375/en/dataset/test-dataset with TEST and test - there also get indexed twice in the search page" shevski 2726 confusing logic on data preview formats ckan demo phase 5 enhancement toby new 2012-07-22T09:38:47Z 2012-07-23T08:08:26Z "1. If a user enters the wrong format on a file that can be previewed - it simply won't be previewed (e.g. a CSV or XML file that can be filled in with JSON in format will just not work or check this 2. If I incorrectly edit format to one that data preview will try to preview it will work even for a format that it doesn't accept (sometimes) e.g. this PDF file I changed the metadata to HTML http://s031.okserver.org:2375/en/dataset/test-dataset/resource/9d27a9d9-36ec-460e-9edb-6dff7ba4fc28" shevski 2735 Dataset order on user page ckan demo phase 5 enhancement toby assigned 2012-07-23T17:01:17Z 2012-08-07T14:20:45Z "I think the datasets on user pages http://s031.okserver.org:2375/user/me should be ordered by latest updated (with most recent at the top) instead of in alphabetical order. What do you think?" shevski 2748 add 'add new resource' button to sidebar ckan demo phase 5 enhancement new 2012-07-25T10:41:18Z 2012-08-09T09:15:44Z "When editing a resource you see the current and any other existing resources in sidebar (e.g. see http://s031.okserver.org:2375/en/dataset/format-error-test/resource_edit/d1eac556-c16f-44af-8148-5e3467b57cf8?inner_span=True) Would be good to have a pretty 'add new' slightly transparent resource folder/pointer undearneath - letting you add resources from the end resource page" shevski 2768 normalise excel to xls ckan demo phase 5 enhancement toby new 2012-07-26T16:16:22Z 2012-08-07T15:01:03Z so that data proxy works shevski 2786 target blank HTML downloads ckan demo phase 5 enhancement new 2012-08-01T11:16:54Z 2012-08-01T11:16:54Z e.g. if I click on download here: http://s031.okserver.org:2375/dataset/example-dataset/resource/d8797e51-b497-46ca-a274-8675533d110b can it take me to a new tab instead of navigating away from ckan? shevski 2795 Check validation of HTML, CSS, JS ckan demo phase 5 enhancement new 2012-08-02T10:25:28Z 2012-08-02T10:25:28Z Ensure that we are being standards compliant toby 2813 Confusing sidebar on demo dataset page ckan demo phase 5 enhancement toby new 2012-08-08T10:03:26Z 2012-08-08T17:03:39Z "On a dataset page on demo.ckan.org, the left sidebar is confusing. * It starts with some random links. Actually they are links to groups which the dataset is in, but this isn't clear. * The sidebar elements that are actually part of the dataset are 'Datset extent' and 'License', so these should be right at the top (if they belong in the sidebar at all), instead of which they are right at the bottom in the junk part of the page (i.e. probably lower than the bottom of the main page, and hence lower than anyone will scroll)." markw 2821 Featured Items on Filter ckan demo phase 5 enhancement new 2012-08-09T09:28:24Z 2012-08-09T09:44:52Z "Problem: On /apps page in the Filter Results box there is a ""Only show featured items"" checkbox, on selection it comes up with 0 solutions. Expected: Presumably an admin can create ""featured items"" so that they can be randomly selected on front page (is this correct)? If there are no ""featured items"" in the whole database can this check box be hidden? Bug is: no UI or obvious way to create featured items. Also the checkbox looks un-styled" danieljohnlewis 2854 no way to see which dataset an app is related to on /apps page ckan demo phase 5 enhancement new 2012-08-15T16:38:48Z 2012-08-15T16:38:48Z e.g. from http://s031.okserver.org:2375/apps?page=1 you cannot edit a related item as an admin or view what dataset the item is related to shevski 2899 Step 2 > Save & add another bug ckan demo phase 5 enhancement new 2012-08-30T13:15:19Z 2012-08-30T13:15:19Z "Goto: 1. Add dataset 2. Get to step 2 3. Click ""Save & add another"" without adding any data 4. Ends on step 3 Should: fail to validate and remain on step 2" johnmartin 2904 Show more formats/groups/tags shouldn't display if there aren't more ckan demo phase 5 enhancement new 2012-09-05T14:14:04Z 2012-09-05T14:14:04Z On the search results page when there are no more formats, groups or tags to display it should not display the show more links. johnmartin 2905 Add dataset URL key behaviour ckan demo phase 5 enhancement new 2012-09-05T14:28:03Z 2012-09-05T14:28:03Z "On step 1 for adding a dataset can we add a ajax ping that checks if the URL that is generated by the JS is taken? Behaviour should be as follows: 1. User types title in box (as current) 2. URL key is regexed by JS (as current) 3. The JS does a ajax call to check if the generated URL key is valid. The ajax callback returns a JSON dump which contains what the URL key should be (e.g. if 'test' was taken it returns 'test-1') 4. If the URL key returned in the ajax is different than the one generated by the JS then: 4a. If the user hasn't clicked the edit button then: it automatically updates the URL key field with the URL key supplied in the ajax 4b. If the user has clicked the edit button then: a form error next to the input appears saying that the URL key is taken and within that error is suggests the URL key returned by the ajax as alternative (the user can click said URL key within the error to automatically choose that one) The URL key isn't reserved by the ajax and upon form submission the usual validation should still apply." johnmartin 2908 Dataset related with image should retain aspect ratio ckan demo phase 5 enhancement new 2012-09-06T11:32:42Z 2012-09-06T11:32:42Z At the moment if you have an related item with an image associated to a dataset that appears in the side column it's resized but doesn't keep it's original aspect ratio. johnmartin 2909 User profile headers are a bit weird ckan demo phase 5 enhancement new 2012-09-06T11:35:48Z 2012-09-06T11:35:48Z They seem to be not exactly well thought out. Needs some thought on better display of the information that they contain. johnmartin 2910 User listings are not very compelling ckan demo phase 5 enhancement new 2012-09-06T11:39:47Z 2012-09-06T11:39:47Z They are currently are just a small image / name. They should be: Larger image, Full name (username), description, edits/dataset johnmartin 2547 ODS Initial data sets ckan opendatasuite 2 enhancement shevski assigned 2012-06-18T10:55:06Z 2012-06-26T10:13:34Z " == Requirements == The ODS demo site will need data adding, initially as fixtures but it would also be useful if we started evaluating datasets that we can ship with ODS installations (at least in the UK) from places such as DGU and ONS. May wish to create a ticket for making sure the datasets within the system are reset every X hours. Perhaps. == Interface == None == User Stories == * As a new system administrator for an ODS instance, I don't want to have a site devoid of any data. Geographically relevant datasets would be welcomed. * As a bizdev person I would like to be able to demonstrate how ODS works with real datasets. == Tasks == [ ] Identify relevant sources for datasets [ ] Pick datasets [ ] Set them up for import == Estimates == " ross 2867 1.8 url_for error ckan enhancement new 2012-08-16T10:57:49Z 2012-08-16T14:44:39Z "From: <thedatahub.org@s077.okserver.org> Date: Thu, Aug 16, 2012 at 10:30 AM Subject: WebApp Error: <class 'routes.util.GenerationException'>: url_for can only return a string, got unicode instead: https://ckannet-storage.commondatastorage.googleapis.com/file/fd3e00d9-eb0d-479b-8303-38909e5400ca?Signature=cmuU3CzRHQ86F3aZ0gljxv%2B3IqQ%3D&Expires=1345109432&AWSAccessKeyId=GOOGC6OU3AYPNY47B66M&x-goog-meta-uploaded-by=d5c8ed25-70de-4035-b29d-ddbe363913c6 To: ckan-sysadmin@okfn.org, kindly@gmail.com URL: http://ckan.net/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca Module weberror.errormiddleware:162 in __call__ << __traceback_supplement__ = Supplement, self, environ sr_checker = ResponseStartChecker(start_response) app_iter = self.application(environ, sr_checker) return self.make_catching_iter(app_iter, environ, sr_checker) except: >> app_iter = self.application(environ, sr_checker) Module beaker.middleware:73 in __call__ << self.cache_manager) environ[self.environ_key] = self.cache_manager return self.app(environ, start_response) >> return self.app(environ, start_response) Module beaker.middleware:155 in __call__ << headers.append(('Set-cookie', cookie)) return start_response(status, headers, exc_info) return self.wrap_app(environ, session_start_response) def _get_session(self): >> return self.wrap_app(environ, session_start_response) Module routes.middleware:131 in __call__ << r'\1', oldpath) response = self.app(environ, start_response) # Wrapped in try as in rare cases the attribute will be gone already >> response = self.app(environ, start_response) Module pylons.wsgiapp:125 in __call__ << controller = self.resolve(environ, start_response) response = self.dispatch(controller, environ, start_response) if 'paste.testing_variables' in environ and hasattr(response, >> response = self.dispatch(controller, environ, start_response) Module pylons.wsgiapp:324 in dispatch << if log_debug: log.debug(""Calling controller class with WSGI interface"") return controller(environ, start_response) def load_test_env(self, environ): >> return controller(environ, start_response) Module ckan.lib.base:239 in __call__ << try: res = WSGIController.__call__(self, environ, start_response) finally: model.Session.remove() >> res = WSGIController.__call__(self, environ, start_response) Module pylons.controllers.core:221 in __call__ << return response(environ, self.start_response) response = self._dispatch_call() if not start_response_called: self.start_response = start_response >> response = self._dispatch_call() Module pylons.controllers.core:172 in _dispatch_call << req.environ['pylons.action_method'] = func response = self._inspect_call(func) else: if log_debug: >> response = self._inspect_call(func) Module pylons.controllers.core:107 in _inspect_call << func.__name__, args) try: result = self._perform_call(func, args) except HTTPException, httpe: if log_debug: >> result = self._perform_call(func, args) Module pylons.controllers.core:60 in _perform_call << """"""Hide the traceback for everything above this method"""""" __traceback_hide__ = 'before_and_this' return func(**args) def _inspect_call(self, func): >> return func(**args) Module ckan.controllers.storage:190 in file << return fapp(request.environ, self.start_response) else: h.redirect_to(file_url) >> h.redirect_to(file_url) Module ckan.lib.helpers:57 in redirect_to << if are_there_flash_messages(): kw['__no_cache__'] = True return _redirect_to(url_for(*args, **kw)) def url(*args, **kw): >> return _redirect_to(url_for(*args, **kw)) Module ckan.lib.helpers:79 in url_for << # fix ver to include the slash kw['ver'] = '/%s' % ver my_url = _routes_default_url_for(*args, **kw) kw['__ckan_no_root'] = no_root return _add_i18n_to_url(my_url, locale=locale, **kw) >> my_url = _routes_default_url_for(*args, **kw) Module routes.util:265 in url_for << if not isinstance(url, str) and url is not None: raise GenerationException(""url_for can only return a string, got "" ""unicode instead: %s"" % url) if url is None: raise GenerationException( >> ""unicode instead: %s"" % url) GenerationException: url_for can only return a string, got unicode instead: https://ckannet-storage.commondatastorage.googleapis.com/file/fd3e00d9-eb0d-479b-8303-38909e5400ca?Signature=cmuU3CzRHQ86F3aZ0gljxv%2B3IqQ%3D&Expires=1345109432&AWSAccessKeyId=GOOGC6OU3AYPNY47B66M&x-goog-meta-uploaded-by=d5c8ed25-70de-4035-b29d-ddbe363913c6 CGI Variables CKAN_CURRENT_URL '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' CKAN_LANG 'en' CKAN_LANG_IS_DEFAULT True CONTENT_TYPE '; charset=utf-8' DOCUMENT_ROOT '/etc/apache2/htdocs' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT '*/*' HTTP_ACCEPT_CHARSET 'utf-8;q=0.7,iso-8859-1;q=0.2,*;q=0.1' HTTP_CONNECTION 'close' HTTP_HOST 'ckan.net' HTTP_USER_AGENT 'Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)' PATH_INFO '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' PATH_TRANSLATED '/etc/ckan/datahub/apache.wsgi/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' REMOTE_ADDR '127.0.0.1' REMOTE_PORT '37236' REQUEST_METHOD 'GET' REQUEST_URI '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' SCRIPT_FILENAME '/etc/ckan/datahub/apache.wsgi' SERVER_ADDR '127.0.0.1' SERVER_ADMIN '[no address given]' SERVER_NAME 'ckan.net' SERVER_PORT '80' SERVER_PROTOCOL 'HTTP/1.0' SERVER_SIGNATURE '<address>Apache/2.2.22 (Ubuntu) Server at ckan.net Port 80</address>\n' SERVER_SOFTWARE 'Apache/2.2.22 (Ubuntu)' WSGI Variables application <beaker.middleware.CacheMiddleware object at 0x7f5312a51650> beaker.cache <beaker.cache.CacheManager object at 0x7f5312a51790> beaker.get_session <bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x7f5312a516d0>> beaker.session {'_accessed_time': 1345109433.273328, '_creation_time': 1345109433.273328} mod_wsgi.application_group 'ckan.net|' mod_wsgi.callable_object 'application' mod_wsgi.handler_script '' mod_wsgi.input_chunked '0' mod_wsgi.listener_host '' mod_wsgi.listener_port '8080' mod_wsgi.process_group 'datahub' mod_wsgi.request_handler 'wsgi-script' mod_wsgi.script_reloading '1' mod_wsgi.version (3, 3) paste.cookies (<SimpleCookie: >, '') paste.registry <paste.registry.Registry object at 0x7f5303b05610> paste.throw_errors True pylons.action_method <bound method StorageController.file of <ckan.controllers.storage.StorageController object at 0x7f530891fa10>> pylons.controller <ckan.controllers.storage.StorageController object at 0x7f530891fa10> pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'} pylons.pylons <pylons.util.PylonsContext object at 0x7f52f864f850> pylons.routes_dict {'action': u'file', 'controller': u'ckan.controllers.storage:StorageController', 'label': u'file/fd3e00d9-eb0d-479b-8303-38909e5400ca'} repoze.who.logger <logging.Logger object at 0x7f53127b2b50> repoze.who.plugins {'openid': <OpenIdIdentificationPlugin 139994769488336>, 'friendlyform': <FriendlyFormPlugin 139994769488528>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator object at 0x7f5312d893d0>, 'auth_tkt': <AuthTktCookiePlugin 139994771858704>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f5312d89350>} routes.route <routes.route.Route object at 0x7f531296e690> routes.url <routes.util.URLGenerator object at 0x7f52f82cd450> webob._parsed_query_vars (GET([]), '') webob.adhoc_attrs {'language': 'en-us'} wsgi process 'Multi process AND threads (?)' wsgi.file_wrapper <built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f52f5870cd8> wsgi.version (1, 1) wsgiorg.routing_args (<routes.util.URLGenerator object at 0x7f52f82cd450>, {'action': u'file', 'controller': u'ckan.controllers.storage:StorageController', 'label': u'file/fd3e00d9-eb0d-479b-8303-38909e5400ca'}) " ross 2362 Improve plugin documentaion, including examples. ckan ckan-v1.9 enhancement toby accepted 2012-05-01T16:13:11Z 2012-07-12T09:30:20Z Improve documentation for plugins and add examples. Could use ckanext-example as a base. kindly 2550 User types ckan datahub-july enhancement icmurray assigned 2012-06-18T11:06:30Z 2012-09-03T10:51:37Z "== Requirements == In the data hub plugin we require the ability to differentiate users between those that have paid for a service, and those that haven't. The distinction isn't boolean as there may be levels of service for paid users, so it may be that we need a 'type' of user where there are various grades of 'paid' which are likely to be strings (specific to the data hub). == Required interface == Once changes have been made to the user schema, for a given user we want to be able to: * determine if they have a paid or a free account, and * get a string name of the type of paid account. Care should be taken to ensure that the 'paid' status of the user cannot be set through the API and only by the datahub plugin. == User Stories == User stories related to the management, setting and changing of a user's payment level, as well as historical information on payments should be done as part of the work that includes actually allowing purchases. For now it is adequate that we can manually control these things through paster commands. Payments types should be linear as I don't believe for this type of service a pick-and-mix modular model would work well. Organizations will inherit the payment level of their owner, so currently there is no requirement for it to affect organizations at all. * As a sysadmin I would like to be able to use a paster command to manually set a user's payment level, or remove it entirely. * As a sysadmin I would like to be able to run a paster command to view a list of users who have a payment plan, grouped by the plan that they have. * As a sysadmin I would like to be able to use the API to change the payment status of a specific user through user_create and user_update. This shouldn't be available to anybody else. * As a user, and only if I have one, I'd like to see my current payment level on my user profile page. == Tasks == [x] Tests [x] Plugin based migration [x] Code [x] Model [x] API [x] Documentation == Estimates == " ross