id type owner reporter milestone status resolution summary description PosixTime ModifiedTime
1785 enhancement seanh seanh ckan-v1.9 accepted Replace 'Revisions' page with site-wide activity stream Replace the /revision page with a /activity page showing a site-wide activity stream of all activities in the site. Or perhaps the site-wide activity stream can go somewhere on the front page of the site instead? 1328632458000000 1340724312000000
2336 enhancement seanh seanh ckanbuild accepted Move Jenkins' install script into ckan core so it can be versioned 1335792914000000 1340639637000000
2341 enhancement seanh seanh ckan-v1.9 accepted Fix Jenkins issue when testing branches that contain model changes 1335876396000000 1340639472000000
2342 enhancement seanh seanh ckan-v1.9 accepted Fix Jenkins issue when testing branches with different solr schema versions 1335876445000000 1340639494000000
2357 enhancement aron.carroll aron.carroll demo phase 5 accepted Create build script for front-end resources Should minify and concatenate JavaScript and CSS as part of #2354 1335886181000000 1343220524000000
2362 enhancement toby kindly ckan-v1.9 accepted Improve plugin documentaion, including examples. Improve documentation for plugins and add examples. Could use ckanext-example as a base. 1335888791000000 1342085420000000
2416 enhancement toby aron.carroll demo phase 5 accepted Normalise resource/data types "Currently we have far too many types that are essentially the same format. The new demo theme is using icons for common types. So we need to normalise the common formats into pretty strings. e.g. application/json, JSON, .json and json all should be output as “json”
See the following Basecamp thread for UI examples and discussion.
https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61906351/comments#comment_179681056" 1337792946000000 1342617802000000
2430 enhancement seanh seanh ckan-v1.9 accepted Look into marking tickets as dependent on other tickets in trac 1337947086000000 1341234346000000
2431 enhancement seanh seanh ckan-v1.9 accepted Trac git integration 1337947128000000 1341234362000000
2466 task seanh seanh ckan-v1.9 accepted Fix spam handling on trac 1338213103000000 1341234539000000
2475 enhancement seanh seanh ckan-v1.9 accepted Upgrade trac.ckan.org to trac 0.12.3 "This is not trivial, requires a database upgrade. But 0.12.3 has some nice new features (multiple vcs repository support, ticket comment editing and nicer preview) and the latest spam filter plugin requires 0.12.
http://trac.edgewall.org/wiki/ChangeLog" 1338317176000000 1341234564000000
2477 enhancement seanh shevski ckan-v1.9 accepted Add extensions section to readthedocs Need a central up to date place for currently supported extensions (closest we have is http://wiki.ckan.org/List_of_Extensions) & their documentation. I suggest this gets added to a dedicated section of docs.ckan.org 1338393244000000 1343124123000000
2478 enhancement seanh seanh ckan-v1.9 accepted Get NeverNotifyUpdaterPlugin installed on trac.ckan.org 1338396734000000 1341234591000000
2493 enhancement aron.carroll toby demo phase 5 accepted Demo site 404 needs some love "templates/error_document_template.html
I've converted the 404 page to the new style but we probably want to serve this as a 'full page' ie content is 100% page width not sidebar and primary content
just go to a non-existant url to see
Discusion:
https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/63374042/
Implementation:
http://s031.okserver.org:2375/404" 1338986081000000 1343135815000000
2516 enhancement seanh seanh ckan-v1.9 accepted Make 'Assign to:' field on trac.ckan.org into a dropdown list there's a setting for this 1339578442000000 1341234822000000
2537 enhancement seanh seanh ckanbuild accepted Test and document ckanbuild "https://github.com/okfn/ckanbuild
Verify that what's there so far still works, write a README explaining how it works" 1339775328000000 1340639830000000
2538 enhancement seanh seanh ckanbuild accepted Add multiple-instance support to ckanbuild "Probably use ansible to do this. To create an instance, create a dir at /etc/ckan/MYSITE, and put MYSITE.wsgi, MYSITE.ini and who.ini files in it. Also put a MYSITE file in /etc/apache2/sites-available. See the example files already present in ckanbuild. Booting a new site should be a single command.
May not handle the postgres/solr/elastic-search side of things yet, could just require the user to set these up herself first and then pass them as args to the create-instance command." 1339775499000000 1340639836000000
2539 enhancement seanh seanh ckanbuild accepted Investigate the existing ckan debian package for ckanbuild Do we want to build on top of the existing debian packaging code? Or throw it away and start fresh? 1339775661000000 1340639845000000
2540 enhancement seanh seanh ckanbuild accepted Implement a way of upgrading ckan sites using ckanbuild When there are multiple ckan sites installed on a single server via ckanbuild, there needs to be some way of upgrading them all to a new ckan version at once. 1339775740000000 1340639850000000
2541 enhancement seanh seanh ckanbuild accepted Add non-core extensions to ckanbuild We want some extensions from outside of CKAN core to be included in ckanbuild. These would be pip installed into the virtualenv before packaging the debian package. Decide which extensions to include. 1339775826000000 1340639856000000
2542 enhancement seanh seanh ckanbuild accepted Create jenkins job to run ckanbuild, and run tests It should run the script to create the debian package, boot a VM, install the debian package on the VM, boot a CKAN instance, then run the tests. 1339775888000000 1340639863000000
3021 enhancement johnmartin johnmartin accepted Logout doesn't work without JS "Essentially, the functionality should be as follows:
- Add logout link that has `.js-hide` attached to it within the header that isn't hidden within a dropdown
See http://plus.google.com/ (when logged in) with and without JS to see an example of the actual sign-out working without JS" 1353410350000000 1353411968000000
140 enhancement rgrp ckan-backlog new News section on front page "Have a news section (suggest as a sidebar item).
News section will link to latest 3/4 blog posts on CKAN from blog.okfn.org.
Details:
* Suggest pulling via rss or similar.
* Will want to cache this ...
Cost: 4h?" 1254902541000000 1265625159000000
331 enhancement rgrp ckan-backlog new Timezone of CKAN timestamps should be configurable "Revisions are timestamped using the server's clock, which may not relate to the expected timezone for the site. e.g. the Norway site has a server on GMT. No timezone info is displayed either.
Would like to set timezone for a CKAN instance to use in rendering revision timestamps. For example, use CET or EST timezone." 1275302440000000 1339774701000000
351 enhancement dread ckan-backlog new Homepage: list new, updated and 'hot' packages "Have a simpler list of exciting data, as opposed to the big revision list.
For example:
{{{
Hot data
===========
New packages: package1, package2, package3
Updated resources: package1, package2, package3
Popular packages:
}}}" 1276595816000000 1339774677000000
369 enhancement shudson@… ckan-backlog new """Package Listing Key"" should appear on Tag results" "Currently there's a nice legend titled ""Package Listing Key"" that appears in right side of ""Browse Packages"" results. The same key should show on other search results like when searching for a tag." 1279821634000000 1339774666000000
370 enhancement shudson@… ckan-backlog new Use better email encryption for author_email and maintainer_email "The JavaScript email encryption used is not very reassuring. Google's MailHide is a much better solution that is easily implemented.
http://www.google.com/recaptcha/mailhide/
Check on the Mailhide API where there are even some Python libraries already built." 1279821819000000 1339774649000000
653 requirement dread ckan-backlog new Trackback links for packages "When people link to a package, a track-back link is auto-created. (Similar system as for blogs).
As suggested by Tim Davies:
Allowing some form of ‘track back’ against datasets
When a non-technical user comes to look at a dataset it would be really useful for them to be able to see if anyone has created an interface interpretation of it already.
I found quite a few cases in research of end-users struggling to make sense of a dataset when good interfaces to that data had already been built and blogged about, but without there being any link from the dataset listing to those data uses.
Accepting track backs could also make it easier for technical users to find blog posts / shared code etc. relating to a given dataset.
" 1285062025000000 1339774636000000
737 enhancement dread ckan-backlog new Markdown syntax summary page "I suggest we produce a quick Markdown cheat-sheet page, showing the key runes: e.g. create a title and quote some text. This page can link to the full Markdown docs for advanced users.
A user going to the Markdown docs that we link will have to read a couple of pages of the raison-d'etre of Markdown before he gets to the syntax. And it's not very easy to read, and being white on black it looks like proper geek stuff.
" 1287766749000000 1323170239000000
811 defect cygri ckan-backlog new Extra field editing form layout breaks when there are long field names The layout of the editing section for extra fields breaks when a field name is slightly too long. Field names jump over to the right. See http://ckan.net/package/edit/dbpedia for examples. 1289994812000000 1323170289000000
812 defect cygri ckan-backlog new Package edit form only allows three extra fields "= Rationale =
The package edit form is restricted to three extra fields. To enter more than three fields, one has to save the package and hit edit again (or hit preview).
= Implementation =
A mechanism similar to the one for resources (where you can add lines as you go) would solve this. So, have a button that adds more extra field rows via JS. (Extra fields don't need up/down buttons that the Resource table has)
Nice to have: a blank field is added when you tab from the last filled-in field in the table." 1289995010000000 1311176917000000
818 requirement cygri ckan-backlog new Rethinking the author and maintainer fields "The semantics of the Author and Maintainer fields are really unclear at the moment. This leads to very inconsistent usage. Also, perhaps Name and Email are not the only fields that are needed for a contact.
Here is a table that shows the current usage of these fields in CKAN:
http://richard.cyganiak.de/2010/ckan/ckan-ppl.html
We note several problems:
* Author and Maintainer are often the same
* Author and Maintainer are often used interchangeably
* People really want to specify URLs for the contacts and stick them into random places because there is no field for it
* Multiple comma-separated names in a single field
I'm not sure what to do about this, but a redesign is necessary in my opinion.
Some ideas:
* Remove the maintainer field?
* Make really clear that Author doesn't refer to the metadata on CKAN, but to the original data
* Add an “author URL” field?" 1290003524000000 1339774621000000
837 enhancement rgrp ckan-backlog new CKAN integration with freebase gridworks / google refine "Thread: http://lists.okfn.org/pipermail/ckan-discuss/2010-November/000718.html
=== Scenario 1 ===
1. User installs Refine and CKAN extension for refine
2. On booting refine and asked to load data they can choose from any data package on CKAN.net (or any other CKAN instance)
3. They edit the dataset on Refine
4. On save (or perhaps as a separate option) they are prompted as to whether they wish to sync the dataset back to CKAN (either as a new package or as a new resource on the existing package)
NB: for the dataset sync back some form of ""CKAN"" storage would be
required (we already have storage.ckan.net running but a closer
integration would be required)
=== Scenario 2 ===
1. User visits a package on CKAN.net (or another CKAN instance)
2. There is a button on the page ""View and edit this dataset in Google Refine""
3. Click button -- ask them if they have Google refine installed
* Yes: instructions for loading dataset into refine
* No: load dataset in hosted version of google refine (we could run this)
4. User edits dataset and hits save. As in previous scenario they are prompted to sync the dataset." 1291140609000000 1339774605000000
895 defect memespring ckan-backlog new Add version number (or simular) to css/js includes query string "Updates to css after a new deploy don't come through without a hard refresh. Adding the version number to the include urls will solve this e.g.
mycssfile.css?v=12345678" 1294343382000000 1339774593000000
924 enhancement dread ckan-backlog new Search box has no search button "The search box at the top-right of CKAN's page doesn't have a 'go' button. I feel that a larger percentage of users expect a 'go' or 'search' button on the right-hand side of the box to press to start searching. Techies tend to know the keyboard shortcut of pressing 'carriage-return' but it might be better to follow standard practise on this.
Examples with 'search' button: Internet Explorer, Firefox, Google, Amazon, trac
Examples without: ?" 1295867533000000 1323170436000000
989 enhancement kindly pudo ckan-future new Extending the model from plugins "We need to support extending the model from plugins. This could involve:
* Adding a plugin hook to extend the mapper
* Adding an upgrade hook for plugin schema migrations
* Documenting how this is to be done
* Find a way to avoid conflicts " 1297689724000000 1340034311000000
1009 enhancement pudo rgrp ckan-backlog new Improvements to user accounts sytem " * Forgot password (email a new password)
* Confirm email
* Do not show register page if you are logged in (redirect to home page)
* ticket:1010 - listing of users.
* Do not use /user for general user account home page (either user normal user page /user/{id} or /user/myaccount)
" 1298635991000000 1310128574000000
1077 enhancement kindly rgrp ckan-backlog new Move to simpler vdm system "== Option 1: 'Changeset' Model ==
See ticket:1135 for vdm ticket. This would involve a) moving to changeset in vdm b) doing the migration in ckan to support this.
Have developed a new ""changeset"" based model for revisioning in vdm.
=== Implementation ===
* The main challenge with this change is schema and data migration
Every revisioned object has a revision_id and revision attribute.
Approximate algorithm:
{{{
Revision -> Changeset
for revtype in [PackageRevision, ...]:
for pkgrev in package_revision:
changeset = lookupchangeset(package_revision)
ChangeObject(cset, (table, id), dictize(pkgrev))
}}}
Question:
* does pkg include tags attributes or not? or we have to dictize, pkgrev, pkg2tagrev, and tag. Probably the latter.
== Option 2: Simplify Revision Object Model ==
Just use a simpler vdm, see ticket:1136 (move to !SessionExtension) and ticket:1137 (remove need for statefulness in vdm).
== Discussion ==
Advantage of Option 1 versus 2:
* Easier support for pending state and similar behaviour
* No need to introduce new tables (and hence migrations) when making something revisioned (or not).
=== Disadvantages ===
* Migration is required
* More difficult to query revision history.
* Could be addressed by having !ChangeObject have separate cols for table name and id but would likely be more difficult.
* Performance (?)
* Have one big !ChangeObject table to query when looking at changed objects rather than many revision tables.
* Not sure this is a biggie as even with Revision model biggest revision object tables are probably on the order of the !ChangeObject table
== Conclusion ==
Implement Option 2 and leave Option 1 for present.
Option 1 includes Option 2 so it seems that that is required in either case (so we may as well with Option 2).
Option 1 requires significant effort (esp migration) so leave for present and then review the situation at some later date." 1302304464000000 1340034345000000
1096 defect rufuspollock pudo ckan-future new [super] CKAN Hosted "Many users of CKAN want to have their own instance without much effort. Setting these up in separate places is a maintenance nightmare, we should much rather have some tenant separation in core CKAN. Some ideas:
* introduce model.Site and c.site
* site has: custom CSS, extra_template_path, title, languages list, package_form, group_form (all configured via web UI)
* Subdomain detector to activate sites.
* use site in Authorizer instead of System, have a NullSite for global things
* allow cross-site search
* packages are in a list of sites, m:n rather than 1:n
* list of sites is string-based, can contain sites not in site table to express harvested external material which is not editable locally. " 1303235062000000 1339774484000000
1101 enhancement sebbacon ckan-backlog new Integrate googlanalytics into site nav "There's a stats plugin (e.g. at http://trac.ckan.org/ticket/832).
Output from the googleanalytics plugin should append to that page, if the stats plugin is present.
Possibly the stats plugin and the googleanalytics plugin should be merged?
Finally, if the stats plugin is active, then a link to the stats page should be added to the main site footer." 1303393926000000 1339774582000000
1120 enhancement tsm ckan-backlog new Atom feeds of each tag "Tags could/should have an Atom feed. This would mean that every edit to relevant packages could be easily monitored. See [1].
[1] http://lists.okfn.org/pipermail/ckan-discuss/2011-May/001162.html" 1304309285000000 1339774568000000
1134 CREP amercader ckan-backlog new CREP0003: Description and Configuration of Harvesters "'''Proposer''': Adrià Mercader
== Abstract ==
The new harvester interface allows to create harvesters for different
sources, but right now harvesters don't have many ways to describe and
configure themselves. We need a way of allowing them to:
* Expose their type and other details so they can be used internally
and on the UI.
* Define configuration settings for particular harvester instances.
== The Problem ==
=== Harvester description ===
The current UI for adding and editing harvest sources is the same used
in ckanext-dgu, and thus the 3 harvester types used in DGU to harvest
various GEMINI realted sources are hardcoded in the form. The form will
be migrated to a DGU-independent one, so we need the harvesters to
provide all the necessary data. There is a current {{{get_type}}} method
that returns the harvester type, but for make it compatible with the DGU
forms, it returns a machine-readable string (e.g. ""CSW Server""), making
it error prone.
=== Arbitrary configuration ===
In the current implementation, when the harvest process is started,
ckanext-harvest looks for all the available plugins that implement the
{{{IHarvester}}} interface and calls the appropiate methods for the
current stage ({{{gather_stage}}},{{{fetch_stage}}},{{{import_stage}}}).
At these stages, harvesters have no way of applying arbitrary
configuration options, so all harvesters of the same type behave on the
same way.
For instance, the CKAN harvester needs a way to define the API version
to use when harvesting remote instances (Right now, the version 2 is
hardcoded on the code).
== Specification ==
=== Harvester description ===
Harvesters will need to provide the following information so the UI form
can be built:
* name: machine-readable name (e.g. ""waf""). This will be the value
stored in the database, and the one used by ckanext-harvest to
call the appropiate harvester.
* title: human-readable name (e.g. ""Web Accessible Folder (WAF)"").
This will appear in the form's select box.
* description: a description of what the harvester does (e.g. ""A Web
Accessible Folder (WAF) displaying a list of GEMINI 2.1
documents""). This will appear on the form as a guidance to the
user.
The way to provide it will be an {{{info}}} method that all harvesters
must implement, which will return a dictionary with the previous
elements:
{{{
{
'name': 'csw',
'title': 'CSW Server',
'description': 'A server that implements OGC's Catalog Service
for the Web (CSW) standard'
}
}}}
=== Arbitrary configuration ===
As different harvesters will have very different needs, we need to
provide a way to persist arbitrary configuration flags for each harvest
source. The more flexible way given the current architecture in my
opinion would be to store the configuration options as a JSON encoded
object as a property of the harvest source (There already is an unused
DB field called {{{config}}} in the database) (Maybe using JsonType?).
This will mean adding an extra field in the harvest source form to allow
entering the configuration. This could be just a simple text field where
users enter the JSON encoded object or a more clever mechanism (i.e an
""Add a configuration flag"" link that adds two new text fields for the
key and value for each flag, and a mechanism to later build the JSON
object). In any case, this should probably be hidden in an ""Advance
options"" section.
== Why do it this way ==
=== Harvester description ===
The {{{info}}} method would provide a single point to get all the
information related to the harvester, and future properties could be
added to the dictionary returned without having to modify the interface.
=== Arbitrary configuration ===
There is an already existing {{{config}}} field in the database, so we
won't need to change the model.
Harvesters could access the config object at any of the stages. Of
course they could provide default values in their implementations so
users don't need to enter them everytime.
== Implementation plan ==
=== Deliverables ===
=== Risks and mitigations ===
The highest risk on the harvesters {{{info}}} method side is that
harvester implementation don't offer one of the necessary properties
(namely name and title). This could fire a warning when showing the
UI form or using the CLI.
=== Participants ===
Adrià Mercader to do it.
=== Progress ===
None yet." 1305108868000000 1339774554000000
1144 enhancement timmcnamara ckan-backlog new Support DSPL "DSPL, the Dataset Publishing Language, is being promoted by Google for its ""Google Public Data Explorer"" system. It is an XML format with metadata.
The format is described on the [http://code.google.com/apis/publicdata/docs/developer_guide.html developer docs ofthe Google Code site].
Google provides a [http://code.google.com/apis/publicdata/docs/dsplgen.html Python script which reads CSV data and generates DSPL]
Sample from http://code.google.com/apis/publicdata/docs/dspl_sample.html:
{{{
#!xml
My statistics
Some very interesting statistics about countries
http://www.stats-bureau.com/mystats/info.html
Bureau of Statistics
http://www.stats-bureau.com
Geography
Social indicators
Population indicators
Poverty & income
Health
Population
Size of the resident population.
Country
My list of countries
Country name
The official name of the country
State
US states
Gender
Gender, Male or Female
Genders
Both genders
unemployment rate
The percent of the labor force that is unemployed, not seasonally
adjusted.
http://www.bls.gov/cps/cps_htgm.htm
true
}}}
" 1305763609000000 1339774517000000
1145 enhancement timmcnamara ckan-backlog new Support the Handle System "The Handle System is an initiative to provide persistent references for resources. That is, it's basically a proxy system for preventing link rot.
Its documentation is here: http://www.handle.net/. Servers running CKAN could host a ""Local Handle Service"", which redirects a hash of a resource to an actual URL.
Some suggested use cases:
- Researcher would like to cite where data came from
- Agencies would like to have a way to prevent vendor
lock-in from CKAN if they decide to move to another
platform" 1305764775000000 1339774502000000
1152 enhancement amercader amercader ckan-backlog new True support for generic CSW servers "The CSW harvesters implemented at the moment were developed with the DGU
project in mind, and they assume all remote CSW servers to implement the
Gemini 2 specification. Gemini 2 is the profile defined in the UK for
INSPIRE complying metadata, so obviously catalogs from other countries
or non-INSPIRE complying ones won't be able to be harvested.
The changes needed to support generic CSW servers (i.e. those
implementing the ISO 19139 profile) are:
* Handling the validators (right now are hardcoded in the harvester
code). This probably involves issues discussed in the CREP 3
(ticket #1134)
* Changes in the model to adapt the specification to ISO 19139
* Renaming objects and classes which are now Gemini-centric
List of CSW servers tested:
https://spreadsheets.google.com/spreadsheet/ccc?key=0Atp3cZFjuIOAdDBVQWRINnlfN1d0b2lleHVEdjBSb2c&hl=en_US&authkey=CNu4hsEB#gid=0" 1306141334000000 1313411822000000
1163 enhancement rgrp rgrp ckan-backlog new Improvements to Storage Extension "Storage is now working but there are
* Integrate with Resources (e.g. create a resource for each file upload and give option to associate with a package)
* Should we introduce rule that files *not* associated with a Resource are periodically deleted?
* Allow setting of a file name/path before upload
* Allow for file overwriting/deleting etc (how should this work -- do we want to allow this sort of thing)
* Integrate local file upload stuff in api/auth/*
== Different Backend Issues ==
Local file store is rather different from 'remote' storage in various ways:
* For remote you don't want to use many buckets as there are bucket limits while for local you want to. Should we there have a single path that users provide which we then partition differently for different backends.
" 1306408778000000 1310133808000000
1165 enhancement nils.toedtmann ckan-future new Add multi-site support to ckan "Currently, each ckan site needs its own ckan wsgi process. That eats a lot of resources where many ckan sites are served from one machine (e.g. eu3).
That would dramatically change if a ckan process could behave like multiple ckans (e.g. like Apache's """", or tracd). Depending on the ""Host:"" header in the HTTP1.1 request, it would choose which local ckan ini file to obey.
I see two ways to constitute the map hostname-to-ini-file map:
* ckan reads a set of ini files, and each ini file declares which servers names it is responsible for
* In a global ini file, there are directives mapping servernames to ini files.
In either case there should be a global ckan ini having the default settings for all local ckan sites. Each site ini could be very short then, just having e.g. title, name, database credentials, active plugins etc." 1306413667000000 1339774466000000
1179 enhancement timmcnamara ckan-backlog new Support tag aliases "A small number of tags are near-duplicates of each other.
Perhaps we could support word stemming from NLTK and/or manual tag aliases:
statistics
statistik
...
survey
surveying
surveys" 1307429221000000 1339774332000000
1182 defect timmcnamara ckan-backlog new "Comments from deleted packages appear in ""Recent Comments"" feed" "When a package has been deleted, say for spam moderation, comments still appear in the recent comments section.
This is a problem because non-admin users will be shown a warning that they're not authorised to view the package if they click on the link.
At CKAN.net currently, this affects the most recent comment." 1307658251000000 1339774319000000
1184 enhancement timmcnamara ckan-backlog new Support Wuala as CKAN storage option "Most of CKANs storage options are tied to the USA. This brings concerns of data security for some organisations who may wish to adopt the system. [http://wua.la Wuala] is a distributed file system that stores data in a peer-to-peer manner. The company behind it, LaCie sells storage for a fee. However, they also enable clients to have 'free' storage space when machines act as a storage node.
In order to be a storage node, a machine needs to be online for more than 14% of the time - roughly 4h per day. Most CKAN servers are likely to have a far greater uptime than this.
Supporting Wuala would go some way to enabling CKAN to be used in a secure manner. That is, CKAN could be promoted for organisational use where there is lots of data to be stored and large geographic distances to be managed. There is a [http://www.wuala.com/Wuala%20API/Sample%20Clients/Python/wualaAPI/ Python client available] and a fairly [http://www.youtube.com/watch?v=3xKZ4KGkQY8 long Google Tech Talk] that overviews the system." 1308034751000000 1339774306000000
1185 defect timmcnamara ckan-backlog new Administrators can't delete packages from web UI "Administrators have ""View"", ""Edit"" and ""History"" tabs. However, I can't see a way to delete a package from the web UI.
Version: CKAN.net as of today" 1308111818000000 1339774289000000
1188 enhancement nickstenning ckan-backlog new Allow diffing against initial (blank) package version "Currently the history page only allows diffing between different versions of a package, but there doesn't appear to be any easy way to see the changes introduced by the first version of a package.
I'm requesting the ability to diff against a ""blank slate"" initial state of a project, so I can see the content of the first project commit.
Not sure if this is a vdm feature, so I'm putting this ticket in against ckan." 1308153160000000 1339774275000000
1198 enhancement dread ckan-backlog new Publisher hierarchy "'Publisher' entities in the model. They are hierarchical.
'User-Publisher' connections with one or more roles (e.g. drafter, moderator).
Authorization settings can control who can set what values in a 'published by' type field.
Publishers and User-Publishers available to read in the API.
Future tickets will provide:
* API to write Publishers and User-Publishers
* UI to edit Publishers and User-Publishers
(This feature deprecates authorization groups)" 1308820592000000 1339774200000000
1201 enhancement kindly ckan-backlog new seperate out logic in atom feeds to logic layer. Simplify the logic in the atom feed an make all feeds use logic layer to return lists. 1308928892000000 1310124297000000
1203 defect johnglover rolf ckan-backlog new "Moderated edits: html code shows as ""changed"" although it is not" "I've installed the Moderated Edits extension (ckanext-moderatededits) and am editing a package imported from IATIregistry.org, with an extra field which contains a bit of HTML.
The editor indicates the field has changed, although the content hasn't (see screenshot). All I can find so far is a minor difference: in the field content, there is a code — and in the rendered table that is an —" 1309274970000000 1313401579000000
1227 enhancement timmcnamara ckan-backlog new Display packages' tags in search results "In when displaying search results, it would be useful to also display the tags of a package. Sometimes it's difficult to infer the scope of what the package does from the title and the first sentence of the description. Tags are quite concise way to display rich information.
ENV=datacatalos.org, with CKAN 1.4.2a" 1311034262000000 1339774147000000
1232 enhancement thejimmyg ckan-backlog new [super] Interface improvements "Child tickets:
* #1194 ""Welcome back"" message for newly registered user
* #1202 Links to datapkg utility don't lead to info about it
* #925 Change the search box icon to remove the down arrow
* #923 Search box doesn't work in leaderboard page in stats extension
* #1034 Flash message cached
* #737 Markdown syntax summary page
* #811 Extra field editing form layout breaks when there are long field names
" 1311178296000000 1315948536000000
1233 enhancement thejimmyg ckan-backlog new [super] Improve wiki-style functionality for history "At the moment we have a good revisioning system but a poor history interface. We need to improve this in a number of areas:
* #191 Searching by modification date
* #193 Searching by time-related field
* #301 Package discussion pages
* #1236 Package history page should provide links to pages at particular revisions, similar to the wikipedia pages
* #1236 Viewing old revisions or unmoderated changes should have a message at the top of the package page
* Other improvements as per my word doc." 1311179392000000 1315948668000000
1235 enhancement thejimmyg ckan-future new [super] Search Improvements "Child tickets:
* #234 UI Review - Autocomplete package names & tags in search
* #193 Searching by time-related field
* #191 Searching by modification date
* #905 Unable to search with accented characters in package names
* #906 Ability to search without accents for accented words
* #924 Search box has no search button
Broadly speaking though we need to choose PostgreSQL, Solr or something else. We don't want to invest our time maintaining two search backends with a limited abstraction layer between the two." 1311182641000000 1311182641000000
1257 enhancement dread ckan-backlog new Anti-Spam tools "We are getting more and more spam on ckan.net and we need to improve our strategy of combating it. It is bad because google ranks who we link to (which we do want for legitimate links), and our front page contains the latest package edits, so spam is immediately very visible.
Spam:
* Package creation
* Package edit
* User creation
* Group creation
Systems to consider:
* Automatic
* captcha
* bayesian scoring
* Sysadmin
* a tool to quickly analyse and remove packages, edits and users
* Helpful users
* button to click indicating spam, leading to any/all of:
* add TODO item (which emails package admin and sysadmins)
* quarantine the package/user so it doesn't show up on the front page and a warning is shown when you view it
General thoughts:
* We should learn from the wikipedia tools, as they are the experts
* This is a serious problem that is only going to get worse and is a threat to the whole site." 1312283565000000 1339774129000000
1259 enhancement johnglover pudo ckan-backlog new """Add a row"" for Extras on Package form" "The default package form offers 4 empty extras fields. Like the resource section, it should have an ""add more"" button to add another row. " 1312302693000000 1312907056000000
1260 enhancement pudo ckan-backlog new Remove duplicate functions from _util.html There seems to be both a list view for dictized and non dictized data structures for all entities in _util.html at the moment. Probably in the back of someone's mind already, but cleanup here would be nice. 1312366652000000 1313401499000000
1261 defect pudo ckan-backlog new Investigate dots in extras search It seems that searching for extras_foo:value works with solr, but extras_foo.bar:value doesn't. No theory on why. 1312366768000000 1312366768000000
1262 enhancement pudo ckan-backlog new "Enforce ""create-user"" permission" This does not seem to have any implications at the moment, it should lock down registration and remove all related links. 1312375296000000 1323090112000000
1273 requirement amercader ckan-backlog new Create docs for API v3 1313412083000000 1313412083000000
1278 enhancement amercader ckan-backlog new Refactor authorized_query calls "There are some functions that still use the Auhtorizer().authorized_query method:
{{{
./ckan/controllers/authorization_group.py:24: query = ckan.authz.Authorizer().authorized_query(c.user, model.AuthorizationGroup)
./ckan/lib/base.py:237: groups = ckan.authz.Authorizer.authorized_query(c.user, model.Group,
./ckan/lib/search/sql.py:55: q = authz.Authorizer().authorized_query(username, model.Group)
./ckan/lib/search/sql.py:118: q = authz.Authorizer().authorized_query(self.options.get('username'), model.Package)
./ckan/logic/action/get.py:154: query = Authorizer().authorized_query(user, model.Group, model.Action.EDIT)
./ckan/tests/test_authz.py:158: q = self.authorizer.authorized_query(self.notadmin.name, model.Package)
./ckan/tests/test_authz.py:353: q = self.authorizer.authorized_query(self.notmember.name, model.Package)
./ckan/tests/test_authz.py:357: q = self.authorizer.authorized_query(self.member.name, model.Package)
./ckan/tests/functional/test_authorization_group.py:44: group_count = Authorizer.authorized_query(u'russianfan', model.AuthorizationGroup).count()
}}}" 1313415177000000 1313415177000000
1286 enhancement rgrp ckan-backlog new Remove remaining formalchemy stuff "Stuff I've spotted:
* forms/*
* template/group/edit_form.html
* template/package/edit_form.html
This can go once new DGU form is in." 1314116996000000 1342436420000000
1288 defect dread ckan-backlog new Package edit/creation can't include 'relationships' field "When you create or edit a package (via the API), you aren't able to specify the relationships it has. (If you do you get 409 {{{ {""__junk"": [""The input field __junk was not expected.""]} }}})
The normal way to create relationships is via /api/rest/relationships/ and this works. But when you GET a package, the dictionary lists all relationship details. So this bug creates a problem for editing a package that has relationships - you want to GET it, make any edits and then PUT it back. The work-around is to delete the 'relationships' key from the dict before you PUT it back.
== Options ==
Ideally, CKAN would read the 'relationships' key and make the necessary changes. This is a chunk of work.
Another good option is to allow an unchanged 'relationships' value, but barf it is edited. This is also a chunk of work.
A bad option would be to just ignore the 'relationships' value, since users will get frustrated changing this value and wonder why it never saves, not understanding it is different to all the rest, without error message.
A final option is to get rid of relationships altogether." 1314203695000000 1339774098000000
1311 enhancement rgrp rgrp ckan-backlog new Modal user register and login form "Subticket of: #1294
Rather than having to visit a dedicated page it would be good if registration and login could be done from a modal form (separate or combined ...).
== Why ==
* It could be used from dataset creation page in situations where user needs to be registered / logged in to create a dataset (so we could allow someone to start creating a dataset and only get them to login at the end ...)
* It allows for quicker and easier logging in
== Implementation ==
* See Friedrich's work on the datahub" 1315297227000000 1315297227000000
1326 enhancement thejimmyg ckan-backlog new Write a set of auth plugin functions to integrate with Druapl Ticket #787 described join auth between CKAN and Drupal. The authentication part is live and implemented. This ticket is a placeholder for work that will be needed in the new auth system to link authorization functions to Drupal. It is dependent on the groups refactor. 1315821084000000 1315821084000000
1336 defect johnglover dread ckan-backlog new License fudge "cset:4b59ab34137d ckan/logic/action/get.py:
{{{
- isopen = model.Package.get_license_register()[license_id].isopen()
- result_dict['isopen'] = isopen
+ try:
+ isopen = model.Package.get_license_register()[license_id].isopen()
+ result_dict['isopen'] = isopen
+ except KeyError:
+ # TODO: create a log message this error?
+ result_dict['isopen'] = False
}}}
This change hides problems with the license server and returns potentially incorrect values for openness.
This has been noted as 'temporary fix' but seems to be forgotten about, since it has been merged to default and gone into release 1.4.3.
I suggest the licenses are cached (I thought this was already the case when CKAN first requests them after start-up?). I suggest failure would return 503." 1315912057000000 1323173073000000
1343 enhancement rgrp rgrp ckan-backlog new [super] User related improvements (login, user pages etc) " * Disallow account creation via openid - #1386
* Require email field - #1319
* Require email confirmation to be activated (?)
* Improvements to user page (e.g. show activity and more info about user) - #1396
* Modal user login - #1311" 1316017098000000 1318528138000000
1352 enhancement amercader ckan-backlog new Use logic functions instead of as_dict when indexing entities "The current search implementation uses the output of the the as_dict method of the domain Package object to update the index
https://bitbucket.org/okfn/ckan/src/56c79e3fc44c/ckan/lib/search/index.py#cl-48
It also uses package_to_api1 in the SynchronousSearch plugin:
https://bitbucket.org/okfn/ckan/src/f9dfb0506594/ckan/lib/search/__init__.py#cl-93
This prevents extensions from being able to index custom properties (e.g. faceting by custom extras not included in the model).
The search should use the logic function to get the package properties:
{{{
get_action('package_show')(context,data_dict)
}}}
" 1316615397000000 1339774086000000
1355 defect amercader ckan-backlog new Package extras property does not include the newly created ones "The extras in the package object sent to the extensions after editing (https://bitbucket.org/okfn/ckan/src/01efd5649c10/ckan/logic/action/update.py#cl-226) do not include the newly added.
" 1317034126000000 1339774056000000
1382 defect thejimmyg thejimmyg ckan-backlog new Deleted resources are present for harvested package Perhaps the importer deletes them before re-importing. We shouldn't have deleted resources, so let's investigate. 1318337889000000 1318337889000000
1384 task rgrp shevski ckan-backlog new CKAN wiki needs updating to refer to thedatahub.org instead of ckan.net and datasets instead of packages Most articles still refer and link to ckan.net, wiki.ckan.net and to packages 1318414077000000 1318414077000000
1403 enhancement zephod zephod ckan-backlog new Refactor groups index page "Groups are listed alphabetically with paging - not an ideal user experience. We would like to list groups in order of 'popularity': The number of datasets they contain.
Following this chain of thought, then, it would be nice to rearrange the groups table by clicking on column headers and having it sort by that column.
Furthermore, then, we'd like to implement a full-fledged groups search feature (if this is at all feasible).
The forthcoming groups refactor will probably have some bearing on this task.
" 1318847512000000 1318847566000000
1406 enhancement zephod ckan-backlog new Re-enable RSS subscriptions "RSS 'subscribe' buttons appeared in many places on the site but were not very helpful. They took (confused) users pointed to the raw feed code, and Google Reader could not understand the feed. Safari, however, could interpret it correctly.
Their presentation needs to be clear and consistent; the RSS feed really needs testing in a variety of readers; and we need to decide exactly which items should get a feed. (Package updates? Groups?)" 1318861327000000 1320930088000000
1411 enhancement zephod rgrp ckan-backlog new Force resource format to be lower case (also mimetype) "Format should be lowercase. Automatically lower case (for extra points have a bit of javascript to force lower case when entering).
* Even more points: do a update on thedatahub repo to make all format lower case (or script this as an update?)" 1319319604000000 1319319604000000
1414 enhancement shevski ckan-backlog new track user log-ins on thedatahub.org "Set up tracking for user logins so that we have stats about how many active users of thedatahub exist
want to be able to see who logged in the the last x months" 1319454782000000 1319454782000000
1423 enhancement markbrough ckan-backlog new Edit resources suggestions " * Description vs Name - Edit Resources view is showing the name of the package rather than the description, and a lot (all?) of the packages before the upgrade don't have names, so might be good to swap this round again, e.g.: http://thedatahub.org/dataset/edit/iati-registry
* Moving resources - Moving them up or down the list used to be quite useful if you had a lot of resources that you might want to leave on the resources page, but only one or two that were actually current and that you wanted to draw attention to. This doesn't exist any more on CKAN but I think it would be good to add it back in." 1319641906000000 1338203678000000
1424 enhancement dread ckan-backlog new Openness notice should be clearer "ckan-discuss discussion suggests changes to the 'openness' indicator
( http://lists.okfn.org/pipermail/ckan-discuss/2011-October/001786.html )
Dataset view page:
* If there is an explicit but non-OKD compliant license, such as CC-BY-NC, then this should be stated explicitly, perhaps: “This dataset is Not Open. License: Creative Commons Attribution Noncommerical. This is not an open license as it does not meet the Open Knowledge Definition.”
* If the license is marked as “Other::License Not Specified”, then this should be stated explicitly, perhaps:
“This dataset is Not Open. It is published without an explicit license, the publisher reserves all rights to the dataset.”
* 3. If the license field was left empty by the contributor of the Data Hub record, then again this should be stated explicitly, perhaps: “This dataset is Not Open. The license of this dataset is unknown or unspecified. Start an enquiry on IsItOpenData »
* There is a bug so that non-open licenses doesn't have an openness notice.
* If downloadable resources are not available, this should not affect 'openness' - check this has been removed." 1319648089000000 1319648089000000
1429 enhancement rgrp rgrp ckan-backlog new Provide DOIs for datasets in a CKAN instance "DOI = digital object identifier = http://www.doi.org/
As a Publisher I want a DOI for my dataset so that it can be cited by and linked to by others in a standard and easy way.
== Details ==
* Probably implement as extension rather than core" 1319977305000000 1319977305000000
1432 enhancement rgrp ckan-backlog new [super] Data processing system for CKAN and Webstore "Super ticket: #1190
A data processing system which utilizes the Webstore. One could
get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules)" 1320142747000000 1339774041000000
1438 enhancement dread ckan-future new Action API - parameter discovery/checking "Many actions in the Action API require parameters. What params are needed should be listed and checked. Because currently, if you get them wrong you simply get a useless 500 error.
Currently they are listed in the docs, extracted from the code manually.
So you could GET /action/api/package_list to receive not only the help text, but a list of arguments.
And if you send an extra or missing argument then an intelligent error message can be returned.
= implementation =
How about some sort of decorator on the action function:
{{{
@logic_params(id, offset, limit)
def get_package_list(context, data_dict):
...
}}}
This would do the param checking, and is there a way to extract these params from the function? Or do a registration of the logic function?
I'd certainly like to keep the list of the list of params for the function with the function, for ease of reading the code.
Another good thing would be to pass in the params named as themselves, rather than having them contained in the data_dict." 1320161890000000 1338205050000000
1439 enhancement dread ckan-backlog new Action API discoverablility "A good service API needs to be discoverable, so you are not always having to refer to the documentation html.
Maybe /api/action should return a list of actions available? (Currently this returns a 404.)
* It would be nice to sort these into get/create/update/delete.
* #1438 Parameters for each of the actions must be discoverable too
/api/action/{action_name} should also return the help text / parameters allowable. (Currently this returns 400 error)" 1320161970000000 1325474974000000
1457 defect jilly.mathews ckan-future new Bug with DataNL instance " ""when logging into http://register.data.overheid.nl/ with OpenID, the /user/me page gives a 404. CKAN version 1.3.4.""
n the manual it says an API key kan be created via http://test.ckan.net/user/me /. However, when I try the corresponding http://register.data.overheid.nl/user/me, I get a 404 error (not found). " 1320925041000000 1320925041000000
1459 enhancement rgrp rgrp ckan-backlog new Featured Dataset feature "Provide way to mark a dataset as featured. Featured database show up on the front page.
TODO: detail this more." 1321113012000000 1321113012000000
1466 enhancement thejimmyg ckan-backlog new Need to support https login for multiple instances as part of the CKAN package install 1321375978000000 1328529062000000
1534 enhancement rgrp ckan-backlog new Change revisions to record userid rather than username "The use of username is problematic because username's can change.
* Change all revision creation code to use user id (simplest is to change c.author field in lib/base.py (?))
* (?) Add a field ipaddr for ip address of anonymous users? (or just keep putting this in author field on Revision and then acception that those won't match when we do a look up against user table)
* Change user view page to look up against user id rather than name
* Perform migration on existing Revision objects
* Match should probably be against both openid and username when searching Revisions' author field (especially true on CKAN where some people have already changed their username from being their openid)" 1323278790000000 1338205050000000
1535 enhancement dread ckan-backlog new Plump for auth header of: X-CKAN-API-KEY "When using the API, the apikey needs to be supplied in a header called 'Authorization'. Because some proxys / deployments use this header for other things, a configurable header was provided as an alternative, with default ""X-CKAN-API-KEY"".
Rufus suggests having *one* way for this.
a) making this not configurable any more
b) making X-CKAN-API-KEY the default
(keep Authorization allowed, but not documented, for backwards compatibility)" 1323279082000000 1339774019000000
1542 enhancement dread ckan-backlog new Buttons to purge spam datasets and groups "A sysadmin should be able to easily examine a suspect group or package, determine if it was created by a spammer (as opposed to being a legitimate object that has been graffitied by a spammer) and purge it.
The existing two-stage revision delete is currently unreliable and perhaps too laborious.
Olav and Richard have needs along this line." 1323364930000000 1339774000000000
1544 task dread ckan-backlog new delete old git branches "We have 150 odd branches ({{{git branch -a}}}) - most of them old - we should prune them. At very least, branches that have been merged in should be deleted. Look at old branches that haven't been merged in and wonder why.
May be of some use:
{{{
git branch --merged
git branch --no-merged
}}}" 1323702610000000 1323702610000000
1557 enhancement David Rasnik jilly mathews ckan-future new Complete Webstore Preview Extension "Finish any work out standing on web store preview extension to be able to package and release.
Ref James and I going through existing features and trying to mention any polishing that needed doing to get exiting features ready for release with projects such as CKAN hosted." 1324291253000000 1324291253000000
1558 enhancement David Raznik jilly mathews ckan-future new Publisher Tools Summarise final set of requirements for this and finish development and test. Estimated 10 working days. 1324291573000000 1324291573000000
1560 enhancement David Raznik jilly mathews ckan-future new Follow extension "Estimate 2 days to finish dev and test.
David can you add any info needed here?" 1324291879000000 1324291879000000
1561 enhancement David Raznik jilly mathews ckan-future new To do extension Can we finish this ready for release on data hub and CKAN Hosted. 1324291972000000 1324291972000000
1562 enhancement Adria jilly mathews ckan-future new Finish Geo Spatial Estimated 4 weeks of Adria's time. I guess this will need to be broken down into more tickets. This feature is being requested by a number of potential customers and we have some ideas of requirements between Rufus and Jilly for this. This is the most popular new feature we talk about to new clients. 1324292193000000 1324292193000000
1564 enhancement David Raznik jilly mathews ckan-future new Structured Data (Data API) "Basic websotre exists but this may be not what is described yet.
CKAN provides a rich API for the data itself, allowing users to query retrieve and use data instantly from datasets in CKAN without needing to download or process it first." 1324292834000000 1324292834000000
1565 enhancement Rufus Pollock jilly mathews ckan-future new Admin dashboard finished? Is testing complete and ready for release? 1324293092000000 1324293092000000
1567 enhancement David Raznik jilly mathews ckan-future new Finish QA extension Requires change to celeryd. Estimated 4 weeks. 1324293599000000 1324293599000000
1569 enhancement David Raznik jilly mathews ckan-future new Wordpressser How much effort will this be to be ready to use? 1324294056000000 1324294056000000
1572 enhancement David Raznik jilly mathews ckan-future new Meta data Harvester Need to write custom harvesters for each client. Is it worth having one for data hub? 1324294509000000 1324294509000000
1573 enhancement David Raznik jilly mathews ckan-future new Apps and Ideas Estimate 2 weeks for someone to finish and test. 1324294593000000 1324294593000000
1577 defect rgrp dread ckan-backlog new Can't upload file with foreign chars in filename "Looks like uploading a file with foreign characters fails due to encoding reasons.
{{{
URL: http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf
Module weberror.errormiddleware:162 in __call__
<< __traceback_supplement__ = Supplement, self, environ
sr_checker = ResponseStartChecker(start_response)
app_iter = self.application(environ, sr_checker)
return self.make_catching_iter(app_iter, environ, sr_checker)
except:
>> app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<< self.cache_manager)
environ[self.environ_key] = self.cache_manager
return self.app(environ, start_response)
>> return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<< headers.append(('Set-cookie', cookie))
return start_response(status, headers, exc_info)
return self.wrap_app(environ, session_start_response)
def _get_session(self):
>> return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<< environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
response = self.app(environ, start_response)
# Wrapped in try as in rare cases the attribute will be gone already
>> response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<
controller = self.resolve(environ, start_response)
response = self.dispatch(controller, environ, start_response)
if 'paste.testing_variables' in environ and hasattr(response,
>> response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<< if log_debug:
log.debug(""Calling controller class with WSGI interface"")
return controller(environ, start_response)
def load_test_env(self, environ):
>> return controller(environ, start_response)
Module ckan.lib.base:123 in __call__
<< # available in environ['pylons.routes_dict']
try:
return WSGIController.__call__(self, environ, start_response)
finally:
model.Session.remove()
>> return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<< return response(environ, self.start_response)
response = self._dispatch_call()
if not start_response_called:
self.start_response = start_response
>> response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<< req.environ['pylons.action_method'] = func
response = self._inspect_call(func)
else:
if log_debug:
>> response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<< func.__name__, args)
try:
result = self._perform_call(func, args)
except HTTPException, httpe:
if log_debug:
>> result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<< """"""Hide the traceback for everything above this method""""""
__traceback_hide__ = 'before_and_this'
return func(**args)
def _inspect_call(self, func):
>> return func(**args)
Module ckanext.storage.controller:2 in auth_form
Module ckan.lib.jsonp:26 in jsonpify
<< Very much modelled after pylons.decorators.jsonify .
""""""
data = func(*args, **kwargs)
return to_jsonp(data)
>> data = func(*args, **kwargs)
Module ckanext.storage.controller:301 in auth_form
<< method = 'POST'
authorize(method, bucket, label, c.userobj, self.ofs)
data = self._get_form_data(label)
return data
>> authorize(method, bucket, label, c.userobj, self.ofs)
Module ckanext.storage.controller:79 in authorize
<< if method != 'GET':
# do not allow overwriting
if ofs.exists(bucket, key):
abort(409)
# now check user stuff
>> if ofs.exists(bucket, key):
Module ofs.remote.botostore:53 in exists
<< if bucket is None:
return False
return (label is None) or (label in bucket)
def claim_bucket(self, bucket):
>> return (label is None) or (label in bucket)
Module boto.s3.bucket:87 in __contains__
<< def __contains__(self, key_name):
return not (self.get_key(key_name) is None)
def startElement(self, name, attrs, connection):
>> return not (self.get_key(key_name) is None)
Module boto.s3.bucket:144 in get_key
<< response = self.connection.make_request('HEAD', self.name, key_name,
headers=headers,
query_args=query_args)
# Allow any success status (2xx) - for example this lets us
# support Range gets, which return status 206:
>> query_args=query_args)
Module boto.s3.connection:388 in make_request
<< if isinstance(key, Key):
key = key.name
path = self.calling_format.build_path_base(bucket, key)
boto.log.debug('path=%s' % path)
auth_path = self.calling_format.build_auth_path(bucket, key)
>> path = self.calling_format.build_path_base(bucket, key)
Module boto.s3.connection:88 in build_path_base
<< def build_path_base(self, bucket, key=''):
return '/%s' % urllib.quote(key)
class SubdomainCallingFormat(_CallingFormat):
>> return '/%s' % urllib.quote(key)
Module urllib:1222 in quote
<< safe_map[c] = (c in safe) and c or ('%%%02X' % i)
_safemaps[cachekey] = safe_map
res = map(safe_map.__getitem__, s)
return ''.join(res)
>> res = map(safe_map.__getitem__, s)
KeyError: u'\xed'
CGI Variables
AUTH_TYPE 'cookie'
CONTENT_TYPE '; charset=utf-8'
DOCUMENT_ROOT '/htdocs'
GATEWAY_INTERFACE 'CGI/1.1'
HTTP_ACCEPT '*/*'
HTTP_ACCEPT_CHARSET 'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
HTTP_ACCEPT_ENCODING 'gzip,deflate,sdch'
HTTP_ACCEPT_LANGUAGE 'en-US,en;q=0.8'
HTTP_CACHE_CONTROL 'max-age=259200'
HTTP_CONNECTION 'keep-alive'
HTTP_COOKIE 'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt=""f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode""; auth_tkt=""f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode""; ckan_user=elenaibp; ckan_display_name=""Elena Mondo""; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; __utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10.1324298316; __utmc=119670349; __utmz=119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)'
HTTP_HOST 'thedatahub.org'
HTTP_REFERER 'http://thedatahub.org/dataset/edit/budget-library-czeck-republic'
HTTP_USER_AGENT 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7'
HTTP_VIA '1.1 localhost (squid/3.0.STABLE19)'
HTTP_X_FORWARDED_FOR '87.114.74.190'
HTTP_X_REQUESTED_WITH 'XMLHttpRequest'
PATH '/usr/local/bin:/usr/bin:/bin'
PATH_INFO '/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
PATH_TRANSLATED '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
REMOTE_ADDR '193.34.146.142'
REMOTE_PORT '55419'
REMOTE_USER u'elenaibp'
REMOTE_USER_DATA 'userid_type:unicode'
REMOTE_USER_TOKENS ['']
REQUEST_METHOD 'GET'
REQUEST_URI '/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf'
SCRIPT_FILENAME '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py'
SCRIPT_URI 'http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
SCRIPT_URL '/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
SERVER_ADDR '193.34.146.146'
SERVER_ADMIN '[no address given]'
SERVER_NAME 'thedatahub.org'
SERVER_PORT '80'
SERVER_PROTOCOL 'HTTP/1.0'
SERVER_SIGNATURE 'Apache/2.2.14 (Ubuntu) Server at thedatahub.org Port 80\n'
SERVER_SOFTWARE 'Apache/2.2.14 (Ubuntu)'
WSGI Variables
application
beaker.cache
beaker.get_session >
beaker.session {'_accessed_time': 1324298703.071357, '_creation_time': 1324293077.4139669}
mod_wsgi.application_group 'ckan.net|'
mod_wsgi.callable_object 'application'
mod_wsgi.listener_host ''
mod_wsgi.listener_port '80'
mod_wsgi.process_group 'ckan.net'
mod_wsgi.reload_mechanism '1'
mod_wsgi.script_reloading '1'
mod_wsgi.version (2, 8)
paste.cookies (, 'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt=""f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode""; auth_tkt=""f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode""; ckan_user=elenaibp; ckan_display_name=""Elena Mondo""; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; _ _utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10...)|utmcmd=(none)')
paste.registry
paste.throw_errors True
pylons.action_method >
pylons.controller
pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons
pylons.routes_dict {'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'}
repoze.who.identity
repoze.who.logger
repoze.who.plugins {'openid': , 'friendlyform': , 'ckan.lib.authenticator:UsernamePasswordAuthenticator': , 'auth_tkt': , 'ckan.lib.authenticator:OpenIDAuthenticator': }
routes.route
routes.url
webob._parsed_query_vars (GET([]), '')
webob.adhoc_attrs {'language': 'en-us'}
wsgi process 'Multiprocess'
wsgi.file_wrapper
wsgiorg.routing_args (, {'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'})
}}}" 1324317659000000 1325473564000000
1578 enhancement rgrp ckan-backlog new [super] Re-enable and refactor ratings 1324322443000000 1325473015000000
1581 enhancement mark.wainwright@… johnglover ckan-future new Blog post about Google Analytics extension for CKAN "The CKAN Google Analytics extension has been updated to work with the latest version of CKAN, could make for a nice blog post.
Can ping John Glover in January for any details required.
Key link is: http://thedatahub.org/analytics/dataset/top though this should probably move to be under stats (e.g. http://thedatahub.org/stats/usage)" 1324402800000000 1325474274000000
1584 enhancement johnglover johnglover ckan-backlog new QA report improvements - 2.5d "Super: #1594
* qa/{username}
* qa/{groupname}
* paginate QA results
* search / filter QA results
* ~~list organisation report by default, but can disable via config option~~ (done)
* UX tidy up of report pages - hide border if no sidebar, etc" 1324459433000000 1338981975000000
1588 enhancement johnglover johnglover ckan-backlog new QA - Give SPARQL endpoints a 4 star rating "Super: #1594
From Richard Cyganiak on the CKAN Discuss list:
Besides considering the media type of resources, it would also make sense to check for the presence of a SPARQL endpoint. SPARQL endpoints are recorded for more than 300 datasets on the Data Hub using the pseudo-type ""api/sparql"". A few more are recorded with the format ""SPARQL"". I suggest that datasets with such resources should also be considered for the fourth star." 1324480405000000 1325475178000000
1589 enhancement johnglover johnglover ckan-backlog new QA - Give 5 star rating to datasets with link metadata "Super: #1594
From Richard Cyganiak on the CKAN Discuss list:
Regarding the fifth star (is the dataset linked to others?). This cannot be automatically determined just by looking at the format. It either requires inspection of the actual data, or information about links in the metadata. As you're probably aware, we've established conventions for recording information on data links in CKAN [1], as part of the work of the lodcloud group on the Data Hub. Link information is captured for hundreds of datasets. I would claim that we have the majority of four-star datasets covered there, and hence you can determine if they should get the fifth star by checking for the presence of a links:xxx field." 1324480600000000 1325475095000000
1596 enhancement dread ckan-future new Refactor authz roles "Suggestions from rgrp:
* Get rid of Roles, and replace them with direct assignment of actions, even though there are many actions, and extensions can add arbitrary ones.
* Debatable whether we should cut the number of actions to correspond to the three roles defined by the base system.
* Have a method of finding roles (or, in future, actions) relevant to a given protection object (e.g. FILE-UPLOAD(ER) not relevant to Packages)
(This ticket is split off from #1065)" 1324549888000000 1338205019000000
1598 enhancement rgrp ckan-backlog new Reinstate Ratings "Ratings were disabled approximately a year ago because:
* Unclear purpose and UX. What did ratings tell you? How useful were they?
* Spamming (esp by bots: you could submit an anonymous rating via a GET request which caused problems)
Both problems are solvable and it would be nice to have this feature reinstated.
* Purpose: can make this more purposable by limiting to logged in users (or at least distinguishing logged in from non-logged in users)
* Even better we could allow ratings to be made public (I'm interested in what someone else I respect finds important)
* Spamming: limit to logged in users and / or use AJAX over an API to submit ..." 1325177524000000 1325474818000000
1604 enhancement dread ckan-backlog new Get ckanext-moderatededits working with CKAN 1.5+ templates "ckanext-moderatededits requires an old and possibly development version of CKAN. It would be good to update it for later CKAN versions.
According to the README, you need CKAN from branch feature-1141-moderated-edits-ajax but the changelog suggests this branch went into version 1.4.2. So it possibly works with 1.4.2 and 1.4.3(.1). But CKAN 1.5 has revamped templates, so the genshi stream filters definitely don't work.
BTW history_ajax/read_ajax calls have been deprecated in CKAN since 1.5.2a and will need fixing up to use the Action API too as part of this." 1325352429000000 1325352429000000
1606 enhancement dread ckan-backlog new metadata license config option "Add a config option to choose the metadata licence. Set default to Open Database License.
Currently the dataset edit form says ""Important: By submitting content, you agree to release your contributions under the Open Database License."" This is hard-coded, but not suitable for when DGU uses the CKAN form - they use the OGL." 1325501130000000 1339773981000000
1635 enhancement seanh seanh ckan-backlog new Email notifications (e.g. for activity streams) "CKAN should be able to send email notifications to users.
Maybe have a notifications table in the db, and a server-side job that runs periodically and consumes rows from this table, mailing them to the users.
One thing that we may want to send users notifications of is activity stream events. So the activity streams code would have to add rows to the notifications table for the mailer job to consume. But remember that email notifications feature is separate from activity streams - we may want to send notifications of other things as well.
Need to implement (at least some of) #1634 before this can be implemented, in order to have something to send notifications about.
Analysis here: http://ckan.okfnpad.org/27" 1326304587000000 1355141157000000
1642 defect pudo ckan-backlog new Extra link generators generate garbled HTML "I had a package descriptions with URLs that contain ""group:foo"". This produces garbled output as the system tries to generate two sets of links: the outer link and an inner link.
Need to fix the parser.
Text:
Webdienst basierende Bereitstellung von Geobasisdaten der Freien und Hansestadt Hamburg. Folgende Geobasisdaten werden als WebMapTileService (WMT-S) für die Dauer des Wettbewerbs netzbasiert unter der Creative Commons Lizenz zur Verfügung gestellt: Digitale Orthophotos 40 cm Auflösung (Layer: apps4d_DOP40), Digitale Stadtkarte (Layer: apps4d_DISK), Digitale Regionalkarte (Layer: apps4d_DIRK), Digitale Karte 1:5000 (Layer: apps4d_DK5).
Metadateneinträge zu den Daten im PortalU:
* [apps4d_DOP40](http://www.portalu.de/trefferanzeige?docuuid=D6082285-6CAC-49C5-B7ED-B5A72BC17DEC&plugid=/kug-group:kug-iplug-udk-db_hh)
* [apps4d_DISK](http://www.portalu.de/trefferanzeige?docuuid=D2DA3359-3DC9-4CA4-AE9B-7C63D53B220F&plugid=/kug-group:kug-iplug-udk-db_hh)
* [apps4d_DIRK](http://www.portalu.de/trefferanzeige?docuuid=B7DDD599-F677-4C80-BED5-F11D82D5D1DD&plugid=/kug-group:kug-iplug-udk-db_hh)
* [apps4d_DK5](http://www.portalu.de/trefferanzeige?docuuid=2AE6D23E-48A5-4D85-BC0A-160737E0C8D2&plugid=/kug-group:kug-iplug-udk-db_hh)
One fix is quoting the URLs" 1326382171000000 1339773967000000
1643 enhancement shevski ckan-backlog new Add fixed tags to thedatahub for better browsing Similar to publicdata.eu, want to have themed areas such as finance, environment, census, etc and country tags 1326393293000000 1326393293000000
1644 enhancement shevski ckan-backlog new Order default dataset page by most downloaded resources on thedatahub "Instead of alphabetically as we do currently, alternatively by most viewed datasets
for http://thedatahub.org/dataset" 1326393542000000 1326393542000000
1647 enhancement shevski ckan-backlog new add links to ckan discuss & dev to thedatahub In the footer as well as more clearly & directly on the About page 1326673852000000 1326707383000000
1667 enhancement seanh seanh ckan-future new Add an extension point for rendering activity streams "Currently rendering of activity stream events to HTML works by looking up a rendering function in a dictionary that maps activity types ('new package', 'changed group', etc.) to rendering functions that take an activity stream event and return the rendered HTML.
There needs to be an extension point where extensions can register their own rendering functions for particular activity types." 1326796151000000 1338204295000000
1668 defect dread ckan-backlog new repoze version discrepency "There's a discrepency in repoze.who versions between the source and package installs:
* repoze.who - package 1.0.18 vs source 1.0.19
* repoze.who-friendlyform - package 1.0b3 vs source 1.0.8
We get a test failure [1] with the 1.0b3 version (from the ubuntu 10.04 python-repoze.who-plugins package). But we've not noticed any problems on s057 instances (br, no, ie etc) which have the package versions of repoze.who.
The reason the package install uses the earlier packaged versions rather than the ones we'd like is that repoze uses all sorts of horrendous import hacks, making it too difficult to put into our 'ckan-conflict' source package.
James suggests we 'do something horrible like dynamically patch repoze on CKAN import'.
[1] http://buildbot.okfn.org/builders/builder-ckan/builds/1371/steps/shell/logs/stdio
ERROR: ckan.tests.functional.test_user.TestUserController.test_user_create_unicode" 1326801746000000 1326801746000000
1679 enhancement dread ckan-backlog new Default roles problem "The 'editor', 'anon_editor' and 'reader' roles are intended to have immutable actions. This was designed to prevent their names being subverted - e.g. an editor should always be able to edit! It also meant that when we add Actions (e.g. DELETE-PACKAGE) then it can be added sensibly to these roles in an upgrade just by changing the defaults table (ckan/model/authz.py).
The problem is that this immutability is only enforced on 'db upgrade'. So you can happily change the editor role using the paster command and it works, right up until you do an upgrade and realise permissions are different.
We should stop the paster commands being able to edit these roles. Or get rid of the immutability completely. Views?" 1326823042000000 1339773923000000
1697 enhancement rgrp ckan-backlog new A Configurable list of states for a Dataset "Currently have 'active' and 'deleted' suggest also:
* 'draft'
* 'hidden'
(Do we need both). Also write out workflows related to these." 1327400630000000 1338204189000000
1717 enhancement shevski ckan-backlog new [super] Search UX improvements "1. Make it possible to search by tag (e.g. by typing tag:csv into the search bar and clicking enter, it should add the 'csv' tag facet to the search)
2. Rename and standardise the list of format tags, on search page this should also be called 'Format' instead of 'res_format' (in the right hand side bar on search page).
3. Make it possible to view full list of tags, formats and groups by clicking on the name. From here you should be table to click on a classification and go back to a search page faceted by that classification. E.g. from search page, click on 'tags', on tag page click on 'london' or whatever, and be navigated back to search page with search within 'london' tag only. Or y'know, a better way of doing it.
4. More standard classifications, such as 'Location' and 'Theme' - like on publicdata.eu
5. Blue search button should be displayed in line with the search bar, not underneath
6. Datasets should be displayed in order of most viewed or downloaded instead of alphabetically. For alphabetic search we could consider adding a way to facet by first letter of dataset name" 1327603981000000 1330088539000000
1740 enhancement seanh ckan-future new Get rid of `from module import ...` "It's really bad to do {{{from module import *}}} and CKAN has a lot of them. I suggest a three-pronged approach:
1. Don't add any more of them.
2. When you're programming if you see an easy opportunity to remove one then do so.
3. At some point we should task someone to go through the code and remove them all (which is what this ticket is for), but this will be a big job and may break things.
We should also get rid of most or all of the {{{from module import foo}}} and {{{from module import foo, bar}}} statements.
I think the right thing to do is just {{{import module}}} and then use {{{module.foo}}} in your code. But if you find yourself doing {{{module.foo.bar}}} then you may have a code smell.
See: http://docs.python.org/dev/howto/doanddont.html" 1328094369000000 1328094884000000
1745 enhancement rgrp ckan-v1.9 new Dataset search UX improvements as of Jan 2012 "Changes to make search both more exploratory and more satisfying to use
* Search query build - #1603
* Ability to add new facet fields ""live""
* That is add fields which then contain faceted options (a bit like data.hri.fi)
* (??) Autocomplete / drop down on search (i.e. search while you type)
* Dubious about value / cost ratio here
Probably would involve to pure JS and HTML implementation.
== Implementation ==
Probably require
* API changes to expose solr style API directly #1737
" 1328224941000000 1340033358000000
1747 enhancement seanh ckan-backlog new Expire old activities Currently the activity streams database tables just get longer and longer over time. Do we want to eventually delete the oldest activities, to keep the length of the table within limits? 1328446589000000 1339773859000000
1748 enhancement seanh seanh ckan-future new Make activity streams conform to http://activitystrea.ms/ standard "At the very least we should make our JSON output conform to their spec:
http://activitystrea.ms/" 1328450784000000 1338204337000000
1750 enhancement seanh ckan-backlog new Move ckan/lib/activity.py into the model Move ckan/lib/activity.py moved to into the model - say ckan/model/activity_extension.py, because it's so tightly knit with the model code, whereas most of the lib code is used in the controllers. 1328465888000000 1339773840000000
1782 enhancement zephod rgrp ckan-future new Improvements to datasets page UX (Feb 2012) " * Relationships in sidebar [WONTFIX yet without some link to documentation or datastructure, something for me to render?]
* ~~Icons for *types* of resources in resource list~~
* List of 'Owners' (administrators) at top right of sidebar (a la group page)
* Much prefer owner to administrator ..." 1328609441000000 1338205232000000
1787 enhancement dread ckan-future new [super] Improve RESTful API " * Lists of entities should be full URLs, rather than just the names
* Discoverability - /api/v3/rest should list the entity types that can be listed
This could be v3 of the RESTful interface." 1328702082000000 1328702082000000
1789 enhancement seanh ckan-backlog new Implement a tag_update() logic action function "So users can rename a tag and/or move it between vocabularies using the API.
Currently we have create_tag() and delete_tag(), but if you were to 'update' a tag by deleting it and then recreating it all the datasets that had that tag will have lost it and you'll have to re-add it to them all.
What should happen to datasets that have the tag, if the tag gets moved between vocabularies? All the datasets just keep the tag with the new vocabulary? This will become a problem if/when we support 'radio button'-style vocabularies (where each dataset can only have one tag from the vocabulary)." 1328805413000000 1339773812000000
1790 enhancement dread ckan-future new Click to delete tags, rather than have all existing tags in the tag text box "From Pablo:
Editing the tags field is clumsy when there are too many tags. Could
show existing effectively as tags (like delicious), then allow clicks
to delete. New tags added via text box." 1328888674000000 1328888674000000
1800 refactor seanh ckan-future new Tidy up *_list() and *_search() functions in ckan/logic/action/get.py "For consistency all the *_list() functions should list objects only and not accept an optional search query. There should be *_search() functions whenever search is needed.
Currently it is inconsistent, e.g. package_list() and resource_list() do not accept a search query option and there are package_search() and resource_search(), but user_list() does accept a query and there is no user_search(). tag_list() also accepts a query, and there is also a tag_search() that apparently duplicates the search functionality." 1329405129000000 1338204886000000
1815 task amercader amercader ckan-future new Reenable Sparql endpoint on publicdata.eu "Estimate: ?
The 4store Sparql endpoint has been down for a long time.
It needs to be reenabled and we need to check that the necessary informtion is pushed when creating /editing a dataset." 1329757723000000 1338204641000000
1818 enhancement amercader amercader ckan-future new Spatial metadata editor "Estimate 2.5d
Right now users need to manually enter a geojson value in the 'spatial' extra field in order to define a geometry for a dataset.
We need a map widget on the edit form that allows users to draw the geometry on top of a map. This will be based on the draw functions offered by OpenLayers [1], [2].
Things to take into account:
* Integration with the form sections.
* Allow some degree of configuration (default extent, limit geometry types...)
[1] http://openlayers.org/dev/examples/draw-feature.html
[2] http://openlayers.org/dev/examples/modify-feature.html
" 1329763941000000 1338203195000000
1823 enhancement rgrp ckan-backlog new Spring clean bin directory Huge number of accumulated (and likely unnecessary) scripts in /bin directory. 1329773331000000 1338203554000000
1824 enhancement seanh ckan-future new Add vocabulary pages "For a free tag foo you can visit the page at /tag/foo and see a list of all the datasets that have the tag foo, and when the tag appears on dataset view pages etc. it's linked to this tag page.
We should do the same thing for vocabulary tags. A tag bar in vocabulary baz should be hyperlinked to a page /tag/baz/bar, or perhaps /vocab/baz/bar." 1329845089000000 1338204958000000
1827 enhancement dread ckan-backlog new 'Register' link should be hidden if you not allowed to register "{{{
I have just deny visitors the create-user permission:
sudo -u ckanstd paster --plugin=ckan roles deny reader create-user -c /etc/ckan/std/std.ini
sudo -u ckanstd paster --plugin=ckan roles deny anon_editor create-user -c /etc/ckan/std/std.ini
and after restarting, the register link is *not* hidden, but now when you access the register page, it shows you this message ""Unauthorized to create a user"" (when not logged in). But anyway that is an improvement.
}}}" 1329924939000000 1339773730000000
2200 enhancement seanh seanh ckan-future new Add vocabulary_id option to tag_show() logic action function Currently tag_show() only works with free tags, it's not possible to get a tag_show for a vocabulary tag. 1330533678000000 1338204900000000
2235 enhancement rgrp ckan-future new Group drop down on dataset edit should use chosen and sort groups by name 1331907357000000 1338204726000000
2244 enhancement toby toby ckan-future new clean up auth "* cache admins etc
* aim to move check_access so independent of logic function
* general spruce up" 1332177771000000 1338205402000000
2247 enhancement zephod ckan-backlog new Resource preview glitch in some browsers "From Ira:
Preview for google spreadsheets are not displaying correctly for me in Firefox v.10.0.02, fine in Chrome.
http://i.imgur.com/KJaqz.png" 1332246614000000 1332246614000000
2257 enhancement toby toby ckan-v1.9 new cleanup template vars "look at reducing what is pulled into templates eg ckan.lib.helpers
ensure that these changes don't break existing extensions etc" 1332513307000000 1340097071000000
2258 enhancement rgrp ckan-future new Customizable contributor agreement " * Customize text at bottom of forms
* Also need to make clear that this does not apply to the data itself (that is covered by the license you choose on your dataset ...)" 1332751549000000 1338204747000000
2265 enhancement dread ckan-future new 'More Like This' for a dataset "When viewing a dataset, it would be nice to show a couple of 'Related Datasets'. i.e. ones that are similar.
SOLR has a feature for finding documents similar to a particular document, called 'More Like This'.
We would like this for DGU." 1332865220000000 1339771350000000
2268 enhancement thejimmyg amercader ckan-backlog new UKLP Use same schema as form for harvested records Datasets created via the harvesting (UKLP ones) and the ones created via the form use a different schema. That could lead to problems. We need to discuss the best approach. 1333108606000000 1333108606000000
2277 refactor icmurray icmurray ckan-backlog new Use the new atom feeds in IATI Atom feeds have been implemented in core in #1593. The IATI-specific implementation can now be removed when IATI move to CKAN >= 1.6.1 1333386628000000 1333386628000000
2279 enhancement rgrp rgrp ckan-v1.8 new Write tutorial for our DataStore API / Data Explorer "Unfortunately ES is a little bit complex / opaque to newcomers. Need to write a simple tutorial walking through this.
This is documentation / example only (no code!)" 1333482263000000 1338205426000000
2282 enhancement toby toby ckan-future new log api calls for users "This was part of #2251 but is now it's own ticket.
we need clarity on this. The use cases so far have been possible limiting api calls for users.
so we want to monitor the user or ip address
what about the api calls?
do we want to record all calls or just some? do we want to record the type of call and to what level of detail?
recording the url just seems to specific to me." 1334074725000000 1338210829000000
2287 enhancement seanh ckan-future new Consistent datetime and time period strings localization "Find all places where date, times and time periods are rendered as strings in CKAN. Make them all go through the same code path (e.g. the helper function in helpers.py) with localization support.
For example, on the user index page right now there are time period strings like ""less than 1 month"" that currently do not get translated (and the way they're implemented doesn't look i18n-friendly).
ISO date format everywhere might be a good idea.
Need to consider local timezone issues.
Python standard library and Babel may have useful helpers for this.
Should the local timezone displayed be a global setting per CKAN instance? Or should it change depending on the location of the user who is viewing the page?" 1334254123000000 1339411445000000
2291 enhancement kindly rgrp ckan-v1.9 new DataStorer incorrectly attempts to store JSON in DataStore "See http://thedatahub.org/dataset/ecb13156-cf11-4e91-b51c-f287d154276e/resource/5db67915-40cb-467e-9d72-194ea4973670
This was marked as json format (and is JSON) but appears to have been imported as a quasi-CSV. Either this hsould not be imported at all or it should be imported as JSON ... (which is straightforward to do).
Assigning to David for triage and review." 1334431147000000 1340636737000000
2298 enhancement icmurray icmurray ckan-v1.9 new Add sort-by controls to the search results page "Sort-by functionality was exposed through the package controller in [1]. But no controls were added to the search-page.
* What should the sortBy controls/widget look like?
* Which fields should be exposed?
The above commit is in the release-1.6.1 branch, so this work is dependant upon release-1.6.1 from being merged into master.
[1] https://github.com/okfn/ckan/commit/8685c6000d1cb211928b4dbc63990fb72d884f8c" 1334569162000000 1340635947000000
2301 enhancement seanh seanh ckan-future new Replace old-style string formatting with format() method (at least in strings marked for translation) "Some strings in CKAN are passed to gettext with more than one %s in them, this is no good as translators may need to change the order of substituted words.
Strings should not use the old style % substitution, they should use the new format method in Python 2.6+, e.g.:
'This {food} is {adjective}.'.format(food='spam', adjective='absolutely horrible')
http://docs.python.org/release/2.6.8/tutorial/inputoutput.html#fancier-output-formatting
We should probably only change strings that are marked for translation and that contain multiple %s, don't want to make too much work or cause too many strings to change and need to be re-translated." 1334579264000000 1340635812000000
2310 enhancement icmurray icmurray ckan-backlog new Refactor the search-query construction in feeds.py "The feeds controller, used to construct atom feeds, duplicates code found in the package controller's search action (in order to construct the custom feed).
Refactor this to remove duplication." 1334592091000000 1337159386000000
2320 enhancement rgrp ckan-v1.9 new Introduce new state: archived Some datasets become obsolete: e.g. http://datahub.io/dataset/uncdb Do not want to delete them but want them marked as archived or deprecated or something. 1335211949000000 1340624119000000
2321 enhancement rgrp ckan-v1.9 new Use resource names/titles in URLs "Would make them much nicer but requires:
* Ensure uniqueness (or deal with ambiguity)
* (?) Slug/names versus titles
* Aside: what is currently called name should probably be title
" 1335236412000000 1340624174000000
2324 enhancement amercader ckan-v1.9 new Link API docs to relevant version not just latest "The link to the API docs in the footer always points to the latests docs, which may not be correct for the particular CKAN version.
Also, remove the 'API' link (to /api/1) as pointless and misleading" 1335351532000000 1345199411000000
2326 enhancement toby toby ckan-future new dataset permalinks "create permalinks
eg \dataset\~a5-DQ\nice-name
~XXXXX defines the permalink the nice-name is ignored
* packages table needs a new field for this data
* existing datasets need updating
* new datasets need the identifier creating
* routing needs updating
* needs to work for all urls like edit
Once this is implemented we can decide when we will use permalinks vs the normal urls." 1335371946000000 1338212799000000
2328 enhancement rgrp ckan-v1.9 new Link to Catalog API for each Dataset, Group and Tag At the moment no way for a normal user to find JSON versions of this material. Would it be worth adding back a link somewhere on the page? 1335442501000000 1340624331000000
2333 enhancement rgrp ckan-v1.9 new Introduce new resource type datastore "This is a resource that is *just* in the datastore. At the moment there are hacky ways to do this (create resource with meaningless URL etc) but we should do this properly:
* New Option: DataStore Table in add a resource form
* URL field is disabled (or points at /api/data or something rather pointless)
* Download button is disabled
* Better would be to find a way to serialize an ES table or stream it ...
" 1335651360000000 1340624473000000
2350 enhancement icmurray icmurray ckan-v1.9 new Hooks in package controller for validating form data "When the validation schema was moved from the package controller into the logic layer, the schema's role was changed from that of processing a form and validating data; to just validating data. Whilst is makes sense to have a schema for validating data as it comes in and leaves the logic layer; there's no longer the hook available to PackageController subclasses to do form validation/processing.
Add a hook into _save_new() and save_edit() prior to the logic-layer call which subclasses can then override.
NB - form processing/validation *is* different to validating data. For example, many data can be inferred from a user's input; or there may be aspects of the form that need to be filled in, but do not make sense to pass as data into the logic-layer, eg. agreeing to T&Cs.
- [ ] Add hooks
- [ ] Use in ckanext-example" 1335881761000000 1340635966000000
2363 enhancement toby kindly ckan-v1.8 new Documentation of best caching practice. Need better documentation on best practices in making page loads faster for non logged in users. 1335889017000000 1340099794000000
2382 task amercader amercader ckan-future new Investigate options for basic geocoding A simple way for geocoding place names would be very useful, e.g in the spatial search, defining a geometry for a dataset (on the form or bulk) 1337017160000000 1338205325000000
2395 defect dread ckan-backlog new paster db clean/init don't work when spatial extension enabled "If you have a spatial enabled database then if you don't disable the spatial extension in the CKAN config temporarily then you get errors when you run paster db clean and paster db init.
Can't you just modify the clean and init functions to run without extensions enabled?
The wider problem is that extensions do their own inits every time you do load_environment, which seems crazy and inefficient to me, since this occurs every time a request comes into CKAN. But that is another problem/ticket." 1337159793000000 1339771313000000
2406 defect kindly markw ckan-v1.9 new Errors in source install doc "There are a few errors or unclarities in the source install doc - see
http://lists.okfn.org/pipermail/ckan-dev/2012-May/002203.html
http://lists.okfn.org/pipermail/ckan-dev/2012-May/002204.html
" 1337595093000000 1340636804000000
2407 enhancement amercader markw ckan-v1.9 new 'Access denied' message is unhelpful "This chap tried to use the API to add a dataset to a group and got the uninformative message 'Access denied':
http://lists.okfn.org/pipermail/ckan-dev/2012-May/002229.html
The problem was he didn't have access to the group. A message like 'You don't have access to the group ""LODcloud""' would have been clearer." 1337612500000000 1340633440000000
2411 enhancement toby markw ckan-future new .ini file is a mess The .ini file needs cleaning up in assorted ways which Toby will expand upon. 1337686900000000 1338210872000000
2415 defect kindly markw ckan-v1.9 new Confusing message on attempting to register new account with upper case "On registering a new account, if you try to use a capital letter in your 'Login', you get the unhelpful error message:
""The form contains invalid entries: Name: Url must be purely lowercase alphanumeric (ascii) characters and these symbols: -_""
What does 'Name' or 'Url' have to do with anything? It means 'Login must be ...'
But even better would be to make the login case-insensitive and allow capitals to be entered.
" 1337779189000000 1341268304000000
2426 enhancement kindly seanh ckan-v1.9 new Get rid of CKAN's formalchemy dependency 1337946462000000 1340616095000000
2427 enhancement kindly seanh ckan-v1.9 new Reduce cruft installed into CKAN's virtualenv due to Pylons 1337946507000000 1341268323000000
2433 defect toby toby ckan-v1.8 new API uses name not id for some version 3 calls see https://github.com/okfn/ckan/pull/5 1337957648000000 1340099820000000
2479 enhancement markw ckan-v1.9 new Meanings of Author and Maintainer fields are unclear "CKAN's default schema has fields for Author ('The name of the main contact, for enquiries about this particular dataset') and 'Maintainer ('another important contact person'). The distinction is not clear. Also the fields are often left blank.
The roles that seems most important are those of the original owner/publisher of the data, and the person maintaining the CKAN record/copy of it. So I suggest:
(1) Rename the fields 'Owner' and 'Maintainer'.
(2) Change the explanatory text for the relevant fields:
Owner: 'The person or organisation who create/collect/publish the data in this dataset.'
Owner e-mail: 'E-mail address for enquiries to the Owner named above.'
Maintainer: 'The person maintaining this dataset on [name of CKAN instance], if different from the above.'
Maintainer e-mail: 'E-mail address for enquiries to the Maintainer named above.'
(3) When a logged-in user creates a new dataset, the main form should have a checkbox, checked by default, marked 'I am the maintainer of this dataset'. If checked, the Maintainer name and e-mail fields are populated from the user's profile.
" 1338452898000000 1339771115000000
2480 enhancement markw ckan-v1.9 new Better message when dataset has no resources "If a dataset has no resources the resources list currently says
'(none)'.
Here is a suggested improvement, provided that a maintainer is named: 'There are no data resources here yet. For information about this data, contact the dataset maintainer.'" 1338453093000000 1339771086000000
2483 enhancement markw ckan-v1.9 new Non-local resources should not have Download links "At present, a resource which is just a URL link to an external resource has a 'Download' button on the resource page. This gives the misleading impression that the resource is stored locally.
This is related to another small UI issue: I think the URL of a resource should be much more prominent, not buried in the 'Additional Information' table.
Suggested fix:
* Put the URL prominently at the top of the resource page (above the preview) (and make it link to the resource)
* Disable the Download button unless the resource is stored locally.
" 1338468734000000 1339771043000000
2485 defect icmurray nils.toedtmann ckan-v1.9 new Encourage leak containment by limiting the number of requests a CKAN process serves "CKAN has [ticket:1345 memory leaks]. They can be contained by limiting the time-to-live of a ckan process. An easy way to achieve this is to limit the number of requests a ckan server process can serve before it gets killed and replaced.
One should ...
* research ways to limit requests-per-process with the different web servers (see below for a start);
* explain these safeguards in the CKAN documentation and encourage users to apply them;
* consider the helper script '''[https://github.com/okfn/ckan/blob/master/ckan_deb/usr/bin/ckan-create-instance ckan-create-instance]''' to generate Apache configs with '''WSGIDaemonProcess ... maximum-requests=XY''' being active instead of [https://github.com/okfn/ckan/blob/master/ckan_deb/usr/lib/ckan/common.sh#L262 commented out]. Or at least let it warn the user to use '''!MaxRequestsPerChild''';
* Investigate current CKAN deployments whether they suffer from mem leaks, and if so contain them.
----
How to limit requests-per-process
* Apache:
* Use '''WSGIDaemonProcess''' with '''maximum-requests=50''' or whatever limit is appropriate. We did this sucessfully on [http://trac.okfn.org/ticket/904 datahub.io] and the [http://trac.okfn.org/ticket/1245 ckan farm]
* (This need to be verified) Without WSGIDaemonProcess, '''!MaxRequestsPerChild 50''' should achieve the same.
* nginx/supervisord: to be researched.
" 1338551229000000 1340636028000000
2486 defect seanh ckan-backlog new Should be able to use . in dataset names 1338655583000000 1339770978000000
2499 enhancement toby icmurray ckan-v1.8 new Documentation for the internal analytics "There's no docs for setting up the internal analytics, nor what it provides.
== Setup ==
Add: `ckan.tracking_enabled = true` to .ini file
Run: `paster tracking -c dev.ini`
== Related links ==
http://trac.ckan.org/ticket/2251" 1339070297000000 1339070297000000
2508 enhancement seanh ckan-backlog new Make it possible to run CKAN tests for each language "Mistakes in translated strings can cause CKAN to crash or otherwise not work, but it's not practical to manually test every page and function of CKAN in every language that we have new translations for before a CKAN release. It'd be great if the tests could automatically be run for each language.
This is probably a big job, we would have to get the tests to respect a language setting in the ini file, check for any individual test cases that specify the language (e.g. in the URL), and also fix test cases that look for specific English words in HTML output, etc.
In the meantime, a good stop-gap solution might be a script that tests for common mistakes in the po files." 1339411335000000 1339770771000000
2524 enhancement kindly kindly ckan-ecportal new If there are no translation files for selected language fall back to default lang. If a user selects a language there are no mo files for then an error is raised. Revert to default language instead. 1339609048000000 1340117608000000
2529 enhancement rgrp ckan-v1.9 new DataHub (or CKAN) widgets "Simple widgets in pure JS. For example:
* Count of datasets in a group (could generalise to a query but not sure how useful that is ...)
* Embeddable list of top X (5) datasets for a given query
* Embeddable list of *my* datasets
Either these live at: {site}/widgets and we have some kind of generator (form where I choose my group, or my query).
Or: we have this attached to areas of site where relevant.
Can combine the 2 so that the latter links to the former. Think first will be easier to do and possibly more useful long-term (e.g. can just link people to that page).
Cf. http://okfnlabs.org/ckanjs/" 1339750049000000 1340624917000000
2530 enhancement kindly rgrp ckan-v1.9 new DataHub purge fails on some revisions "See http://datahub.io/ckan-admin/trash and try to purge revisions (*not* datasets). It will fail on some of the revisions with errors like:
Problem purging revision 391db9e8-df57-4e0e-8fe6-d4e0c2318344: (IntegrityError) update or delete on table ""revision"" violates foreign key constraint ""group_extra_revision_revision_id_fkey"" on table ""group_extra_revision"" DETAIL: Key (id)=(391db9e8-df57-4e0e-8fe6-d4e0c2318344) is still referenced from table ""group_extra_revision"". 'DELETE FROM revision WHERE revision.id = %(id)s' {'id': u'391db9e8-df57-4e0e-8fe6-d4e0c2318344'}" 1339750498000000 1341268280000000
2531 enhancement rgrp ckan-backlog new New state option: archived / deprecated "Deleted means things will get purged at some point.
Archived means they stay around but get hidden from search results and a big warning notice gets displayed saying this is archived / deprecated.
@richard cyganiak ..." 1339750787000000 1339770649000000
2543 enhancement icmurray ckan-v1.9 new facet.sort is not available in the package_search action "Not all solr facet parameters are available through the pcakage_search action. In particular, `facet.sort` has been asked for; but this ticket should check to see if there are other parameters that would be easy to add too.
See: http://wiki.apache.org/solr/SimpleFacetParameters#facet.sort" 1340013335000000 1340633091000000
2555 enhancement toby aron.carroll demo phase 5 new Demo site needs a breadcrumb helper Something to make building breadcrumbs a bit nicer 1340026983000000 1342618384000000
2572 enhancement toby toby ckan-v1.9 new clean up stats plugin attempt to disengage the stats plugin from core as much as possible 1340105054000000 1340899524000000
2573 enhancement icmurray ckan-future new package_search does not allow solr's per-field facet parameters "Solr allows its facet parameters to be specified on a per-field basis, eg. `facet.limit` applies to ''all'' facet fields, but solr allows it to be overriden for a specific field, eg. `facet.tags.limit`.
We don't support this at the moment because we have a whitelist of valid solr query parameters that we accept. See `ckan.lib.search.query.VALID_SOLR_PARAMETERS`." 1340112439000000 1340633101000000
2579 enhancement toby toby ckan-v1.9 new move sort_by functions into lib.helpers "make these more available but keep existing functionality so not to break any users
remove_field()
drill_down_url()
etc" 1340281798000000 1340899337000000
2582 enhancement rgrp ckan-v1.9 new Do not hide notes / readme on dataset pages "Current we hide most of readme and then let users reveal it. Stop doing this and if necessary add a quick link down to resources section. (Maybe also rename resources to Data and Resources ...?)
Aside: believe I have mentioned this somewhere a month + ago but could not find the ticket." 1340312340000000 1340625009000000
2583 enhancement toby toby demo phase 5 new make sure that we implement authentication where needed in development many auth checks may have been lost we need to check they are still working etc 1340359478000000 1342086274000000
2585 enhancement seanh ckan-v1.9 new Escape solr control characters in search queries, add advanced search screen "Suggestion from David Read:
We noticed that some search queries produce unexpected search results in CKAN, due to them containing special characters. For example if you were to search for ""Spend over £25,000 - NHS Leeds"" then it would not come up with the dataset with that exact name. It was excluding datasets with the word ""NHS"" due to the dash/minus sign. It works fine if you escape the minus sign: ""Spend over £25,000 \- NHS Leeds"".
So in data.gov.uk I've added escaping of such control characters in
our plugin and this useful routine:
http://fragmentsofcode.wordpress.com/2010/03/10/escape-special-characters-for-solrlucene-query/
Perhaps you would consider providing this in CKAN core in future?
I think there is an occasional case when power users would want to use the special characters - brackets, +, -, boolean operators etc. but maybe these could be reserved for an 'advanced search' screen?" 1340360773000000 1340625078000000
2603 refactor icmurray icmurray ckan-v1.9 new Remove deprecated 'fields' parameter from resource_search The fields parameter of resource_search was deprecated when fixing #2438. It can be removed in release 1.9, and the action tidied up as a result. 1340730601000000 1340730601000000
2621 refactor icmurray icmurray ckan-v1.9 new Remove the deprecated 'fields' parameter from tag_search and tag_autocomplete "This was deprecated in 1.8 as it wasn't accessible via GET requests due to being a dict. See #2439
In a future release of CKAN (probably 1.9) it can be removed.
Internal uses of it were removed in #2439, but there are tests that still use it." 1340900569000000 1340900569000000
2622 defect seanh new Login fails in Opera 12 "Try to login to CKAN using Opera 12, get ""Login failed. Bad username or password. (Or if using OpenID, it hasn't been associated with a user account.)""" 1340902602000000 1340902602000000
2625 enhancement seanh ckan-v1.9 new Add i18n strings from non-core but supported extensions to ckan.pot file Have to decide which non-core extensions are going to be supported first. 1341236903000000 1341236903000000
2635 enhancement dread new Non-destructive SOLR reindex "You can't run the search-index reindex on a live server because it will give us bad results for 2 to 3 hours while it runs. Can there be an option that doesn't delete the entire index at the start?
Instead it could just delete any items that don't exist any more, then delete them and regenerate them one by one. So the total number of datasets doesn't change much." 1341829394000000 1341829394000000
2656 defect seanh dread new Feed with few results has bad paging link, causing exception "This page http://thedatahub.org/feeds/custom.atom?q=wombat has 0 results and contains a link to http://thedatahub.org/feeds/custom.atom?q=wombat&page=0 which the page=0 causes this exception:
{{{
ckan.lib.search.common.SearchError'>: SOLR returned an error running query
Error: ""'start' parameter cannot be negative""
}}}" 1342001112000000 1342001112000000
2663 enhancement toby toby ckan-v1.9 new h.resource_display_name needs love "This function is shit and needs cleaning up and a doc string
description is markdown and should be treated properly
either we should truncate all or leave it to the templates but work universally
url if no name / desc this is in demo-theme branch" 1342017746000000 1342017746000000
2673 enhancement rgrp new simplify set of options for resources Far too many resource options. Lets restrict back to data file and API. Visualizations etc can either get linked in description or in the Related items. 1342300559000000 1342300559000000
2679 enhancement icmurray icmurray ckan-v1.9 new Change default behaviour of TemplateController.view to 404. "The current behaviour of TemplateController.view() (which is the fallback controller should all others fail) is to attempt to render (as a genshi template) the requested file.
Although this may be a feature that some instances want. In general, it leads to:
- 500s when attempting to access a normal template (eg - http://datahub.io/importer/preview)
- A way of inadvertantly serving things you may not want to serve. (Small risk, as it needs to be renderable as a genshi template).
Solution:
- Change the controller to 404
- Ensure there's a way for existing ckan instances to override that behaviour should they need it." 1342436133000000 1342436133000000
2683 enhancement seanh new Add no-cache header to _tracking API call's response to make sure it doesn't get cached 1342446577000000 1342446577000000
2688 enhancement ross new Allow ordering of groups in WUI Currently the group_index page just shows the entire list of groups, forcing the ordering to be by name. It would be better if it could be sortable by name (or reversed) or by package_count (or reversed) 1342520875000000 1342520875000000
2708 enhancement kindly toby ckan-v1.9 new limit extra data for package/group show " context['package_limits'] = { 'tags': 5, <- get first 5
'extras': 0, <- get all
}
only get what you ask for have to be explicit
context['group_limits'] = {} only main item
start with datasets/groups expand if we like it" 1342622420000000 1342622420000000
2709 enhancement icmurray markw new Atom feeds are undocumented There doesn't seem to be any documentation yet for Atom feeds. 1342624310000000 1342626212000000
2718 enhancement toby shevski demo phase 4 new can't add dataset to more than one group "trying to add a dataset to another group means it's no longer part of the first group
http://demo.ckan.org/dataset/edit/afterfibre" 1342780550000000 1344544203000000
2719 defect dread new Feeds controller does not catch NotAuthorized exception Results in bad user experience and WebApp errors emailed out. Seen in 1.7.1 1342872863000000 1342872863000000
2721 defect toby shevski demo phase 4 new deleted groups should not show on 'Add to Groups' dropdown "Groups previously deleted still show up in the add dataset process in step 3 'Additional info'
http://s031.okserver.org:2375/dataset/new_metadata/ff" 1342948632000000 1344544214000000
2725 enhancement toby shevski demo phase 5 new Case sensitivity on tags "My feeling is that 'country-US' and 'country-us' should be the same tag. However currently tags with caps are treated differently
see http://s031.okserver.org:2375/en/dataset/test-dataset
with TEST and test - there also get indexed twice in the search page" 1342949667000000 1343030773000000
2726 enhancement toby shevski demo phase 5 new confusing logic on data preview formats "1. If a user enters the wrong format on a file that can be previewed - it simply won't be previewed (e.g. a CSV or XML file that can be filled in with JSON in format will just not work or check this
2. If I incorrectly edit format to one that data preview will try to preview it will work even for a format that it doesn't accept (sometimes) e.g. this PDF file I changed the metadata to HTML http://s031.okserver.org:2375/en/dataset/test-dataset/resource/9d27a9d9-36ec-460e-9edb-6dff7ba4fc28" 1342949927000000 1343030906000000
2728 defect toby shevski demo phase 4 new deleted group shows on search index - for admins 'test-group', which has been deleted,shows up on main search page under groups - and can be filtered by - see http://s031.okserver.org:2375/dataset?groups=test-group 1342950784000000 1345023944000000
2729 enhancement kindly shevski ckan-backlog new searching for tags:[tag] works but tag:[tag] doesn't "which is confusing since you can only search for one tag like this at a time. I.e.
tags:economics,cvs or tags:economics, csv or tags:economics+CSV doesn't work for example; therefore tag:economics, should also work!
http://s031.okserver.org:2375/dataset?q=tags%3Aeconomics&sort=relevance+asc" 1342951109000000 1342951176000000
2731 enhancement markw new Some sites permanently 'down for maintenance' "A large number of XXX.ckan.net sites give the following message:
""This Site is Down for Maintenance
We apologize for the inconvenience.
~ The Open Knowledge Foundation sysadmins.""
The message is unhelpful and patently false - the sites do not exist. Some of them were supposed to have been redirected to a relevant group at thedatahub.org in this ticket (now closed):
http://trac.okfn.org/ticket/933
However, the redirection only seems to have worked in one case, http://si.ckan.net.
The problem still affects the following sites - the first 4 of which have supposedly been merged:
* http://lt.ckan.net
* http://hu.ckan.net
* http://be.ckan.net
* http://gr.ckan.net
* http://nz.ckan.net
* http://bg.ckan.net
* http://fi.ckan.net
* http://katalogdanych.centrumcyfrowe.pl
* http://pl.ckan.net
* http://southampton.ckan.net
* http://sl.ckan.net
Please sort this out by redirecting, removing the sites, giving a more helpful (and accurate) failure message, etc, as appropriate." 1343045168000000 1343051608000000
2733 enhancement johnglover johnglover ckan-v1.9 new Datastore logic functions "== Where does the data go? ==
In a postgres database configured by the ckan.datastore_write_url config option which is a sqlalchemy url.
The user should have rights to create tables.
== Whats the api like? ==
We will just implement it as logic functions like the rest of CKAN and will part of core. After that we may add some nicer api functions that use these but that is a secondary concern.
== What are the initial logic functions? ==
* datastore_create
* datastore_delete
* datastore_show
== What is the JSON input format for datastore_create ==
To begin with it can have the following keys. It is fairly consistent with Max Ogdens' gut servers. Except adds resource_id.
{{{
{
resource_id: resource_id # the data is going to be stored against.
fields: a list of dictionaries of fields/columns and their extra metadata.
records: a list of dictionaries of the data eg [{""dob"": ""2005"", ""some_stuff"": ['a', b']}, ..]
}
}}}
* The first row will be used to guess types not in the fields and the guessed types will be added to the headers permanently. Consecutive rows have to conform to the field definitions.
* rows: can be empty so that you can just set the fields
* fields are optional but needed if you want to do type hinting or add extra information for certain columns or to explicitly define ordering.
eg: [{""id"": ""dob"", ""type"": ""timestamp"" }, {""id"": ""some_stuff"", ""type"": ""text""}, ...].
A header items values can not be changed after it has been defined nor can the ordering of them be changed. They can be extended though.
* Any error results in total failure!! For now pass back the actual error.
* Should be transactional
== What json does datastore_delete take? ==
{{{
{
resource_id: resource_id # the data is going to be deleted.
filters: dictionary of matching conditions to delete
e.g {'key1': 'a. 'key2': 'b'} this will be equivalent to ""delete from table where key1 = 'a' and key2 = 'b' "".
No filters (either not present or not defined) then delete the table. If we want truncate then add truncate: true to truncate the table.
}
}}}
== What json does datastore_search take? ==
{{{
{
resource_id: resource_id # the data is going to be selected.
filters : dictionary of matching conditions to select
e.g {'key1': 'a. 'key2': 'b'} this will be equivalent to ""select * from table where key1 = 'a' and key2 = 'b' ""
q: full text query
limit: limit the amount of rows to size default 100
offset: offset the amount of rows
fields: list of fields return in that order, defaults (empty or not present) to all fields in fields order.
sort: comma separated field names with ordering e.g ""fieldname1, fieldname2 desc""
}
}}}
Some free code: https://gist.github.com/3163864
== What json does datastore_search return? ==
{{{
{
fields: same type as datastore_create accepts (i.e. with metadata)
offset: The same offset that was supplied in datastore_show
limit: The original limit
filters: The filters that were applied in data_show
total: # total matching records without size or offset
records: [same as data_create] # list of matching results
}
}}}
On error will return:
{{{
{
__error__: … sql error …
}
}}}
== What types are allowed? ==
Aim to support as many postgres/postgis types that have string representations.
http://www.postgresql.org/docs/9.1/static/datatype.html
http://www.postgresql.org/docs/9.1/static/sql-createdomain.html
== IDs ==
Each row in a table will be given an _id column which has an id generated by us which you can use in queries.
== Other Features ==
Each row will store the _full_text index of all the data in the row. At some later point there will most likely be a way to index fields add constraints etc." 1343058886000000 1343656105000000
2745 defect amercader ckan-v1.9 new Password reset returns an exception if the key parameter is missing "Instead of showing a notice, the password reset page throws an exception if the key parameter is missing:
{{{
Module ckan.controllers.user:329 in perform_reset
c.reset_key = request.params.get('key')
if not mailer.verify_reset_link(user_obj, c.reset_key):
h.flash_error(_('Invalid reset key. Please try again.'))
abort(403)
if not mailer.verify_reset_link(user_obj, c.reset_key):
Module ckan.lib.mailer:100 in verify_reset_link
if not user.reset_key or len(user.reset_key) < 5:
return False
return key.strip() == user.reset_key
return key.strip() == user.reset_key
AttributeError: 'NoneType' object has no attribute 'strip'
}}}
Apart from the obvious fix of checking for the 'key' parameter, it seems like is quite common to get these reset urls without the key parameter, so I suspect some email clients might strip the query params when building the links. We could avoid this problem by making the key part of the url instead of a param:
http://thedatahub.org/en/user/reset/3086e91c-fe09-4a98-92e1-19de67a9ac9d/b4c2d03fa8
instead of:
http://thedatahub.org/en/user/reset/3086e91c-fe09-4a98-92e1-19de67a9ac9d?key=b4c2d03fa8
" 1343145931000000 1343145931000000
2748 enhancement shevski demo phase 5 new add 'add new resource' button to sidebar "When editing a resource you see the current and any other existing resources in sidebar
(e.g. see http://s031.okserver.org:2375/en/dataset/format-error-test/resource_edit/d1eac556-c16f-44af-8148-5e3467b57cf8?inner_span=True)
Would be good to have a pretty 'add new' slightly transparent resource folder/pointer undearneath - letting you add resources from the end resource page" 1343212878000000 1344503744000000
2751 enhancement toby toby demo phase 5 new check translations for full demo site need to check everything gets translated - sean did this before so will have info 1343216443000000 1344243046000000
2758 enhancement toby toby ckan-v1.9 new file storage gives error if config not available but no useful user information "We get an error which should be improved the actual problem is this but not passed to user
⇝ KeyError: 'ofs.impl'
View as: Interactive (full) | Text (full) | XML (full)
Module ckan.controllers.storage:2 in auth_form view
Module ckan.lib.jsonp:26 in jsonpify view
>> data = func(*args, **kwargs)
Module ckan.controllers.storage:407 in auth_form view
>> authorize(method, bucket, label, c.userobj, self.ofs)
Module ckan.controllers.storage:200 in ofs view
>> StorageAPIController._ofs_impl = get_ofs()
Module ckan.controllers.storage:71 in get_ofs view
>> storage_backend = config['ofs.impl']
Module paste.registry:146 in __getitem__ view
>> return self._current_obj()[key]
KeyError: 'ofs.impl'" 1343287709000000 1343287709000000
2761 enhancement seanh ckan-v1.8 new Document all the errors you can get when setting up filestore, and how to fix them "Add it to a 'Troubleshooting' section on the filestore page: http://docs.ckan.org/en/ckan-1.7.1/filestore.html
For the error messages and their solutions, see various threads on ckan-dev" 1343302566000000 1343302566000000
2762 defect seanh ckan-v1.8 new test_related.py crashes "/home/seanh/Projects/ckan/ckan/ckan/tests/functional/test_related.py
ImportError (cannot import name assert_regexp_matches)" 1343303753000000 1343303753000000
2763 defect seanh ckan-v1.8.1 new Multilingual tests failing test_multilingual_plugin.TestDatasetTermTranslation.test_dataset_index_translation, test_multilingual_plugin.TestDatasetTermTranslation.test_group_read_translation both failing for me on master 1343303819000000 1350303864000000
2768 enhancement toby shevski demo phase 5 new normalise excel to xls so that data proxy works 1343319382000000 1344351663000000
2771 enhancement seanh seanh ckan-v1.8 new Documentation and examples for IDatasetForm and IGroupForm "Add minimal, working IDatasetForm and IGroupForm example extensions to core, with tests.
The IDatasetForm example should use tag vocabularies (two birds with one stone)
The IDatasetForm and IGroupForm docs are not very good (and are somewhat spread around different doc chapters), fix them up, and reference the new working examples.
Tab Vocabularies docs should reference IDatasetForm example.
When using convert_to/from_extras() you have to remove any free extras from the form or it won't work, this needs to be documented (in the docstring maybe)
There have been recent changes to the schemas that IDatasetForm and IGroupForm use, make sure the docs are up to date." 1343392238000000 1350303564000000
2773 enhancement markw markw ckan-v1.9 new About page needs improving The about page for the DataHub (thedatahub.org/about) could be improved. More importantly the default about page for a generic CKAN instance should be completely different - focus more on Open Data rather than the community hub idea (as this is more relevant for most installations) and remove specific references to tdh. 1343646795000000 1343646795000000
2775 enhancement toby aron.carroll demo phase 4 new Add bin/less to paster serve command "Ideally the ./bin/less command would be run when the server is started.
* Also it would be good to have a paster command to build the production CSS with {{{ ./bin/less --production }}
* The command could also detect missing node binaries and redirect to the documentation." 1343685686000000 1344543962000000
2777 enhancement icmurray new bug: user attributes 1343726363000000 1343726363000000
2780 enhancement toby shevski demo phase 4 new way for admins to undelete datasets "Since admins can see deleted datasets - there should be a way for them
a) to know they are currently deleted & not viewable by normal users - ticket: #2779
b) way to undelete such datasets - this ticket
I suggest a button on the edit form instead of the delete button
i.e. remove normal delete button with 'deleted dataset, only admins can view' with undelete button next to the message?" 1343737248000000 1345023811000000
2784 defect icmurray icmurray new model dictize sensitive data The model dictize layer doesn't consistently remove sensitive data from the dictized models. It should use the current context to decide whether to include sensitive data or not. 1343814685000000 1343814685000000
2786 enhancement shevski demo phase 5 new target blank HTML downloads e.g. if I click on download here: http://s031.okserver.org:2375/dataset/example-dataset/resource/d8797e51-b497-46ca-a274-8675533d110b can it take me to a new tab instead of navigating away from ckan? 1343819814000000 1343819814000000
2788 enhancement amercader amercader ckan-v1.9 new Speed improvements on creating/updating and indexing "Specially needed when importing large numbers of datasets.
Profiling the import command from the harvesting extension has shown some areas where improvements could be made.
" 1343832992000000 1343832992000000
2790 enhancement kindly toby demo phase 4 new logic.action.user_show is slow "This is a very slow call it would benefit from the sort of speed-ups that package-search received
for me locally this is taking 6 seconds for rufus using the datahub data I have. I think a lot of this is the dataset retrival/dictization
can we just grab json blobs from solr?
also is it possible to specify a sort order/paging?
I've put this as a demo-theme ticket as it is an big issue on the demo we are at 25 second page loads - which i can get down to about 8.8 secs so this is the main pain point now" 1343852483000000 1345023734000000
2795 enhancement toby demo phase 5 new Check validation of HTML, CSS, JS Ensure that we are being standards compliant 1343903128000000 1343903128000000
2796 enhancement mark.wainwright ross new Need a datahub one-pager "A one-pager explaining what the datahub is and with howto/examples for new users. This would make it much easier to explain the value in using the datahub for storing data.
" 1343924916000000 1345129495000000
2810 enhancement kindly ckan-future new heroku ckan support Get ckan working on heroku 1344364858000000 1344364858000000
2813 enhancement toby markw demo phase 5 new Confusing sidebar on demo dataset page "On a dataset page on demo.ckan.org, the left sidebar is confusing.
* It starts with some random links. Actually they are links to groups which the dataset is in, but this isn't clear.
* The sidebar elements that are actually part of the dataset are 'Datset extent' and 'License', so these should be right at the top (if they belong in the sidebar at all), instead of which they are right at the bottom in the junk part of the page (i.e. probably lower than the bottom of the main page, and hence lower than anyone will scroll)." 1344420206000000 1344445419000000
2815 defect seanh seanh ckan-v1.8.1 new db_to_form_package_schema() strips tracking summary, isopen If an IDatasetForm plugin with a db_to_form_schema() based on db_to_form_package_schema() (which is in turn based on default_package_schema()) is in use then the 'tracking_summary' dict and the 'isopen' bool get stripped from package dicts during validation, e.g. during package_show(), and these values are then not available to templates. 1344444427000000 1350303821000000
2820 defect danieljohnlewis demo phase 5 new English Language: Visualization -> Visualisation "Problem: In the English version (which has a UK flag, indicating British English), the word ""Visualization"" is used. For an example see the ""Filter by type"" drop down on the /apps page.
Expected: This should be ""Visualisation"" in British English. Any instances of ""Visualize"" should be changed to ""Visualise"" too.
" 1344504455000000 1344504455000000
2821 enhancement danieljohnlewis demo phase 5 new Featured Items on Filter "Problem: On /apps page in the Filter Results box there is a ""Only show featured items"" checkbox, on selection it comes up with 0 solutions.
Expected: Presumably an admin can create ""featured items"" so that they can be randomly selected on front page (is this correct)? If there are no ""featured items"" in the whole database can this check box be hidden?
Bug is: no UI or obvious way to create featured items. Also the
checkbox looks un-styled" 1344504504000000 1344505492000000
2822 enhancement toby toby demo phase 4 new Resource additional info titles format/i18n "the title for additional info should be translated
capitalised etc" 1344504620000000 1344543985000000
2823 enhancement toby toby demo phase 5 new resource additional info title order "Order the items so that none user fields are first from ticket #2707
" 1344504773000000 1344504773000000
2828 enhancement toby shevski demo phase 4 new Draft datasets are confusing - tickets need creating "reported as editing datasets incorrect
e.g. clicking on edit here http://s031.okserver.org:2375/dataset/ff
takes you to create dataset page http://s031.okserver.org:2375/dataset/edit/ff
but this is the correct behaviour of a draft dataset
We need to show draft datasets correctly
proper tickets need making for the different issues after review of issues with them - who can see, where, admins and viewing, orgs too etc" 1344506178000000 1344547324000000
2829 enhancement johnglover toby ckan-v1.9 new Archiver fails on 403 http response "Had this issue with the archiver on my local machine need to be logged in (I am admin) to see via web front end
{{{
$ paster archiver update -c ../ckan/development.ini
2012-08-09 11:01:37,636 INFO [ckanext.archiver.commands] Archival of dataset resource data added to celery queue: opencontext-chogha-mish-fauna (1 resources)
2012-08-09 11:01:37,671 INFO [ckanext.archiver.commands] Getting dataset metadata: south-african-national-gov-budget-2012-13
2012-08-09 11:01:37,900 INFO [ckan.lib.base] /api/action/package_show render time 0.043 seconds
Traceback (most recent call last):
File ""/home/toby/okfn/pyenv/bin/paster"", line 8, in
load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py"", line 104, in run
invoke(command, command_name, options, args[1:])
File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py"", line 143, in invoke
exit_code = runner.run(args)
File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py"", line 238, in run
result = self.command()
File ""/home/toby/okfn/pyenv/src/ckanext-archiver/ckanext/archiver/commands.py"", line 98, in command
response = app.post(api_url + '/package_show', data)
File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py"", line 262, in post
expect_errors=expect_errors)
File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py"", line 243, in _gen_request
return self.do_request(req, status=status)
File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py"", line 406, in do_request
self._check_status(status, res)
File ""/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py"", line 439, in _check_status
res.body))
paste.fixture.AppError: Bad response: 403 Forbidden (not 200 OK or 3xx redirect for /api/action/package_show)
{""help"": ""Return the metadata of a dataset (package) and its resources.\n\n :param id: the id or name of the dataset\n :type id: string\n\n :rtype: dictionary\n\n "", ""success"": false, ""error"": {""message"": ""Access denied"", ""__type"": ""Authorization Error""}}
}}}" 1344508484000000 1344508484000000
2830 enhancement toby toby demo phase 4 new Need method to undelete groups need controller action and front-end method 1344509408000000 1344547341000000
2831 enhancement aron.carroll ckan 2.0 new Create a limited subset of markdown that's supported "Allowing people to use the full range of markdown results in extremely messy output across the site. I'd suggest limiting support to only a subset of common use cases.
* Allow all inline elements, this allows bold, italic, code and links.
* Allow lists.
Disallow
* Horizontal Rules
* Headings
* Block quote and code (this may turn out to be useful and so could be included)
This way you get markdowns paragraph handling and a few inline styles without breaking the entire layout of the page.
Here's the full syntax if anyone is interested http://daringfireball.net/projects/markdown/syntax" 1344512467000000 1344512467000000
2833 enhancement aron.carroll demo phase 5 new Load module templates before calling .initialize() "I think this would be a nice feature for remote loading templates if the options.template value ends in "".html"".
{{{
ckan.module('my-module', {
options: {
template: 'my-template.html'
},
initialize: function () {
this.template // This is the loaded template.
}
});
}}}" 1344531939000000 1344531939000000
2834 enhancement aron.carroll demo phase 4 new Defer all publish calls until all modules have been initialised "Currently if a module uses sandbox.publish() in its initilaize() function then any modules initialised afterwards will miss the event.
We should queue all calls to .publish() in the pubsub module in an array until all modules have loaded then iterate over and execute them." 1344532093000000 1344856692000000
2835 enhancement aron.carroll demo phase 5 new Client module needs a template loading method "{{{
Client#getTemplate(name, params, success, error);
}}}
Where params, success and error are optional arguments. test/index.html already has an implementation called loadFixture()." 1344532233000000 1344532233000000
2838 enhancement seanh ckan-v1.9 new Context variables accepted by action functions need to be documented "I was doing this:
{{{
context = {'model': base.model, 'session': base.model.Session,
'user': toolkit.c.user or toolkit.c.author,
'extras_as_string': True}
group_dict = logic.get_action('group_show')(context,
{'id': group_id})
}}}
in an extension and one of the group_dicts fields, one that uses convert_to/from_extras, was coming out with the wrong value. It took me ages to realise that I had to pass {{{'extras_as_string': True}}} in the context. I don't think this or other context variables are documented anywhere." 1344630911000000 1344630911000000
2844 enhancement rgrp new SQL-only (no solr) version of CKAN " * ~~Search needs to run of local DB (again)~~
* paster db clean attemps to connect to SOLR (still works as does db first but then excepts which is not nice UX)" 1344859168000000 1345454527000000
2846 defect seanh ckan-v1.8.1 new Organizations allows you to set an organization as the parent organization of itself that doesn't make any sense 1344860536000000 1350303844000000
2850 enhancement aron.carroll demo phase 4 new Add JSON support for IE7 "The JavaScript just isn't going to work without it. Look into either the newer JSON3:
http://bestiejs.github.com/json3/
or the old faithful:
https://github.com/douglascrockford/JSON-js/" 1344940692000000 1344940692000000
2851 enhancement ross new Preview of PDF tries to connect to datastore "e.g. http://datahub.io/dataset/ccc-gistemp/resource/80ebdbd6-d91b-4fef-9db4-d3dfbd7e868e
CKAN attempts to render the PDF as a datastore tabular date even though the mimetype of the PDF is correctly specified." 1345032766000000 1345032766000000
2854 enhancement shevski demo phase 5 new no way to see which dataset an app is related to on /apps page e.g. from http://s031.okserver.org:2375/apps?page=1 you cannot edit a related item as an admin or view what dataset the item is related to 1345048728000000 1345048728000000
2857 defect toby shevski demo phase 4 new no UI to follow another user although option on dashboard "e.g. second option in left hand sidebar is follow another user' http://s031.okserver.org:2375/user/dashboard but if you click on that and then any user, there's no follow button
to me the dashboard user stuff is highly unfinished and probably should not be included unless someone is very keen for it" 1345049983000000 1345107097000000
2859 defect icmurray icmurray ckan-v1.8 new Fix the build "The requires files have been removed. These were required by buildkit.
We've lost the information regarding which packages were already in ubuntu repos, which conflicted and which were missing. But perhaps we can just package everything up." 1345109217000000 1345109217000000
2861 enhancement toby shevski demo phase 4 new Add qa breadcrumb can we add in the breadcrumb navigation to qa pages so that you can go back to http://s031.okserver.org:2375/qa from sub pages such as http://s031.okserver.org:2375/qa/dataset/five_stars 1345114046000000 1345483270000000
2862 enhancement toby markw demo phase 4 new Revised revised groups description "Revised text for 'What are groups?' box at demo.ckan.org/group (after discussion with IB re #2812):
What are groups?
Groups allow you to group users and data together so that they are easier to manage. Group owners can assign roles and authorisations, giving each project or department control of its own data publishing.
Users can browse or facet by groups, which could be an organisation (for example, the Department of Health) or topic (e.g. Transport, Health), making it easier to find the data they are looking for." 1345114322000000 1345115072000000
2863 defect toby shevski demo phase 4 new non-admin users can edit other's draft datasets edit button shows up for daniel lewis for http://s031.okserver.org:2375/dataset/ff 1345114479000000 1345201502000000
2867 enhancement ross new 1.8 url_for error "From:
Date: Thu, Aug 16, 2012 at 10:30 AM
Subject: WebApp Error: : url_for can only return a string, got unicode instead: https://ckannet-storage.commondatastorage.googleapis.com/file/fd3e00d9-eb0d-479b-8303-38909e5400ca?Signature=cmuU3CzRHQ86F3aZ0gljxv%2B3IqQ%3D&Expires=1345109432&AWSAccessKeyId=GOOGC6OU3AYPNY47B66M&x-goog-meta-uploaded-by=d5c8ed25-70de-4035-b29d-ddbe363913c6
To: ckan-sysadmin@okfn.org, kindly@gmail.com
URL: http://ckan.net/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca
Module weberror.errormiddleware:162 in __call__
<< __traceback_supplement__ = Supplement, self, environ
sr_checker = ResponseStartChecker(start_response)
app_iter = self.application(environ, sr_checker)
return self.make_catching_iter(app_iter, environ, sr_checker)
except:
>> app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<< self.cache_manager)
environ[self.environ_key] = self.cache_manager
return self.app(environ, start_response)
>> return self.app(environ, start_response)
Module beaker.middleware:155 in __call__
<< headers.append(('Set-cookie', cookie))
return start_response(status, headers, exc_info)
return self.wrap_app(environ, session_start_response)
def _get_session(self):
>> return self.wrap_app(environ, session_start_response)
Module routes.middleware:131 in __call__
<< r'\1', oldpath)
response = self.app(environ, start_response)
# Wrapped in try as in rare cases the attribute will be gone already
>> response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<
controller = self.resolve(environ, start_response)
response = self.dispatch(controller, environ, start_response)
if 'paste.testing_variables' in environ and hasattr(response,
>> response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<< if log_debug:
log.debug(""Calling controller class with WSGI interface"")
return controller(environ, start_response)
def load_test_env(self, environ):
>> return controller(environ, start_response)
Module ckan.lib.base:239 in __call__
<< try:
res = WSGIController.__call__(self, environ, start_response)
finally:
model.Session.remove()
>> res = WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<< return response(environ, self.start_response)
response = self._dispatch_call()
if not start_response_called:
self.start_response = start_response
>> response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<< req.environ['pylons.action_method'] = func
response = self._inspect_call(func)
else:
if log_debug:
>> response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<< func.__name__, args)
try:
result = self._perform_call(func, args)
except HTTPException, httpe:
if log_debug:
>> result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<< """"""Hide the traceback for everything above this method""""""
__traceback_hide__ = 'before_and_this'
return func(**args)
def _inspect_call(self, func):
>> return func(**args)
Module ckan.controllers.storage:190 in file
<< return fapp(request.environ, self.start_response)
else:
h.redirect_to(file_url)
>> h.redirect_to(file_url)
Module ckan.lib.helpers:57 in redirect_to
<< if are_there_flash_messages():
kw['__no_cache__'] = True
return _redirect_to(url_for(*args, **kw))
def url(*args, **kw):
>> return _redirect_to(url_for(*args, **kw))
Module ckan.lib.helpers:79 in url_for
<< # fix ver to include the slash
kw['ver'] = '/%s' % ver
my_url = _routes_default_url_for(*args, **kw)
kw['__ckan_no_root'] = no_root
return _add_i18n_to_url(my_url, locale=locale, **kw)
>> my_url = _routes_default_url_for(*args, **kw)
Module routes.util:265 in url_for
<< if not isinstance(url, str) and url is not None:
raise GenerationException(""url_for can only return a string, got ""
""unicode instead: %s"" % url)
if url is None:
raise GenerationException(
>> ""unicode instead: %s"" % url)
GenerationException: url_for can only return a string, got unicode instead: https://ckannet-storage.commondatastorage.googleapis.com/file/fd3e00d9-eb0d-479b-8303-38909e5400ca?Signature=cmuU3CzRHQ86F3aZ0gljxv%2B3IqQ%3D&Expires=1345109432&AWSAccessKeyId=GOOGC6OU3AYPNY47B66M&x-goog-meta-uploaded-by=d5c8ed25-70de-4035-b29d-ddbe363913c6
CGI Variables
CKAN_CURRENT_URL '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca'
CKAN_LANG 'en'
CKAN_LANG_IS_DEFAULT True
CONTENT_TYPE '; charset=utf-8'
DOCUMENT_ROOT '/etc/apache2/htdocs'
GATEWAY_INTERFACE 'CGI/1.1'
HTTP_ACCEPT '*/*'
HTTP_ACCEPT_CHARSET 'utf-8;q=0.7,iso-8859-1;q=0.2,*;q=0.1'
HTTP_CONNECTION 'close'
HTTP_HOST 'ckan.net'
HTTP_USER_AGENT 'Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)'
PATH_INFO '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca'
PATH_TRANSLATED '/etc/ckan/datahub/apache.wsgi/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca'
REMOTE_ADDR '127.0.0.1'
REMOTE_PORT '37236'
REQUEST_METHOD 'GET'
REQUEST_URI '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca'
SCRIPT_FILENAME '/etc/ckan/datahub/apache.wsgi'
SERVER_ADDR '127.0.0.1'
SERVER_ADMIN '[no address given]'
SERVER_NAME 'ckan.net'
SERVER_PORT '80'
SERVER_PROTOCOL 'HTTP/1.0'
SERVER_SIGNATURE 'Apache/2.2.22 (Ubuntu) Server at ckan.net Port 80\n'
SERVER_SOFTWARE 'Apache/2.2.22 (Ubuntu)'
WSGI Variables
application
beaker.cache
beaker.get_session >
beaker.session {'_accessed_time': 1345109433.273328, '_creation_time': 1345109433.273328}
mod_wsgi.application_group 'ckan.net|'
mod_wsgi.callable_object 'application'
mod_wsgi.handler_script ''
mod_wsgi.input_chunked '0'
mod_wsgi.listener_host ''
mod_wsgi.listener_port '8080'
mod_wsgi.process_group 'datahub'
mod_wsgi.request_handler 'wsgi-script'
mod_wsgi.script_reloading '1'
mod_wsgi.version (3, 3)
paste.cookies (, '')
paste.registry
paste.throw_errors True
pylons.action_method >
pylons.controller
pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons
pylons.routes_dict {'action': u'file', 'controller': u'ckan.controllers.storage:StorageController', 'label': u'file/fd3e00d9-eb0d-479b-8303-38909e5400ca'}
repoze.who.logger
repoze.who.plugins {'openid': , 'friendlyform': , 'ckan.lib.authenticator:UsernamePasswordAuthenticator': , 'auth_tkt': , 'ckan.lib.authenticator:OpenIDAuthenticator': }
routes.route
routes.url
webob._parsed_query_vars (GET([]), '')
webob.adhoc_attrs {'language': 'en-us'}
wsgi process 'Multi process AND threads (?)'
wsgi.file_wrapper
wsgi.version (1, 1)
wsgiorg.routing_args (, {'action': u'file', 'controller': u'ckan.controllers.storage:StorageController', 'label': u'file/fd3e00d9-eb0d-479b-8303-38909e5400ca'})
" 1345114669000000 1345128279000000
2869 defect shevski demo phase 4 new user profile not linked on disqus until page refresh "Reported by Daniel:
Problem: DISQUS comments don't appear with profile details (inc. username).
How to reproduce:
1) Go to a dataset, e.g. I went to http://s031.okserver.org:2375/dataset/ff
2) I logged into DISQUS using the Google Authentication, e.g. in my case that is okfn.daniel.lewis@gmail.com
3) I made a comment and submitted
4) It comes up with the comment, but no linked profile until the page gets refreshed" 1345114714000000 1345114714000000
2870 enhancement seanh ross new 1.8 tag_list not defined "From:
Date: Thu, Aug 16, 2012 at 3:20 AM
Subject: WebApp Error: : ""tag_list"" not defined
To: ckan-sysadmin@okfn.org, kindly@gmail.com
URL: http://ckan.net/dataset/uk-postboxes
Module weberror.errormiddleware:162 in __call__
<< __traceback_supplement__ = Supplement, self, environ
sr_checker = ResponseStartChecker(start_response)
app_iter = self.application(environ, sr_checker)
return self.make_catching_iter(app_iter, environ, sr_checker)
except:
>> app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<< self.cache_manager)
environ[self.environ_key] = self.cache_manager
return self.app(environ, start_response)
>> return self.app(environ, start_response)
Module beaker.middleware:155 in __call__
<< headers.append(('Set-cookie', cookie))
return start_response(status, headers, exc_info)
return self.wrap_app(environ, session_start_response)
def _get_session(self):
>> return self.wrap_app(environ, session_start_response)
Module routes.middleware:131 in __call__
<< r'\1', oldpath)
response = self.app(environ, start_response)
# Wrapped in try as in rare cases the attribute will be gone already
>> response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<
controller = self.resolve(environ, start_response)
response = self.dispatch(controller, environ, start_response)
if 'paste.testing_variables' in environ and hasattr(response,
>> response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<< if log_debug:
log.debug(""Calling controller class with WSGI interface"")
return controller(environ, start_response)
def load_test_env(self, environ):
>> return controller(environ, start_response)
Module ckan.lib.base:239 in __call__
<< try:
res = WSGIController.__call__(self, environ, start_response)
finally:
model.Session.remove()
>> res = WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<< return response(environ, self.start_response)
response = self._dispatch_call()
if not start_response_called:
self.start_response = start_response
>> response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<< req.environ['pylons.action_method'] = func
response = self._inspect_call(func)
else:
if log_debug:
>> response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<< func.__name__, args)
try:
result = self._perform_call(func, args)
except HTTPException, httpe:
if log_debug:
>> result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<< """"""Hide the traceback for everything above this method""""""
__traceback_hide__ = 'before_and_this'
return func(**args)
def _inspect_call(self, func):
>> return func(**args)
Module ckan.controllers.package:322 in read
<< template = template[:template.index('.') + 1] + format
return render(template, loader_class=loader)
def comments(self, id):
>> return render(template, loader_class=loader)
Module ckan.lib.base:153 in render
<< try:
return cached_template(template_name, render_template,
loader_class=loader_class)
except ckan.exceptions.CkanUrlException, e:
raise
>> loader_class=loader_class)
Module pylons.templating:249 in cached_template
<< return content
else:
return render_func()
>> return render_func()
Module ckan.lib.base:102 in render_template
<< if loader_class == NewTextTemplate:
return literal(stream.render(method=""text"", encoding=None))
return literal(stream.render(method=method, encoding=None,
>> return literal(stream.render(method=""text"", encoding =None))
Module genshi.core:183 in render
<< method = self.serializer or 'xml'
generator = self.serialize(method=method, **kwargs)
return encode(generator, method=method, encoding=encoding, out=out)
def select(self, path, namespaces=None, variables=None):
>> return encode(generator, method=method, encoding=encoding, out =out)
Module genshi.output:57 in encode
<< _encode = lambda string: string
if out is None:
return _encode(''.join(list(iterator)))
for chunk in iterator:
out.write(_encode(chunk))
>> return _encode(''.join(list(iterator)))
Module genshi.output:569 in __call__
<< def __call__(self, stream):
strip_markup = self.strip_markup
for event in stream:
if event[0] is TEXT:
data = event[1]
>> for event in stream:
Module genshi.core:288 in _ensure
<< # unchanged
yield event
for event in stream:
yield event
>> for event in stream:
Module genshi.core:288 in _ensure
<< # unchanged
yield event
for event in stream:
yield event
>> for event in stream:
Module genshi.filters.transform:686 in _unmark
<< def _unmark(self, stream):
for mark, event in stream:
kind = event[0]
if not (kind is None or kind is ATTR or kind is BREAK):
>> for mark, event in stream:
Module genshi.filters.transform:1145 in __call__
<< :param stream: The marked event stream to filter
""""""
for mark, event in stream:
yield mark, event
if mark is ENTER:
>> for mark, event in stream:
Module genshi.filters.transform:714 in __call__
<< stream = iter(stream)
next = stream.next
for mark, event in stream:
if mark is None:
yield mark, event
>> for mark, event in stream:
Module genshi.filters.transform:682 in _mark
<< def _mark(self, stream):
for event in stream:
yield OUTSIDE, event
>> for event in stream:
Module genshi.core:288 in _ensure
<< # unchanged
yield event
for event in stream:
yield event
>> for event in stream:
Module genshi.filters.transform:686 in _unmark
<< def _unmark(self, stream):
for mark, event in stream:
kind = event[0]
if not (kind is None or kind is ATTR or kind is BREAK):
>> for mark, event in stream:
Module genshi.filters.transform:1145 in __call__
<< :param stream: The marked event stream to filter
""""""
for mark, event in stream:
yield mark, event
if mark is ENTER:
>> for mark, event in stream:
Module genshi.filters.transform:714 in __call__
<< stream = iter(stream)
next = stream.next
for mark, event in stream:
if mark is None:
yield mark, event
>> for mark, event in stream:
Module genshi.filters.transform:682 in _mark
<< def _mark(self, stream):
for event in stream:
yield OUTSIDE, event
>> for event in stream:
Module genshi.core:288 in _ensure
<< # unchanged
yield event
for event in stream:
yield event
>> for event in stream:
Module genshi.filters.transform:686 in _unmark
<< def _unmark(self, stream):
for mark, event in stream:
kind = event[0]
if not (kind is None or kind is ATTR or kind is BREAK):
>> for mark, event in stream:
Module genshi.filters.transform:1145 in __call__
<< :param stream: The marked event stream to filter
""""""
for mark, event in stream:
yield mark, event
if mark is ENTER:
>> for mark, event in stream:
Module genshi.filters.transform:714 in __call__
<< stream = iter(stream)
next = stream.next
for mark, event in stream:
if mark is None:
yield mark, event
>> for mark, event in stream:
Module genshi.filters.transform:682 in _mark
<< def _mark(self, stream):
for event in stream:
yield OUTSIDE, event
>> for event in stream:
Module genshi.core:288 in _ensure
<< # unchanged
yield event
for event in stream:
yield event
>> for event in stream:
Module genshi.filters.transform:686 in _unmark
<< def _unmark(self, stream):
for mark, event in stream:
kind = event[0]
if not (kind is None or kind is ATTR or kind is BREAK):
>> for mark, event in stream:
Module ckanext.googleanalytics.plugin:93 in download_adder
<< [downloaded %s times]'''
count = None
for mark, (kind, data, pos) in stream:
if mark and kind == START:
href = data[1].get('href')
>> for mark, (kind, data, pos) in stream:
Module genshi.filters.transform:714 in __call__
<< stream = iter(stream)
next = stream.next
for mark, event in stream:
if mark is None:
yield mark, event
>> for mark, event in stream:
Module genshi.filters.transform:682 in _mark
<< def _mark(self, stream):
for event in stream:
yield OUTSIDE, event
>> for event in stream:
Module genshi.core:288 in _ensure
<< # unchanged
yield event
for event in stream:
yield event
>> for event in stream:
Module genshi.filters.transform:686 in _unmark
<< def _unmark(self, stream):
for mark, event in stream:
kind = event[0]
if not (kind is None or kind is ATTR or kind is BREAK):
>> for mark, event in stream:
Module genshi.filters.transform:1175 in __call__
<< """"""
callable_value = hasattr(self.value, '__call__')
for mark, (kind, data, pos) in stream:
if mark is ENTER:
if callable_value:
>> for mark, (kind, data, pos) in stream:
Module genshi.filters.transform:714 in __call__
<< stream = iter(stream)
next = stream.next
for mark, event in stream:
if mark is None:
yield mark, event
>> for mark, event in stream:
Module genshi.filters.transform:682 in _mark
<< def _mark(self, stream):
for event in stream:
yield OUTSIDE, event
>> for event in stream:
Module genshi.core:288 in _ensure
<< # unchanged
yield event
for event in stream:
yield event
>> for event in stream:
Module genshi.filters.transform:686 in _unmark
<< def _unmark(self, stream):
for mark, event in stream:
kind = event[0]
if not (kind is None or kind is ATTR or kind is BREAK):
>> for mark, event in stream:
Module genshi.filters.transform:1145 in __call__
<< :param stream: The marked event stream to filter
""""""
for mark, event in stream:
yield mark, event
if mark is ENTER:
>> for mark, event in stream:
Module genshi.filters.transform:714 in __call__
<< stream = iter(stream)
next = stream.next
for mark, event in stream:
if mark is None:
yield mark, event
>> for mark, event in stream:
Module genshi.filters.transform:682 in _mark
<< def _mark(self, stream):
for event in stream:
yield OUTSIDE, event
>> for event in stream:
Module genshi.template.base:605 in _include
<< from genshi.template.loader import TemplateNotFound
for event in stream:
if event[0] is INCLUDE:
href, cls, fallback = event[1]
>> for event in stream:
Module genshi.template.base:565 in _flatten
<< elif kind is EXPR:
result = _eval_expr(data, ctxt, vars)
if result is not None:
# First check for a string, otherwise the iterable test
>> result = _eval_expr(data, ctxt, vars)
Module genshi.template.base:277 in _eval_expr
<< if vars:
ctxt.push(vars)
retval = expr.evaluate(ctxt)
if vars:
ctxt.pop()
>> retval = expr.evaluate(ctxt)
Module genshi.template.eval:178 in evaluate
<< __traceback_hide__ = 'before_and_this'
_globals = self._globals(data)
return eval(self.code, _globals, {'__data__': data})
>> return eval(self.code, _globals, {'__data__': data})
Module ?:51 in
<<
Tags
${tag_list(c.pkg_dict.get('tags', ''))}
>> ${tag_list(c.pkg_dict.get('tags', ''))}
Module genshi.template.eval:309 in lookup_name
<< val = BUILTINS.get(name, val)
if val is UNDEFINED:
val = cls.undefined(name)
return val
>> val = cls.undefined(name)
Module genshi.template.eval:410 in undefined
<< """"""Raise an ``UndefinedError`` immediately.""""""
__traceback_hide__ = True
raise UndefinedError(key, owner=owner)
>> raise UndefinedError(key, owner=owner)
UndefinedError: ""tag_list"" not defined
CGI Variables
CKAN_CURRENT_URL '/dataset/uk-postboxes'
CKAN_LANG 'en'
CKAN_LANG_IS_DEFAULT True
CKAN_PAGE_CACHABLE True
CONTENT_TYPE '; charset=utf-8'
DOCUMENT_ROOT '/etc/apache2/htdocs'
GATEWAY_INTERFACE 'CGI/1.1'
HTTP_ACCEPT 'text/html,text/plain,text/xml'
HTTP_ACCEPT_CHARSET 'ISO-8859-1,utf-8;q=0.7,*;q=0.7'
HTTP_ACCEPT_ENCODING 'gzip'
HTTP_ACCEPT_LANGUAGE 'vi,en-us;q=0.7,en;q=0.3'
HTTP_CONNECTION 'close'
HTTP_HOST 'ckan.net'
HTTP_USER_AGENT 'coccoc/1.0 ()'
PATH_INFO '/dataset/uk-postboxes'
PATH_TRANSLATED '/etc/ckan/datahub/apache.wsgi/dataset/uk-postboxes'
REMOTE_ADDR '127.0.0.1'
REMOTE_PORT '47065'
REQUEST_METHOD 'GET'
REQUEST_URI '/dataset/uk-postboxes'
SCRIPT_FILENAME '/etc/ckan/datahub/apache.wsgi'
SERVER_ADDR '127.0.0.1'
SERVER_ADMIN '[no address given]'
SERVER_NAME 'ckan.net'
SERVER_PORT '80'
SERVER_PROTOCOL 'HTTP/1.0'
SERVER_SIGNATURE 'Apache/2.2.22 (Ubuntu) Server at ckan.net Port 80\n'
SERVER_SOFTWARE 'Apache/2.2.22 (Ubuntu)'
WSGI Variables
application
beaker.cache
beaker.get_session >
beaker.session {'_accessed_time': 1345083636.079552, '_creation_time': 1345083636.079552}
mod_wsgi.application_group 'ckan.net|'
mod_wsgi.callable_object 'application'
mod_wsgi.handler_script ''
mod_wsgi.input_chunked '0'
mod_wsgi.listener_host ''
mod_wsgi.listener_port '8080'
mod_wsgi.process_group 'datahub'
mod_wsgi.request_handler 'wsgi-script'
mod_wsgi.script_reloading '1'
mod_wsgi.version (3, 3)
paste.cookies (, '')
paste.registry
paste.throw_errors True
pylons.action_method >
pylons.controller
pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons
pylons.routes_dict {'action': u'read', 'controller': u'package', 'id': u'uk-postboxes'}
repoze.who.logger
repoze.who.plugins {'openid': , 'friendlyform': , 'ckan.lib.authenticator:UsernamePasswordAuthenticator': , 'auth_tkt': , 'ckan.lib.authenticator:OpenIDAuthenticator': }
routes.route
routes.url
webob._parsed_query_vars (GET([]), '')
webob.adhoc_attrs {'language': 'en-us'}
wsgi process 'Multi process AND threads (?)'
wsgi.file_wrapper
wsgi.version (1, 1)
wsgiorg.routing_args (, {'action': u'read', 'controller': u'package', 'id': u'uk-postboxes'})
" 1345114732000000 1345114732000000
2881 defect seanh ckan-v1.8.1 new Remove or improve 'Domain Model' docs page "http://docs.ckan.org/en/ckan-1.8/domain-model.html
Currently not very useful. Also these pages:
http://docs.ckan.org/en/latest/domain-model-dataset.html
http://docs.ckan.org/en/latest/domain-model-resource.html
http://docs.ckan.org/en/latest/domain-model-task-status.html
This just seems very incomplete to me (surely there is more to the domain model the dataset, resource and task status?) and very likely to not get updated as changes are mode to the code. Would anyone seriously refer to this rather than just looking at the source code?
If we want to have something along these lines in the docs, I suggest adding docstrings to packages, modules, classes and methods and then using autodoc to pull them into sphinx." 1345539917000000 1345626056000000
2882 defect seanh ckan-v1.8.1 new Remove User Stories Overview from docs? "Why is this here? Is it up to date? Is anyone keeping it up to date? Does this belong in some sort of CKAN planning or roadmap document rather than in the documentation?
http://docs.ckan.org/en/latest/user-stories.html" 1345540723000000 1345540723000000
2883 enhancement seanh ckan-v1.8.1 new Add high level dev overview to 'For CKAN Developers' section of docs Here's a draft: https://gist.github.com/3414107 1345546355000000 1345629890000000
2884 defect seanh ckan-v1.8.1 new Lock icon shown next to unknown licenses Set the license of a dataset to blank (the first option) or to 'License Not Specified' or 'Other (Non-Commercial)' and a lock icon appears next to the license on the dataset read page. Shouldn't this be a question mark or something? 1345564873000000 1345564873000000
2885 enhancement seanh ckan-v1.9 new Labels (editor, admin) not translated in authorization pages for datasets or groups 1345625351000000 1345625524000000
2886 enhancement ross ckan 2.0 new Configurable related items "Related items (Apps & Ideas) have a collection of types, which are currently fixed. Some of these types are ambiguous, such as ideas in that they may not have a link - and if they do it is likely to be to a blog post - another existing type.
Whilst the URL is required, and we think this should stay required, we should also allow users to change the types found in Apps & Ideas to a shortened list that suits their requirements." 1345625718000000 1345625718000000
2887 defect seanh ckan-v1.8.1 new """Welcome to CKAN!"" on front page is untranslatable" "ckan/templates/home/index.html contains:
Welcome to ${g.site_title}!
which means that just ""Welcome to"" ends up in the ckan.po files for
translation. This makes the phrase untranslatable in languages that have a different word ordering than English where the site title needs to be somewhere other than at the end of the sentence." 1345627079000000 1345627233000000
2888 enhancement kindly new Datapreview in Iframe Try and put datapreview in Iframe. 1345745461000000 1345745461000000
2889 enhancement seanh ckan-v1.9 new Support JSON values in extras, when returning JSON from API "For example, edit the warandpeace dataset and add an extra with key ""comments"" and value ""[{u'date': u'Tue Aug 21 13:01:48 GMT 2012', u'comment': u'testing ckan', u'userId': u'11702'}, {u'date': u'Wed Aug 22 13:01:48 GMT 2012', u'comment': u'testing ckan again', u'userId': u'11702'}]"", now visit API URLs such as:
/api/search/dataset?sort=metadata_modified+desc&all_fields=1&limit=10
/api/rest/dataset/warandpeace
/api/action/package_show?id=warandpeace
The value of the extra comes out as a string in the dataset JSON, rather than having its contents unpacked into a list of JSON objects." 1345811737000000 1345811737000000
2890 enhancement seanh ckan-v1.8.1 new Collect data previews and data store docs in one chapter "Currently there is this page:
http://docs.ckan.org/en/latest/data-viewer.html
which covers Recline Data Explorer and other kinds of data preview in CKAN. It is under the Publishing Datasets section in the documentation. I had to to a search for 'recline' to find it.
Separately there is this page: http://docs.ckan.org/en/ckan-1.7.1/datastore.html which covers datastore, datastorer, and the data api.
I suggest collecting this together in one chapter called 'Data Previews'. If I understand it right the general gist would be:
CKAN has builtin previews of data resources on resource pages, enabled by default.
Images, Google Documents, and web page resources will be loaded into embedded iframes for preview.
Text-like files will be displayed raw.
CSV or Excel files uploaded to CKAN will be previewed using Recline Data Explorer.
Additionally, you can enable CKAN's DataStore, requires you to install ElasticSearch and nginx and put datastore.enabled=1 in your ini file. Lets you use the Data API to query data.
Does having DataStore enabled mean you get preview of more types of resources? Any resource that's available via the Data API will be previewed using Recline,
You can install ckanext-datastorer, and then CSV and Excel files _linked to_ as CKAN resources will be previewed using Recline also. Requires celeryd." 1346149236000000 1346175867000000
2891 defect seanh ckan-v1.8.1 new Update or remove DataStore google presentation "Currently this page:
http://docs.ckan.org/en/ckan-1.8/datastore.html
has an embedded Google docs presentation about data store which contains some outdated information, e.g. links to datastoreclient which was merged into ckanclient.
Suggest moving the google presentation into the documentation proper, so it works with grep etc." 1346149423000000 1346154177000000
2892 defect seanh ckan-v1.8.1 new IDatasetForm's form_to_db_schema() breaks API? "It looks like when an IDatasetForm plugin with a form_to_db_schema() method is active, then the posted data dict is validated against the schema even if the user is posting to an API endpoint like /api/action/package_update etc. This breaks the API, for example 'name' and 'title' are optional keys for package_update (you can supply only the 'id'), but the schema validation will fail if either are not present in the posted dict.
It may be we need different schema for package create and package update, currently IDatasetForm has just the one form_to_db_schema() method." 1346154125000000 1346154163000000
2893 defect seanh ckan-v1.8.1 new Tags missing from form_to_db_package_schema() "With an IDatasetForm plugin active with a form_to_db_package_schema() method that returns a schema based on ckan/logic/schema.py:form_to_db_package_schema() I'm getting validation errors like this:
[ckan.controllers.api] Validation error: ""{'__type': 'Validation Error', '__junk': [u'The input field __junk was not expected.']}""
They seem to be caused when a data dict with key 'tags' is passed to e.g. the package_update API, for example in ckan/tests/functional/api/model/test_vocabulary.py there are various tests that get this error when they call package_update (if you run the tests with a suitable IDatasetForm plugin active).
In ckan/logic/schema.py:form_to_db_package_schema() 'tags' is for some reason deliberately removed from the schema." 1346155864000000 1346155864000000
2894 defect seanh ckan-v1.8.1 new Empty lists get stripped from package dicst in package_show() "If there's an IDatasetForm plugin active with a form_to_db_schema(), then packages with e.g. no extras get the 'extras' key entirely stripped by the validation in package_show, which causes some tests to crash, e.g. in ckan/tests/functional/api/test_activity.py (if the tests are run with a suitable IDatasetForm plugin active).
If a package has no extras the dict returned by package_show should still have an 'extras' key with an empty list as the value.
Suspect this also effects packages with no tags, no groups, etc." 1346157634000000 1346157634000000
2895 enhancement rgrp new QA pages (e.g. broken resource links) are not paginated And as a result timeout on e.g. the datahub ... 1346175485000000 1346175485000000
2896 defect seanh ckan-v1.8.1 new resource_create crashes on ValidationErrors with KeyError: 'resources' "ckan/logic/action/create.py:resource_create():
try:
pkg_dict = _get_action('package_update')(context, pkg_dict)
except ValidationError, e:
errors = e.error_dict['resources'][-1]
raise ValidationError(errors)
This assumes that package_update always returns an error dict with 'resources' in it. This assumption doesn't hold." 1346257754000000 1346257754000000
2897 defect seanh ckan-v1.8.1 new db_to_form_package_schema() crashes historical dataset page "If there's an IDatasetForm plugin active with a db_to_form_schema() method based on db_to_form_package_schema(), then viewing historical versions of datasets (e.g. http://127.0.0.1:5000/dataset/dasd%402012-08-30T12%3A26%3A18.524191) crashes.
Reason is that revison_id and revision_timestamp need to be added to db_to_form_package_schema(), something like:
schema.update({'revision_id': [validators.ignore_missing, unicode]})
schema.update({'revision_timestamp':
[validators.ignore_missing, unicode]})
" 1346330495000000 1346330495000000
2898 defect johnmartin demo phase 5 new Looses data entered on step 1 of create dataset after login "To reproduce:
1. Enter details for new dataset without being logged in
2. Hit login page for just before step 2
3. Once logged in you end up back at the empty add dataset form (instead of expected 2nd step of dataset edit)" 1346332094000000 1346332120000000
2899 enhancement johnmartin demo phase 5 new Step 2 > Save & add another bug "Goto:
1. Add dataset
2. Get to step 2
3. Click ""Save & add another"" without adding any data
4. Ends on step 3
Should: fail to validate and remain on step 2" 1346332519000000 1346332519000000
2900 defect seanh ckan-v1.8.1 new snippets/package_list.html template crashes on dataset that has no 'notes' "Sometimes a dataset dict comes through package_show() with 'notes': None and then the 'notes' key gets stripped from the dict when validation happens (validation only happens if there's an IDatasetForm plugin with a db_to_form_schema() method) and then package_list.html template crashes, e.g. on user profile pages.
Fix in the template? Or in the schema?
The template also crashes on packages with no tags (empty list gets stripped from dict) and probably other empty fields as well" 1346335207000000 1346335788000000
2901 enhancement johnmartin demo phase 5 new Language Dropdown bug in footer with IE7 "The language dropdown in Internet Explorer 7 behaves oddly. Essentially I think the problem is with the way that the dropdown decides to go into ""drop-up mode"" (e.g there aint enough screen space below the initial state so it has to drop up)." 1346754674000000 1346754674000000
2902 enhancement ross ckan 2.0 new genshi is used in Group controller "The Group controller uses the genshi module where it may be better for it to use Markup() in the template.
See https://github.com/okfn/ckan/pull/117/files#r1524863" 1346782230000000 1346782230000000
2904 enhancement johnmartin demo phase 5 new Show more formats/groups/tags shouldn't display if there aren't more On the search results page when there are no more formats, groups or tags to display it should not display the show more links. 1346854444000000 1346854444000000
2905 enhancement johnmartin demo phase 5 new Add dataset URL key behaviour "On step 1 for adding a dataset can we add a ajax ping that checks if the URL that is generated by the JS is taken? Behaviour should be as follows:
1. User types title in box (as current)
2. URL key is regexed by JS (as current)
3. The JS does a ajax call to check if the generated URL key is valid. The ajax callback returns a JSON dump which contains what the URL key should be (e.g. if 'test' was taken it returns 'test-1')
4. If the URL key returned in the ajax is different than the one generated by the JS then:
4a. If the user hasn't clicked the edit button then: it automatically updates the URL key field with the URL key supplied in the ajax
4b. If the user has clicked the edit button then: a form error next to the input appears saying that the URL key is taken and within that error is suggests the URL key returned by the ajax as alternative (the user can click said URL key within the error to automatically choose that one)
The URL key isn't reserved by the ajax and upon form submission the usual validation should still apply." 1346855283000000 1346855283000000
2906 defect dominik demo phase 5 new Wrong flags in language dropdown "The flags in the dropdown on details pages are not right.
You can see the error on http://s031.okserver.org:2375/dataset/malawi-aid-projects/resource/b717c20e-2006-4ad4-82d2-59b57ebc1ab0." 1346861144000000 1346862889000000
2908 enhancement johnmartin demo phase 5 new Dataset related with image should retain aspect ratio At the moment if you have an related item with an image associated to a dataset that appears in the side column it's resized but doesn't keep it's original aspect ratio. 1346931162000000 1346931162000000
2909 enhancement johnmartin demo phase 5 new User profile headers are a bit weird They seem to be not exactly well thought out. Needs some thought on better display of the information that they contain. 1346931348000000 1346931348000000
2910 enhancement johnmartin demo phase 5 new User listings are not very compelling They are currently are just a small image / name. They should be: Larger image, Full name (username), description, edits/dataset 1346931587000000 1346931587000000
2911 enhancement kindly new Internal documentation of Organization Groups A summary/user story doc of how organizations and groups are expected to work. 1346941384000000 1346941384000000
2912 defect seanh ckan-v1.8.1 new ckanext/organizations: Neither public or private initially selected when adding dataset When adding a dataset with the organizations extension on, neither the Public or the Private radio button is initially selected. Public should be selected. 1347271076000000 1350303897000000
2913 defect seanh ckan 2.0 new IGroupForm has package_form() method Should be group_form() I think 1347374739000000 1350303779000000
2914 defect seanh ckan-v1.8.1 new Add form_to_db_schema_options() to IDatasetForm and IGroupForm This optional method is supported when creating or updating groups and packages and is used in DefaultGroupForm and DefaultPackageForm, but seems to be missing from the interfaces. Should be added with docstring. 1347445795000000 1350303928000000
2915 refactor seanh ckan 2.0 new Refactor form_to_db_schema_options() "Having two methods form_to_db_schema(self) and form_to_db_schema_options(self, options) seems unnecessary, why not just have form_to_db_schema(self, options=None)?
Fixing this might break existing extensions although I don't think any are using form_to_db_schema_options() so it should be okay.
The same thing goes for db_to_form_schema_options().
Also why are we passing a dict 'options' that always contains the same three keys 'api', 'context' and 'type'? Why not three params api=None, contenxt=None, type=None?" 1347446269000000 1347447588000000
2917 enhancement johnmartin ross ckan 2.0 new Organization admins can delete themselves Organization administrators can delete themselves from the user management pages. We should disable this on the front-end as well as the back-end. 1347451843000000 1347455636000000
2919 defect seanh ckan-v1.8 new Remove IController from docs Still in docs but seems gone from code 1347461253000000 1347461253000000
2920 defect seanh ckan 2.0 new Genshi text template message extractor no longer needed? "In setup.py, do we still need:
('templates/**.txt', 'genshi', {
'template_class': 'genshi.template:TextTemplate'
}),
in message_extractors? Doesn't like there are any txt files in there
anymore." 1347530249000000 1347530249000000
2921 defect seanh ckan 2.0 new Add docstring to top of lib/extract.py file "I think it couldn't hurt for this module to have a docstring at the top
of the file explaining what the module is for. I know from setup.py that
it's there to provide the extract_ckan() string extractor function for
Babel, but I think looking at the file on its own it's not so obvious.
Also a couple of other small fixes:
jinja2_cleaner looks like a helper function only meant to be used by
extract_ckan(), might make things clearer if it was called
_jinja2_cleaner(), just so it doesn't look like a public function the
module wants to export.
jinja_extensions seems to be a module-level variable that is only used
in one function, could be moved into the function, unless you think
there might be more functions that want it in future." 1347530407000000 1347530407000000
2922 defect seanh ckan 2.0 new Better docstring for CKANInternationalizationExtension "I'm unsure about what's going on here. As I understand it, when we run
`python setup.py extract_messages` it's going to use the extract_ckan()
function from ckan/lib/extract.py to process the HTML files. For the
HTML files that are jinja2 templates, extract_ckan() will call
jinja2_cleaner() which will call regularise_html() on the strings. So
the strings are regularised when they are extracted from the source
files.
But then we have the parse() method of CkanInternationalizationExtension
also calling regularise_html(). I don't get what's happening here. Why
do the strings need to be regularised twice?
My guess is that CkanInternationalizationExtension is used when the
strings are extracted from the templates at runtime, and they need to be
regularised at this time in order to match them against the regularised
strings in the mo files to find the translations to output?
Maybe CkanInternationalizationExtension needs a better docstring saying
what it does?" 1347530507000000 1347530507000000
2923 defect seanh ckan 2.0 new Change regularise -> regularize The function is called regularise_html(), can't remember what file it's in. 1347530582000000 1347530582000000
2924 defect seanh ckan 2.0 new Better docs for trans js command, and add to release process "Add a better docstring to for trans js command explaining what it does and why, and how to use it.
Also add this command to the CKAN Release Process as it's needed there" 1347530671000000 1347530671000000
2925 defect seanh ckan 2.0 new Remove trans mangle paster command? "- Is `trans mangle` really necessary? If you upload a pot file to
Transifex, it can generate a po file for you with 100% strings
translated into a fictional pseudo language where everything is really
long strings of unicode characters. I found this worked well for
coverage testing, and also tests handling of unicode and long strings
all over the place." 1347530768000000 1347530768000000
2926 enhancement seanh ckan 2.0 new I*Form tests for per-type templates "Finish the tests for the per-package-type custom template support in IPackageForm, IGroupForm and IOrganizationForm (i.e. the new_template(), read_template() etc. methods).
The new way that template inheritance works in jinja looks like it should allow this to be tested nicely without duplicating entire templates in the example extensions." 1347552081000000 1347552081000000
2927 enhancement seanh ckan 2.0 new Test new package/group/organization type URLs support IPackageForm, IGroupForm and IOrganizationForm allow you to define e.g. new package types with new top-level URLs, but the example extensions don't have tests for this yet. 1347552153000000 1347552153000000
2928 enhancement seanh ckan 2.0 new Run CKAN tests with example_i*form extensions enabled Before releasing CKAN 2.0 we need to run all the CKAN tests with a modified test-core.ini with the example_idatasetform, example_igroupform and example_iorganizationform plugins enabled. If any tests fail, fix the bugs. This needs to be done for each release so add it to the release process. 1347552334000000 1347552334000000
2929 enhancement seanh ckan 2.0 new Remove is_fallback() from I*Form extension interfaces? "If a plugin wants to take over the default package type for example, it can return ['dataset'] from its package_types() method. I don't see how there can be a package type that is not the default 'package' and is not returned by the package_types() method of a plugin, because the URLs for such a package type would not be mapped. Am I missing something or is is_fallback() not needed?
Same for IGroupForm and IOrganizationForm" 1347552597000000 1347552597000000
2931 enhancement seanh ckan 2.0 new Better docstring for app_globals.py "''' The application's Globals object ''' is not very informative.
" 1347891194000000 1347891378000000
2932 enhancement seanh ckan 2.0 new Add docstring to system_info.py What is the system_info table for? It looks like a way to override config file settings in the database, perhaps so that admins can use a web interface to change site title etc. Could do with a docstring explaining the intended purpose of the table. 1347893429000000 1347893429000000
2933 enhancement seanh ckan 2.0 new Update theming docs "There's quite a lot of custom stuff going on, with all the custom jinja extensions, and the use of Fanstatic plus custom fanstatic extensions. People who are new to CKAN wanting to write CKAN themes are likely to be confused. I think the whole lot could do with documentation, maybe in the Theming and Customizing chapter in the sphinx docs.
I suggest completely replacing the contents of doc/theming.rst, and putting a 'legacy theming' link at the bottom linking to a doc/legacy_theming.rst file with the old contents." 1347895272000000 1347895272000000
2934 enhancement Dominik new Webtests for CKAN "The tests in CKAN only test single pages (functional tests) or single methods (unit tests). In order to make sure that the whole system still works as expected, we need tests that cover whole processes.
An example user story to be tested would be:
""As a user with create permissions I want to be able to log in, create a new resource, save it and then make sure that it is in the system.""
or:
""As a user I want to use the search functionality and open a resource from the search page.""
These user stories require a different testing system with interaction and processes. " 1347966647000000 1348060201000000
2936 defect seanh ckan-v1.8 new Updating a group via the API clears its packages "If the group dict that you post to the API does not have any 'packages' key then all the group's packages get removed. I think it would be better if you could just update e.g. the group's description without having to also post the list of packages, and apparently this is how other update API actions work.
Might be worth checking all the update API actions for this behaviour, and making sure they're all consistent" 1348048066000000 1348048066000000
2938 enhancement Toby dominik new Explain bundling in doc/resources.rst "Since it makes sense to bundle resources into one js file that are commonly used together, there should be some explanations on how separate js scripts are merged into one file for production.
Explanations could include how groups are handled or different resource files." 1348488835000000 1348488835000000
2939 enhancement toby toby new Orgs are groups holder for ticket number 1348564034000000 1348564034000000
2943 enhancement dominik new Chrome does not resize preview Chrome does not resize iframe after a full refresh/ on first load 1349089686000000 1349090759000000
2944 enhancement dominik new Recline preview does not work in Opera Opera cannot show a recline preview. I shows 0 records. 1349090802000000 1349090802000000
2945 enhancement dominik new Pdf preview does not load in IE The pdf preview does not load in IE 9. 1349090869000000 1349090869000000
2947 enhancement dominik new Redirect to the resource page from /resource/{res-id} "Use story:
As a user of the datasore I want to be able to put the resource id without the dataset name in a url and see the resource page.
Proposed redirect:
/resource/{res-id} -> dataset/foo-bar/resource/{res-id}" 1349097325000000 1349097325000000
2948 enhancement dominik new Negative range breaks datastore backend Recline allows negative ranges. The backend returns an error 500 for that which breaks the recline preview. 1349101854000000 1349101865000000
2949 defect amercader amercader new Reenable Data API button on the new theme The checks to show or not the button need to be updated for the latest datastore version 1349107464000000 1349107464000000
2951 enhancement icmurray ckan 2.0 new Paster command for building css from less "With the latest template, css and js changes in 2.0, there are a number of things that need preparation prior to a production deployment. One of these is:
- the less files should be compiled to css (main.css, not just main.debug.css)
This ticket is to provide a paster command which will compile the .less into the main.css file. The idea of the paster command is that it will be run by developers, and they will check in the resulting .css files. It will also provide a convenient mechanism for production scenarios with node.js installed on the production server, as they will be able to compile the .less there as well.
## Background
The .css files that need to be served are built using `less`. The css files that are generated *are* checked-in to the repo; but they are a build artifact. So the general workflow is:
- commit changes to the less files
- build the main.css file and commit
This paster command slots into the above workflow for convenience.
## Notes
- doc/frontend-development.rst
- bin/less
to run this paster command, `Node` will be required. So that dependency should be checked.
## The paster command
This is the proposed behaviour of the paster command:
- ensure `custom.less` does not exist
- for each colour in{fuchsia,green,maroon,red}:
- generate a `custom.less` file for $colour. There's a paster command that does this already: `paster color .
- generate the `css` from the `less` files. ie - the equivelant of running `bin/less --production`.
- this will generate a `main.css` file, which should be renamed to `$color.css`.
- ensure `custom.less` does not exist
- generate the `css` from the `less` files. This will create a `main.css` with the default colour scheme.
- call the paster command that minifies css and js files. (This command does not exist yet, see #2950)" 1349118740000000 1349175030000000
2952 enhancement seanh icmurray ckan 2.0 new incorporate javascriopt translations into translations workflow There are now javascript translations. There's a paster command, `trans js`. This is a ticket to ensure that using `trans js` is documented in the normal ckan translation workflow (which is followed after a feature freeze, prior to release). And to see if there's any need for further incorporation into the current translation process. 1349118945000000 1349118945000000
2957 task dominik new New datastore on postgres prior to 9.0 The ``GRANT SELECT ON ALL`` and ``ALTER DEFAULT PRIVILEGES`` are not available in Postgres prior to version 9.0. 1349292214000000 1349292214000000
2958 defect seanh ckan-v1.8 new Uploading files with unicode characters in filename fails in CKAN 1.7 and 1.8 "e.g. 2012_08(주요국가).xls
I tested in CKAN 2.0 and it seemed to work fine there." 1349342976000000 1349342976000000
2959 defect icmurray icmurray ckan 2.0 new Changing a Group's name through the action api disassociates it from its datasets in the index "Repro:
- Create a new Group, named ""test-group"".
- Add a dataset to it.
- Verify the dataset belongs to the group by visiting the Group's read-page
- Update the Group through the action api (group_update), using the uid in the ""id"" field, and a new name in the ""name"" field.
- Visit the group's read-page. The list of datasets will be empty.
This was an issue when editing a Group through the web interface, which was fixed in [1]. However it only fixes the issue in the group controller.
[1] https://github.com/okfn/ckan/commit/dbe25d8b8d7fabfc40c5d794a920b91cec349335" 1349363935000000 1349363935000000
2960 task dominik ckan-backlog new Private datasets are not private At the moment everyone can read all resources. There is no support for hiding resources that are in private datasets. 1349694255000000 1349694255000000
2962 enhancement dominik ckan-backlog new Search across multiple ckan instances "Could be done by:
- using the solr distributed search
- difficult set up
- merging result sets from apis
- make sure that results can be merged properly (score, facets, ...)" 1349736622000000 1349736622000000
2963 defect amercader ckan-v1.8 new Timeout on tag pages with lots of datasets "e.g. http://thedatahub.org/tag/lod
Tags with less datasets work fine (e.g. http://thedatahub.org/tag/railways)" 1350295999000000 1350295999000000
2964 defect seanh ckan 2.0 new Last organization admin can remove herself "If you are the only admin of an organization you can edit the organization's members and demote yourself, then the org has no admins and no one (except sysadmins maybe) can edit it.
Last admin should not be able to remove or demote herself.
Also applies to groups." 1350296058000000 1350296058000000
2965 defect amercader ckan-v1.8 new Stats extension broken on 1.8 " * Graphs not showing (looks like a flot related file is missing)
* Wrong groups counts (e.g. Data Explorer Examples show 1800 datasets when it onlu has 8)" 1350296141000000 1350296157000000
2966 enhancement seanh ckan 2.0 new 'Add' button text is wrong when editing organization members e.g. if I just changed a member's capacity I am not adding anything 'save' is better maybe 1350296149000000 1350296149000000
2967 defect seanh ckan 2.0 new Organization members edit page reloads after demoting self Edit an organizations members page and demote yourself from organization admin, after saving the members edit page reloads even though you are no longer an admin and should no longer be able to access this page. 1350296227000000 1350296227000000
2968 defect seanh ckan 2.0 new Anyone can access organization members page The button will not show if you are not authorized but browse to /organization/members/foo and you can edit the members, it does stop you when you try to save your changes, but you shouldn't be able to get to the page at all 1350296355000000 1350297070000000
2969 defect seanh ckan 2.0 new Group members page 500s The group members page (e.g. /group/members/roger) seems broken, IDs instead of user names are shown for the members, and clicking on a member 500s 1350296454000000 1350296454000000
2970 defect seanh ckan 2.0 new Organization and group member links use id not name e.g. it the 'Members' button links to /organization/members/0a44... instead of /organization/members/foobar 1350296531000000 1350296531000000
2971 enhancement seanh ckan 2.0 new """Are ytou sure you want to delete this member?"" should say which member" when deleting members from groups and orgs 1350296840000000 1350296840000000
2972 enhancement seanh ckan 2.0 new Remove any imports of authz.py and delete file it is no longer used 1350297442000000 1350297442000000
2973 enhancement seanh ckan 2.0 new Move new_authz.py into logic/auth/__init__.py "Makes sense to keep all auth stuff under logic/auth.
Also decide which functions from new_authz.pu should be marked private with _.
Also decide which functions from new_authz.py are really helpers for the auth functions in get.py, update.py and delete.py, and should maybe moved to another file, e.g. logic/auth/helpers.py" 1350297545000000 1350297545000000
2974 enhancement seanh ckan 2.0 new General of all auth functions "From Toby:
A general cleanup of all auth functions (in logic/auth) to check their fitness, error messages, etc." 1350297612000000 1350297612000000
2975 enhancement seanh ckan 2.0 new Tests for auth functions and new actions "From Toby: tests around the auth functions and new actions - probably needs an improved testing mechanism and things like test data creation on via actions
" 1350297660000000 1350297660000000
2976 enhancement seanh ckan 2.0 new Polish group and organization member pages "From Toby: A little polish to the member pages and probably a little hardening of
the controller and logic actions" 1350297709000000 1350297709000000
2977 enhancement seanh ckan 2.0 new Fix user autocomplete on group and organization member pages "From Toby: @johnmartin the user autocomplete on
member add needs fixing it gets data but does not understand what to do
with it
" 1350297749000000 1350297749000000
2978 enhancement seanh ckan 2.0 new Tests for permissions for organizations and groups "From Toby: checking the logic around the who can do what from the user stories -
via some tests may be a good approach
" 1350297806000000 1350297806000000
2979 enhancement seanh ckan 2.0 new Requesting membership to groups and organizations "From Toby: The requesting membership user stories and implementation remain needed
" 1350297844000000 1350297844000000
2981 refactor seanh ckan 2.0 new Remove config from jinja templates "From Toby: remove the config from jinja2 templates get all such stuff via g. (before 2.0)
" 1350297960000000 1350297960000000
2982 refactor seanh ckan 2.0 new Move functionality from controllers into template helpers "From Toby: strip more functionality from controllers and add via template helpers
eg activity streams
" 1350298002000000 1350298002000000
2983 refactor seanh ckan 2.0 new Refactor lib/base.py to remove circular import issues "From Toby: c) refactor lib/base.py to remove the circular import issues (render
functions to lib/render.py - would fix much of that)
" 1350298044000000 1350298044000000
2984 refactor seanh ckan 2.0 new Auto-populate context with user, model, session, etc. "From Toby: make the context auto-populated with things like user/model/session if
not user supplied - especially for extensions (pre 2.0) maybe via a helper
or else a decorator on the action - helper seems nicer maybe done via
get_action()" 1350298101000000 1350298101000000
2985 refactor seanh ckan 2.0 new Make lib/helpers.py more template-specific "From Toby: make lib/helpers more template specific and less generally shared with
none template uses (some functions are shared but many shouldn't be) -
again help avoid circular imports
" 1350298188000000 1350298188000000
2986 refactor seanh ckan 2.0 new Make lib/base.py more like it was originally intended "From Toby: make lib/base.py be more like it was originally intended eg defines
stuff that is shared eg _ but maybe try to include stuff like
render/get_action if the circular import stuff can be solved - which should
be doable if hard and likely needs an extra file or two for the externals _
etc
" 1350298245000000 1350298245000000
2987 refactor seanh ckan 2.0 new Remove all direct calls to logic action and auth functions From Toby 1350298287000000 1350298287000000
2988 enhancement seanh ckan 2.0 new UI functional tests for CKAN 2.0 Afaik the new CKAN 2.0 frontend has no functional tests 1350298935000000 1350298935000000
2989 enhancement seanh ckan 2.0 new """Add dataset to organization"" should auto-select the organization" "'Add dataset to organization' button, when you get through to the third stage of the new dataset form the organization you came from is not selected.
I don't see any option to choose the group when adding or updating a dataset, but if I add a dataset via the ""Add dataset to group"" button on a group's page, then the dataset seems to get added to that group. (And I can also add/remove existing datasets by editing the group.) I wonder if organizations should work the same way, instead of having an Organization drop-down when creating or updating a dataset. The add dataset page needs to somehow indicate that you're adding a dataset to a certain group or organization though, doesn't currently." 1350299052000000 1350299052000000
2990 enhancement seanh ckan 2.0 new Fix descriptions of groups and organizations on /groups and /organizations pages 1350299091000000 1350299091000000
2991 enhancement seanh ckan 2.0 new Add blocks to header.html template Currently if you want to customise the header.html template you have to copy the whole thing, it isn't broken up into blocks like other templates are. 1350385820000000 1350385820000000
2992 enhancement johnmartin new Delete resource should send me back to edit dataset page 1350407956000000 1350407956000000
2993 defect seanh ckan 2.0 new """logged_in"" and ""visitor"" show in user list at /users" 1350466922000000 1350484826000000
2994 enhancement seanh ckan 2.0 new Add dataset and user popovers throughout the site "If you hover the mouse over a dataset or user in an activity stream, you now get a popover with some info about that dataset or user and a follow/unfollow button.
These popovers should appear wherever datasets or users are listed, throughout the site." 1350483964000000 1350483964000000
2995 enhancement seanh ckan 2.0 new Popovers for resources, groups, organizations, tags... "If you hover the mouse over a dataset or user in an activity stream, you now get a popover with some info about that dataset or user and a follow/unfollow button.
It would be nice to add similar popovers for other types of object in CKAN, e.g. resources, groups, organizations, tags..." 1350484041000000 1350484041000000
2996 enhancement seanh ckan 2.0 new Fix behaviour of View Profile button in user popover when already on user's profile "The user popovers that appear when you hover the mouse over a user in an activity stream include a View Profile button that takes you to that user's profile page. If you are already on that user's profile page then it simply reloads the same page, but it sends you to the Datasets tab if you were on another tab.
Possible solutions:
Don't show button if already on user's page?
Button just makes popover disappear, if already on user's page?
Button does reload the page, but reloads the same tab (datasets, followers, activity stream) that you were on.
Marking this low priority" 1350484238000000 1350484238000000
2997 enhancement seanh ckan 2.0 new Add activity streams to dataset pages Add an activity stream tab to dataset pages, like we have on user profile pages. Dataset activity streams are already implemented in the backend. 1350484306000000 1350484306000000
2998 enhancement seanh ckan 2.0 new Add activity streams to group pages Before we do this we should implement smarter group activity streams (i.e. what activities should appear in a group's activity stream?) and following of groups. 1350484394000000 1350484394000000
2999 enhancement seanh ckan 2.0 new Add activity streams to organization pages Before we do this we need to merge the organizations branch into master, we need to implement smarter activity streams for organizations, and we need to implement following of organizations. 1350484457000000 1350484457000000
3000 enhancement seanh ckan 2.0 new Add number of followers to dataset pages User profile pages now show some interesting stats about the user: num. datasets, num. edits and num. followers. Would be nice to add some similar info to dataset pages. At least num. followers can be done (already implemented in backend). 1350484560000000 1350484560000000
3001 defect seanh ckan 2.0 new Multilingual plugin crashes CKAN on add dataset when some languages are default "Enable the multilingual plugins:
ckan.plugins = stats synchronous_search multilingual_dataset multilingual_group multilingual_tag
and set your default language to one not supported by the multilingual plugin, e.g.
ckan.locale_default = cs_CZ
now run CKAN and try to add a dataset:
File '/home/seanh/Projects/ckan171/ckan/ckanext/multilingual/plugin.py', line 141 in before_index
text_field_items['text_' + default_lang].extend(all_terms)
KeyError: 'text_cs_CZ'
It doesn't matter what language you are viewing the site in in your browser, the default language setting in the ini file determines whether it crashed or not.
A number of supported languages are defined at the top of ckanext/multilingual/plugin.py. I think if the default language is not one of these it crashes.
I think this affects all versions of CKAN since the multilingual plugin was added so at least 1.7, 1.8 and 2.0" 1350579048000000 1350579048000000
3002 defect amercader ckan-v1.8 new API v1/2 'legacy' search parameters must be escaped before they are put into a Solr query string "Just to track @tauberer patch on Github.
Would be nice to write a test for it. Probably going to 1.8.1" 1350639142000000 1350639142000000
3003 enhancement amercader ckan-v1.8 new New pg databases should be created with UTF8 encoding rather than system default "Just to track @tauberer patch on Github.
Probably going to 1.8.1" 1350639199000000 1350639199000000
3005 enhancement seanh seanh ckan 2.0 new Add following/unfollowing of groups: model, API, frontend, tests 1351013744000000 1351013744000000
3006 enhancement seanh ckan-v1.8.1 new Update translations from Transifex There are some new translations already on the CKAN 1.8 resource on Transifex already (e.g. 100% Norwegian translation I think). We should at least pull these from Transifex for 1.8.1, and may as well do a call for translations too. 1351506144000000 1351506144000000
3007 enhancement seanh ckan 2.0 new Adding a dataset creates multiple activities Adding a new dataset creates multiple activity stream activities, e.g. seanh created the dataset foo, seanh add the resource bar to the dataset foo, and seanh updated the dataset foo are all created when I add a new dataset. I wonder if these can be collapsed into a single activity. 1351516507000000 1351517019000000
3010 enhancement seanh ckan 2.0 new Pin images don't appear in data explorer "When minified files are in use (set debug = false in ini file) the pin images for points don't show up in the data explorer map view when viewing a resource with geolocation data.
set debug = true will fix the problem by using unminified files instead but also turns on a bunch of debug stuff you wouldn't want on a production site.
middleware.py, around line 73, is where the decision to turn minified files on or off based on the debug setting is made. This can be hacked on a production site to run with unminified files but otherwise run on production mode thereby getting around the issue with the pins. But that doesn't fix the underlying bug." 1351705506000000 1351705506000000
3011 enhancement johnglover johnglover ckan 2.0 new Recline fixes and updates for CKAN 2.0 Fixes and updates to Recline / Datapreview for CKAN 2.0. 1352288455000000 1352288455000000
3012 enhancement toby toby new data.gov auth 1352393409000000 1352393409000000
3013 defect dominik dominik new common-error-messages is unreadable "Since the update of the doc theme, the page became unreadable.
http://docs.ckan.org/en/latest/common-error-messages.html" 1352553505000000 1352553505000000
3014 defect seanh ckan 2.0 new Crash when deleting a non-empty vocabulary "From Knud Möller:
when I try to delete a non-empty tag vocabulary via the API (through HTTP), I get an internal server error. Checking
the logs, this turns out to be a consistency error raised by sqlalchemy:
Error - : (IntegrityError) update or delete on table ""vocabulary"" violates
foreign key constraint ""tag_vocabulary_id_fkey"" on table ""tag""
DETAIL: Key (id)=(21421955-7560-467c-af30-9f790b73e6ae) is still referenced from table ""tag"".
'DELETE FROM vocabulary WHERE vocabulary.id = %(id)s' {'id': u'21421955-7560-467c-af30-9f790b73e6ae'}
URL: http://33.33.33.10:5000/api/action/vocabulary_delete
The error makes sense, but I'm wondering if it would be useful to extend the API to also allow the deletion of
non-empty vocabularies, possibly via a parameter (not sure what best practice in API design is). At the very least, it
would be cool if the error message coming back in the response had more information in it." 1352803808000000 1352803808000000
3016 enhancement johnmartin johnmartin ckan 2.0 new CKAN 2.0 template tweaks Just a ticket to keep track of a few suggested template changes. 1352813417000000 1352813417000000
3017 enhancement johnglover johnglover ckan 2.0 new New stable branch for the demo server 1352819088000000 1352819088000000
3018 enhancement johnmartin johnmartin ckan 2.0 new Load more in activity streams Activity streams should be able to load more than 15 items within them. Suggest the default amount of loading is around 30 and then click to load more. 1352900051000000 1355140950000000
3019 defect seanh ckan 2.0 new Cannot delete dataset extras Deleting extras in the web interface is broken 1352918678000000 1352918678000000
3020 enhancement seanh new Update CKAN coding standards 1353096711000000 1353096711000000
3023 enhancement amercader amercader new New methods on IPackageController to provide access to the data_dict Extension hooking into the edit and create methods of the IPackageController interface receive the package object. This may not include all the fields that came from the form. The new extension points will pass the validated data_dict so extensions can have access to it 1353603657000000 1353603657000000
3025 enhancement amercader amercader new Add requests to core requirements Because yes please 1354052124000000 1354052124000000
3026 enhancement amercader amercader new Support icons on nav_named_link nav_named_link won't include the nice icon even when passing the icon keyword param. 1354301516000000 1354301516000000
3027 enhancement kindly kindly new solr for 2.0 "change mm
support solr 3 and 4
add *_date field" 1354587729000000 1354587729000000
3028 enhancement seanh seanh ckan 2.0 new Feature: dashboard activity stream filtering https://github.com/okfn/ckanext-pdeu/issues/13 1355141244000000 1355141244000000
3030 enhancement toby new clean up helper functions 1355834038000000 1355834038000000
84 enhancement kindly rgrp ckan-future assigned Revert support on versioned objects "Basic revert in the classic wiki form is already support by purging a Revision. However may wish to support:
1. Cases where multiple objects changed in a revision but only want to revert 1 (low priority)
2. Want to revert but have reversion as a new revision of that object.
Seems low priority at present.
Cost: ?" 1248339543000000 1340626385000000
143 enhancement thejimmyg dread ckan-backlog assigned Most active users listed on homepage Display league of users' recent activity on homepage. 1255010373000000 1312372381000000
235 enhancement tobes dread ckan-v1.9 assigned Resource format normalization and detection "Try to gather proper MIME information for all package resources in CKAN. This is a shared ticket with dcat-tools (https://bitbucket.org/pudo/dcat-tools), i.e. opendatasearch.org. This can then also be used by ckanrdf, the CKAN RDF conversion service.
Sub-tasks:
* Create a Google Spreadsheet with two Worksheets: ""MIME-Mappings"", i.e. ""CSV"" -> ""text/csv"" and ""Name mappings"", i.e. ""text/csv"" -> ""Comma-Separated Spreadsheet"".
* Collect and map surface forms from all CKANs
* Access this via Swiss and apply, store as a PackageResource extra field pending #826 (Resource extras).
* Add heuristics for format auto-detections:
* Map well-known file extensions
* Recognize obvious magic (Zip, Tar)
* Peek into Zipfile/Tarfiles
* Define a convention for generic data types (many CKAN packages have only ""Spreadsheet"" defined, either detect specific type or set MIME to */tabular-data or similar)
* See also: #816 (Autocomplete for the resource format field)" 1263827604000000 1340627624000000
250 enhancement icmurray dread ckan-v1.9 assigned RDF link in Atom feed Add link to RDF representation of a package in our Atom feed. 1266507695000000 1340631430000000
253 enhancement dread ckan-backlog assigned Package relationships "= Overview =
Functionality to formally associate packages. We see a need for specific parent-child, inheriting or dependency relations. Not only should this help navigation between packages in the web interface, but it also provides a mechanism to automatically pull dependencies when downloading a data package, in a similar manner as we see in software package management.
= Examples =
1. There are 27 packages in data.gov.uk to do with the Data4NR's Health Poverty Index. There is currently no common link between these, unless you search for 'HPI' (which also brings up House Price Index), or look under tag 'health' (which also has 600 other results). There should be a link on each HPI package page to navigate to the other 'sibling' HPI packages, and to a 'root' package that has info about the set. This could be partially achieved using the existing tag or group concepts, but a more explicit/official/obvious marking of their relationship could be beneficial.
2. In ckan.net is freedict, a collection of translation dictionaries. You could make each dictionary a child package and use this system. But it would probably be better to make each dictionary a different resource in the same package. (There are other ideas to denote a resource as the data making up a 'portion' of package, or a 'whole' of the package, to help people downloading datasets in the software package style.)
3. OSM has had some Naptan data imported (bus stops), with special permission - i.e. a more liberal license. It would be useful to show this link on both OSM and Naptan packages in CKAN: OSM 'derives from' Naptan with a comment about the license change. I'm not sure this is useful to an automatic download or use of these datasets, but may aid exploration on the CKAN website and understanding the provenance of the bus stop data on it.
4. IPCC collection of data linked / mirrored. Not sure if there are useful relationships here?
5. Dracos gets postbox locations from crowd sourcing and OSM. We could say Dracos 'derives from' OSM.
See more examples discussed here: http://trac.ckan.org/ticket/253
= Implementation =
This is split into four tickets:
* Model: ticket:254
* Read in WUI: ticket:255
* Edit in WUI: ticket:256
* API: ticket:257
No need for write access to be provided API for the moment.
This ticket also encompasses ticket:169 (Package derivations) and ticket:176 (Package dependencies)." 1266854721000000 1339774726000000
256 requirement dread ckan-backlog assigned Package relationships - 3. Edit in WUI "WUI:
* Editable as part of package or separately? (e.g. like authz)
* Do we normalize to only one type name of the pair?
* Do we allow create of relationship from both ends (e.g. only from dependency to dependent or either way?)
" 1266928561000000 1339774714000000
277 enhancement zephod dread ckan-backlog assigned Set some config options / settings in WUI (extension) "== Use case ==
As a ckan administrator I want to easily change options about the CKAN install.
== Implementation ==
=== Settings to be in DB ===
Suggested:
{{{
## Title of site (using in several places including templates and tag
ckan.site_title = CKAN
## Logo image to use (replaces site_title string on front page if defined)
ckan.site_logo = http://assets.okfn.org/p/ckan/img/ckan_logo_box.png
## Site tagline / description (used on front page)
ckan.site_description =
## Used in creating some absolute urls (such as rss feeds, css files) and
## dump filenames
ckan.site_url =
## Favicon (default is the CKAN software favicon)
ckan.favicon = http://assets.okfn.org/p/ckan/img/ckan.ico
## An 'id' for the site (using, for example, when creating entries in a common search index)
## If not specified derived from the site_url
# ckan.site_id = ckan.net
## API url to use (e.g. in AJAX callbacks)
## Enable if the API is at a different domain
# ckan.api_url = http://www.ckan.net
## html content to be inserted just before