{22} Trac tickets (2647 matches)

Results (1 - 100 of 2647)

1 2 3 4 5 6 7 8 9 10 11
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1785 enhancement seanh seanh ckan-v1.9 accepted Replace 'Revisions' page with site-wide activity stream

Replace the /revision page with a /activity page showing a site-wide activity stream of all activities in the site. Or perhaps the site-wide activity stream can go somewhere on the front page of the site instead?

1328632458000000 1340724312000000
#2336 enhancement seanh seanh ckanbuild accepted Move Jenkins' install script into ckan core so it can be versioned 1335792914000000 1340639637000000
#2341 enhancement seanh seanh ckan-v1.9 accepted Fix Jenkins issue when testing branches that contain model changes 1335876396000000 1340639472000000
#2342 enhancement seanh seanh ckan-v1.9 accepted Fix Jenkins issue when testing branches with different solr schema versions 1335876445000000 1340639494000000
#2357 enhancement aron.carroll aron.carroll demo phase 5 accepted Create build script for front-end resources

Should minify and concatenate JavaScript? and CSS as part of #2354

1335886181000000 1343220524000000
#2362 enhancement toby kindly ckan-v1.9 accepted Improve plugin documentaion, including examples.

Improve documentation for plugins and add examples. Could use ckanext-example as a base.

1335888791000000 1342085420000000
#2416 enhancement toby aron.carroll demo phase 5 accepted Normalise resource/data types

Currently we have far too many types that are essentially the same format. The new demo theme is using icons for common types. So we need to normalise the common formats into pretty strings. e.g. application/json, JSON, .json and json all should be output as “json”

See the following Basecamp thread for UI examples and discussion.

https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61906351/comments#comment_179681056

1337792946000000 1342617802000000
#2430 enhancement seanh seanh ckan-v1.9 accepted Look into marking tickets as dependent on other tickets in trac 1337947086000000 1341234346000000
#2431 enhancement seanh seanh ckan-v1.9 accepted Trac git integration 1337947128000000 1341234362000000
#2466 task seanh seanh ckan-v1.9 accepted Fix spam handling on trac 1338213103000000 1341234539000000
#2475 enhancement seanh seanh ckan-v1.9 accepted Upgrade trac.ckan.org to trac 0.12.3

This is not trivial, requires a database upgrade. But 0.12.3 has some nice new features (multiple vcs repository support, ticket comment editing and nicer preview) and the latest spam filter plugin requires 0.12.

http://trac.edgewall.org/wiki/ChangeLog

1338317176000000 1341234564000000
#2477 enhancement seanh shevski ckan-v1.9 accepted Add extensions section to readthedocs

Need a central up to date place for currently supported extensions (closest we have is http://wiki.ckan.org/List_of_Extensions) & their documentation. I suggest this gets added to a dedicated section of docs.ckan.org

1338393244000000 1343124123000000
#2478 enhancement seanh seanh ckan-v1.9 accepted Get NeverNotifyUpdaterPlugin installed on trac.ckan.org 1338396734000000 1341234591000000
#2493 enhancement aron.carroll toby demo phase 5 accepted Demo site 404 needs some love

templates/error_document_template.html

I've converted the 404 page to the new style but we probably want to serve this as a 'full page' ie content is 100% page width not sidebar and primary content

just go to a non-existant url to see

Discusion:

https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/63374042/

Implementation:

http://s031.okserver.org:2375/404

1338986081000000 1343135815000000
#2516 enhancement seanh seanh ckan-v1.9 accepted Make 'Assign to:' field on trac.ckan.org into a dropdown list

there's a setting for this

1339578442000000 1341234822000000
#2537 enhancement seanh seanh ckanbuild accepted Test and document ckanbuild

https://github.com/okfn/ckanbuild

Verify that what's there so far still works, write a README explaining how it works

1339775328000000 1340639830000000
#2538 enhancement seanh seanh ckanbuild accepted Add multiple-instance support to ckanbuild

Probably use ansible to do this. To create an instance, create a dir at /etc/ckan/MYSITE, and put MYSITE.wsgi, MYSITE.ini and who.ini files in it. Also put a MYSITE file in /etc/apache2/sites-available. See the example files already present in ckanbuild. Booting a new site should be a single command.

May not handle the postgres/solr/elastic-search side of things yet, could just require the user to set these up herself first and then pass them as args to the create-instance command.

1339775499000000 1340639836000000
#2539 enhancement seanh seanh ckanbuild accepted Investigate the existing ckan debian package for ckanbuild

Do we want to build on top of the existing debian packaging code? Or throw it away and start fresh?

1339775661000000 1340639845000000
#2540 enhancement seanh seanh ckanbuild accepted Implement a way of upgrading ckan sites using ckanbuild

When there are multiple ckan sites installed on a single server via ckanbuild, there needs to be some way of upgrading them all to a new ckan version at once.

1339775740000000 1340639850000000
#2541 enhancement seanh seanh ckanbuild accepted Add non-core extensions to ckanbuild

We want some extensions from outside of CKAN core to be included in ckanbuild. These would be pip installed into the virtualenv before packaging the debian package. Decide which extensions to include.

1339775826000000 1340639856000000
#2542 enhancement seanh seanh ckanbuild accepted Create jenkins job to run ckanbuild, and run tests

It should run the script to create the debian package, boot a VM, install the debian package on the VM, boot a CKAN instance, then run the tests.

1339775888000000 1340639863000000
#3021 enhancement johnmartin johnmartin accepted Logout doesn't work without JS

Essentially, the functionality should be as follows:

  • Add logout link that has .js-hide attached to it within the header that isn't hidden within a dropdown

See http://plus.google.com/ (when logged in) with and without JS to see an example of the actual sign-out working without JS

1353410350000000 1353411968000000
#84 enhancement kindly rgrp ckan-future assigned Revert support on versioned objects

Basic revert in the classic wiki form is already support by purging a Revision. However may wish to support:

  1. Cases where multiple objects changed in a revision but only want to revert 1 (low priority)
  2. Want to revert but have reversion as a new revision of that object.

Seems low priority at present.

Cost: ?

1248339543000000 1340626385000000
#143 enhancement thejimmyg dread ckan-backlog assigned Most active users listed on homepage

Display league of users' recent activity on homepage.

1255010373000000 1312372381000000
#235 enhancement tobes dread ckan-v1.9 assigned Resource format normalization and detection

Try to gather proper MIME information for all package resources in CKAN. This is a shared ticket with dcat-tools (https://bitbucket.org/pudo/dcat-tools), i.e. opendatasearch.org. This can then also be used by ckanrdf, the CKAN RDF conversion service.

Sub-tasks:

  • Create a Google Spreadsheet with two Worksheets: "MIME-Mappings", i.e. "CSV" -> "text/csv" and "Name mappings", i.e. "text/csv" -> "Comma-Separated Spreadsheet".
  • Collect and map surface forms from all CKANs
  • Access this via Swiss and apply, store as a PackageResource? extra field pending #826 (Resource extras).
  • Add heuristics for format auto-detections:
    • Map well-known file extensions
    • Recognize obvious magic (Zip, Tar)
    • Peek into Zipfile/Tarfiles?
  • Define a convention for generic data types (many CKAN packages have only "Spreadsheet" defined, either detect specific type or set MIME to */tabular-data or similar)
  • See also: #816 (Autocomplete for the resource format field)
1263827604000000 1340627624000000
#250 enhancement icmurray dread ckan-v1.9 assigned RDF link in Atom feed

Add link to RDF representation of a package in our Atom feed.

1266507695000000 1340631430000000
#253 enhancement dread ckan-backlog assigned Package relationships

Overview

Functionality to formally associate packages. We see a need for specific parent-child, inheriting or dependency relations. Not only should this help navigation between packages in the web interface, but it also provides a mechanism to automatically pull dependencies when downloading a data package, in a similar manner as we see in software package management.

Examples

  1. There are 27 packages in data.gov.uk to do with the Data4NR's Health Poverty Index. There is currently no common link between these, unless you search for 'HPI' (which also brings up House Price Index), or look under tag 'health' (which also has 600 other results). There should be a link on each HPI package page to navigate to the other 'sibling' HPI packages, and to a 'root' package that has info about the set. This could be partially achieved using the existing tag or group concepts, but a more explicit/official/obvious marking of their relationship could be beneficial.
  1. In ckan.net is freedict, a collection of translation dictionaries. You could make each dictionary a child package and use this system. But it would probably be better to make each dictionary a different resource in the same package. (There are other ideas to denote a resource as the data making up a 'portion' of package, or a 'whole' of the package, to help people downloading datasets in the software package style.)
  1. OSM has had some Naptan data imported (bus stops), with special permission - i.e. a more liberal license. It would be useful to show this link on both OSM and Naptan packages in CKAN: OSM 'derives from' Naptan with a comment about the license change. I'm not sure this is useful to an automatic download or use of these datasets, but may aid exploration on the CKAN website and understanding the provenance of the bus stop data on it.
  1. IPCC collection of data linked / mirrored. Not sure if there are useful relationships here?
  1. Dracos gets postbox locations from crowd sourcing and OSM. We could say Dracos 'derives from' OSM.

See more examples discussed here: http://trac.ckan.org/ticket/253

Implementation

This is split into four tickets:

No need for write access to be provided API for the moment.

This ticket also encompasses ticket:169 (Package derivations) and ticket:176 (Package dependencies).

1266854721000000 1339774726000000
#256 requirement dread ckan-backlog assigned Package relationships - 3. Edit in WUI

WUI:

  • Editable as part of package or separately? (e.g. like authz)
  • Do we normalize to only one type name of the pair?
  • Do we allow create of relationship from both ends (e.g. only from dependency to dependent or either way?)
1266928561000000 1339774714000000
#277 enhancement zephod dread ckan-backlog assigned Set some config options / settings in WUI (extension)

Use case

As a ckan administrator I want to easily change options about the CKAN install.

Implementation

Settings to be in DB

Suggested:

## Title of site (using in several places including templates and <title> tag
ckan.site_title = CKAN

## Logo image to use (replaces site_title string on front page if defined)
ckan.site_logo = http://assets.okfn.org/p/ckan/img/ckan_logo_box.png

## Site tagline / description (used on front page)
ckan.site_description = 

## Used in creating some absolute urls (such as rss feeds, css files) and 
## dump filenames
ckan.site_url =

## Favicon (default is the CKAN software favicon)
ckan.favicon = http://assets.okfn.org/p/ckan/img/ckan.ico


## An 'id' for the site (using, for example, when creating entries in a common search index) 
## If not specified derived from the site_url
# ckan.site_id = ckan.net

## API url to use (e.g. in AJAX callbacks)
## Enable if the API is at a different domain
# ckan.api_url = http://www.ckan.net

## html content to be inserted just before </body> tag (e.g. google analytics code)
## NB: can use html e.g. <strong>blah</strong>
## NB: can have multiline strings just indent following lines
# ckan.template_footer_end = 

NB: these will still need to be stored somewhere for loading on initialization. do this in db init function ...

Settings / Options / KeyValues? Table

Columns:

  • [namespace]: ? only if KeyValues? (for settings this would then always be settings)
  • key
  • label
  • value (json)
  • type (e.g. date and to specify in advance what type should be)
  • description
  • tags: ?? (for grouping ...)

Loading settings from DB

Do this in ckan/config/environment.py

WUI

  • /ckan-admin/settings
  • Show label, plus description plus text field

Depends

  • Would be part of ckan-admin section and hence build on ticket:833 (Administrative dashboard)
1269274861000000 1318247121000000
#285 enhancement rgrp assigned Paginate list of packages on tag read page

Is this worth doing? On hmg.ckan.net start to have a lot of packages with a given tag ...

1270664606000000 1340631923000000
#301 enhancement rgrp assigned Package discussion pages

A package discussion page is like a wikipedia discussion page: an editable free text page for people to have discussion/post comments about a given package.

It provides a way for people to make suggestions about a package without needing access to main package.

1272301033000000 1340632055000000
#728 requirement amercader johnbywater ckan-backlog assigned CSW Harvesting shall be optimised in respect of reharvesting only records that have changed

Hi Will, this is important again because some CSW servers we use have over 300 documents in. Could you take a look at modifying the filter please?

1287675340000000 1310124784000000
#763 enhancement dread ckan-future assigned Read-only mode - Setup

Admin configures entering read-only mode in one of two places:

  • CKAN config file (e.g. ckan.ini)
  • environment variable from Apache config

Once enabled, no writes can occur to the database (including user ratings and other usage stats).

1288091506000000 1338206204000000
#765 enhancement dread ckan-backlog assigned Read-only mode - API usage

All writes to the API are captured and you are returned an error explaining the reason.

Possible errors:

  • 503 temporary maintenance
  • 403 forbidden (if server if permanently read-only)
1288091897000000 1338206123000000
#794 requirement amercader johnbywater ckan-backlog assigned Investigate reconciling UKLP Publisher and Provider with DGU

This needs more analysis, but the GEMINI2 attribute "metadata point of contact" must be reconciled with the registered publisher (or agent).

This might also be used to filter records harvested from a CSW source, but filtering also needs more analysis, as does distinction between agent and provider.

1289227811000000 1311179581000000
#909 enhancement pudo ckan-backlog assigned DCat importer for CKAN

Write an importer that supports most well known variants of DCat in importing the data as CKAN packages. In particular, the following sources should be supported:

  • CKANrdf generated exports
  • opengov.se RDF (not really DCat)
  • Sunlight Nationdatacatalog as harvested by dcat-tools.
1295265958000000 1346669602000000
#948 enhancement dread ckan-future assigned Highlight (to a sysadmin) which packages are deleted

When a customer logs in as a sysadmin then he/she see all packages, including deleted and pending ones. These are hidden to the average user, but the sysadmin has no idea of this until he clicks on the package and sees at the bottom 'state: deleted'.

It should be more obvious than that on the search view - an icon, message or crossed-out name to packages are deleted.

1296646369000000 1338206280000000
#979 enhancement kindly dread ckan-backlog assigned Edit Resource extras in the API

Follows on from #826. We can now edit resource extras in the WUI (to some extent - see #978 for remaining issues) and we can view resource extras in the API, but we can't yet edit them in the API.

1297429777000000 1315222244000000
#1041 enhancement thejimmyg thejimmyg ckan-backlog assigned Start Using the CKAN Wiki for Tutorial-style documentation

For example, I will document the following:

I'd love if someone else would write:

  • An authorisation tutorial covering the core model, the command line tools and examples of every possible way of using the system
  • A HOWTO guide with screenshots for adding a package
1300284715000000 1312372367000000
#1062 defect johnglover sebbacon ckan-backlog assigned Data preview encoding error

The preview of "Species Misc Turtle Download" at http://ckan.net/package/taxonconcept results in the following error:

Unable to Preview - Had an error from dataproxy: Data Transformation Error (Data transformation failed. Reason: 'utf8' codec can't decode byte 0x8b in position 1: unexpected code byte

1301396143000000 1311773731000000
#1069 enhancement tobes rgrp assigned Stub datasets (request for datasets)

Idea is to have stubs for datasets that someone wants but don't yet exist (or haven't been discovered) in the way one has stub pages on a wiki.

We could do this within the existing model by a slight 'abuse' - create a dataset and mark it with a special tag e.g. todo.does-not-yet-exist or similar ...

(Just as we have datasets listed that exist but aren't available ...)

Alternative would be to have a request for datasets subsystem.

I prefer the stub dataset model because it's simpler, provides a simple workflow (as a dataset is found or comes into existence), and the package page provides a natural space in which to accumulate information about what is wanted and what exists.

Implementation

  • Agree a new dedicated tag. e.g. todo.does-not-exist
1301666919000000 1340632215000000
#1130 enhancement lucychambers assigned First time users

Send users to FAQ first time on CKAN

1304938761000000 1340633514000000
#1135 enhancement kindly rgrp assigned Changeset model for vdm

Move to Changeset model for vdm.

A changeset model is like an Audit-Log model in which we just record Changesets with Change-Objects rather than have Revision-Objects for each Object that is revisioned.

This change would also incorporate significant simplication of vdm.

1305209986000000 1340632267000000
#1136 enhancement kindly rgrp assigned Move to SessionExtension in vdm

When vdm was created there was no SessionExtension so we use MapperExtension for doing revisioning. Now that SessionExtension? exists we should use it. We can also follow the existing SQLAlchemy recipe: <http://www.sqlalchemy.org/docs/orm/examples.html?highlight=versioning#versioned-objects>

1305210855000000 1340632980000000
#1137 enhancement kindly rgrp assigned Remove need for statefulness in vdm

Statefulness, especially statefulness for relation (esp m2m) is cause of most of the complexity in vdm. It is required because, atm, revision objects have FKs to continuity objects.

This ticket proposes the following changes:

NB: this could be limited just to case of join tables (leaving state stuff on other tables)

  • Remove FKs from revision to continuity (or allow for them to be nullable).
    • We could just limit this to m2m stuff
  • Delete of an object leads to:
    • Deletion of continuity object
    • Adding an entry in revision table with state set to deleted (we retain state on revision table)

If this is done we will no longer need to worry about filtering on state on relationships as join table will only contain "active" relationships.

If we do this on all tables we remove need for any state awareness in client (e.g. no need to filter tables on active state).

The only disadvantage of this change is that undeletion becomes more problematic (we have to recreate some continuity objects).

1305211628000000 1340631974000000
#1168 enhancement thejimmyg dread ckan-backlog assigned Test system for deb packaging

Get buildbot to:

  • build the deb packages
  • install them into a fresh virtual machine
  • run smoke tests on the installed ckan
1306441994000000 1330990423000000
#1171 enhancement mark.wainwright dread ckan 2.0 assigned Citation instructions on dataset and resource view pages

Some sort of citation helper. Something small on the dataset and resource page that would show how to cite.

wwaites: Some related thoughts on this from opb: http://homepages.inf.ed.ac.uk/opb/papers/ssdbm2006.pdf

timclicks: I'm looking at Dataverse for the first time[0]. It seems very popular in the social sciences. I noticed that there is a recommended citation for each dataset. For example, [1] is has this one: "Targeted Input Programme (TIP) 2000-01", http://hdl.handle.net/1902.1/SSC-MWI-TIP2000-01-M1 V1 [Version]"

Implementation

Add a small box at bottom of dataset / resource page (or in sidebar on dataset page) with title "Cite this" with contents like:

%title. %author. Retrieved %date. %site_title.

For resource: %title = %dataset_title. %resource_name.

Could also add export to ref managers (e.g. to bibtex) but that is for later.

1306920799000000 1347358705000000
#1240 enhancement kindly rgrp ckan-backlog assigned [super] API v4

(Just creating this ticket as somewhere to keep notes)

  • Decide on REST api versus action API
    • Do we want to support both?
  • Tidying
    • Unify on /api/v{version num}/... structure (do we want a default option that points to current default? e.g. /api/default/ ...)
    • extras merged into normal field list in package
    • Get rid of /rest/ so just have api/v1/package
    • Get rid of separation of search api from 'rest' api
      • Propose that GET on REST index is search e.g. /package/?q=...
        • This is also resolves issue whereby GET at root returns whole package set (a *bad* idea) as this would now become the matchall search query (with a default limit on items returned)
  • Resource read/write in API (separate from package)
    • Does this need authorization work?
  • user/account API - read/write
  • Remove autocomplete -- can just use search
    • Do not worry about backwards compat as should only be used in our js (if others using it too bad!)
1311525660000000 1325473312000000
#1244 enhancement dread assigned Notes field carriage-returns converted to CRLF

When you edit a package in the web form, if the notes field had \n as the End Of Line symbol, it gets lost when you preview or save the package, and the notes field is displayed all on one line.

This can be seen when editing annakarenina (as created by 'paster create-test-data'). The diff shows for example:

- Some test notes
+ Some test notes
?                +

but it would more clearly be shown as:

- Some test notes\n
+ Some test notes
?                ++

This is a significant problem with DGU, since a lot comes in via the API.

It's not clear what we should do about it. We could standardise on \n or \r\n when the form submission comes in. Do different browsers on different platforms do different things with EOLs?

Analysis

Displaying the package: the Markdown processor respects both EOLs when displaying the field, putting each line in a <p> tag.

Creating the package edit form: placed into <textfield>.

Browser displaying package edit form: <textfield> displays \n and \r\n as EOL. But \n\n gets compressed to one EOL. But on submission, both are returned as \r\n.

Receiving the edited package: Somewhere along the line the EOL gets converted to \n\n.

1311689456000000 1340191253000000
#1255 enhancement kindly kindly ckan-backlog assigned Drupal consistancy checks.

Make a robust way to make sure the drupal database is consistent with the ckan data.

1312219968000000 1313400054000000
#1285 enhancement dread ckan-future assigned Errors cause emails

Currently a sysadmin gets an email when an exception is not caught. But there are occasions when we DO want to catch an exception so we can fail nicely for the user, but the sysadmin STILL gets an email to know to fix something. e.g. if there is an exception when search indexes a package. You want to catch the exception and still run any other notify calls.

1314116944000000 1338206151000000
#1287 enhancement thejimmyg dread ckan-backlog assigned NAVL validation errors - Junk fields should be listed explicitly

When you create a package, but specify a key that is not allowed (e.g. 'relationships') then you get error message:

{"__junk": ["The input field __junk was not expected."]}

It should mention the actual key which is not expected. e.g.

{"relationships": ["The input field 'relationships' was not expected."]}

Kindly said that James' version of NAVL was better in this respect, so this might be best solved by moving to that.

1314203102000000 1330990459000000
#1314 enhancement dread ckan-backlog assigned ckanclient search - generator improvements

Apparently the search generator always makes two requests, even if you don't want to see the search results, which might be slow. Can this be optimised?

Maybe we should also provide a second search function that doesn't use the generator - the original simple search function (that leaves the user to deal with limit & offset).

1315395410000000 1340191233000000
#1317 defect dread ckan-backlog assigned password reset - improve user search

In password reset, it gets confused if you have two similar users. This is because with the string the user provides, it searches several fields, not just name but also fullname and email address, allowing you to search for these. But only name is unique.

Specific problem: Ira searches for "Irina" then it finds both: <User name=irina fullname=Irina [email protected] ] and <User name=shevski fullname=Ira email=> (I think)

Maybe need to choose which field it searches?

1315415539000000 1340191221000000
#1322 enhancement dread assigned Action API improvements

Focusing on improving Action API as the v3 API:

  • have an optional parameter of the data_dict called "options". Options would contain items that would get passed into the context. e.g. {"options" : {"ref_package_by": "id"}}.
  • instead of using API version to change the way packages are referenced, use the ref_package_by.
  • All package_show, group_show etc. to accept an object 'name' as an alternative to object 'id'.
  • Action API is v3 of api, replacement for v1 & v2. Default for most urls is still v1, but if url is /api/action then default to v3.

Next steps:

  • Add search API (package, resource,
  • Add Util API
  • Clarify JSONP still applies
  • Add doc strings, clarifying parameters
1315474749000000 1340191088000000
#1328 defect [email protected] assigned Unicode & paster commands

A possible bug in CKAN when I tried deleting users using "paster --plugin=ckan user delete" command.

To reproduce the bug do the following:

  1. Create a user with an ID (which in my case was a user's full name)

that contains non-unicode caracters like Norwegian "æ", "ø", or "å".

  1. Make sure that you can see something like the example below:

(pyenv) [email protected]:$ paster --plugin=ckan user Users: name=Rustæm

  1. Then try deleting the user with following command:

(pyenv) [email protected]:$ paster --plugin=ckan user delete "Rustæm"

You should now get a python encoding error. I know that this is quite rare case, but in our case it caused some trouble. Could you guys have a look at this bug?

CKAN ver. 1.3.3.

1315823110000000 1340191065000000
#1358 enhancement zephod rgrp ckan-backlog assigned Generate configuration documentation automatically from the deployment_ini_tmpl file

At the moment documentation of config options is duplicated between source (deployment_ini_tmpl in ckan/config which is used to generate user ini file) and the docs.

Suggest we write a script that automatedly generates reference documentation for the config from the source.

May be obsoleted by #277 (some config in db)

1317076350000000 1318257823000000
#1366 defect dread ckan-future assigned Search inside extra fields

SOLR search doesn't support searching for part of an extra field, but it does for other fields.

i.e. title="One Two Three" matches q=one AND q=title:one and geographic_coverage="England Scotland" matches q=England BUT NOT q=geographic_coverage:England

This problem emerged when we went to SOLR in #1275 (CKAN 1.5a). Tests were skipped.

This is could be a problem for DGU and maybe elsewhere.

1317290992000000 1338206707000000
#1393 enhancement johnglover dread ckan-backlog assigned Don't skip search tests

Now we don't use postgres search, all the tests involving search now don't need to be skipped when running on sqlite. Should help coders spot earlier if these tests break.

1318505453000000 1320153590000000
#1447 defect kindly dread ckan-backlog assigned disk space leakage

Periodically we see some CKAN servers fall over because they run out of disk space. We need to find out if there is a common cause and fix it.

One problem in the past has been file handles running out when creating lots of tiny files in the data directory.

Another problem has been several enourmous backups being created every day - pdeu on eu25.

1320666843000000 1340727330000000
#1458 enhancement rgrp rgrp ckan-future assigned Support previewing kml files in data viewer

Super ticket: #1151 (viewing geo data)

E.g. preview of http://thedatahub.org/dataset/louisvillecrime should bring up a map

1320936488000000 1340632932000000
#1460 enhancement rgrp assigned Improve extensions documentation

Current extensions documentation needs some work: http://docs.ckan.org/en/latest/plugins.html

  • Queue extension section may now be out of date (?)
  • Think about how it integrates with https://github.com/okfn/ckanext-example (especially tutorial and example extension)
  • Document all plugin points (auto-extract from CKAN source??)
1321114523000000 1340191001000000
#1489 enhancement dread ckan-backlog assigned Updating example theme/extension

ckanext-example needs updating for CKAN 1.5:

  • theme changes
  • new forms

About: 'ckanext-exampletheme' was created in Spring 2011 as an example CKAN extension that showed how to customise the look & operation of CKAN. This moved to github and renamed 'ckanext-example'.

1322137920000000 1324292384000000
#1507 enhancement zephod rgrp ckan-backlog assigned Minor fixes to dataset add on Group edit form - 0.5d

Group edit dataset add form needs some work

  • Dataset name is not cleared when you add
  • No way to remove item from list of datasets to be added if I make a mistake
  • (2nd Apr 2012) It now seems that option to add multiple datasets at once has disappeared (perhaps during the CSS/HTML refactor ...)
1323088429000000 1338205220000000
#1509 defect dread assigned Mis-dated old revisions of datasets

e.g. http://thedatahub.org/dataset/osm%402011-07-12T12%3A16%3A47.590358 gives:

This is the current revision of this dataset, as edited at 2011-07-12 12:16.

but it should say the date 2011-07-12 (which is the data being displayed).

The problem is looking at PackageRevision?, when it might be a tag or extra.

1323090398000000 1340190814000000
#1550 enhancement ross ckan-backlog assigned Allow simple auth via the API

It should be possible to pass userid/username and api key and obtain a response from CKAN for external services that use CKAN auth. Those services shouldn't be talking to the DB directly.

1324049369000000 1346670055000000
#1571 enhancement rgrp jilly mathews ckan-backlog assigned [super] Issues Extension

"Proper" issues extension.

Overview

  • This would be an extension and improvement of existing todo extension - #842 (https://github.com/okfn/ckanext-issues)
    • Current extension does work but lacks polish
    • Should we polish or do significant improvements?
  • Functionality - see etherpad

Steps

  • (Rename todo(s) -> issue(s) in existing extension)
1324294225000000 1330863183000000
#1652 enhancement kindly ross ckan-backlog assigned How we intergrate with Drupal Multiligual?
  • Analysis of how/where we can integrate with Drupal Multilingual
  • Is it likely to access CKAN via /en/data/... or /data/en/... (not sure, probably former) and we need to make sure that it ends up at /en/...
  • Check with Ian for other issues.
1326709894000000 1340187535000000
#1661 defect dread ckan-backlog assigned Wrong Routes version installed by CKAN package

Jaakko Louhio reported that the wrong Routes version got installed during CKAN's package install.

He is using Ubuntu 10.04 and he believes it install Routes 1.12.3 instead of what we use which is 1.11.

1326718061000000 1339773949000000
#1684 enhancement ross ckan-backlog assigned Remove all config from ckanext-archiver

ckanext-archiver currently has a settings file (and a default) and it should be passed in all relevant information from the context.

Remove all settings (ARCHIVE_DIR and MAX_CONTENT_LENGTH and others) and pass them in from CKAN.

1326983821000000 1346670037000000
#1689 enhancement kindly dread assigned List deleted datasets in API

The admin extension allows deleted datasets to be viewed, but there is no equivalent in the API. The package_list API call filters to just 'state=active' datasets.

'state' could be a parameter on the package_list call

  • Easy to call and implement
  • But getting permissions right for the different state values may not be easy - maybe a separate logic function is better?
1327062214000000 1340190040000000
#1749 enhancement seanh ckan-future assigned Allow creating activity details through API

Currently the activity_create() logic action function only lets you create top-level activity stream items, and not their related activity details. It should handle activity details via nested dicts.

1328465406000000 1338213933000000
#1778 enhancement seanh seanh ckan-future assigned Replace classmethod's with module-level functions

In many places we have @classmethods where simple module-level functions would do (and would be more idiomatic in Python), e.g. the get() classmethods in the model/ classes.

Doing it with module functions lets us type module.function() instead of module.Class.method().

In many cases we're importing classes directly with from module import Class and then doing Class.method(), but from module import foo is bad style: http://docs.python.org/dev/howto/doanddont.html

1328543614000000 1340623743000000
#1831 enhancement ross ckan-backlog assigned Login with email address

Users should be able to log in using either their username, or their email address, both of which are unique.

Will require a change to UsernamePasswordAuthenticator? in ckan.lib.authenticator.py and possible a useful User.by_email in the user model if it doesn't already exist.

1330073906000000 1346670504000000
#1832 enhancement dread ckan-backlog assigned dataset purge API

Purging datasets (deleting them fully, not just changing the state to 'deleted') is important for users testing dataset creation over the API on a test CKAN instance.

Without this, they need to resort to more difficult methods such as:

  • cleaning and reloading the database
  • setting the test datasets to state 'deleted' and also appending a suffix '_00' and incrementing the number until there is no clash of names.

Requested for NHSIC.

Implementation

  • This could slot into the Action API.
  • Of course we would need to ensure the user's had been given the specific right to purge.
  • I suggest we log the full details of the dataset being purged.
1330077745000000 1339773706000000
#2197 defect zydio ckan-backlog assigned Storage Metadata API: add/update not working with local file storage (Pairtree)

If OFS is configured with Pairtree to use a local file storage, all POST requests to add/update metadata ( /api/storage/metadata/{label} ) will fail.

This is due to the use of BotoOFS specific private methods in StorageAPIController.set_metadata(), eg: self.ofs._require_bucket(bucket), self.ofs._get_key(b, label), self.ofs._update_key_metadata(k, metadata) ... those methods can't be found in POTFS and this causes errors. The API should use only OFSInterface methods, or should conditionally make calls based off the actual type of self.ofs.

PS: I did set "ckan" as "Component" in the ticket because storage has been integrated back into the core in CKAN 1.6

1330421377000000 1346662128000000
#2203 enhancement kindly rgrp ckan-v1.9 assigned Data Viewer support for PDF
  • Boot viewer for format/mimetype: application/pdf, application/x-pdf, application/acrobat, applications/vnd.pdf, text/pdf, text/x-pdf
  • propose we use pdf.js - https://github.com/mozilla/pdf.js
  • We could also try iframing for browsers with native support but this seems more complex
1330860293000000 1348568961000000
#2234 enhancement seanh ckan-future assigned Write a CKAN extension for pulling items from RSS/Atom feeds into CKAN templates

You configure the extension with some RSS and/or Atom feeds, it automatically reads items from these feeds and makes them available in the template context, you write a custom template to e.g. display 'news' items from a Wordpress blog on your front page.

This extension might be simpler and less fragile than ckanext-wordpresser, and also more generally useful.

Details

  • Mark Pilgrim's Universal Feed Parser might be useful for reading the feeds
  • Feed items should probably be cached somewhere

Suggested

  • The news item 'widget' should be wrapped in a known class so that it can be styled easily regardless of the format of any HTML entry.
  • For non-HTML formatted items (Atom should tell you the content type of the entry) maybe we should have a template for rendering each item along with any enclosures that it might reference
  • Caching is pretty crucial and should probably obey the ttl of the feed.
1331902755000000 1346669567000000
#2284 enhancement ross ckan-backlog assigned Local queries/views/viz in Related stuff.

From #2204

not sure how much we have thought through using this for storing queries / views / visualizations coming from our data viewer. In particular, wonder if this necessitates some kind of support for arbitrary json data ...

### Comments

In the simplest form, right now all I need to store is store the url to the resource plus hashbang config for recline. That would be super easy ...

1334218700000000 1346669513000000
#2294 defect kindly rgrp ckan-v1.9 assigned Unauthorized action on Data API gets 302 rather than 401 or similar

Well known issue that 401 *in API* end up with 302 redirect to login page (which makes no sense and is very confusing for clients).

Fix this by ensuring that error messsages on API get passed through correctly.

Minor because affect is annoying but not terminal

1334432104000000 1340624003000000
#2329 enhancement rgrp ckan 2.0 assigned Add back in RSS/Atom links on relevant pages

E.g. on dataset page, on revision page, on user page and on search results.

1335442540000000 1346664271000000
#2334 enhancement kindly rgrp ckan-v1.9 assigned Improved data import

Placeholder for thinking about this ...

  • Allow users to configure info for CSV import (e.g. separators, quotes etc)
1335651662000000 1340624538000000
#2335 defect seanh seanh ckan-v1.9 assigned Unicode characters don't work in CKAN ini files

You get a UnicodeDecodeError?, looks like it tries to decode the ini file as ascii. This is a problem because someone might try to set their site description to something with an umlaut in it, for example.

1335789726000000 1340624635000000
#2337 enhancement seanh seanh ckan-v1.9 assigned Sort out ini file confusion in docs

For example if you follow the source install instructions then you have development.ini, then when you go onto Post-Installation Setup it talks about std.ini. People don't realise that these files are the same.

Either fix the docs or fix CKAN to use same ini file names in package and source installs/

1335797876000000 1340624600000000
#2348 enhancement ross ckan-future assigned Java client library for CKAN

Start a Java library for interacting with the CKAN Action API.

Currently requires support for Groups, Tags, Resources, and Search.

An example app would be very useful.

1335879980000000 1346670024000000
#2370 enhancement seanh seanh ckan-v1.9 assigned Add examples of all config settings in default config file

The default config file should contains examples (maybe commented out) for all config variables. Makes it easier to edit the config. For example openid_enabled is not in there.

1336042237000000 1340624662000000
#2378 enhancement ross ckan-future assigned Extract metadata directly from resources that contain it.

Some files that might be uploaded as resources on a dataset, such as some image formats, already contain metadata. For example some jpeg files might contain Exif ( http://en.wikipedia.org/wiki/Exchangeable_image_file_format ) data.

[ ] Obtain list of embedded metadata 'standards'

[ ] Where the metadata is available it should be added as extra fields on the resource that is uploaded.

[ ] HTML?

[ ] Image formats (exif/gif etc)

[ ] PDF

[ ] Eventually consider ...

[ ] MS Office documents

1336651088000000 1346669985000000
#2404 enhancement rgrp ckan-future assigned Auto-screenshot for related items

As a User I want an image generated for my related item automatically so that I do not have to take picture myself and upload it somewhere

Implementation: (?) Use phantomjs or something like http://snapito.com/ Investigate for 1.8, will be needed for TDH Oct.

1337583347000000 1346669526000000
#2405 defect icmurray rgrp ckan-v1.9 assigned Improvements to data viewer embed
  • Decent powered by icon for the Embed for the DataHub?? (make this text customizable and allow html ...)
    • Current icon and text looks really bad
  • Embed link uses uuid rather than dataset name (this goes to perma-url question ...)
  • Data viewer embed url includes all the resource info (not needed -- just url and backend info ...)
  • Embed extension urls do not respect original domain but instead use site_url (I'm guessing). E.g. datahub.io domain ends up with links to embed on thedatahub.org
1337583565000000 1340635981000000
#2412 defect zephod icmurray ckan-sprint-2012-06-25 assigned More than one resource invalidatiing breaks dataset edit form

When attempting to add more than one resource at once, if more than one resource invalidates, this results in a js error, leaving the form in an inconsistent state.

Repro:

  1. Go to /dataset/new
  2. Add a new resource. Fill in one of the fields with an invalid value. eg - last_modified, or size...
  3. Add another resource, doing the same thing: make one of the fields invalid.
  4. Try to save the dataset.
  5. The entered resource information will be lost, and a js error "Uncaught Error: Can't add the same model to a set twice,: backbone.js:586" will be thrown.
1337717148000000 1338203234000000
#2446 enhancement johnmartin aron.carroll demo phase 4 assigned Create demo dataset history/comparison page

This is a low priority for the demo site.

Discussion:

https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62414120/comments

Implementation:

http://s031.okserver.org:2375/dataset/history/adur_district_spending

1338210738000000 1352658819000000
#2451 enhancement johnmartin aron.carroll demo phase 5 assigned Create demo user list page

Discussion:

https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62268474/comments

Implementation:

http://s031.okserver.org:2375/user

1338211213000000 1352658902000000
#2457 enhancement johnmartin aron.carroll demo phase 4 assigned Create demo tags list page

This includes the tag page as well for now.

Discussion:

https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62998445/comments

Implementation:

http://s031.okserver.org:2375/en/tag

1338211735000000 1352658878000000
#2461 enhancement toby aron.carroll demo phase 4 assigned Create demo revisions page

I believe this is to be merged in with site activity.

Discussion:

https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts

Implementation:

http://s031.okserver.org:2375/en/revision

1338212055000000 1344543838000000
#2473 enhancement kindly kindly ckan-v1.9 assigned Make datstorer store field ordering in _meta field

Recline views should have a default table order, being the same as the csv that was imported.

1338293492000000 1340636871000000
#2513 enhancement ross ckan-backlog assigned Dataproxy should not default to utf8

Unless explicitly told by the source web server the dataproxy should not assume that the content it has can be encoded as UTF-8. Even though the chars from 128 - 255 overlap an attempt to decode some byte array as utf8 will fail whenever a latin1 char whose bitpattern has the MSB set.

This will mean that the UTF8Recoder can be more rigid in its acceptance of data, Postel aside.

1339575820000000 1346669646000000
#2520 defect seanh seanh ckan-v1.9 assigned Document undocumented config options

There are 21 undocumented config options in CKAN, some of which are not mentioned in the config file template either:

ckan.admin.name ckan.admin.email ckan.default.group_type ckan.page_cache_enabled ckan.cache_enabled ckan.cache_expires ckan.extra_resource_fields ckan.extra_resource_group_fields ckan.storage.key_prefix ckan.storage.max_content_length ckan.feeds.authority_name * ckan.feeds.date * ckan.feeds.author_name * ckan.feeds.author_link * ckan.mail_from ckan.gravatar_default * ckan.plugins ckan.api_url ckan.auth.profile ckan.datastore.enabled ckan.tracking_enabled

There are also some options that are in the default deployment.ini even though they're deprecated:

ckan.async_notifier carrot_messaging_library ckan.build_search_index_synchronously

See email to ckan-dev from David Read: http://lists.okfn.org/pipermail/ckan-dev/2012-June/002447.html

It'd be best if the docs could be automatically pulled from the source into sphinx using autodoc, see #1358

1339588368000000 1340624908000000
#2535 enhancement rgrp assigned SSL certificate for DataHub + https by default

DataHub? is increasingly used and we should ensure it uses ssl as part of general security.

See also #1446 (Need to support https login for multiple instances as part of the CKAN package install)

1339758027000000 1346662082000000
#2546 requirement ross ckan-backlog assigned ODS Managing homepage content

Requirements

Require the ability for users to control some level of content that is visible on the home page of their ODS installation. This may be through RSS/Atom feeds (see #2234) or another mechanism but should result in admins being able to change blocks of text on their homepage.

This should not be configuration, but accessible through WUI.

Interface

None

User Stories

  • As a system administrator I want to have control over content displayed on the front page beyond featured/popular items.

  • As a system administrator I don't want to manage content through having to write an extension.

Tasks

[ ] Analysis

Estimates

1340016842000000 1346663437000000
#2547 enhancement shevski ross opendatasuite 2 assigned ODS Initial data sets

Requirements

The ODS demo site will need data adding, initially as fixtures but it would also be useful if we started evaluating datasets that we can ship with ODS installations (at least in the UK) from places such as DGU and ONS.

May wish to create a ticket for making sure the datasets within the system are reset every X hours. Perhaps.

Interface

None

User Stories

  • As a new system administrator for an ODS instance, I don't want to have a site devoid of any data. Geographically relevant datasets would be welcomed.
  • As a bizdev person I would like to be able to demonstrate how ODS works with real datasets.

Tasks

[ ] Identify relevant sources for datasets

[ ] Pick datasets

[ ] Set them up for import

Estimates

1340016906000000 1340705614000000
1 2 3 4 5 6 7 8 9 10 11
Note: See TracReports for help on using and creating reports.