CKAN: Ticket Query

#1317: password reset - improve user search

dread — Wed, 07 Sep 2011 17:12:19 GMT

In password reset, it gets confused if you have two similar users. This is because with the string the user provides, it searches several fields, not just name but also fullname and email address, allowing you to search for these. But only name is unique.

Specific problem: Ira searches for "Irina" then it finds both: <User name=irina fullname=Irina email=irina.bolychevsky@okfn.org ] and <User name=shevski fullname=Ira email=> (I think)

Maybe need to choose which field it searches?

#2686: enabling datastore & data API breaks recline

shevski — Tue, 17 Jul 2012 09:06:51 GMT

First I noticed that the gold prices dataset preview was not displaying & has data API enabled Secondly I tried enabling datastore for http://datahub.io/dataset/adur_district_spending/resource/281dffa6-ea9b-4446-be41-05dced06591f and after I saved the preview no longer worked. Unticking the datastore & data api checkbox brought it back

Is this a known issue?

#1447: disk space leakage

dread — Mon, 07 Nov 2011 11:54:03 GMT

Periodically we see some CKAN servers fall over because they run out of disk space. We need to find out if there is a common cause and fix it.

One problem in the past has been file handles running out when creating lots of tiny files in the data directory.

Another problem has been several enourmous backups being created every day - pdeu on eu25.

#1832: dataset purge API

dread — Fri, 24 Feb 2012 10:02:25 GMT

Purging datasets (deleting them fully, not just changing the state to 'deleted') is important for users testing dataset creation over the API on a test CKAN instance.

Without this, they need to resort to more difficult methods such as:

cleaning and reloading the database
setting the test datasets to state 'deleted' and also appending a suffix '_00' and incrementing the number until there is no clash of names.

Requested for NHSIC.

Implementation

This could slot into the Action API.
Of course we would need to ensure the user's had been given the specific right to purge.
I suggest we log the full details of the dataset being purged.

#1314: ckanclient search - generator improvements

dread — Wed, 07 Sep 2011 11:36:50 GMT

Apparently the search generator always makes two requests, even if you don't want to see the search results, which might be slow. Can this be optimised?

Maybe we should also provide a second search function that doesn't use the generator - the original simple search function (that leaves the user to deal with limit & offset).

#1571: [super] Issues Extension

jilly mathews — Mon, 19 Dec 2011 11:30:25 GMT

"Proper" issues extension.

Etherpad: http://ckan.okfnpad.org/issues
Code: https://github.com/okfn/ckanext-issues

Overview

This would be an extension and improvement of existing todo extension - #842 (https://github.com/okfn/ckanext-issues)
- Current extension does work but lacks polish
- Should we polish or do significant improvements?
Functionality - see etherpad

Steps

~~(Rename todo(s) -> issue(s) in existing extension)~~

#1240: [super] API v4

rgrp — Sun, 24 Jul 2011 16:41:00 GMT

(Just creating this ticket as somewhere to keep notes)

Decide on REST api versus action API
- Do we want to support both?
Tidying
- Unify on /api/v{version num}/... structure (do we want a default option that points to current default? e.g. /api/default/ ...)
- extras merged into normal field list in package
- Get rid of /rest/ so just have api/v1/package
- Get rid of separation of search api from 'rest' api
  - Propose that GET on REST index is search e.g. /package/?q=...
    - This is also resolves issue whereby GET at root returns whole package set (a *bad* idea) as this would now become the matchall search query (with a default limit on items returned)
Resource read/write in API (separate from package)
- Does this need authorization work?
user/account API - read/write
Remove autocomplete -- can just use search
- Do not worry about backwards compat as should only be used in our js (if others using it too bad!)

#1661: Wrong Routes version installed by CKAN package

dread — Mon, 16 Jan 2012 12:47:41 GMT

Jaakko Louhio reported that the wrong Routes version got installed during CKAN's package install.

He is using Ubuntu 10.04 and he believes it install Routes 1.12.3 instead of what we use which is 1.11.

#1489: Updating example theme/extension

dread — Thu, 24 Nov 2011 12:32:00 GMT

ckanext-example needs updating for CKAN 1.5:

theme changes
new forms

About: 'ckanext-exampletheme' was created in Spring 2011 as an example CKAN extension that showed how to customise the look & operation of CKAN. This moved to github and renamed 'ckanext-example'.

#1168: Test system for deb packaging

dread — Thu, 26 May 2011 20:33:14 GMT

Get buildbot to:

build the deb packages
install them into a fresh virtual machine
run smoke tests on the installed ckan

#2197: Storage Metadata API: add/update not working with local file storage (Pairtree)

zydio — Tue, 28 Feb 2012 09:29:37 GMT

If OFS is configured with Pairtree to use a local file storage, all POST requests to add/update metadata ( /api/storage/metadata/{label} ) will fail.

This is due to the use of BotoOFS specific private methods in StorageAPIController.set_metadata(), eg: self.ofs._require_bucket(bucket), self.ofs._get_key(b, label), self.ofs._update_key_metadata(k, metadata) ... those methods can't be found in POTFS and this causes errors. The API should use only OFSInterface methods, or should conditionally make calls based off the actual type of self.ofs.

PS: I did set "ckan" as "Component" in the ticket because storage has been integrated back into the core in CKAN 1.6

#1041: Start Using the CKAN Wiki for Tutorial-style documentation

thejimmyg — Wed, 16 Mar 2011 14:11:55 GMT

For example, I will document the following:

A specific guide on using mercurial when working with our branching and merging policy
- DONE - http://wiki.ckan.net/Becoming_a_CKAN_Developer
Policy document on how we use tickets, plan sprints and make releases
- DONE - http://wiki.ckan.net/Becoming_a_CKAN_Developer
How to package software as .deb file for project deployment

I'd love if someone else would write:

An authorisation tutorial covering the core model, the command line tools and examples of every possible way of using the system
A HOWTO guide with screenshots for adding a package

#277: Set some config options / settings in WUI (extension)

dread — Mon, 22 Mar 2010 16:21:01 GMT

Use case

As a ckan administrator I want to easily change options about the CKAN install.

Implementation

Settings to be in DB

Suggested:

## Title of site (using in several places including templates and <title> tag
ckan.site_title = CKAN
## Logo image to use (replaces site_title string on front page if defined)
ckan.site_logo = http://assets.okfn.org/p/ckan/img/ckan_logo_box.png
## Site tagline / description (used on front page)
ckan.site_description =
## Used in creating some absolute urls (such as rss feeds, css files) and
## dump filenames
ckan.site_url =
## Favicon (default is the CKAN software favicon)
ckan.favicon = http://assets.okfn.org/p/ckan/img/ckan.ico
## An 'id' for the site (using, for example, when creating entries in a common search index)
## If not specified derived from the site_url
# ckan.site_id = ckan.net
## API url to use (e.g. in AJAX callbacks)
## Enable if the API is at a different domain
# ckan.api_url = http://www.ckan.net
## html content to be inserted just before </body> tag (e.g. google analytics code)
## NB: can use html e.g. <strong>blah</strong>
## NB: can have multiline strings just indent following lines
# ckan.template_footer_end =

NB: these will still need to be stored somewhere for loading on initialization. do this in db init function ...

Settings / Options / KeyValues? Table

Columns:

[namespace]: ? only if KeyValues? (for settings this would then always be settings)
key
label
value (json)
type (e.g. date and to specify in advance what type should be)
description
tags: ?? (for grouping ...)

Loading settings from DB

Do this in ckan/config/environment.py

WUI

/ckan-admin/settings
Show label, plus description plus text field

Depends

Would be part of ckan-admin section and hence build on ticket:833 (Administrative dashboard)

#2554: Research Virtuoso cartridges

ross — Mon, 18 Jun 2012 13:37:25 GMT

Look into writing a cartridge for importing CKAN data into a Virtuoso quadstore

http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSponger#How Does It Work?

#1684: Remove all config from ckanext-archiver

ross — Thu, 19 Jan 2012 14:37:01 GMT

ckanext-archiver currently has a settings file (and a default) and it should be passed in all relevant information from the context.

Remove all settings (ARCHIVE_DIR and MAX_CONTENT_LENGTH and others) and pass them in from CKAN.

#765: Read-only mode - API usage

dread — Tue, 26 Oct 2010 11:18:17 GMT

All writes to the API are captured and you are returned an error explaining the reason.

Possible errors:

503 temporary maintenance
403 forbidden (if server if permanently read-only)

#2590: Publisher dashboard

ross — Mon, 25 Jun 2012 10:03:37 GMT

Need proper user stories but ...

Publisher admins/editors may need a more useful group read page showing things like:

The current search

Recent activity

People within the group

Followers

Others?

#256: Package relationships - 3. Edit in WUI

dread — Tue, 23 Feb 2010 12:36:01 GMT

WUI:

Editable as part of package or separately? (e.g. like authz)
Do we normalize to only one type name of the pair?
Do we allow create of relationship from both ends (e.g. only from dependency to dependent or either way?)

#253: Package relationships

dread — Mon, 22 Feb 2010 16:05:21 GMT

Overview

Functionality to formally associate packages. We see a need for specific parent-child, inheriting or dependency relations. Not only should this help navigation between packages in the web interface, but it also provides a mechanism to automatically pull dependencies when downloading a data package, in a similar manner as we see in software package management.

Examples

There are 27 packages in data.gov.uk to do with the Data4NR's Health Poverty Index. There is currently no common link between these, unless you search for 'HPI' (which also brings up House Price Index), or look under tag 'health' (which also has 600 other results). There should be a link on each HPI package page to navigate to the other 'sibling' HPI packages, and to a 'root' package that has info about the set. This could be partially achieved using the existing tag or group concepts, but a more explicit/official/obvious marking of their relationship could be beneficial.

In ckan.net is freedict, a collection of translation dictionaries. You could make each dictionary a child package and use this system. But it would probably be better to make each dictionary a different resource in the same package. (There are other ideas to denote a resource as the data making up a 'portion' of package, or a 'whole' of the package, to help people downloading datasets in the software package style.)

OSM has had some Naptan data imported (bus stops), with special permission - i.e. a more liberal license. It would be useful to show this link on both OSM and Naptan packages in CKAN: OSM 'derives from' Naptan with a comment about the license change. I'm not sure this is useful to an automatic download or use of these datasets, but may aid exploration on the CKAN website and understanding the provenance of the bus stop data on it.

IPCC collection of data linked / mirrored. Not sure if there are useful relationships here?

Dracos gets postbox locations from crowd sourcing and OSM. We could say Dracos 'derives from' OSM.

See more examples discussed here: http://trac.ckan.org/ticket/253

Implementation

This is split into four tickets:

Model: ticket:254
Read in WUI: ticket:255
Edit in WUI: ticket:256
API: ticket:257

No need for write access to be provided API for the moment.

This ticket also encompasses ticket:169 (Package derivations) and ticket:176 (Package dependencies).

#2546: ODS Managing homepage content

ross — Mon, 18 Jun 2012 10:54:02 GMT

Requirements

Require the ability for users to control some level of content that is visible on the home page of their ODS installation. This may be through RSS/Atom feeds (see #2234) or another mechanism but should result in admins being able to change blocks of text on their homepage.

This should not be configuration, but accessible through WUI.

Interface

None

User Stories

As a system administrator I want to have control over content displayed on the front page beyond featured/popular items.

As a system administrator I don't want to manage content through having to write an extension.

Tasks

[ ] Analysis

Estimates

#2732: New file upload functionality

ross — Mon, 23 Jul 2012 15:53:09 GMT

We should simplify upload and storage of files, initially only to local storage with archiver eventually being fixed to archive data externally. WIP pad is http://ckan.okfnpad.org/uploads

Simplifying uploads

Currently uploads are too painful/difficult/fiddly to use and/or configure. We want to simplify uploads so that they are done directly to the CKAN server, without support for remote services (S3 etc) and/or the dependencies it introduces.

We want to fix:

File uploads themselves
Storage of uploaded files
Notification of the upload to other components

File uploads

Things file upload should do:

Allow sysadmin to disable
Allow auth'ed users to upload
Store whatever they send on disk, and store DB entry linking the file to the person
When creating the resource, the user should be able to choose from all of the files they have uploaded but not yet associated with a resource. This will allow for bulk upload and then a delayed association. Whenver a user creates a resource they either upload a file now, or see previously uploaded files.

Can we do the upload asynchronously and then associate the
uploaded key with the resource before the save ? What happens
if the user tries to submit before asymc upload finishes ? Should
we delay them?

The upload workflow should look like...

File upload should be a straightforward file upload with normal auth checks and normal processing of the posted data.
1. When called via ajax then the ID of the newly created file should be returned,
2. When called via WUI then it should also be given the url to redirect to after the file upload has been handled - the id will be passed as a query param.
The resource save should check whether it has a file id and in that case updates the file object to point to the resource.

This should enable:

Separate file upload into a user's temporary store, either individually or as a batch.
Creating resources and simply choosing from previously uploaded, unassigned files
Adding files/data to a resource after the fact.

File storage

File storage should be local to the CKAN install, and not a remote service. Any archiving to remove storage providers should be outside of the main request.

File storage should:

allow moving data, a sysadmin should be able to move the storage root and change configuration and have the system continue running (i.e. don't store absolute paths).
provide maintainability, it should be easy to determine which old files are not associated with resources and thus can be cleaned up.
allow for collection of information (i.e. estimate of storate space used)
check whether there is enough space and handling the conequences cleanly
ensure files to be written only underneath its own root folder, checks should be made after any path generation that the file begins with the location of the file storage.
Have a configurable maximum accepted blob size during upload.
Should store what meta-data was provided with the upload, such as mimetype.

Somewhere in the DB we should store ...

Column	Notes
id	An identifier
owner	The owning user, who uploaded the file
path	The path (from the 'storage root') to the file
size	The size in bytes of the file on disk
mimetype	The mimetype of the file, as provided by the uploader
upload_date	When the data was uploaded
resource	The ID of the resource it belongs to. A unidirectional relationship.
archived_url	The URL where this file has been archived

Generating paths should try and separate the files, perhaps based on username of the owner, or some other mechanism to avoid a single folder full of files.

Notifications

We need to make sure that it is possible to notify other components within the system that an upload has taken place, or at least make it easy for them to be notified. The primary use case for this is to notify the component that will translate/upload certain formats to the data store.

We could do this based on the post-upload update to the file model (i.e. when we record the total received size of the file).

#1287: NAVL validation errors - Junk fields should be listed explicitly

dread — Wed, 24 Aug 2011 16:25:02 GMT

When you create a package, but specify a key that is not allowed (e.g. 'relationships') then you get error message:

{"__junk": ["The input field __junk was not expected."]}

It should mention the actual key which is not expected. e.g.

{"relationships": ["The input field 'relationships' was not expected."]}

Kindly said that James' version of NAVL was better in this respect, so this might be best solved by moving to that.

#143: Most active users listed on homepage

dread — Thu, 08 Oct 2009 13:59:33 GMT

Display league of users' recent activity on homepage.

#1507: Minor fixes to dataset add on Group edit form - 0.5d

rgrp — Mon, 05 Dec 2011 12:33:49 GMT

Group edit dataset add form needs some work

Dataset name is not cleared when you add
No way to remove item from list of datasets to be added if I make a mistake
(2nd Apr 2012) It now seems that option to add multiple datasets at once has disappeared (perhaps during the CSS/HTML refactor ...)

#1831: Login with email address

ross — Fri, 24 Feb 2012 08:58:26 GMT

Users should be able to log in using either their username, or their email address, both of which are unique.

Will require a change to UsernamePasswordAuthenticator? in ckan.lib.authenticator.py and possible a useful User.by_email in the user model if it doesn't already exist.

#2284: Local queries/views/viz in Related stuff.

ross — Thu, 12 Apr 2012 08:18:20 GMT

From #2204

not sure how much we have thought through using this for storing queries / views / visualizations coming from our data viewer. In particular, wonder if this necessitates some kind of support for arbitrary json data ...

### Comments

In the simplest form, right now all I need to store is store the url to the resource plus hashbang config for recline. That would be super easy ...

#1652: How we intergrate with Drupal Multiligual?

ross — Mon, 16 Jan 2012 10:31:34 GMT

Analysis of how/where we can integrate with Drupal Multilingual

Is it likely to access CKAN via /en/data/... or /data/en/... (not sure, probably former) and we need to make sure that it ends up at /en/...

Check with Ian for other issues.

#1358: Generate configuration documentation automatically from the deployment_ini_tmpl file

rgrp — Mon, 26 Sep 2011 22:32:30 GMT

At the moment documentation of config options is duplicated between source (deployment_ini_tmpl in ckan/config which is used to generate user ini file) and the docs.

Suggest we write a script that automatedly generates reference documentation for the config from the source.

May be obsoleted by #277 (some config in db)

#979: Edit Resource extras in the API

dread — Fri, 11 Feb 2011 13:09:37 GMT

Follows on from #826. We can now edit resource extras in the WUI (to some extent - see #978 for remaining issues) and we can view resource extras in the API, but we can't yet edit them in the API.

#1255: Drupal consistancy checks.

kindly — Mon, 01 Aug 2011 17:32:48 GMT

Make a robust way to make sure the drupal database is consistent with the ckan data.

#1393: Don't skip search tests

dread — Thu, 13 Oct 2011 11:30:53 GMT

Now we don't use postgres search, all the tests involving search now don't need to be skipped when running on sqlite. Should help coders spot earlier if these tests break.

#2513: Dataproxy should not default to utf8

ross — Wed, 13 Jun 2012 08:23:40 GMT

Unless explicitly told by the source web server the dataproxy should not assume that the content it has can be encoded as UTF-8. Even though the chars from 128 - 255 overlap an attempt to decode some byte array as utf8 will fail whenever a latin1 char whose bitpattern has the MSB set.

This will mean that the UTF8Recoder can be more rigid in its acceptance of data, Postel aside.

#1062: Data preview encoding error

sebbacon — Tue, 29 Mar 2011 10:55:43 GMT

The preview of "Species Misc Turtle Download" at http://ckan.net/package/taxonconcept results in the following error:

Unable to Preview - Had an error from dataproxy: Data Transformation Error (Data transformation failed. Reason: 'utf8' codec can't decode byte 0x8b in position 1: unexpected code byte

#1550: Allow simple auth via the API

ross — Fri, 16 Dec 2011 15:29:29 GMT

It should be possible to pass userid/username and api key and obtain a response from CKAN for external services that use CKAN auth. Those services shouldn't be talking to the DB directly.

#2607: 'Upload a file' appears on resource form when storage not enabled

seanh — Wed, 27 Jun 2012 13:30:08 GMT

if the user tries to upload a file they will get "Failed to get credentials for storage upload. Upload cannot proceed"

Maybe add a test for it this time, this bug has appeared and reappeared before

#1414: track user log-ins on thedatahub.org

shevski — Mon, 24 Oct 2011 11:13:02 GMT

Set up tracking for user logins so that we have stats about how many active users of thedatahub exist want to be able to see who logged in the the last x months

#1201: seperate out logic in atom feeds to logic layer.

kindly — Fri, 24 Jun 2011 15:21:32 GMT

Simplify the logic in the atom feed an make all feeds use logic layer to return lists.

#2729: searching for tags:[tag] works but tag:[tag] doesn't

shevski — Sun, 22 Jul 2012 09:58:29 GMT

which is confusing since you can only search for one tag like this at a time. I.e. tags:economics,cvs or tags:economics, csv or tags:economics+CSV doesn't work for example; therefore tag:economics, should also work!

http://s031.okserver.org:2375/dataset?q=tags%3Aeconomics&sort=relevance+asc

#1668: repoze version discrepency

dread — Tue, 17 Jan 2012 12:02:26 GMT

There's a discrepency in repoze.who versions between the source and package installs:

repoze.who - package 1.0.18 vs source 1.0.19
repoze.who-friendlyform - package 1.0b3 vs source 1.0.8

We get a test failure [1] with the 1.0b3 version (from the ubuntu 10.04 python-repoze.who-plugins package). But we've not noticed any problems on s057 instances (br, no, ie etc) which have the package versions of repoze.who.

The reason the package install uses the earlier packaged versions rather than the ones we'd like is that repoze uses all sorts of horrendous import hacks, making it too difficult to put into our 'ckan-conflict' source package.

James suggests we 'do something horrible like dynamically patch repoze on CKAN import'.

[1] http://buildbot.okfn.org/builders/builder-ckan/builds/1371/steps/shell/logs/stdio ERROR: ckan.tests.functional.test_user.TestUserController?.test_user_create_unicode

#2395: paster db clean/init don't work when spatial extension enabled

dread — Wed, 16 May 2012 09:16:33 GMT

If you have a spatial enabled database then if you don't disable the spatial extension in the CKAN config temporarily then you get errors when you run paster db clean and paster db init.

Can't you just modify the clean and init functions to run without extensions enabled?

The wider problem is that extensions do their own inits every time you do load_environment, which seems crazy and inefficient to me, since this occurs every time a request comes into CKAN. But that is another problem/ticket.

#1606: metadata license config option

dread — Mon, 02 Jan 2012 10:45:30 GMT

Add a config option to choose the metadata licence. Set default to Open Database License.

Currently the dataset edit form says "Important: By submitting content, you agree to release your contributions under the Open Database License." This is hard-coded, but not suitable for when DGU uses the CKAN form - they use the OGL.

#1544: delete old git branches

dread — Mon, 12 Dec 2011 15:10:10 GMT

We have 150 odd branches (git branch -a) - most of them old - we should prune them. At very least, branches that have been merged in should be deleted. Look at old branches that haven't been merged in and wonder why.

May be of some use:

git branch --merged
git branch --no-merged

#1647: add links to ckan discuss & dev to thedatahub

shevski — Mon, 16 Jan 2012 00:30:52 GMT

In the footer as well as more clearly & directly on the About page

#1343: [super] User related improvements (login, user pages etc)

rgrp — Wed, 14 Sep 2011 16:18:18 GMT

Disallow account creation via openid - #1386
Require email field - #1319
- Require email confirmation to be activated (?)
Improvements to user page (e.g. show activity and more info about user) - #1396
Modal user login - #1311

#1717: [super] Search UX improvements

shevski — Thu, 26 Jan 2012 18:53:01 GMT

Make it possible to search by tag (e.g. by typing tag:csv into the search bar and clicking enter, it should add the 'csv' tag facet to the search)

Rename and standardise the list of format tags, on search page this should also be called 'Format' instead of 'res_format' (in the right hand side bar on search page).

Make it possible to view full list of tags, formats and groups by clicking on the name. From here you should be table to click on a classification and go back to a search page faceted by that classification. E.g. from search page, click on 'tags', on tag page click on 'london' or whatever, and be navigated back to search page with search within 'london' tag only. Or y'know, a better way of doing it.

More standard classifications, such as 'Location' and 'Theme' - like on publicdata.eu

Blue search button should be displayed in line with the search bar, not underneath

Datasets should be displayed in order of most viewed or downloaded instead of alphabetically. For alphabetic search we could consider adding a way to facet by first letter of dataset name

#1578: [super] Re-enable and refactor ratings

rgrp — Mon, 19 Dec 2011 19:20:43 GMT

#1232: [super] Interface improvements

thejimmyg — Wed, 20 Jul 2011 16:11:36 GMT

Child tickets:

#1194 "Welcome back" message for newly registered user
#1202 Links to datapkg utility don't lead to info about it
#925 Change the search box icon to remove the down arrow
#923 Search box doesn't work in leaderboard page in stats extension
#1034 Flash message cached
#737 Markdown syntax summary page
#811 Extra field editing form layout breaks when there are long field names

#1233: [super] Improve wiki-style functionality for history

thejimmyg — Wed, 20 Jul 2011 16:29:52 GMT

At the moment we have a good revisioning system but a poor history interface. We need to improve this in a number of areas:

#191 Searching by modification date
#193 Searching by time-related field
#301 Package discussion pages
#1236 Package history page should provide links to pages at particular revisions, similar to the wikipedia pages
#1236 Viewing old revisions or unmoderated changes should have a message at the top of the package page
Other improvements as per my word doc.

#1432: [super] Data processing system for CKAN and Webstore

rgrp — Tue, 01 Nov 2011 10:19:07 GMT

Super ticket: #1190

A data processing system which utilizes the Webstore. One could get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules)

#1326: Write a set of auth plugin functions to integrate with Druapl

thejimmyg — Mon, 12 Sep 2011 09:51:24 GMT

Ticket #787 described join auth between CKAN and Drupal. The authentication part is live and implemented. This ticket is a placeholder for work that will be needed in the new auth system to link authorization functions to Drupal. It is dependent on the groups refactor.

#2277: Use the new atom feeds in IATI

icmurray — Mon, 02 Apr 2012 17:10:28 GMT

Atom feeds have been implemented in core in #1593. The IATI-specific implementation can now be removed when IATI move to CKAN >= 1.6.1

#1352: Use logic functions instead of as_dict when indexing entities

amercader — Wed, 21 Sep 2011 14:29:57 GMT

The current search implementation uses the output of the the as_dict method of the domain Package object to update the index

https://bitbucket.org/okfn/ckan/src/56c79e3fc44c/ckan/lib/search/index.py#cl-48

It also uses package_to_api1 in the SynchronousSearch? plugin:

https://bitbucket.org/okfn/ckan/src/f9dfb0506594/ckan/lib/search/__init__.py#cl-93

This prevents extensions from being able to index custom properties (e.g. faceting by custom extras not included in the model).

The search should use the logic function to get the package properties:

get_action('package_show')(context,data_dict)

#370: Use better email encryption for author_email and maintainer_email

shudson@… — Thu, 22 Jul 2010 18:03:39 GMT

The JavaScript? email encryption used is not very reassuring. Google's MailHide? is a much better solution that is easily implemented.

http://www.google.com/recaptcha/mailhide/

Check on the Mailhide API where there are even some Python libraries already built.

#653: Trackback links for packages

dread — Tue, 21 Sep 2010 09:40:25 GMT

When people link to a package, a track-back link is auto-created. (Similar system as for blogs).

As suggested by Tim Davies:

Allowing some form of ‘track back’ against datasets When a non-technical user comes to look at a dataset it would be really useful for them to be able to see if anyone has created an interface interpretation of it already.

I found quite a few cases in research of end-users struggling to make sense of a dataset when good interfaces to that data had already been built and blogged about, but without there being any link from the dataset listing to those data uses. Accepting track backs could also make it easier for technical users to find blog posts / shared code etc. relating to a given dataset.

#331: Timezone of CKAN timestamps should be configurable

rgrp — Mon, 31 May 2010 10:40:40 GMT

Revisions are timestamped using the server's clock, which may not relate to the expected timezone for the site. e.g. the Norway site has a server on GMT. No timezone info is displayed either.

Would like to set timezone for a CKAN instance to use in rendering revision timestamps. For example, use CET or EST timezone.

#1145: Support the Handle System

timmcnamara — Thu, 19 May 2011 00:26:15 GMT

The Handle System is an initiative to provide persistent references for resources. That is, it's basically a proxy system for preventing link rot.

Its documentation is here: http://www.handle.net/. Servers running CKAN could host a "Local Handle Service", which redirects a hash of a resource to an actual URL.

Some suggested use cases:

Researcher would like to cite where data came from
Agencies would like to have a way to prevent vendor lock-in from CKAN if they decide to move to another platform

#1179: Support tag aliases

timmcnamara — Tue, 07 Jun 2011 06:47:01 GMT

A small number of tags are near-duplicates of each other.

Perhaps we could support word stemming from NLTK and/or manual tag aliases:

statistics statistik ... survey surveying surveys

#1184: Support Wuala as CKAN storage option

timmcnamara — Tue, 14 Jun 2011 06:59:11 GMT

Most of CKANs storage options are tied to the USA. This brings concerns of data security for some organisations who may wish to adopt the system. Wuala is a distributed file system that stores data in a peer-to-peer manner. The company behind it, LaCie? sells storage for a fee. However, they also enable clients to have 'free' storage space when machines act as a storage node.

In order to be a storage node, a machine needs to be online for more than 14% of the time - roughly 4h per day. Most CKAN servers are likely to have a far greater uptime than this.

Supporting Wuala would go some way to enabling CKAN to be used in a secure manner. That is, CKAN could be promoted for organisational use where there is lots of data to be stored and large geographic distances to be managed. There is a Python client available and a fairly long Google Tech Talk that overviews the system.

#1144: Support DSPL

timmcnamara — Thu, 19 May 2011 00:06:49 GMT

DSPL, the Dataset Publishing Language, is being promoted by Google for its "Google Public Data Explorer" system. It is an XML format with metadata.

The format is described on the developer docs ofthe Google Code site.

Google provides a Python script which reads CSV data and generates DSPL

Sample from http://code.google.com/apis/publicdata/docs/dspl_sample.html:

<?xml version="1.0" encoding="UTF-8"?>
<dspl xmlns="http://schemas.google.com/dspl/2010"
    xmlns:geo="http://www.google.com/publicdata/dataset/google/geo"
    xmlns:geo_usa="http://www.google.com/publicdata/dataset/google/geo/us"
    xmlns:time="http://www.google.com/publicdata/dataset/google/time"
    xmlns:quantity="http://www.google.com/publicdata/dataset/google/quantity"
    xmlns:entity="http://www.google.com/publicdata/dataset/google/entity">
  <import namespace="http://www.google.com/publicdata/dataset/google/time"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/quantity"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/entity"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/geo"/>
  <info>
    <name>
      <value>My statistics</value>
    </name>
    <description>
      <value>Some very interesting statistics about countries</value>
    </description>
    <url>
      <value>http://www.stats-bureau.com/mystats/info.html</value>
    </url>
  </info>
  <provider>
    <name>
      <value>Bureau of Statistics</value>
    </name>
    <url>
      <value>http://www.stats-bureau.com</value>
    </url>
  </provider>
  <topics>
    <topic id="geography">
      <info>
        <name><value>Geography</value></name>
      </info>
    </topic>
    <topic id="social_indicators">
      <info>
        <name><value>Social indicators</value></name>
      </info>
      <topic id="population_indicators">
        <info>
          <name><value>Population indicators</value></name>
        </info>
      </topic>
      <topic id="poverty_and_income">
        <info>
          <name><value>Poverty & income</value></name>
        </info>
      </topic>
      <topic id="health">
        <info>
          <name><value>Health</value></name>
        </info>
      </topic>
    </topic>
  </topics>
  <concepts>
    <!-- As noted in the tutorial, this concept should extend quantity:amount.-->
    <concept id="population">
      <info>
        <name>
          <value>Population</value>
        </name>
        <description>
          <value>Size of the resident population.</value>
        </description>
      </info>
      <topic ref="population_indicators"/>
      <type ref="integer"/>
    </concept>
    <!-- This country concept is defined for educational purposes only. A country
    concept exists in the Google geo dataset. See:
    http://code.google.com/apis/publicdata/docs/canonical/geo.html -->
    <concept id="country" extends="geo:location">
      <info>
        <name>
          <value>Country</value>
        </name>
        <description>
          <value>My list of countries</value>
        </description>
      </info>
      <type ref="string"/>
      <property id="name">
        <info>
          <name><value xml:lang="en">Country name</value></name>
          <description>
            <value xml:lang="en">The official name of the country</value>
          </description>
        </info>
        <type ref="string"/>
      </property>
      <table ref="countries_table"/>
    </concept>
    <!-- This US state concept is defined for educational purposes only. A US state
      concept exists in the Google geo US dataset. See:
      http://code.google.com/apis/publicdata/docs/canonical/geo.us.html -->
    <concept id="state" extends="geo:location">
      <info>
        <name>
          <value>State</value>
        </name>
        <description>
          <value>US states</value>
        </description>
      </info>
      <type ref="string"/>
      <property concept="country" isParent="true"/>
      <table ref="states_table"/>
    </concept>
    <concept id="gender" extends="entity:entity">
      <info>
          <name>
          <value>Gender</value>
        </name>
        <description>
          <value>Gender, Male or Female</value>
        </description>
        <pluralName><value>Genders</value></pluralName>
        <totalName><value>Both genders</value></totalName>
      </info>
      <type ref="string"/>
      <table ref="genders_table"/>
    </concept>
    <concept id="unemployment_rate" extends="quantity:rate">
      <info>
        <name>
          <value>unemployment rate</value>
        </name>
        <description>
          <value>The percent of the labor force that is unemployed, not seasonally
            adjusted.</value>
        </description>
        <url><value>http://www.bls.gov/cps/cps_htgm.htm</value></url>
      </info>
      <topic ref="social_indicators"/>
      <type ref="float"/>
      <attribute id="is_percentage">
        <type ref="boolean"/>
        <value>true</value>
      </attribute>
    </concept>
  </concepts>
  <slices>
    <slice id="countries_slice">
      <dimension concept="country"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <table ref="countries_slice_table"/>
    </slice>
    <slice id="states_slice">
      <dimension concept="state"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <metric concept="unemployment_rate"/>
      <table ref="states_slice_table"/>
    </slice>
    <slice id="countries_gender_slice">
      <dimension concept="country"/>
      <dimension concept="gender"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <table ref="countries_gender_slice_table"/>
    </slice>
  </slices>
  <tables>
    <table id="countries_table">
      <column id="country" type="string"/>
      <column id="name" type="string"/>
      <column id="latitude" type="float"/>
      <column id="longitude" type="float"/>
      <data>
        <file format="csv" encoding="utf-8">countries.csv</file>
      </data>
    </table>
    <table id="countries_slice_table">
      <column id="country" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
      <data>
        <file format="csv" encoding="utf-8">country_slice.csv</file>
      </data>
    </table>
    <table id="states_table">
      <column id="state" type="string"/>
      <column id="name" type="string"/>
      <column id="country" type="string">
        <value>US</value>
      </column>
      <column id="latitude" type="float"/>
      <column id="longitude" type="float"/>
      <data>
        <file format="csv" encoding="utf-8">states.csv</file>
      </data>
    </table>
    <table id="states_slice_table">
      <column id="state" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
      <column id="unemployment_rate" type="float"/>
      <data>
        <file format="csv" encoding="utf-8">state_slice.csv</file>
      </data>
    </table>
    <table id="genders_table">
      <column id="gender" type="string"/>
      <column id="name" type="string"/>
      <data>
        <file format="csv" encoding="utf-8">genders.csv</file>
      </data>
    </table>
    <table id="countries_gender_slice_table">
      <column id="country" type="string"/>
      <column id="gender" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
      <data>
        <file format="csv" encoding="utf-8">gender_country_slice.csv</file>
      </data>
    </table>
  </tables>
</dspl>

#1823: Spring clean bin directory

rgrp — Mon, 20 Feb 2012 21:28:51 GMT

Huge number of accumulated (and likely unnecessary) scripts in /bin directory.

#2486: Should be able to use . in dataset names

seanh — Sat, 02 Jun 2012 16:46:23 GMT

#924: Search box has no search button

dread — Mon, 24 Jan 2011 11:12:13 GMT

The search box at the top-right of CKAN's page doesn't have a 'go' button. I feel that a larger percentage of users expect a 'go' or 'search' button on the right-hand side of the box to press to start searching. Techies tend to know the keyboard shortcut of pressing 'carriage-return' but it might be better to follow standard practise on this.

Examples with 'search' button: Internet Explorer, Firefox, Google, Amazon, trac Examples without: ?

#2962: Search across multiple ckan instances

dominik — Mon, 08 Oct 2012 22:50:22 GMT

Could be done by:

using the solr distributed search
- difficult set up
merging result sets from apis
- make sure that results can be merged properly (score, facets, ...)

#818: Rethinking the author and maintainer fields

cygri — Wed, 17 Nov 2010 14:18:44 GMT

The semantics of the Author and Maintainer fields are really unclear at the moment. This leads to very inconsistent usage. Also, perhaps Name and Email are not the only fields that are needed for a contact.

Here is a table that shows the current usage of these fields in CKAN: http://richard.cyganiak.de/2010/ckan/ckan-ppl.html

We note several problems:

Author and Maintainer are often the same
Author and Maintainer are often used interchangeably
People really want to specify URLs for the contacts and stick them into random places because there is no field for it
Multiple comma-separated names in a single field

I'm not sure what to do about this, but a redesign is necessary in my opinion.

Some ideas:

Remove the maintainer field?
Make really clear that Author doesn't refer to the metadata on CKAN, but to the original data
Add an “author URL” field?

#2247: Resource preview glitch in some browsers

zephod — Tue, 20 Mar 2012 12:30:14 GMT

From Ira: Preview for google spreadsheets are not displaying correctly for me in Firefox v.10.0.02, fine in Chrome. http://i.imgur.com/KJaqz.png

#1286: Remove remaining formalchemy stuff

rgrp — Tue, 23 Aug 2011 16:29:56 GMT

Stuff I've spotted:

forms/*
template/group/edit_form.html
template/package/edit_form.html

This can go once new DGU form is in.

#1260: Remove duplicate functions from _util.html

pudo — Wed, 03 Aug 2011 10:17:32 GMT

There seems to be both a list view for dictized and non dictized data structures for all entities in _util.html at the moment. Probably in the back of someone's mind already, but cleanup here would be nice.

#1598: Reinstate Ratings

rgrp — Thu, 29 Dec 2011 16:52:04 GMT

Ratings were disabled approximately a year ago because:

Unclear purpose and UX. What did ratings tell you? How useful were they?
Spamming (esp by bots: you could submit an anonymous rating via a GET request which caused problems)

Both problems are solvable and it would be nice to have this feature reinstated.

Purpose: can make this more purposable by limiting to logged in users (or at least distinguishing logged in from non-logged in users)
- Even better we could allow ratings to be made public (I'm interested in what someone else I respect finds important)
Spamming: limit to logged in users and / or use AJAX over an API to submit ...

#2310: Refactor the search-query construction in feeds.py

icmurray — Mon, 16 Apr 2012 16:01:31 GMT

The feeds controller, used to construct atom feeds, duplicates code found in the package controller's search action (in order to construct the custom feed).

Refactor this to remove duplication.

#1403: Refactor groups index page

zephod — Mon, 17 Oct 2011 10:31:52 GMT

Groups are listed alphabetically with paging - not an ideal user experience. We would like to list groups in order of 'popularity': The number of datasets they contain.

Following this chain of thought, then, it would be nice to rearrange the groups table by clicking on column headers and having it sort by that column.

Furthermore, then, we'd like to implement a full-fledged groups search feature (if this is at all feasible).

The forthcoming groups refactor will probably have some bearing on this task.

#1278: Refactor authorized_query calls

amercader — Mon, 15 Aug 2011 13:32:57 GMT

There are some functions that still use the Auhtorizer().authorized_query method:

./ckan/controllers/authorization_group.py:24:        query = ckan.authz.Authorizer().authorized_query(c.user, model.AuthorizationGroup)
./ckan/lib/base.py:237:        groups = ckan.authz.Authorizer.authorized_query(c.user, model.Group,
./ckan/lib/search/sql.py:55:        q = authz.Authorizer().authorized_query(username, model.Group)
./ckan/lib/search/sql.py:118:        q = authz.Authorizer().authorized_query(self.options.get('username'), model.Package)
./ckan/logic/action/get.py:154:    query = Authorizer().authorized_query(user, model.Group, model.Action.EDIT)
./ckan/tests/test_authz.py:158:        q = self.authorizer.authorized_query(self.notadmin.name, model.Package)
./ckan/tests/test_authz.py:353:        q = self.authorizer.authorized_query(self.notmember.name, model.Package)
./ckan/tests/test_authz.py:357:        q = self.authorizer.authorized_query(self.member.name, model.Package)
./ckan/tests/functional/test_authorization_group.py:44:        group_count = Authorizer.authorized_query(u'russianfan', model.AuthorizationGroup).count()

#1406: Re-enable RSS subscriptions

zephod — Mon, 17 Oct 2011 14:22:07 GMT

RSS 'subscribe' buttons appeared in many places on the site but were not very helpful. They took (confused) users pointed to the raw feed code, and Google Reader could not understand the feed. Safari, however, could interpret it correctly.

Their presentation needs to be clear and consistent; the RSS feed really needs testing in a variety of readers; and we need to decide exactly which items should get a feed. (Package updates? Groups?)

#1584: QA report improvements - 2.5d

johnglover — Wed, 21 Dec 2011 09:23:53 GMT

Super: #1594

qa/{username}
qa/{groupname}
paginate QA results
search / filter QA results
~~list organisation report by default, but can disable via config option~~ (done)
UX tidy up of report pages - hide border if no sidebar, etc

#1588: QA - Give SPARQL endpoints a 4 star rating

johnglover — Wed, 21 Dec 2011 15:13:25 GMT

Super: #1594

From Richard Cyganiak on the CKAN Discuss list:

Besides considering the media type of resources, it would also make sense to check for the presence of a SPARQL endpoint. SPARQL endpoints are recorded for more than 300 datasets on the Data Hub using the pseudo-type "api/sparql". A few more are recorded with the format "SPARQL". I suggest that datasets with such resources should also be considered for the fourth star.

#1589: QA - Give 5 star rating to datasets with link metadata

johnglover — Wed, 21 Dec 2011 15:16:40 GMT

Super: #1594

From Richard Cyganiak on the CKAN Discuss list:

Regarding the fifth star (is the dataset linked to others?). This cannot be automatically determined just by looking at the format. It either requires inspection of the actual data, or information about links in the metadata. As you're probably aware, we've established conventions for recording information on data links in CKAN [1], as part of the work of the lodcloud group on the Data Hub. Link information is captured for hundreds of datasets. I would claim that we have the majority of four-star datasets covered there, and hence you can determine if they should get the fifth star by checking for the presence of a links:xxx field.

#1198: Publisher hierarchy

dread — Thu, 23 Jun 2011 09:16:32 GMT

'Publisher' entities in the model. They are hierarchical.

'User-Publisher' connections with one or more roles (e.g. drafter, moderator).

Authorization settings can control who can set what values in a 'published by' type field.

Publishers and User-Publishers available to read in the API.

Future tickets will provide:

API to write Publishers and User-Publishers
UI to edit Publishers and User-Publishers

(This feature deprecates authorization groups)

#1429: Provide DOIs for datasets in a CKAN instance

rgrp — Sun, 30 Oct 2011 12:21:45 GMT

DOI = digital object identifier = http://www.doi.org/

As a Publisher I want a DOI for my dataset so that it can be cited by and linked to by others in a standard and easy way.

Details

Probably implement as extension rather than core

#2960: Private datasets are not private

dominik — Mon, 08 Oct 2012 11:04:15 GMT

At the moment everyone can read all resources. There is no support for hiding resources that are in private datasets.

#1535: Plump for auth header of: X-CKAN-API-KEY

dread — Wed, 07 Dec 2011 17:31:22 GMT

When using the API, the apikey needs to be supplied in a header called 'Authorization'. Because some proxys / deployments use this header for other things, a configurable header was provided as an alternative, with default "X-CKAN-API-KEY".

Rufus suggests having *one* way for this. a) making this not configurable any more b) making X-CKAN-API-KEY the default

(keep Authorization allowed, but not documented, for backwards compatibility)

#1355: Package extras property does not include the newly created ones

amercader — Mon, 26 Sep 2011 10:48:46 GMT

The extras in the package object sent to the extensions after editing (https://bitbucket.org/okfn/ckan/src/01efd5649c10/ckan/logic/action/update.py#cl-226) do not include the newly added.

#1288: Package edit/creation can't include 'relationships' field

dread — Wed, 24 Aug 2011 16:34:55 GMT

When you create or edit a package (via the API), you aren't able to specify the relationships it has. (If you do you get 409 {"__junk": ["The input field __junk was not expected."]} )

The normal way to create relationships is via /api/rest/relationships/ and this works. But when you GET a package, the dictionary lists all relationship details. So this bug creates a problem for editing a package that has relationships - you want to GET it, make any edits and then PUT it back. The work-around is to delete the 'relationships' key from the dict before you PUT it back.

Options

Ideally, CKAN would read the 'relationships' key and make the necessary changes. This is a chunk of work.

Another good option is to allow an unchanged 'relationships' value, but barf it is edited. This is also a chunk of work.

A bad option would be to just ignore the 'relationships' value, since users will get frustrated changing this value and wonder why it never saves, not understanding it is different to all the rest, without error message.

A final option is to get rid of relationships altogether.

#812: Package edit form only allows three extra fields

cygri — Wed, 17 Nov 2010 11:56:50 GMT

Rationale

The package edit form is restricted to three extra fields. To enter more than three fields, one has to save the package and hit edit again (or hit preview).

Implementation

A mechanism similar to the one for resources (where you can add lines as you go) would solve this. So, have a button that adds more extra field rows via JS. (Extra fields don't need up/down buttons that the Resource table has)

Nice to have: a blank field is added when you tab from the last filled-in field in the table.

#1644: Order default dataset page by most downloaded resources on thedatahub

shevski — Thu, 12 Jan 2012 18:39:02 GMT

Instead of alphabetically as we do currently, alternatively by most viewed datasets

for http://thedatahub.org/dataset

#1424: Openness notice should be clearer

dread — Wed, 26 Oct 2011 16:54:49 GMT

ckan-discuss discussion suggests changes to the 'openness' indicator ( http://lists.okfn.org/pipermail/ckan-discuss/2011-October/001786.html )

Dataset view page:

If there is an explicit but non-OKD compliant license, such as CC-BY-NC, then this should be stated explicitly, perhaps: “This dataset is Not Open. License: Creative Commons Attribution Noncommerical. This is not an open license as it does not meet the Open Knowledge Definition.”
If the license is marked as “Other::License Not Specified”, then this should be stated explicitly, perhaps: “This dataset is Not Open. It is published without an explicit license, the publisher reserves all rights to the dataset.”
3. If the license field was left empty by the contributor of the Data Hub record, then again this should be stated explicitly, perhaps: “This dataset is Not Open. The license of this dataset is unknown or unspecified. Start an enquiry on IsItOpenData? »
There is a bug so that non-open licenses doesn't have an openness notice.
If downloadable resources are not available, this should not affect 'openness' - check this has been removed.

#140: News section on front page

rgrp — Wed, 07 Oct 2009 08:02:21 GMT

Have a news section (suggest as a sidebar item).

News section will link to latest 3/4 blog posts on CKAN from blog.okfn.org.

Details:

Suggest pulling via rss or similar.
Will want to cache this ...

Cost: 4h?

#2531: New state option: archived / deprecated

rgrp — Fri, 15 Jun 2012 08:59:47 GMT

Deleted means things will get purged at some point.

Archived means they stay around but get hidden from search results and a big warning notice gets displayed saying this is archived / deprecated.

@richard cyganiak ...

#1466: Need to support https login for multiple instances as part of the CKAN package install

thejimmyg — Tue, 15 Nov 2011 16:52:58 GMT

#1077: Move to simpler vdm system

rgrp — Fri, 08 Apr 2011 23:14:24 GMT

Option 1: 'Changeset' Model

See ticket:1135 for vdm ticket. This would involve a) moving to changeset in vdm b) doing the migration in ckan to support this.

Have developed a new "changeset" based model for revisioning in vdm.

Implementation

The main challenge with this change is schema and data migration

Every revisioned object has a revision_id and revision attribute.

Approximate algorithm:

Revision -> Changeset
for revtype in [PackageRevision, ...]:
    for pkgrev in package_revision:
        changeset = lookupchangeset(package_revision)
        ChangeObject(cset, (table, id), dictize(pkgrev))

Question:

does pkg include tags attributes or not? or we have to dictize, pkgrev, pkg2tagrev, and tag. Probably the latter.

Option 2: Simplify Revision Object Model

Just use a simpler vdm, see ticket:1136 (move to SessionExtension) and ticket:1137 (remove need for statefulness in vdm).

Discussion

Advantage of Option 1 versus 2:

Easier support for pending state and similar behaviour
No need to introduce new tables (and hence migrations) when making something revisioned (or not).

Disadvantages

Migration is required
More difficult to query revision history.
- Could be addressed by having ChangeObject have separate cols for table name and id but would likely be more difficult.
Performance (?)
- Have one big ChangeObject table to query when looking at changed objects rather than many revision tables.
  - Not sure this is a biggie as even with Revision model biggest revision object tables are probably on the order of the ChangeObject table

Conclusion

Implement Option 2 and leave Option 1 for present.

Option 1 includes Option 2 so it seems that that is required in either case (so we may as well with Option 2).

Option 1 requires significant effort (esp migration) so leave for present and then review the situation at some later date.

#1750: Move ckan/lib/activity.py into the model

seanh — Sun, 05 Feb 2012 18:18:08 GMT

Move ckan/lib/activity.py moved to into the model - say ckan/model/activity_extension.py, because it's so tightly knit with the model code, whereas most of the lib code is used in the controllers.

#1203: Moderated edits: html code shows as "changed" although it is not

rolf — Tue, 28 Jun 2011 15:29:30 GMT

I've installed the Moderated Edits extension (ckanext-moderatededits) and am editing a package imported from IATIregistry.org, with an extra field which contains a bit of HTML.

The editor indicates the field has changed, although the content hasn't (see screenshot). All I can find so far is a minor difference: in the field content, there is a code &#8212 and in the rendered table that is an —

#1311: Modal user register and login form

rgrp — Tue, 06 Sep 2011 08:20:27 GMT

Subticket of: #1294

Rather than having to visit a dedicated page it would be good if registration and login could be done from a modal form (separate or combined ...).

Why

It could be used from dataset creation page in situations where user needs to be registered / logged in to create a dataset (so we could allow someone to start creating a dataset and only get them to login at the end ...)
It allows for quicker and easier logging in

Implementation

See Friedrich's work on the datahub

#737: Markdown syntax summary page

dread — Fri, 22 Oct 2010 16:59:09 GMT

I suggest we produce a quick Markdown cheat-sheet page, showing the key runes: e.g. create a title and quote some text. This page can link to the full Markdown docs for advanced users.

A user going to the Markdown docs that we link will have to read a couple of pages of the raison-d'etre of Markdown before he gets to the syntax. And it's not very easy to read, and being white on black it looks like proper geek stuff.

#2508: Make it possible to run CKAN tests for each language

seanh — Mon, 11 Jun 2012 10:42:15 GMT

Mistakes in translated strings can cause CKAN to crash or otherwise not work, but it's not practical to manually test every page and function of CKAN in every language that we have new translations for before a CKAN release. It'd be great if the tests could automatically be run for each language.

This is probably a big job, we would have to get the tests to respect a language setting in the ini file, check for any individual test cases that specify the language (e.g. in the URL), and also fix test cases that look for specific English words in HTML output, etc.

In the meantime, a good stop-gap solution might be a script that tests for common mistakes in the po files.

#1336: License fudge

dread — Tue, 13 Sep 2011 11:07:37 GMT

cset:4b59ab34137d ckan/logic/action/get.py:

-            isopen = model.Package.get_license_register()[license_id].isopen()
-            result_dict['isopen'] = isopen
+            try:
+                isopen = model.Package.get_license_register()[license_id].isopen()
+                result_dict['isopen'] = isopen
+            except KeyError:
+                # TODO: create a log message this error?
+                result_dict['isopen'] = False

This change hides problems with the license server and returns potentially incorrect values for openness.

This has been noted as 'temporary fix' but seems to be forgotten about, since it has been merged to default and gone into release 1.4.3.

I suggest the licenses are cached (I thought this was already the case when CKAN first requests them after start-up?). I suggest failure would return 503.

#1261: Investigate dots in extras search

pudo — Wed, 03 Aug 2011 10:19:28 GMT

It seems that searching for extras_foo:value works with solr, but extras_foo.bar:value doesn't. No theory on why.

#1101: Integrate googlanalytics into site nav

sebbacon — Thu, 21 Apr 2011 13:52:06 GMT

There's a stats plugin (e.g. at http://trac.ckan.org/ticket/832).

Output from the googleanalytics plugin should append to that page, if the stats plugin is present.

Possibly the stats plugin and the googleanalytics plugin should be merged?

Finally, if the stats plugin is active, then a link to the stats page should be added to the main site footer.

#1009: Improvements to user accounts sytem

rgrp — Fri, 25 Feb 2011 12:13:11 GMT

Forgot password (email a new password)
Confirm email
Do not show register page if you are logged in (redirect to home page)
ticket:1010 - listing of users.
- Do not use /user for general user account home page (either user normal user page /user/{id} or /user/myaccount)

#1163: Improvements to Storage Extension

rgrp — Thu, 26 May 2011 11:19:38 GMT

Storage is now working but there are

Integrate with Resources (e.g. create a resource for each file upload and give option to associate with a package)
- Should we introduce rule that files *not* associated with a Resource are periodically deleted?
Allow setting of a file name/path before upload
Allow for file overwriting/deleting etc (how should this work -- do we want to allow this sort of thing)
Integrate local file upload stuff in api/auth/*

Different Backend Issues

Local file store is rather different from 'remote' storage in various ways:

For remote you don't want to use many buckets as there are bucket limits while for local you want to. Should we there have a single path that users provide which we then partition differently for different backends.

#351: Homepage: list new, updated and 'hot' packages

dread — Tue, 15 Jun 2010 09:56:56 GMT

Have a simpler list of exciting data, as opposed to the big revision list.

For example:

Hot data
===========
New packages: package1, package2, package3
Updated resources: package1, package2, package3
Popular packages:

#1604: Get ckanext-moderatededits working with CKAN 1.5+ templates

dread — Sat, 31 Dec 2011 17:27:09 GMT

ckanext-moderatededits requires an old and possibly development version of CKAN. It would be good to update it for later CKAN versions.

According to the README, you need CKAN from branch feature-1141-moderated-edits-ajax but the changelog suggests this branch went into version 1.4.2. So it possibly works with 1.4.2 and 1.4.3(.1). But CKAN 1.5 has revamped templates, so the genshi stream filters definitely don't work.

BTW history_ajax/read_ajax calls have been deprecated in CKAN since 1.5.2a and will need fixing up to use the Action API too as part of this.

#1411: Force resource format to be lower case (also mimetype)

rgrp — Sat, 22 Oct 2011 21:40:04 GMT

Format should be lowercase. Automatically lower case (for extra points have a bit of javascript to force lower case when entering).

Even more points: do a update on thedatahub repo to make all format lower case (or script this as an update?)