CKAN: Ticket Query

#1830: OpenID disablable

dread — Thu, 23 Feb 2012 13:02:17 GMT

Allow OpenID to be disabled as a login option.

Currently (CKAN 1.6) there is a config option openid_enabled which stops the login option being displayed, but:

In the User edit form you can still add your OpenID
The repoze.who.openid middleware is still required - can it be avoided?

#1829: Changing back to English prints the flash message in the previous non-English language

dread — Thu, 23 Feb 2012 12:37:40 GMT

On the homepage click "francais" and then "English". The flash message reads "Le langage a été fixé à: français" when it should say "The language is now: English".

#1828: Upgrade to Recline v0.3

rgrp — Wed, 22 Feb 2012 20:41:44 GMT

Recline v0.3 has introduced some breaking changes in API and usage. Upgrade to use it.

#1826: weight queries so that title is more important than rest of body

kindly — Wed, 22 Feb 2012 15:24:20 GMT

Currently everything is considered equally when doing a normal search. Use dismax when there is not a fielded search to get round this. This is part of the work for #1822 as the dismax option is needed for multilingual search.

#1825: Write docs for vocabularies API

seanh — Tue, 21 Feb 2012 17:25:22 GMT

Done on branch feature-1698-tag-taxonomies

#1822: Weight mulilingual searches correctly

kindly — Mon, 20 Feb 2012 21:08:00 GMT

Dismax query across languages. Titles and selected languages, weighted more highly.

#1821: create internal multilingual extension that brings together all multilingual features

kindly — Mon, 20 Feb 2012 20:54:27 GMT

This extension point needs to combine.

Making sure the correct data get put in search index.
Make the search queries are modified to weight multilingual search correctly.
View pages, including search results are translated.
Bring together all search schema modifications.

#1817: Importer for EC publisher info

johnglover — Mon, 20 Feb 2012 18:24:50 GMT

write importer for publisher info (currently in JSON format), add to ckanext-ecportal
add group translations to translations table
run on test server

#1813: Upadate ckanext-pdeu to work with CKAN 1.6

amercader — Mon, 20 Feb 2012 16:58:48 GMT

Tasks include:

Estimate 4d

Update requirements (git)
Search facets error
Error on dataset read (extras in context)
Url generator for rdf
Make templates work with latest theme
Use dicts in templates (isopen())

#1809: Catch request exceptions in archiver link_checker task

johnglover — Mon, 20 Feb 2012 13:57:47 GMT

Some request exceptions are currently not being caught (see the celery log on thedatahub for examples)

#1807: Recently added/updated datasets for EC portal

seanh — Mon, 20 Feb 2012 12:52:06 GMT

For EC Portal need a page (or HTML snippet embedded in the front page?) listing recently added or updated datasets.

Could be generated from the activity streams

Should it list datasets only, or other activities as well?

May be related to #1785 (Replace 'Revisions' page with site-wide activity stream)

#1806: make drop down language selector

toby — Mon, 20 Feb 2012 12:36:33 GMT

language chooser should look like the ec one

#1804: CKAN mounted at URL - changing language problem

dread — Fri, 17 Feb 2012 13:22:36 GMT

e.g. http://189.9.137.65/dados/ clicking on Deutsch link is http://189.9.137.65/dados/locale?locale=de&return_to=%2Fdados%2F&hash=1dc17c315c419df850da0dd3599eefa9da76fbeb and redirect goes to http://189.9.137.65/dados/dados/?__cache=97995106 so /dados/dados/ when it should be /dados.

Affects CKAN 1.6b only (not yet released).

#1802: Investigate CKAN deployment on Windows

dread — Thu, 16 Feb 2012 16:25:12 GMT

See how possible it is to run CKAN on a Windows server.

Plan:

Get CKAN code and dependencies installed on Windows
- ideally as similar to standard package install as possible
- virtualenv/pip should work
- try Cygwin to help
- SOLR and Postgres could potentially run on other servers if necessary.
Try to run on Apache
See if there are any issues

#1799: To login as a different user or register, you need to logout first

dread — Thu, 16 Feb 2012 14:33:40 GMT

When you are logged in, you cannot login as a different user before first logging out. (This is a restricting in repoze.who, which gives priority to an existing recognised identity over another one you supply.)

Yet, the CKAN login form appears when you try to do something you're not allowed to, even if you are logged in already. You login with a different identity and nothing happens - you remain as the first identity - this is incorrect.

(It is also a little confusing to be sent to the log-in page, although we do have a flash message explaining, and you might want to relogin as a superuser if you can.)

Whilst in this area, it would be worth fixing the problem for developers where you are logged in, do "paster db clean" and then try to log in - AuthTkt? recognises your old cookie but the User object doesn't exist, so you get an error logging in. In this instance we should tell people to log-out.

#1797: [super] Webstore v2 - Integrated and based on ElasticSearch

rgrp — Wed, 15 Feb 2012 11:43:05 GMT

Main tasks:

Integrated seamlessly into CKAN (authorization etc)
Based off ElasticSearch?

Etherpad: http://ckan.okfnpad.org/webstore

Tasks

~~New API point (config + controller) [0.75d]~~
- /api/data/{resource-id}
~~Authorization [0.5d]~~
~~Documentation [0.25d]~~
~~Upgrade data viewer to use ES [0.5d]~~
~~Convert webstorer to new setup - http://github.com/okfn/ckanext-webstorer [1d] - #1753~~
- See below
"Create DataStore? Resource" in Resource creation (??)
- Just creates a resource with blank url and datastore enabled (webstore_url non null)
~~Tick box to enable DataStore? in Resource edit form (for time being would just set webstore_url to any non-null/non-empty value)~~
- Also need to use this info in DataStore? extension?
~~Create nice instructions on Resource page about using the DataStore?~~
~~Rename webstore -> datastore~~
~~Deploy [0.5d]~~

Details

Layout on ES

elasticsearch/{index}/{type-id}
index = ckan-{ckan-id}
type-id = resource-id (1)

Request path

Req => Nginx => CKAN App (authz) => req to elasticsearch and returns results

See as an example: https://github.com/okfn/elastic-proxy

Questions

How do we decide what to use in the viewer e.g. Webstore data versus DataProxy?
- Check for existence of webstore_url on Resource (should really be webstore_enabled boolean or similar, however we stick with url for the time being because it exists and we avoid migration). If exists use Webstore o/w don't.
How do we decide when webstore is activated
- Ans: see previous.
Do we (and how do we) allow for resources which are just for webstores
- Ans: we have to set webstore_url = True ... on save. Or add a big button saying: "Create Blank Resource" ...
How does webstorer work ...
- CSV import - NS: make sure we add e.g. rownum or id = rownum ...
- Separate mini-service (webstorer.thedatahub.org ...)
How do we selectively disable recline where not appropriate (e.g. query support makes no sense for DataProxy?)
- Ans: do not bother at present. Focus on Webstore route.
Where does geodata support come in (in Recline or separately)
- This is a separate issue

User Story 1

User -> CKAN instance -> Add Dataset -> Adds a Resource via uploading a CSV file -> Save

=> Result = New Dataset + Resource + Data API to resource and nice Preview of the data

Solution:

Allow normal upload process
Webstorer queue task which should get triggered by new resource. Check the resource and if it can convert into the associated resource webstore database.
Update the resource in some way (set webstore_url)

User Stories

User Story 2

User -> CKAN instance -> Add Dataset -> Adds a Resource via linking to some remote file (CSV or otherwise) or uploading some data (CSV or otherwise)

=> Result = New Dataset + Resource + nice Preview of the data (wherever possible)

User Story 3

User -> CKAN instance -> Add Dataset -> Adds a Resource which will be webstore database in CKAN -> User saves data to this data store.

=> Result = New Dataset + Resource (Webstore resource) which can now be written to or read from by the user

Solution: just create a resource with blank url (or better, have a new option to create webstore resource ...)

#1792: [super] api to use same validation and logic as wui forms.

kindly — Mon, 13 Feb 2012 15:21:27 GMT

The api currently uses the default validation schemas regardless of the form you specified on the front end. There should be a way to post through the api using the same validation rules.

#1788: IE7 compatibility problems

zephod — Thu, 09 Feb 2012 14:54:10 GMT

From Antonia Acuno, via Ira:

It appears that the current layout for the datahub does not work in IE7, which would mean a problem for us as most gov uses it and is one of those we have to support...any views?

Tested in several gov departments computers, so a common enough issue.

http://imgur.com/ip4xa

From Rufus: We haven't officially supported IE7 so far but can put engineering effort into this if absolutely needed. I note that IE 7 is now 3y out of date and is likely a security risk. Obviously this is something to ask Antonio about.

#1775: Enable adding many tags at once to controlled vocabularies via API

seanh — Mon, 06 Feb 2012 15:10:56 GMT

We already have API calls for adding one tag or removing one tag at a time to/from a vocab. Want more convenient ways to add or remove many in one call, or just pass in a list of tags to completely replace a vocab's current list.

Add tags_create() function that calls the existing tag_create() many times?
Also tags_delete()?
Or just let tag_create() and tag_delete() accept a list?
Add tags argument to vocabulary_update() to pass in a list of tags to replace the vocab's current tags? (But it should still be possible to simply rename a vocab without changing its tags and without having to pass in the tag list.)

#1757: DGU Hide administrators from public view

ross — Mon, 06 Feb 2012 12:19:35 GMT

On publisher view page - hide administrators from public view - only logged in admins of group should see them.

#1753: Last fixes of Webstorer + Deploy on the DataHub

rgrp — Mon, 06 Feb 2012 11:32:12 GMT

Upgrade to new DataStore? system
Better support for unicode etc
? Rename to DataStorer?
Deploy

#1744: UX improvements to dataset edit as of Dec 2011

rgrp — Thu, 02 Feb 2012 23:17:08 GMT

Super ticket: #1506

~~Show edit summary form again, at least on edit pages (and prefill it for create to something sensible like: Creating dataset {title} {name})~~ [Done #1506]
~~Fix drop down for resource types in resource add/edit to limit to fixed set of options (as per http://wiki.ckan.org/Domain_Model/Resource)~~
- ~~See also http://lists.okfn.org/pipermail/ckan-discuss/2011-December/001860.html~~
Allow reordering of resources (or: and this is somewhat more complex but possibly simpler: allow editing of a resource_order field and we use that to sort resources (this is more like wordpress page_order -- these two options could become the same if we just hijack the existing position field on the join table and get rid of ordereredlist extension)) [DONE #1506]
~~Radically simplify set of licenses~~ [DONE #1359]
~~"Unsaved changes" warning briefly when you click "Save" #1735~~
~~"Unsaved changed" typo on dataset edit page~~

#1670: Publisher profile documentation

ross — Tue, 17 Jan 2012 12:15:49 GMT

Super #1669

Analysis of all permissions for users given their publishers to ensure that we have covered all of the bases and haven't missed any use-cases.

#1653: Change URLs for multilingual site

ross — Mon, 16 Jan 2012 10:43:10 GMT

To support multiple languages we should have an easy way to specify the language as part of the URL, so that URLs are both specific and we also reduce the dependency on the session.

Analysis [1d] - Find the best way of implementing this and how everyone else does their language URLs.

Write Middleware + update url_for to take account of the language. [2d]

Document the language setup, and how to replicate it. [1d]

#1526: [super] Update and test existing DGU package form

icmurray — Tue, 06 Dec 2011 12:08:33 GMT

Re-create behaviour of old form without using form-alchemy. And unit test.

[X] write high-level functional tests for the form. Creation / edition / validation.

Original estimate: 6 days (3 days x 2)

Time spent: 3 days

Time remaining: 0 days

[X] write the html

Original estimate: 2 days (1 day x 2)

Time spent: 1.3 days

Time remaining: 0 days

[X] modify the existing validation schema

Original estimate: 2 days (1 day x 2)

Time spent: 1.3 days

Time remaining: 1 days

[X] split resources into types: individual, time-series, and additional.

Original estimate: 2 days

Time spent: 2.5 days

Time remaining: 0 days

[X] contact information is derived from Publisher, but can be altered for each dataset. #1617

Original estimate: 4 days

Time spent: 0 days

Time remaining: 4 days

(Waiting on publisher integration)

[X] foi contact information is derived from user, but can be altered for each dataset. #1617

Original estimate: 1 days

Time spent: 0 days

Time remaining: 1 days

(Waiting on publisher integration)

[X] provide stubs for the theme and sub-themes #1618

Original estimate: 1 days

Time spent: 0 days

Time remaining: 0 days

[X] provide stubs for the publisher hierarchy #1619

Original estimate: 2 days

Time spent: 0.5 days

Time remaining: 1.5 days

Note - the groups refactor is essentially ready to be used. And this would be a good test case for it.

[X] don't allow package-type selection when editing a package

Can't change a dataset from containing time-series resources to one containing individual resources or visa-versa. (#1620)

Original estimate: 0.5 days

Time spent: 0 days

Time remaining: 0 days

#1506: [super] UX improvements to dataset edit, create and search + group + user page (Dec 2011)

rgrp — Mon, 05 Dec 2011 12:31:46 GMT

Historical Parent Tickets

~~Parent-ticket: #1450~~
~~Parent-ticket: #1032~~

Step Zero

~~COMPLETELY REDESIGN resource edit process~~

Dataset Edit-Resources

~~Force format field to be lowercase~~
- ~~Modify backend to serve it up in that format~~
~~Subticket #978: Add extra fields to resources in Web UI~~
~~Fix display of large datasets (http://test.ckan.net/dataset/europeana-lod)~~
~~When submitting forms with errors, open the correct resource pane.~~
~~No hint is given that the time format for a resource is in ISO, eg 2005-01-01T10:10:10~~
~~Resource description field to become the markdown editor we use everywhere else.~~

Dataset Settings

~~Subticket #1648: Clarify edit terminology~~
~~Merge Groups and Tags into main Settings section (no point having them on their own and most people will miss this).~~
~~Fix (i.e. allow / show) name editing on dataset edit.~~
- ~~Fix the slugify so it does not overwrite if i change title after fixing name.~~
- Make slugify encourage people to use shorter names. Either by cutting off (not so good) or showing a warning and saying something like 'hey you may want to cut this down. Hit edit to customize the dataset name now'
~~Autocomplete the author/maintainer box (requires API extension?)~~ [WONTFIX what? it's not like that field is linked to users of the site or anything]

Dataset View

~~Fix animation glitch in expanding description~~
~~Can we make the list of files look more like a table? Lose space between files? Lose rounded corners?~~
~~Verify that there are no more (none) placeholder <td> objects (i believe they've been refactored out)~~

Dataset Create

~~"Upload or link data files, APIs and other materials related to your dataset."~~
~~Include the description markdown editor again (keep it small)~~
~~Licence should maybe be restored~~
- ~~"(Don't worry if you don't know what your licence is)"~~
~~Show the log message field~~
- ~~Beautify~~
~~Pre-populate the log message field?~~
~~? Show Home Page field when creating dataset~~ [WONTFIX don't push it]
~~? Do some Guerilla UX to see whether we need to Redirect to Resources directly after dataset creation?~~ [WONTFIX doesnt make sense with new resource editor awesomeness]

User pages

~~Improvements to user page UX - 1d - #1396~~
~~Broken links for non-Gravatar use icons - 0.25d - #1719~~

Etc

Take before and after screenshots

#1469: Geo-spatial search on the front end

thejimmyg — Tue, 15 Nov 2011 16:58:44 GMT

Now that we have a geospatial search API, we should build a front-end that integrates with the existing search for displaying results.

#1400: Document Archiver Extension

rgrp — Fri, 14 Oct 2011 13:58:25 GMT

Create page on http://wiki.ckan.org/ about setting up a queue (this will merge into core docs when this is stable) - having a queue will now become a generic requirement for a bunch of extensions
README with install instructions in ckanext-archiver
Create an entry in http://wiki.ckan.org/Extensions
Write a blog post

#978: Edit Resource including Extras in Web UI

dread — Fri, 11 Feb 2011 13:06:59 GMT

Part of super-ticket #1506.

Follows on from #826.

We can configure some extra fields in resources and can edit them in the Web UI, but we can't create new columns in the Web UI. Update the WUI to handle tis.

#906: Ability to search without accents for accented words

Stiivi — Thu, 13 Jan 2011 17:26:28 GMT

In Slovakia users are expecting from sites to be able to search without typing accents, for example if they do not have SK keyboard, just US. For example searching for: 'Obyvateľstvo' and 'obyvatelstvo' (population) or 'štatistika' and 'statistika' (statistics) should yield same results.

This should work the other way around as well, as some people might enter entries without accents into CKAN instance and others might search with accents.

For SK language simple ASCII transliteration for searched term and indexed words is sufficient (iconv ASCIITRANSLIT).

#1151: Preview for geographic data should be a map

timmcnamara — Sun, 22 May 2011 05:46:57 GMT

Data viewer / previewer for a resource that has a KML file, or others, such as GeoRSS and GeoJSON, we should provide a map.

Support for KML files: #1458

This is non-trivial for external files as we need a way to jsonify. For files stored locally this is more of a recline issue (and will require a bit of work to either guess columns or allow user to specify them).