Reported by dread, 2 years ago.

When you are logged in, you cannot login as a different user before first logging out. (This is a restricting in repoze.who, which gives priority to an existing recognised identity over another one you supply.)

Yet, the CKAN login form appears when you try to do something you're not allowed to, even if you are logged in already. You login with a different identity and nothing happens - you remain as the first identity - this is incorrect.

(It is also a little confusing to be sent to the log-in page, although we do have a flash message explaining, and you might want to relogin as a superuser if you can.)

Whilst in this area, it would be worth fixing the problem for developers where you are logged in, do "paster db clean" and then try to log in - AuthTkt? recognises your old cookie but the User object doesn't exist, so you get an error logging in. In this instance we should tell people to log-out.

Reported by dread, 2 years ago.

You get an exception if you use the API to search packages and specify a non-ascii character in a field other than q.

For example:

http://catalogue.data.gov.uk/api/search/package?title=%E2%80%93

This "N-dash" (Unicode character 2013) causes this exception:

Module ckan.controllers.api:460 in search
<<                      if ver in u'12':
                               # Otherwise, put all unrecognised ones into the q parameter
                               params = convert_legacy_parameters_to_solr(params)
                           query = query_for(model.Package)
                           results = query.run(params)
>>  params = convert_legacy_parameters_to_solr(params)
Module ckan.lib.search.query:38 in convert_legacy_parameters_to_solr
<<      for search_key in non_solr_params:
               value_obj = legacy_params[search_key]
               value = str(value_obj).replace('+', ' ')
               if search_key == 'all_fields':
                   if value:
>>  value = str(value_obj).replace('+', ' ')
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 30: ordinal not in range(128)

This problem affects CKAN 1.5 and 1.5.1 only.

Reported by rgrp, 2 years ago.

Main tasks:

• Integrated seamlessly into CKAN (authorization etc)
• Based off ElasticSearch?

Etherpad: ​http://ckan.okfnpad.org/webstore

Tasks

• New API point (config + controller) [0.75d]
  • /api/data/{resource-id}
• Authorization [0.5d]
• Documentation [0.25d]
• Upgrade data viewer to use ES [0.5d]
• Convert webstorer to new setup - ​http://github.com/okfn/ckanext-webstorer [1d] - #1753
  • See below
• "Create DataStore? Resource" in Resource creation (??)
  • Just creates a resource with blank url and datastore enabled (webstore_url non null)
• Tick box to enable DataStore? in Resource edit form (for time being would just set webstore_url to any non-null/non-empty value)
  • Also need to use this info in DataStore? extension?
• Create nice instructions on Resource page about using the DataStore?
• Rename webstore -> datastore
• Deploy [0.5d]

Details

Layout on ES

elasticsearch/{index}/{type-id}

index = ckan-{ckan-id}
type-id = resource-id (1)

Request path

Req => Nginx => CKAN App (authz) => req to elasticsearch and returns results

See as an example: ​https://github.com/okfn/elastic-proxy

Questions

1. How do we decide what to use in the viewer e.g. Webstore data versus DataProxy?
   • Check for existence of webstore_url on Resource (should really be webstore_enabled boolean or similar, however we stick with url for the time being because it exists and we avoid migration). If exists use Webstore o/w don't.
2. How do we decide when webstore is activated
   • Ans: see previous.
3. Do we (and how do we) allow for resources which are just for webstores
   • Ans: we have to set webstore_url = True ... on save. Or add a big button saying: "Create Blank Resource" ...
4. How does webstorer work ...
   • CSV import - NS: make sure we add e.g. rownum or id = rownum ...
   • Separate mini-service (webstorer.thedatahub.org ...)
5. How do we selectively disable recline where not appropriate (e.g. query support makes no sense for DataProxy?)
   • Ans: do not bother at present. Focus on Webstore route.
6. Where does geodata support come in (in Recline or separately)
   • This is a separate issue

User Story 1

User -> CKAN instance -> Add Dataset -> Adds a Resource via uploading a CSV file -> Save

=> Result = New Dataset + Resource + Data API to resource and nice Preview of the data

Solution:

1. Allow normal upload process
2. Webstorer queue task which should get triggered by new resource. Check the resource and if it can convert into the associated resource webstore database.
3. Update the resource in some way (set webstore_url)

User Stories

User Story 2

User -> CKAN instance -> Add Dataset -> Adds a Resource via linking to some remote file (CSV or otherwise) or uploading some data (CSV or otherwise)

=> Result = New Dataset + Resource + nice Preview of the data (wherever possible)

User Story 3

User -> CKAN instance -> Add Dataset -> Adds a Resource which will be webstore database in CKAN -> User saves data to this data store.

=> Result = New Dataset + Resource (Webstore resource) which can now be written to or read from by the user

Solution: just create a resource with blank url (or better, have a new option to create webstore resource ...)