Ticket #1190 (new enhancement) — at Initial Version
[super] Data API and Data Processing System
| Reported by: | rgrp | Owned by: | rgrp |
|---|---|---|---|
| Priority: | awaiting triage | Milestone: | ckan-v1.5 |
| Component: | ckan | Keywords: | |
| Cc: | Repository: | ckan | |
| Theme: | none |
Description
For some time (e.g. 1y+!) we have known that we want to integrate some kind of datastore / data processing system with CKAN. As you may recall we've had a CREP in progress on this for some months:
<http://bitbucket.org/okfn/ceps/src/tip/cep/cep-0004.txt>
I think we are now close to having an agreed way ahead. I think we can distinguish 3 modules that are needed:
- A datastore with dataapi.
Suggestion is this would be sqlite based with a simple sql based API. http://ckan.net/api/data/{user|org}/{datastore_name}?q={some-read-sql-query}
- A data processing system which utilizes this datastore. One could
get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules)
- A bootstrap process to convert CKAN resources into new resources
stored in our datastore so that they are accessible via the API.
