Ticket #1190 (new enhancement) — at Version 3

Opened 3 years ago

Last modified 2 years ago

[super] CREP 0004 Data API and Data Processing System

Reported by: rgrp Owned by: rgrp
Priority: awaiting triage Milestone: ckan-v1.5
Component: ckan Keywords:
Cc: Repository: ckan
Theme: none

Description (last modified by rgrp) (diff)

For some time (e.g. 1y+!) we have known that we want to integrate some kind of datastore / data processing system with CKAN. We've had a CREP in progress on this for some months (may copy that here at some point):

<http://bitbucket.org/okfn/ceps/src/tip/cep/cep-0004.txt>

We can distinguish 3 modules that are needed:

  1. "Webstore": A datastore with dataapi - #1208

Suggestion is this would be sqlite based with a simple sql based API. http://ckan.net/api/data/{user|org}/{datastore_name}?q={some-read-sql-query}

  1. A data processing system which utilizes this datastore. One could

get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules)

  1. A bootstrap process to convert CKAN resources into new resources

stored in our datastore so that they are accessible via the API.

More info

Change History

comment:1 Changed 3 years ago by rgrp

  • Description modified (diff)
  • Summary changed from [super] Data API and Data Processing System to [super] CREP 0004 Data API and Data Processing System

comment:2 Changed 3 years ago by rgrp

  • Description modified (diff)

comment:3 Changed 3 years ago by rgrp

  • Description modified (diff)
Note: See TracTickets for help on using tickets.