http://localhost/ticket/1190 <p> For some time (e.g. 1y+!) we have known that we want to integrate some kind of datastore / data processing system with CKAN. We've had a CREP in progress on this for some months (may copy that here at some point): </p> <p> <a class="ext-link" href="http://wiki.ckan.org/CEP0004"><span class="icon">​</span>http://wiki.ckan.org/CEP0004</a> </p> <p> We can distinguish 3 modules that are needed: </p> <ol><li>"Webstore": A datastore with dataapi - <a class="closed ticket" href="http://localhost/ticket/1208" title="enhancement: Webstore: a datastore with web API (closed: fixed)">#1208</a> </li></ol><p> Suggestion is this would be sqlite based with a simple sql based API. <a class="ext-link" href="http://ckan.net/api/data/{user|org}/{datastore_name}?q={some-read-sql-query"><span class="icon">​</span>http://ckan.net/api/data/{user|org}/{datastore_name}?q={some-read-sql-query</a>} </p> <ol start="2"><li>Automated conversion of suitable resources into datastore upon resource creation so that e.g. they are accessible via the API. <a class="closed ticket" href="http://localhost/ticket/1398" title="enhancement: Automated conversion of resource data into webstore (closed: fixed)">#1398</a> </li></ol><ol start="3"><li>A data processing system which utilizes this datastore. One could </li></ol><p> get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules) - <a class="new ticket" href="http://localhost/ticket/1432" title="enhancement: [super] Data processing system for CKAN and Webstore (new)">#1432</a> </p> <h2 id="Moreinfo">More info</h2> <ul><li>Overview diagram: <a class="ext-link" href="https://docs.google.com/drawings/d/1XK7dcpFXNlMzVFgLPYPZUXOsFRPzgCEvS-w7FcvydEQ/edit?hl=en_GB"><span class="icon">​</span>https://docs.google.com/drawings/d/1XK7dcpFXNlMzVFgLPYPZUXOsFRPzgCEvS-w7FcvydEQ/edit?hl=en_GB</a> </li></ul> en-us http://assets.okfn.org/p/ckan/img/ckan_logo_shortname.png http://localhost/ticket/1190 Trac 0.12.3 rgrp Thu, 16 Jun 2011 12:34:30 GMT http://localhost/ticket/1190#comment:1 http://localhost/ticket/1190#comment:1 <ul> <li><strong>description</strong> modified (<a href="/ticket/1190?action=diff&amp;version=1">diff</a>) </li> <li><strong>summary</strong> changed from <em>[super] Data API and Data Processing System</em> to <em>[super] CREP 0004 Data API and Data Processing System</em> </li> </ul> Ticket rgrp Mon, 04 Jul 2011 18:34:40 GMT http://localhost/ticket/1190#comment:2 http://localhost/ticket/1190#comment:2 <ul> <li><strong>description</strong> modified (<a href="/ticket/1190?action=diff&amp;version=2">diff</a>) </li> </ul> Ticket rgrp Wed, 06 Jul 2011 12:05:52 GMT http://localhost/ticket/1190#comment:3 http://localhost/ticket/1190#comment:3 <ul> <li><strong>description</strong> modified (<a href="/ticket/1190?action=diff&amp;version=3">diff</a>) </li> </ul> Ticket rgrp Thu, 13 Oct 2011 18:15:44 GMT http://localhost/ticket/1190#comment:4 http://localhost/ticket/1190#comment:4 <ul> <li><strong>description</strong> modified (<a href="/ticket/1190?action=diff&amp;version=4">diff</a>) </li> </ul> Ticket rgrp Tue, 01 Nov 2011 10:20:27 GMT http://localhost/ticket/1190#comment:5 http://localhost/ticket/1190#comment:5 <ul> <li><strong>description</strong> modified (<a href="/ticket/1190?action=diff&amp;version=5">diff</a>) </li> </ul> Ticket rgrp Tue, 01 Nov 2011 10:33:12 GMT http://localhost/ticket/1190#comment:6 http://localhost/ticket/1190#comment:6 <ul> <li><strong>status</strong> changed from <em>new</em> to <em>closed</em> </li> <li><strong>resolution</strong> set to <em>fixed</em> </li> </ul> <p> A very substantial portion of this got done as part of 0.5 including the webstore <a class="closed ticket" href="http://localhost/ticket/1208" title="enhancement: Webstore: a datastore with web API (closed: fixed)">#1208</a> and the majority of automated conversion <a class="closed ticket" href="http://localhost/ticket/1398" title="enhancement: Automated conversion of resource data into webstore (closed: fixed)">#1398</a>. <a class="new ticket" href="http://localhost/ticket/1432" title="enhancement: [super] Data processing system for CKAN and Webstore (new)">#1432</a> is currently deferred so we're going to close this ticket on basis that item (3) is now out of scope. </p> Ticket rgrp Wed, 07 Dec 2011 15:08:42 GMT http://localhost/ticket/1190#comment:7 http://localhost/ticket/1190#comment:7 <ul> <li><strong>description</strong> modified (<a href="/ticket/1190?action=diff&amp;version=7">diff</a>) </li> </ul> Ticket