http://localhost/ticket/2733 <h2 id="Wheredoesthedatago">Where does the data go?</h2> <p> In a postgres database configured by the ckan.datastore_write_url config option which is a sqlalchemy url. </p> <p> The user should have rights to create tables. </p> <h2 id="Whatstheapilike">Whats the api like?</h2> <p> We will just implement it as logic functions like the rest of CKAN and will part of core. After that we may add some nicer api functions that use these but that is a secondary concern. </p> <h2 id="Whataretheinitiallogicfunctions">What are the initial logic functions?</h2> <ul><li>datastore_create </li><li>datastore_delete </li><li>datastore_show </li></ul><h2 id="WhatistheJSONinputformatfordatastore_create">What is the JSON input format for datastore_create</h2> <p> To begin with it can have the following keys. It is fairly consistent with Max Ogdens' gut servers. Except adds resource_id. </p> <pre class="wiki">{ resource_id: resource_id # the data is going to be stored against. fields: a list of dictionaries of fields/columns and their extra metadata. records: a list of dictionaries of the data eg [{"dob": "2005", "some_stuff": ['a', b']}, ..] } </pre><ul><li>The first row will be used to guess types not in the fields and the guessed types will be added to the headers permanently. Consecutive rows have to conform to the field definitions. </li><li>rows: can be empty so that you can just set the fields </li><li>fields are optional but needed if you want to do type hinting or add extra information for certain columns or to explicitly define ordering. </li></ul><p> eg: [{"id": "dob", "type": "timestamp" }, {"id": "some_stuff", "type": "text"}, ...]. A header items values can not be changed after it has been defined nor can the ordering of them be changed. They can be extended though. </p> <ul><li>Any error results in total failure!! For now pass back the actual error. </li><li>Should be transactional </li></ul><h2 id="Whatjsondoesdatastore_deletetake">What json does datastore_delete take?</h2> <pre class="wiki">{ resource_id: resource_id # the data is going to be deleted. filters: dictionary of matching conditions to delete e.g {'key1': 'a. 'key2': 'b'} this will be equivalent to "delete from table where key1 = 'a' and key2 = 'b' ". No filters (either not present or not defined) then delete the table. If we want truncate then add truncate: true to truncate the table. } </pre><h2 id="Whatjsondoesdatastore_searchtake">What json does datastore_search take?</h2> <pre class="wiki">{ resource_id: resource_id # the data is going to be selected. filters : dictionary of matching conditions to select e.g {'key1': 'a. 'key2': 'b'} this will be equivalent to "select * from table where key1 = 'a' and key2 = 'b' " q: full text query limit: limit the amount of rows to size default 100 offset: offset the amount of rows fields: list of fields return in that order, defaults (empty or not present) to all fields in fields order. sort: comma separated field names with ordering e.g "fieldname1, fieldname2 desc" } </pre><p> Some free code: <a class="ext-link" href="https://gist.github.com/3163864"><span class="icon">​</span>https://gist.github.com/3163864</a> </p> <h2 id="Whatjsondoesdatastore_searchreturn">What json does datastore_search return?</h2> <pre class="wiki">{ fields: same type as datastore_create accepts (i.e. with metadata) offset: The same offset that was supplied in datastore_show limit: The original limit filters: The filters that were applied in data_show total: # total matching records without size or offset records: [same as data_create] # list of matching results } </pre><p> On error will return: </p> <pre class="wiki">{ __error__: … sql error … } </pre><h2 id="Whattypesareallowed">What types are allowed?</h2> <p> Aim to support as many postgres/postgis types that have string representations. </p> <p> <a class="ext-link" href="http://www.postgresql.org/docs/9.1/static/datatype.html"><span class="icon">​</span>http://www.postgresql.org/docs/9.1/static/datatype.html</a> </p> <p> <a class="ext-link" href="http://www.postgresql.org/docs/9.1/static/sql-createdomain.html"><span class="icon">​</span>http://www.postgresql.org/docs/9.1/static/sql-createdomain.html</a> </p> <h2 id="IDs">IDs</h2> <p> Each row in a table will be given an _id column which has an id generated by us which you can use in queries. </p> <h2 id="OtherFeatures">Other Features</h2> <p> Each row will store the _full_text index of all the data in the row. At some later point there will most likely be a way to index fields add constraints etc. </p> en-us http://assets.okfn.org/p/ckan/img/ckan_logo_shortname.png http://localhost/ticket/2733 Trac 0.12.3 johnglover Mon, 30 Jul 2012 13:48:25 GMT http://localhost/ticket/2733#comment:1 http://localhost/ticket/2733#comment:1 <ul> <li><strong>description</strong> modified (<a href="/ticket/2733?action=diff&amp;version=1">diff</a>) </li> </ul> Ticket