Changes between Initial Version and Version 2 of Ticket #1797


Ignore:
Timestamp:
02/29/12 21:16:59 (2 years ago)
Author:
rgrp
Comment:

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #1797

    • Property Keywords [4d] added

  • Ticket #1797 – Description

    initial v2  
     1Main tasks: 
     2 
     3 * Integrated seamlessly into CKAN (authorization etc) 
     4 * Based off ElasticSearch 
     5 
    16Etherpad: http://ckan.okfnpad.org/webstore 
    27 
    3 == Questions == 
    4  1. Authorization (and url structure) 
    5     * Authorization API? 
    6  2. How do we integrate webstore into main site 
    7    * /dataset/{xyz}/resource/{r-id}/data => /api/rest/dataset/{xyz}/resource/{r-id}/data 
    8    * /dataset/{xyz}/resource/{r-id}/raw <= raw / bulk version of this data ... 
    9  3. What query API do we use  
    10   * We just use elasticsearch 
    11   * https://github.com/okfn/recline/issues/34 
     8= Tasks = 
    129 
    13 == Details == 
     10 * New API point (config + controller) [0.75d] 
     11   * /api/data/{resource-id} 
     12 * Authorization [0.5d] 
     13 * Documentation [0.25d] 
     14 * Upgrade data viewer to use ES [0.5d] 
     15 * Convert webstorer to new setup - http://github.com/okfn/ckanext-webstorer [1d] 
     16  * See below 
     17 * "Create Webstore Resource" in Resource creation (??) 
     18 * Deploy [0.5d] 
    1419 
    15 Layout on ES: 
     20= Details = 
     21 
     22== Layout on ES == 
    1623 
    1724{{{ 
    … …  
    2229}}} 
    2330 
    24 Request path: 
     31== Request path == 
    2532 
    2633Req => Nginx => CKAN App (authz) => req to elasticsearch and returns results 
    2734 
    2835See as an example: https://github.com/okfn/elastic-proxy 
     36 
     37== Questions == 
     38 
     391. How do we decide what to use in the viewer e.g. Webstore data versus DataProxy 
     40    * Check for existence of webstore_url on Resource (should really be webstore_enabled boolean or similar, however we stick with url for the time being because it exists and we avoid migration). If exists use Webstore o/w don't. 
     412. How do we decide when webstore is activated 
     42    * Ans: see previous. 
     433. Do we (and how do we) allow for resources which are just for webstores 
     44    * Ans: we have to set webstore_url = True ... on save. Or add a big button saying: "Create Blank Resource" ... 
     454. How does webstorer work ... 
     46    * CSV import - NS: make sure we add e.g. rownum or __id__ = rownum ... 
     47    * Separate mini-service (webstorer.thedatahub.org ...) 
     485. How do we selectively disable recline where not appropriate (e.g. query support makes no sense for DataProxy) 
     49    * Ans: do not bother at present. Focus on Webstore route. 
     506. Where does geodata support come in (in Recline or separately) 
     51    * This is a separate issue 
     52 
     53== User Story 1 == 
     54 
     55User -> CKAN instance -> Add Dataset -> Adds a Resource via uploading a CSV file -> Save 
     56 
     57=> Result = New Dataset + Resource + Data API to resource and nice Preview of the data 
     58 
     59Solution: 
     60 
     61 1. Allow normal upload process 
     62 2. Webstorer queue task which should get triggered by new resource. Check the resource and if it can convert into the associated resource webstore database. 
     63 3. Update the resource in some way (set webstore_url) 
     64 
     65= User Stories = 
     66 
     67== User Story 2 == 
     68 
     69User -> CKAN instance -> Add Dataset -> Adds a Resource via linking to some remote file (CSV or otherwise) or uploading some data (CSV or otherwise) 
     70 
     71=> Result = New Dataset + Resource + nice Preview of the data (wherever possible) 
     72 
     73 
     74== User Story 3 == 
     75 
     76User -> CKAN instance -> Add Dataset -> Adds a Resource which will be webstore database in CKAN -> User saves data to this data store. 
     77 
     78=> Result = New Dataset + Resource (Webstore resource) which can now be written to or read from by the user 
     79 
     80Solution: just create a resource with blank url (or better, have a new option to create webstore resource ...)