http://localhost/ticket/1574 <p> Allow uploading of files to push the data into webstore. Initially we were going to suggest changes to ckanext-storage but after further analysis we arrived at the conclusion that this should be implemented by ckanext-archiver as it already handles archiving of data from various sources and would be the best place to 'archive' to webstore. </p> <ol><li>A user wants to upload a file to CKAN, and so chooses the file upload option as they do currently with ckanext-storage. The file upload itself should be handled by this code internal to core (see <a class="closed ticket" href="http://localhost/ticket/1608" title="enhancement: Merge storage into Core (closed: fixed)">#1608</a>) but still using ofs [5d] </li></ol><ol start="2"><li>The user is provided the link to the file as currently. </li></ol><ol start="3"><li>The archiver's update task will check configuration to determine if it should upload some mime-types (tabular formats) to webstore. Configuration should also determine where webstore is. (see <a class="closed ticket" href="http://localhost/ticket/1609" title="enhancement: Celery task for ckanext-archiver to write to webstore. (closed: fixed)">#1609</a>) [5d] - took 6 </li></ol><ul><li>It should be stored in webstore using {username}/{resource-id} where {username} is the name of the user uploading the original file. </li></ul><ol start="4"><li>The archiver will continue to move the file into permanent storage as per current practice. </li></ol><h2 id="Tickets">Tickets</h2> <ul><li><a class="closed ticket" href="http://localhost/ticket/1608" title="enhancement: Merge storage into Core (closed: fixed)">#1608</a> Merge storage into Core [5d] </li><li><a class="closed ticket" href="http://localhost/ticket/1609" title="enhancement: Celery task for ckanext-archiver to write to webstore. (closed: fixed)">#1609</a> Celery task for ckanext-archiver to write to webstore. [5d] (took 6) </li><li><a class="closed ticket" href="http://localhost/ticket/1687" title="enhancement: Deploy new storage changes from master to datahub (closed: invalid)">#1687</a>: Deploy to datahub [1d] </li><li><a class="closed ticket" href="http://localhost/ticket/1681" title="enhancement: Core storage document (closed: fixed)">#1681</a>: Core storage documentation [1d?] <ul><li>Internal (perhaps in README somewhere or wiki): how does process work, where does data end up and in what order etc </li><li>External: here's how to configure it and here's what you get ... (goes in main sphinx docs) </li></ul></li></ul> en-us http://assets.okfn.org/p/ckan/img/ckan_logo_shortname.png http://localhost/ticket/1574 Trac 0.12.3 ross Tue, 03 Jan 2012 09:18:18 GMT http://localhost/ticket/1574#comment:1 http://localhost/ticket/1574#comment:1 <p> Related: </p> <blockquote> <p> <a class="closed ticket" href="http://localhost/ticket/1608" title="enhancement: Merge storage into Core (closed: fixed)">#1608</a> Merge storage into Core <a class="closed ticket" href="http://localhost/ticket/1609" title="enhancement: Celery task for ckanext-archiver to write to webstore. (closed: fixed)">#1609</a> Celery task for ckanext-archiver to write to webstore. </p> </blockquote> Ticket ross Tue, 03 Jan 2012 11:30:11 GMT http://localhost/ticket/1574#comment:2 http://localhost/ticket/1574#comment:2 <p> Added <a class="closed ticket" href="http://localhost/ticket/1611" title="enhancement: Implement auth API calls for webstore/external use (closed: duplicate)">#1611</a> as relevant to webstore integration </p> Ticket rgrp Thu, 05 Jan 2012 14:38:56 GMT http://localhost/ticket/1574#comment:3 http://localhost/ticket/1574#comment:3 <p> @ross: Is it possible to update the detailed description of this ticket so it reflects current reality (whatever that is ;-) ) (BTW: you can edit the main descript of the ticket by scrolling to the bottom of this ticket edit page) </p> Ticket ross Fri, 06 Jan 2012 10:52:44 GMT http://localhost/ticket/1574#comment:4 http://localhost/ticket/1574#comment:4 <ul> <li><strong>description</strong> modified (<a href="/ticket/1574?action=diff&amp;version=4">diff</a>) </li> </ul> <p> Updated description. </p> Ticket ross Fri, 06 Jan 2012 10:53:50 GMT http://localhost/ticket/1574#comment:5 http://localhost/ticket/1574#comment:5 <ul> <li><strong>description</strong> modified (<a href="/ticket/1574?action=diff&amp;version=5">diff</a>) </li> </ul> <p> <span class="underline">Some issues to be addressed</span> </p> <ul><li>What about existing content stored on google/s3? </li><li>Metadata for OFS should use username/userid? If user changes username we could just change the pairtree metadata json file, it should not be automated or easy for users to change username though. </li><li>OFS Metadata should contain the form data from CKAN upload </li><li>Remove _owner from OFS metadata (unnecessary) </li></ul><p> <span class="underline">Some issues to still consider</span> </p> <ul><li>Allow .sqlite upload? </li><li>Allow .sql upload? </li></ul> Ticket ross Mon, 09 Jan 2012 12:09:09 GMT http://localhost/ticket/1574#comment:6 http://localhost/ticket/1574#comment:6 <ul> <li><strong>owner</strong> set to <em>ross</em> </li> <li><strong>status</strong> changed from <em>new</em> to <em>assigned</em> </li> </ul> Ticket kindly Thu, 12 Jan 2012 18:26:23 GMT http://localhost/ticket/1574#comment:7 http://localhost/ticket/1574#comment:7 <ul> <li><strong>milestone</strong> changed from <em>ckan-backlog</em> to <em>ckan-v1.6</em> </li> </ul> Ticket rgrp Wed, 18 Jan 2012 16:38:03 GMT http://localhost/ticket/1574#comment:8 http://localhost/ticket/1574#comment:8 <ul> <li><strong>description</strong> modified (<a href="/ticket/1574?action=diff&amp;version=8">diff</a>) </li> </ul> Ticket ross Wed, 18 Jan 2012 16:42:45 GMT http://localhost/ticket/1574#comment:9 http://localhost/ticket/1574#comment:9 <ul> <li><strong>description</strong> modified (<a href="/ticket/1574?action=diff&amp;version=9">diff</a>) </li> </ul> <p> Added related ticket <a class="closed ticket" href="http://localhost/ticket/1681" title="enhancement: Core storage document (closed: fixed)">#1681</a> for documentation/announce to be done asap. </p> Ticket rgrp Wed, 18 Jan 2012 16:53:42 GMT http://localhost/ticket/1574#comment:10 http://localhost/ticket/1574#comment:10 <ul> <li><strong>description</strong> modified (<a href="/ticket/1574?action=diff&amp;version=10">diff</a>) </li> </ul> Ticket ross Fri, 20 Jan 2012 10:56:51 GMT http://localhost/ticket/1574#comment:11 http://localhost/ticket/1574#comment:11 <ul> <li><strong>description</strong> modified (<a href="/ticket/1574?action=diff&amp;version=11">diff</a>) </li> </ul> Ticket rgrp Wed, 15 Feb 2012 20:45:33 GMT http://localhost/ticket/1574#comment:12 http://localhost/ticket/1574#comment:12 <ul> <li><strong>summary</strong> changed from <em>[super] Storage changes</em> to <em>[super] Storage changes including automated save to Webstore</em> </li> </ul> Ticket ross Mon, 28 May 2012 11:02:35 GMT http://localhost/ticket/1574#comment:13 http://localhost/ticket/1574#comment:13 <ul> <li><strong>status</strong> changed from <em>assigned</em> to <em>closed</em> </li> <li><strong>resolution</strong> set to <em>fixed</em> </li> </ul> Ticket