http://localhost/ticket/1397 <p> We want to cache/archive data associated to a resource so it is available if the resource url disappears (and in order to support other processing we may wish to do e.g. webstorer ...) </p> <p> Etherpad: <a class="ext-link" href="http://ckan.okfnpad.org/queue"><span class="icon">​</span>http://ckan.okfnpad.org/queue</a> (most relevant parts inlined here) </p> <h2 id="Preliminaries">Preliminaries</h2> <ul><li>Add task_status table to store qa/archiever/webstore information that does not need to be versioned. - <a class="closed ticket" href="http://localhost/ticket/1363" title="enhancement: Add task_status table to store qa/archiever/webstore information that does ... (closed: fixed)">#1363</a> (and <a class="closed ticket" href="http://localhost/ticket/1371" title="enhancement: Make logic layer functions for working with task_status table (closed: fixed)">#1371</a> - related logic functions) </li></ul><h3 id="Configurationsetupfordaemons">Configuration setup for daemons</h3> <p> Pass config through to workers i.e site_url, user, api_key. Need to make site user account. <a class="closed ticket" href="http://localhost/ticket/1408" title="enhancement: Make site user, so tasks can post information back to ckan. (closed: fixed)">#1408</a> </p> <p> celeryd config: </p> <p> All providers of tasks will add an item to the following entry point: </p> <pre class="wiki">[ckan.tasks] name = ckanext.{name}.tasks:.... </pre><p> celeryconfig.py </p> <pre class="wiki">from pkg_resources import iter_entry_points for entry in iter_entry_points: celeryimports.appen(....) CELERY_IMPORTS = celeryimports </pre><h2 id="WorkItems">Work Items</h2> <ul><li>Resource change notifications in core - Make an IResourceChange and IResourceUrlChange. [1d] [0.75d] - <a class="closed ticket" href="http://localhost/ticket/1383" title="enhancement: Add IResourceChange IResourceUrlChange Interfaces. (closed: fixed)">#1383</a> </li><li>Generate archiving request on resource url change [0.25d][0.25d] - <a class="closed ticket" href="http://localhost/ticket/1399" title="enhancement: Generate archiving request on resource url change (closed: fixed)">#1399</a> </li><li>Make site user account. </li><li>Make entry point system for celery config </li><li>Archiver daemon <a class="closed ticket" href="http://localhost/ticket/891" title="task: Resource download worker daemon (closed: fixed)">#891</a> <ol><li>implement link-check function and task (point 2 from Archiver.update above) [1d] [0.5d] </li><li>Rewrite archiver to use external storage. (decide how!)[3d][~2d] </li></ol></li><li>Write to resource and task status table.[1d][0.75d] </li><li>[Required?] Make archived data available in WUI - <a class="closed ticket" href="http://localhost/ticket/892" title="enhancement: Make stored data available in WUI - 0.5d (closed: fixed)">#892</a> </li><li>Documentation - <a class="closed ticket" href="http://localhost/ticket/1400" title="enhancement: Document Archiver Extension (closed: fixed)">#1400</a> </li></ul> en-us http://assets.okfn.org/p/ckan/img/ckan_logo_shortname.png http://localhost/ticket/1397 Trac 0.12.3 rgrp Thu, 13 Oct 2011 18:17:04 GMT http://localhost/ticket/1397#comment:1 http://localhost/ticket/1397#comment:1 <ul> <li><strong>description</strong> modified (<a href="/ticket/1397?action=diff&amp;version=1">diff</a>) </li> </ul> Ticket rgrp Fri, 14 Oct 2011 13:25:21 GMT http://localhost/ticket/1397#comment:2 http://localhost/ticket/1397#comment:2 <ul> <li><strong>description</strong> modified (<a href="/ticket/1397?action=diff&amp;version=2">diff</a>) </li> </ul> Ticket rgrp Fri, 14 Oct 2011 13:34:24 GMT http://localhost/ticket/1397#comment:3 http://localhost/ticket/1397#comment:3 <ul> <li><strong>description</strong> modified (<a href="/ticket/1397?action=diff&amp;version=3">diff</a>) </li> </ul> Ticket rgrp Fri, 14 Oct 2011 13:34:36 GMT http://localhost/ticket/1397#comment:4 http://localhost/ticket/1397#comment:4 <ul> <li><strong>milestone</strong> changed from <em>ckan-v1.6</em> to <em>ckan-sprint-2011-10-24</em> </li> </ul> Ticket rgrp Fri, 14 Oct 2011 13:46:06 GMT http://localhost/ticket/1397#comment:5 http://localhost/ticket/1397#comment:5 <ul> <li><strong>description</strong> modified (<a href="/ticket/1397?action=diff&amp;version=5">diff</a>) </li> </ul> Ticket rgrp Fri, 14 Oct 2011 14:18:56 GMT http://localhost/ticket/1397#comment:6 http://localhost/ticket/1397#comment:6 <ul> <li><strong>priority</strong> changed from <em>awaiting triage</em> to <em>major</em> </li> <li><strong>description</strong> modified (<a href="/ticket/1397?action=diff&amp;version=6">diff</a>) </li> <li><strong>milestone</strong> changed from <em>ckan-sprint-2011-10-24</em> to <em>ckan-v1.6</em> </li> </ul> Ticket rgrp Fri, 14 Oct 2011 14:21:06 GMT http://localhost/ticket/1397#comment:7 http://localhost/ticket/1397#comment:7 <ul> <li><strong>description</strong> modified (<a href="/ticket/1397?action=diff&amp;version=7">diff</a>) </li> </ul> Ticket kindly Wed, 19 Oct 2011 10:47:18 GMT http://localhost/ticket/1397#comment:8 http://localhost/ticket/1397#comment:8 <ul> <li><strong>description</strong> modified (<a href="/ticket/1397?action=diff&amp;version=8">diff</a>) </li> </ul> Ticket kindly Wed, 19 Oct 2011 10:50:48 GMT http://localhost/ticket/1397#comment:9 http://localhost/ticket/1397#comment:9 <ul> <li><strong>description</strong> modified (<a href="/ticket/1397?action=diff&amp;version=9">diff</a>) </li> </ul> Ticket kindly Mon, 28 May 2012 11:02:00 GMT http://localhost/ticket/1397#comment:10 http://localhost/ticket/1397#comment:10 <ul> <li><strong>status</strong> changed from <em>new</em> to <em>closed</em> </li> <li><strong>resolution</strong> set to <em>fixed</em> </li> </ul> Ticket