Changes between Version 5 and Version 6 of Ticket #1397


Ignore:
Timestamp:
10/14/11 14:18:56 (3 years ago)
Author:
rgrp
Comment:

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #1397

    • Property Priority changed from awaiting triage to major
    • Property Milestone changed from ckan-sprint-2011-10-24 to ckan-v1.6

  • Ticket #1397 – Description

    v5 v6  
    66 
    77 * Add task_status table to store qa/archiever/webstore information that does not need to be versioned. - #1363 (and #1371 - related logic functions) 
     8===  Configuration setup for daemons === 
    89 
    9 == Tasks == 
     10 * Standard ini file 
     11 * Sections are named after daemon / extension. E.g. [my-daemon] 
     12 * Arbitrary values but anticipate at least 2 stanard values: 
     13   * ckan_url e.g. http://thedatahub.org/ 
     14   * ckan_apikey e.g. xxxxxx 
     15   * celery_config 
     16 
     17celeryd config: 
     18 
     19All providers of tasks will add an item to the following entry point: 
     20 
     21{{{ 
     22[ckan.tasks] 
     23name = ckanext.{name}.tasks:.... 
     24}}} 
     25 
     26celeryconfig.py 
     27 
     28{{{ 
     29from pkg_resources import iter_entry_points 
     30for entry in iter_entry_points: 
     31     celeryimports.appen(....) 
     32 
     33CELERY_IMPORTS = celeryimports 
     34}}} 
     35 
     36== Work Items == 
    1037 
    1138 1. Resource change notifications in core - Make an IResourceChange and IResourceUrlChange. [1d] [0.75d] -  #1383 
    12  2. ckanext-archiver implements IResourceUrlChange and sends tasks to celery. [0.25d][0.25d] - ??? 
     39 2. Generate archiving request on resource url change [0.25d][0.25d] - #1399 
    1340 3. Archiver daemon #891 
    1441   1. implement link-check function and task (point 2 from Archiver.update above) [1d] [0.5d] 
    1542   2. Rewrite archiver to use external storage. (decide how!)[3d][~2d] 
    1643 5. Write to resource and task status table.[1d][0.75d] 
    17  6. Make archived data available in WUI - #892 
     44 6. [Required?] Make archived data available in WUI - #892 
     45 7. Documentation - #1400 
    1846 
    1947== Archiver process == 
    2048 
    21 Archiver: 
    22  
    23  0. A resource is added to CKAN 
    24  1. IResourceCreate event generated 
    25  2. IF: resource url points to ckan storage or falls within some other set of exclusion conditions then END else continue 
    26  3. Generate a Archiver.Update task with resource.id 
     49Generate archiving request on resource url change: #  
    2750 
    2851Archiver.update: see #891