CKAN: Ticket Query

#1072: Add filters to authztool

dread — Wed, 06 Apr 2011 16:11:51 GMT

It takes several minutes to print the 'rights' on DGU, which is annoying when you only want to grep for a few lines. Much quicker than grepping is to filter in the query.

#1067: CLI for loading/dumping complete databases

dread — Fri, 01 Apr 2011 08:11:03 GMT

Use 'db dump' and 'db load' for 'pg_dump' and 'psql -f' of a database. Use pylons config to find out database options.

#1066: Default reader role too permissive

dread — Fri, 01 Apr 2011 08:07:30 GMT

The definition of the 'reader' role includes creating packages, which is too permissive for some CKAN instances (e.g. DGU). 'Reader' suggests only reading, so I think this role should avoid creating and editing.

All projects so far want all roles to be able to create users, so this stays as a Reader action for now, as a convenience.

Implementation:

Action.PACKAGE_CREATE removed from reader's default_role_actions
Visitor has a new default role, called 'anon_editor' which can edit packages, but not groups / auth groups - you have to log in for that.
Migration script not needed?
Code comments written, to make clear the suggested policy

#1046: Dictization and the new logic layer

thejimmyg — Thu, 17 Mar 2011 12:24:54 GMT

The stages involved with doing this.

Convert model objects to standard dict format (DONE)
Convert standard dicts to current api formats (DONE)
Make standard dicts savable (DONE)
Validate standard dict format. (DONE)
Authorize actions

#962: Improvements to data preview extension

rgrp — Mon, 07 Feb 2011 09:55:24 GMT

Basic implementation done (and deployed):

https://bitbucket.org/okfn/ckanext-datapreview

However plenty to improve, e.g.

Support more formats (use external systems for preview?)
- json (!)
- html (trivial!)
- sparql
- ...
Do not display preview if no preview

Also suggest reworking to use external services rather than doing preview 'in house' (doing in house places heavy reliance on data proxy service and on converting data to a standard format).

#1059: Loader coping better with poor search indexing

dread — Mon, 28 Mar 2011 11:09:56 GMT

Loader currently checks for same name, but also should check for name_, name etc.

#1061: Orphaned home/license page

dread — Tue, 29 Mar 2011 10:02:48 GMT

No links to home/license and it contains out of date references to knowledgeforge. Remove it.

#1071: Package history API moved to /api/rest/package/revisions

dread — Mon, 04 Apr 2011 17:24:42 GMT

api/rest/package_history is not RESTful or follow API naming conventions. Therefore move it to /api/rest/package/revisions

Also, API docs incomplete.

#937: Record download stats for resources (extension)

rgrp — Sat, 29 Jan 2011 22:31:13 GMT

As a User (especially as a Package Owner/Maintainer?) I want to know how many times a resource has been downloaded (and when).

So let's record download stats (as in clicks on the link for a resource).

Implementation

Use the existing support for this feature in google analytics or piwiki
For google analytics see:
- Integrate to record: http://www.google.com/support/analytics/bin/answer.py?answer=55529
- Accessing analytics data via API (for showing download counts): http://code.google.com/apis/analytics/docs/gdata/gdataDeveloperGuide.html
Limitations only record downloads by browsers with js turned on.
- This is OK I think (in any case machine download via e.g. datapkg does not get downloaded)

Old Spec (do it in CKAN)

Record info of form: resource id (or url?), timestamp
Do this via javascript capturing of onclick event talking to an api
API: /api/resource/{id}/download
- POST to increment (how do we stop spamming -- could use a nonce setup with a random string set on each page load for the js)
- GET to get data back { total: X, day_count: [ [yyyy-mm-dd, count], ... ] }

Questions

Do we record ip addresses (to handle de-botting etc)?
Do we count preview clicks as well?

#1079: Refactor API to use new logic layer and dictization

rgrp — Mon, 11 Apr 2011 08:12:10 GMT

Convert current api saves to the new standard dict format.

#1073: Search index checker

dread — Thu, 07 Apr 2011 14:10:44 GMT

Tool that checks which packages have not been indexed.

Required for DGU: https://trac.dataco.coi.gov.uk/projects/datagov/ticket/940

#1060: Spreadsheet importer tries to import readonly keys

dread — Mon, 28 Mar 2011 11:36:50 GMT

e.g. we just added notes_rendered and that is read in as an extra field. Tests failing in ckanext-importlib

Also related: we are missing lost metadata_created and metadata_modified in the dumps.

#1068: metadata_modified problem

dread — Fri, 01 Apr 2011 10:01:25 GMT

This test has been failing since the clocks changed:

======================================================================
FAIL: ckan.tests.models.test_package.TestPackageRevisions.test_02_metadata_created_and_modified
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dread/hgroot/pyenv-ckan2/lib/python2.6/site-packages/nose-0.11.3-py2.6.egg/nose/case.py", line 186, in runTest
    self.test(*self.arg)
  File "/home/dread/hgroot/ckan2/ckan/tests/models/test_package.py", line 283, in test_02_metadata_created_and_modified
    assert out == exp, (out, exp)
AssertionError: (datetime.datetime(2011, 4, 1, 10, 45, 50, 875509), datetime.datetime(2011, 4, 1, 9, 45, 50, 875509))
----------------------------------------------------------------------