{22} Trac tickets (2647 matches)

Results (1001 - 1100 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1009 enhancement pudo rgrp ckan-backlog new Improvements to user accounts sytem
  • Forgot password (email a new password)
  • Confirm email
  • Do not show register page if you are logged in (redirect to home page)
  • ticket:1010 - listing of users.
    • Do not use /user for general user account home page (either user normal user page /user/{id} or /user/myaccount)
1298635991000000 1310128574000000
#1010 enhancement rgrp rgrp ckan-v1.4-sprint-2 closed fixed List CKAN users in WUI

Should have user listing at /user/ rather than user account page.

  • list users, sorted by number of packages contributed/edited
  • Move user home page to /user/{user-id}
  • Paginated
1298649180000000 1298740889000000
#1011 enhancement sebbacon sebbacon ckan-v1.4-sprint-3 closed fixed Make pluggable AuthzGroups implementation

I have a requirement to allow an external source to define the groups of which a user is a member.

I propose to create an IAuthzUserGroups plugin interface that allows an extension to arbitrarily extend the list of AuthzGroups? that a user is in.

1298819657000000 1299245206000000
#1019 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Webhooks notification service

We propose the following push approach using individual packages:

  1. New, updated and deleted packages are pushed to a url endpoint on Wordpress (WPURL). This endpoint will be a configuration option on the CKAN side. A POST or PUT http method will get used. The payload will be a json document of the following form:

{

payload: The entity data as available from the REST API entity-type: 'Package', operation-type: 'create'|'update'|'delete' # one of these options

}

It will be sent as the body of the request with content-type set to application/json.

  1. The wordpress side will provide a 200 on success. Any other response will be taken as a failure. On failure, submission will be archived and failure logged and notified to system administrator. Submission can be resent later automatically by sysadmin after review.
  1. Pushes will happen continuously and approximately simultaneously with updates (a webhooks type model)
  1. [optional] CKAN side will support configuring authorization for basic authentication if applied on WP side.
  1. [optional] List queue status (including failures) on ckan adminstrative dashboard.
1299166784000000 1299166930000000
#1021 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Config option to disable OpenID

HRI don't like federation, want to login normal way only. Make this a config option and perhaps even mess with runtime repoze config

1299492920000000 1299518828000000
#1022 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Error reporting in CKAN worker API

Report on errors by any worker daemons, send them out via E-Mail

1299493047000000 1299512991000000
#1024 enhancement pudo pudo ckan-v1.4-sprint-3 closed duplicate Lock down all controller actions in CKAN

When a user visits the site, he/she may only see a login box.

1299493428000000 1299668648000000
#1025 enhancement dread dread ckan-v1.4-sprint-3 closed fixed Default authz can be set in config

Currently the default authz for a package is hard-coded to:

 <PackageRole user="visitor" role="editor" context="Package">,
 <PackageRole user="logged_in" role="editor" context="Package">,

This should be configurable in the config, so that you can have a more locked down instance etc.

1299596110000000 1299751045000000
#1026 enhancement dread dread ckan-v1.4-sprint-3 closed fixed cli for creating users

It's handy to be able to create users using the cli (e.g. dgu migration)

1299604652000000 1299605128000000
#1027 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Authorization checks on all controller actions

We want to have authz checks on all controller actions so that we can lock down CKAN to a login-only mode.

1299666256000000 1299682082000000
#1031 enhancement johnlawrenceaspden rgrp ckan-v1.4-sprint-4 closed fixed User lookup API

Add an api for searching users. This is needed for any kind of ajax autocomplete (needed for anywhere we want to add users).

  • API location: /api/util/user/lookup?q=querystr&limit=10
  • Return json objects containing {id: ..., name: ..., fullname: ...}
  • Put in a module called controllers/apiv2/user.py
1299780419000000 1300101520000000
#1032 enhancement rgrp rgrp ckan-v1.6 closed fixed [super] Resources in WUI

Add resources into Web User Interface.

  • Locate at: /dataset/{dataset}/resource/{id}
  • CRUD
  • Authorization

Done:

  • #945 - Richer resources - Resource Groups, new fields, improved UI
  • #1445 - Resource View page in WUI
  • #1450 - Dataset view pages to match

Moved to superticket #1506:

  • #978 - Edit Resource Extras in Web UI
1299782021000000 1330348463000000
#1035 enhancement thejimmyg dread closed wontfix Form impressions given an ID

To counter Cross Site Request Forgery attacks, each form generated is assigned a random number in its url, which must be passed when you submit the form.

(Something to consider for the new form mechanism)

1299857978000000 1338206251000000
#1038 enhancement dread dread ckan-v1.4 closed fixed Authz tool - operate on all packages at once

Add 'package:all' to authz tool to allow mass changes of authz.

1300212788000000 1300212841000000
#1039 enhancement dread dread ckan-v1.4 closed fixed Default user roles read from config

(instead of being hard coded)

1300212822000000 1300212856000000
#1041 enhancement thejimmyg thejimmyg ckan-backlog assigned Start Using the CKAN Wiki for Tutorial-style documentation

For example, I will document the following:

I'd love if someone else would write:

  • An authorisation tutorial covering the core model, the command line tools and examples of every possible way of using the system
  • A HOWTO guide with screenshots for adding a package
1300284715000000 1312372367000000
#1046 enhancement kindly thejimmyg ckan-v1.4-sprint-5 closed fixed Dictization and the new logic layer

The stages involved with doing this.

  • Convert model objects to standard dict format (DONE)
  • Convert standard dicts to current api formats (DONE)
  • Make standard dicts savable (DONE)
  • Validate standard dict format. (DONE)
  • Authorize actions
1300364694000000 1302777668000000
#1048 enhancement dread dread ckan-v1.4-sprint-4 closed fixed Complete making groups versioned
  • Deleting a group changes state to 'deleted' rather than purging it
  • Adding authz tests for deleted groups
1300387655000000 1300702752000000
#1050 enhancement thejimmyg johnlawrenceaspden closed invalid Authz lib improvement and refactor of ckan/lib/authztool.py

Refactor ckan/lib/authztool.py so that the relevant methods are independent of the command line interface.

The extracted methods should live in a new file ckan/authz.py. authztool.py should probably move into cli.py and will just do command line parsing and printing and use ckan/authz.py. The updated web gui for authz will also use this code.

Tests should be made. There's already a file ckan/tests/test_authz.py, which looks like the appropriate place for new tests.

all to go on a branch feature-1050-refactor-authtoolz

Optional extras

  • Rename ckan/authz.py to ckan/lib/authz.py or even ckan/logic/authz.py
1300451937000000 1315394117000000
#1061 enhancement dread dread ckan-v1.4-sprint-5 closed invalid Orphaned home/license page

No links to home/license and it contains out of date references to knowledgeforge. Remove it.

1301392968000000 1301922350000000
#1065 enhancement zephod johnlawrenceaspden ckan-v1.6 closed fixed [super] Change Authorization System

Child tickets

  • #1198 Publisher hierarchy
  • #1050 Authz lib improvement and refactor of ckan/lib/authztool.py
  • #1004 Group creation instructions missing
  • #1099 Strange interactions between two browsers while playing with authz groups
  • #1115 can have two authzgroups with the same name
  • #1133 command line rights manipulation doesn't work
  • #1138 minor navigations behave inconsistently

Old ticket description:

  1. Change name of AuthzGroup? to UserGroup? to reflect what it is for
  1. Get rid of Roles, and replace them with direct assignment of actions, even though there are many actions, and extensions can add arbitrary ones.
    • Debatable whether we should cut the number of actions to correspond to the three roles defined by the base system.
    • Have a method of finding roles (or, in future, actions) relevant to a given protection object (e.g. FILE-UPLOAD(ER) not relevant to Packages)
  1. Change UserGroups? so that they can have a hierarchical structure,

More info on Hierarchy change

e.g. UserGroup? NHS contains the User nhsysadmin, as well as the UserGroups? SURREY and BERKS, which themselves contain users.

One user in SURREY is Simon the Sysadmin, who has permissions on the whole system. His permissions should not leak out to other users or groups, and user permissions generally should not.

Each Group has permissions over various objects.

A user has permissions in his own right, and also has the permissions of his own group, and of all the groups contained in his group, and so on recursively.

Algorithm:

possible(user, action, package):

if user has permission for action on package

or any of have that permission

or any of his groups group-children (but not user-children), and so on recursively have the permission.

1301508331000000 1324550041000000
#1066 enhancement dread dread ckan-v1.4-sprint-5 closed fixed Default reader role too permissive

The definition of the 'reader' role includes creating packages, which is too permissive for some CKAN instances (e.g. DGU). 'Reader' suggests only reading, so I think this role should avoid creating and editing.

All projects so far want all roles to be able to create users, so this stays as a Reader action for now, as a convenience.

Implementation:

  • Action.PACKAGE_CREATE removed from reader's default_role_actions
  • Visitor has a new default role, called 'anon_editor' which can edit packages, but not groups / auth groups - you have to log in for that.
  • Migration script not needed?
  • Code comments written, to make clear the suggested policy
1301645250000000 1301932136000000
#1067 enhancement dread dread ckan-v1.4-sprint-5 closed fixed CLI for loading/dumping complete databases

Use 'db dump' and 'db load' for 'pg_dump' and 'psql -f' of a database. Use pylons config to find out database options.

1301645463000000 1302186503000000
#1069 enhancement tobes rgrp assigned Stub datasets (request for datasets)

Idea is to have stubs for datasets that someone wants but don't yet exist (or haven't been discovered) in the way one has stub pages on a wiki.

We could do this within the existing model by a slight 'abuse' - create a dataset and mark it with a special tag e.g. todo.does-not-yet-exist or similar ...

(Just as we have datasets listed that exist but aren't available ...)

Alternative would be to have a request for datasets subsystem.

I prefer the stub dataset model because it's simpler, provides a simple workflow (as a dataset is found or comes into existence), and the package page provides a natural space in which to accumulate information about what is wanted and what exists.

Implementation

  • Agree a new dedicated tag. e.g. todo.does-not-exist
1301666919000000 1340632215000000
#1070 enhancement rgrp rgrp ckan-v1.5 closed fixed Plan a new domain model and layer architecture for CKAN

See http://wiki.ckan.net/Domain_Model especially section on v2.

  • New domain model is planned but not yet finally agreed.
  • Layer architecture is complete and implemented
1301910940000000 1310117129000000
#1072 enhancement dread dread ckan-v1.4-sprint-5 closed fixed Add filters to authztool

It takes several minutes to print the 'rights' on DGU, which is annoying when you only want to grep for a few lines. Much quicker than grepping is to filter in the query.

1302106311000000 1302106474000000
#1073 enhancement dread dread ckan-v1.4-sprint-5 closed fixed Search index checker

Tool that checks which packages have not been indexed.

Required for DGU: https://trac.dataco.coi.gov.uk/projects/datagov/ticket/940

1302185444000000 1302185825000000
#1074 enhancement rgrp rgrp ckan-v1.5 closed fixed Refactor authz web user interface to have common code and templating

Currently repeat the same template and code across Package Authz, Group Authz, and Authz Group authz.

Having now implemented a new, cleaner setup in ckanext-admin we should port this back into core.

  • Common template code (checkbox template)
  • Logic code (or just common code) for wiring into authz system
  • Look for all places thoroughout the system where usernames, authzgroups or groups need to be typed into boxes, and make sure that they auto-complete appropriately.

Will also deliver a significant improvement in the form of ajax user lookup.

1302271586000000 1314303581000000
#1075 enhancement johnlawrenceaspden rgrp ckan-v1.4-sprint-6 closed fixed Administrative dashboard - Edit Authorization related to System object

Roles on System object are important because admin role on system equates to being a 'sysadmin' (i.e. able to do anything).

  • Make users sysadmin (either as separate action or as part of editing roles on system object)
  • /authz subpage for editing roles on system object
    • Add and update user roles
    • Add and update authz group roles
    • List actions associated to roles at top of page (extra points for checkbox table with editability)
  • Document on http://wiki.ckan.net/Authorization what roles on System object 'mean' esp sysadmin role on System

Related Tickets

  • super ticket: #833
  • authz lib improvement and authztool refactor #1050
1302279799000000 1303227982000000
#1076 enhancement johnlawrenceaspden rgrp ckan-v1.4 closed fixed Improve revision and package purge system

Purging Revisions

  • Delete button displayed on:
    • /revision/list
    • (/package/history)
      • /package/history is problematic because html does not allow nested forms and we already have form for doing diff/comparison.
    • /revision/{id}
  • Delete button submits to delete action on revision and changes revision state to 'deleted'.
    • undelete button now displayed and revisions are marked as deleted in some way (e.g. greyed out?)
  • Sysadmins then visit /ckan-admin/trash which lists all revisions with deleted state. There is a large button: "Empty trash" (irreversible). Click button purges all revisions with deleted state.

Purging Packages

  • Put into deleted state.
  • Listed on /ckan-admin/trash
  • Separate Empty trash button which deletes all associated revisions.
    • Should be separate from Empty trash for revisions

Current system

  • Single purge link on revision listing if a sysadmin which permanently purges the revision and all associated changes (without confirmation atm!)
1302283442000000 1303236302000000
#1077 enhancement kindly rgrp ckan-backlog new Move to simpler vdm system

Option 1: 'Changeset' Model

See ticket:1135 for vdm ticket. This would involve a) moving to changeset in vdm b) doing the migration in ckan to support this.

Have developed a new "changeset" based model for revisioning in vdm.

Implementation

  • The main challenge with this change is schema and data migration

Every revisioned object has a revision_id and revision attribute.

Approximate algorithm:

Revision -> Changeset

for revtype in [PackageRevision, ...]:
    for pkgrev in package_revision:
        changeset = lookupchangeset(package_revision)
        ChangeObject(cset, (table, id), dictize(pkgrev))

Question:

  • does pkg include tags attributes or not? or we have to dictize, pkgrev, pkg2tagrev, and tag. Probably the latter.

Option 2: Simplify Revision Object Model

Just use a simpler vdm, see ticket:1136 (move to SessionExtension) and ticket:1137 (remove need for statefulness in vdm).

Discussion

Advantage of Option 1 versus 2:

  • Easier support for pending state and similar behaviour
  • No need to introduce new tables (and hence migrations) when making something revisioned (or not).

Disadvantages

  • Migration is required
  • More difficult to query revision history.
    • Could be addressed by having ChangeObject have separate cols for table name and id but would likely be more difficult.
  • Performance (?)
    • Have one big ChangeObject table to query when looking at changed objects rather than many revision tables.
      • Not sure this is a biggie as even with Revision model biggest revision object tables are probably on the order of the ChangeObject table

Conclusion

Implement Option 2 and leave Option 1 for present.

Option 1 includes Option 2 so it seems that that is required in either case (so we may as well with Option 2).

Option 1 requires significant effort (esp migration) so leave for present and then review the situation at some later date.

1302304464000000 1340034345000000
#1078 enhancement kindly rgrp ckan-v1.5 closed fixed Refactors WUI controllers and forms to use logic layer
  • Deserialize forms to new dict format.
  • Replace controllers/forms to use dictization.
1302509347000000 1305828973000000
#1079 enhancement kindly rgrp ckan-v1.4-sprint-5 closed fixed Refactor API to use new logic layer and dictization
  • Convert current api saves to the new standard dict format.
1302509530000000 1302777504000000
#1087 enhancement wwaites ckan-sprint-2011-11-21 closed fixed version and contact info api call

a simple api call that returns data like this:

{ "version": ckan_software_version,
  "contact": { "name": "Some Admin", "mbox": "[email protected]" },
  "description": "Site Description",
  "url": "http://canonical.name.ckan.net/"
1302628944000000 1320866159000000
#1089 enhancement dread dread ckan-v1.4-sprint-6 closed fixed Check for "--ckan" when running nosetests

(because if you forget, you get difficult to understand errors, and more than one person has tripped up on this)

1302631189000000 1302631733000000
#1094 enhancement thejimmyg thejimmyg ckan-v1.5 closed duplicate [super] Refactor the Auth System

Here are some proposed changes related to CKAN's authorization system - they aren't very big, but should provide for some forthcoming use cases including #787.

Two man reasons for the changes are:

  • We have a completely refactored architecture now which introduces a logic layer. These Auth changes are designed to better support the way we work with that layer.
  • Different CKAN extension apps may need radically different authentication/authorisation so we need to allow whatever we have to be override-able.

The first two changes revolve around the is_authorized method, which is called by the logic layer to ask whether a particular user (e.g. Bob) is allowed to do a certain action (e.g. edit) on a certain object (e.g. Package).

  1. The first thing the is_authorized method is a hook to a plugin

which *overrides* the current call with its own implementation (note: in previous discussions we have considered allowing a chain of plugins, no longer!)

Reason: authorization can be completely delegated to another system (or partially)

  1. is_authorized method currently takes (username, action, object)

but for action=create_package, the object supplied is System, and for action=edit the object supplied is the package. Instead action should always be the string name of a function in the logic layer and object should always be the object passed to that function. This means our auth system is based around the actual actions we are performing (rather than a model them) and with the actual data that forms the action (rather than a related object). You never need a System object in this model.

  1. Rename these two classes to better reflect what they are
  1. Rename the Editor role to PriveledgeUser? since Editors sometimes can't edit.

Although this sounds a bit radical we already have auth extensions.

Read-only CKAN Web UI

(Additional requirement from #764)

Whilse using CKAN web interface, you are not tempted to edit stuff:

  • You know at all times this CKAN is read-only
  • All editing facilities are still seen but greyed-out with an indication why it is.
1303117973000000 1311173649000000
#1097 enhancement dread dread ckan-v1.4-sprint-6 closed fixed Sidebar hideable

The web interface has a sidebar (#primary) which should be hidden in some pages. This is for QA extension and useful for package new and edit pages. Must be compatible with DGU theme.

1303293416000000 1303293476000000
#1101 enhancement sebbacon ckan-backlog new Integrate googlanalytics into site nav

There's a stats plugin (e.g. at http://trac.ckan.org/ticket/832).

Output from the googleanalytics plugin should append to that page, if the stats plugin is present.

Possibly the stats plugin and the googleanalytics plugin should be merged?

Finally, if the stats plugin is active, then a link to the stats page should be added to the main site footer.

1303393926000000 1339774582000000
#1108 enhancement zephod pudo ckan-sprint-2011-09-12 closed fixed Create a more modern theme for CKAN

CKAN looks a bit aged, it should be styled more modernly and some elements could be re-arranged:

  • Collect user info in top bar
  • re-add the logo to ckan.net
  • Remove tags from main menu, replace with /sitemap.xml

Inpiration:

quora.com, github.com, Google Projects, Google Refine, etc.

CKAN.net or CKAN general theme?

To be decided. Suggest we start with ckan.net specific and then backwards integrate (?). Existing ckan.net theme repo:

https://bitbucket.org/okfn/ckan-ckan.net

1303830790000000 1315140879000000
#1110 enhancement kindly kindly closed wontfix profile ckan

We need to see what areas of ckan are slow.

1303840041000000 1340034394000000
#1112 enhancement dread dread ckan-v1.4-sprint-7 closed fixed Allow searching for all packages
GET api/search/package?q=

returns all packages. This is so you can filter them e.g. by openness, which is not currently possible.

1304007852000000 1304358643000000
#1114 enhancement dread dread ckan-v1.4-sprint-7 closed fixed CLI for viewing search index of a package

To see what lexemes are generated for debug purposes.

1304078353000000 1304085484000000
#1119 enhancement rgrp rgrp ckan-v1.4-sprint-7 closed fixed Fully functional storage extension with file upload

Previous work in #877 and #879 + #853 (storage API). In this ticket:

  • Improve authorization
  • Establish convention for laying out files on disk
  • Add documentation
  • Fix bugs with #879 (does not currently work -- boto may have changed)
1304094382000000 1305037201000000
#1120 enhancement tsm ckan-backlog new Atom feeds of each tag

Tags could/should have an Atom feed. This would mean that every edit to relevant packages could be easily monitored. See [1].

[1] http://lists.okfn.org/pipermail/ckan-discuss/2011-May/001162.html

1304309285000000 1339774568000000
#1121 enhancement dread closed wontfix JSON extras appear in package edit form mangled

It is possible to use the CKAN API to insert JSON format data into package extra values, but this data is displayed in the package edit form.

Example: Package http://ckan.net/package/hbz_unioncatalog in the API the extra value as a list:

"extras": {"publishingInstitution": ["http://lobid.org/organisation/DE-605", "http://lobid.org/organisation/DE-290"...

yet when you edit the package it loses all the quotes and brackets: http://ckan.net/package/edit/hbz_unioncatalog {{{http://lobid.org/organisation/DE-605http://lobid.org/organisation/DE-290... }}} so when you save the package, the list is mangled into a bad string.

1304367504000000 1307358426000000
#1122 enhancement pudo dread closed wontfix JSON Extra data not searchable

It is possible to use the CKAN API to insert JSON format data into package extra values, but this data is not found on searching.

Full text from Pascal:

we encountered a Problem concerning accessing Arrays/Lists.

curl -XGET 'http://ckan.net/api/rest/package/hbz_unioncatalog'

will get you amongst others:

 "extras": {"publishingInstitution":
"[u'http://lobid.org/organisation/DE-605',
u'http://lobid.org/organisation/DE-290',
u'http://lobid.org/organisation/DE-38M',
u'http://lobid.org/organisation/DE-98',
u'http://lobid.org/organisation/DE-38',
u'http://lobid.org/organisation/DE-Kn41',
u'http://lobid.org/organisation/DE-82',
u'http://lobid.org/organisation/DE-107',
u'http://lobid.org/organisation/DE-929',
u'http://lobid.org/organisation/DE-Zw1',
u'http://lobid.org/organisation/DE-832']"}

but if I try to query this:

wget
'http://ckan.net/api/search/package?q=lobid&publishingInstitution="http://lobid.org/organisation/DE-605"'

I get only two packages, among the package "hbz_unioncatalog" is
missing. (These two packages have only one value for
"publishingInstitution").

The "extra/publishingInstitution"-Array was uploaded through a "curl
-XPUT ...
 "extras": {
   "publishingInstitution":[
     "http://lobid.org/organisation/DE-605",
     "http://lobid.org/organisation/DE-290",
     "http://lobid.org/organisation/DE-38M",
     "http://lobid.org/organisation/DE-98",
     "http://lobid.org/organisation/DE-38",
     "http://lobid.org/organisation/DE-Kn41",
     "http://lobid.org/organisation/DE-82",
     "http://lobid.org/organisation/DE-107",
     "http://lobid.org/organisation/DE-929",
     "http://lobid.org/organisation/DE-Zw1",
     "http://lobid.org/organisation/DE-832"
   ]
   },
...
1304367510000000 1306747714000000
#1124 enhancement thejimmyg nils.toedtmann ckan-sprint-2011-12-05 closed fixed push apt package python-ckanext-solr into our debian repository

python-ckanext-solr is already available in http://apt-alpha.ckan.org/datanl-dev, but not yet in http://apt-alpha.ckan.org/debian (that is why we had to [pip-install it for DataGM). Please push into main repo.

1304537793000000 1323168156000000
#1125 enhancement dread nils.toedtmann closed fixed Debian package "ckan" should not depend on "postgresql"

The debian package "ckan" with the two scripts "ckan-create-instance" and "ckan-instance-maintenance" depends against "postgresql". But "ckan-create-instance" is quite handy even when the DB is remote: it creates all the data dirs with the correct permissions, and the ckan and apache configs.

Please add a flag "--without-local-db" to "ckan-create-instance" and remove the postgres dependancy from the debain package.

1304538095000000 1310134813000000
#1126 enhancement dread dread ckan-v1.4-sprint-7 closed fixed Exceptions arising from error page

I'm not completely clear what the use case is for loading the error page in this way, but somehow original_request is None and that creates an unnecessary exception with the logic refactor.

http://ckan.net/error/document?__cache=39020485
...
Module ckan.controllers.error:29 in document
<<          original_response = request.environ.get('pylons.original_response')
               # Bypass error template for API operations.
               if original_request.path.startswith('/api'):
                   return original_response.body
               # Otherwise, decorate original response with error template.
>>  if original_request.path.startswith('/api'):
AttributeError: 'NoneType' object has no attribute 'path'
1304586683000000 1304587078000000
#1130 enhancement lucychambers assigned First time users

Send users to FAQ first time on CKAN

1304938761000000 1340633514000000
#1135 enhancement kindly rgrp assigned Changeset model for vdm

Move to Changeset model for vdm.

A changeset model is like an Audit-Log model in which we just record Changesets with Change-Objects rather than have Revision-Objects for each Object that is revisioned.

This change would also incorporate significant simplication of vdm.

1305209986000000 1340632267000000
#1136 enhancement kindly rgrp assigned Move to SessionExtension in vdm

When vdm was created there was no SessionExtension so we use MapperExtension for doing revisioning. Now that SessionExtension? exists we should use it. We can also follow the existing SQLAlchemy recipe: <http://www.sqlalchemy.org/docs/orm/examples.html?highlight=versioning#versioned-objects>

1305210855000000 1340632980000000
#1137 enhancement kindly rgrp assigned Remove need for statefulness in vdm

Statefulness, especially statefulness for relation (esp m2m) is cause of most of the complexity in vdm. It is required because, atm, revision objects have FKs to continuity objects.

This ticket proposes the following changes:

NB: this could be limited just to case of join tables (leaving state stuff on other tables)

  • Remove FKs from revision to continuity (or allow for them to be nullable).
    • We could just limit this to m2m stuff
  • Delete of an object leads to:
    • Deletion of continuity object
    • Adding an entry in revision table with state set to deleted (we retain state on revision table)

If this is done we will no longer need to worry about filtering on state on relationships as join table will only contain "active" relationships.

If we do this on all tables we remove need for any state awareness in client (e.g. no need to filter tables on active state).

The only disadvantage of this change is that undeletion becomes more problematic (we have to recreate some continuity objects).

1305211628000000 1340631974000000
#1138 enhancement johnlawrenceaspden closed invalid minor navigations behave inconsistently

For Authorization Groups, if you have admin privileges you see view, edit and authz tabs, and if you don't have the necessary privileges you only see the view tab.

For Packages, you see all tabs whatever your permissions, so there's a link you can click on which will redirect you to the login page.

1305279888000000 1316965357000000
#1139 enhancement lucychambers lucychambers ckan-backlog closed fixed Create CKAN Theme Gallery

Take screenshots of existing ckan instances esp those mentioned <http://wiki.ckan.net/Theming> and put on flickr in ckan or ckan-theme group so we can create a gallery ... (both to illustrate theming but also to show ckan instances that are around -- could add to http://wiki.ckan.net/Instances)

1305645859000000 1306941356000000
#1142 enhancement annapowellsmith rgrp ckan-v1.5 closed fixed [super] Major Overhaul and Extension of CKAN Documentation

Child tickets:

  • #1041 Start Using the CKAN Wiki for Tutorial-style documentation
  • #1192 Convert CKAN Sphinx docs into admin/reference manual

Previous super ticket (from 3m ago): http://trac.ckan.org/ticket/927

  • CKAN 1-page overview (for enterprise and for data hackers)
  • Administrator's Guide (including install)
  • Extensions Guide
  • Separate CKAN.net info from Software Documentation (?)

Also now a wiki page with more detail: http://wiki.ckan.net/Documentation_Plans

Minor Items

1305727452000000 1313775665000000
#1143 enhancement dread dread ckan-v1.5-sprint-2 closed fixed Improve stats page
  • Ensure we don't include deleted packages in the stats
  • Some visual improvements:
    • Number of packages:
      • fix x axis to start at first revision
      • fix y axis to start at zero
    • Fix problem with legend to 'Revisions to packages' graph.
    • Hide sidebar in the template, so it isn't right-cropped in the DGU theme
  • Add some testing of the stats lib - results of basic stats
1305734836000000 1307024681000000
#1144 enhancement timmcnamara ckan-backlog new Support DSPL

DSPL, the Dataset Publishing Language, is being promoted by Google for its "Google Public Data Explorer" system. It is an XML format with metadata.

The format is described on the developer docs ofthe Google Code site.

Google provides a Python script which reads CSV data and generates DSPL

Sample from http://code.google.com/apis/publicdata/docs/dspl_sample.html:

<?xml version="1.0" encoding="UTF-8"?>
<dspl xmlns="http://schemas.google.com/dspl/2010"
    xmlns:geo="http://www.google.com/publicdata/dataset/google/geo"
    xmlns:geo_usa="http://www.google.com/publicdata/dataset/google/geo/us"
    xmlns:time="http://www.google.com/publicdata/dataset/google/time"
    xmlns:quantity="http://www.google.com/publicdata/dataset/google/quantity"
    xmlns:entity="http://www.google.com/publicdata/dataset/google/entity">

  <import namespace="http://www.google.com/publicdata/dataset/google/time"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/quantity"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/entity"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/geo"/>
  
  <info>
    <name>
      <value>My statistics</value>
    </name>
    <description>
      <value>Some very interesting statistics about countries</value>
    </description>
    <url>
      <value>http://www.stats-bureau.com/mystats/info.html</value>
    </url>
  </info>

  <provider>
    <name>
      <value>Bureau of Statistics</value>
    </name>
    <url>
      <value>http://www.stats-bureau.com</value>
    </url>
  </provider>

  <topics>
    <topic id="geography">
      <info>
        <name><value>Geography</value></name>
      </info>
    </topic>
    <topic id="social_indicators">
      <info>
        <name><value>Social indicators</value></name>
      </info>
      <topic id="population_indicators">
        <info>
          <name><value>Population indicators</value></name>
        </info>
      </topic>
      <topic id="poverty_and_income">
        <info>
          <name><value>Poverty & income</value></name>
        </info>
      </topic>
      <topic id="health">
        <info>
          <name><value>Health</value></name>
        </info>
      </topic>
    </topic>
  </topics>

  <concepts>
    <!-- As noted in the tutorial, this concept should extend quantity:amount.-->
    <concept id="population">
      <info>
        <name>
          <value>Population</value>
        </name>
        <description>
          <value>Size of the resident population.</value>
        </description>
      </info>
      <topic ref="population_indicators"/>
      <type ref="integer"/>
    </concept>

    <!-- This country concept is defined for educational purposes only. A country
    concept exists in the Google geo dataset. See:

    http://code.google.com/apis/publicdata/docs/canonical/geo.html --> 
    <concept id="country" extends="geo:location">
      <info>
        <name>
          <value>Country</value>
        </name>
        <description>
          <value>My list of countries</value>
        </description>
      </info>
      <type ref="string"/>
      <property id="name">
        <info>
          <name><value xml:lang="en">Country name</value></name>
          <description>
            <value xml:lang="en">The official name of the country</value>
          </description>
        </info>
        <type ref="string"/>
      </property>
      <table ref="countries_table"/>
    </concept>

    <!-- This US state concept is defined for educational purposes only. A US state
      concept exists in the Google geo US dataset. See:

      http://code.google.com/apis/publicdata/docs/canonical/geo.us.html --> 
    <concept id="state" extends="geo:location">
      <info>
        <name>
          <value>State</value>
        </name>
        <description>
          <value>US states</value>
        </description>
      </info>
      <type ref="string"/>
      <property concept="country" isParent="true"/>
      <table ref="states_table"/>
    </concept>

    <concept id="gender" extends="entity:entity">
      <info>
          <name>
          <value>Gender</value>
        </name>
        <description>
          <value>Gender, Male or Female</value>
        </description>
        <pluralName><value>Genders</value></pluralName>
        <totalName><value>Both genders</value></totalName>
      </info>
      <type ref="string"/>
      <table ref="genders_table"/>
    </concept>

    <concept id="unemployment_rate" extends="quantity:rate">
      <info>
        <name>
          <value>unemployment rate</value>
        </name>
        <description>
          <value>The percent of the labor force that is unemployed, not seasonally
            adjusted.</value>
        </description>
        <url><value>http://www.bls.gov/cps/cps_htgm.htm</value></url>
      </info>
      <topic ref="social_indicators"/>
      <type ref="float"/>
      <attribute id="is_percentage">
        <type ref="boolean"/>
        <value>true</value>
      </attribute>
    </concept>

  </concepts>

  <slices>
    <slice id="countries_slice">
      <dimension concept="country"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <table ref="countries_slice_table"/>
    </slice>

    <slice id="states_slice">
      <dimension concept="state"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <metric concept="unemployment_rate"/>
      <table ref="states_slice_table"/>
    </slice>

    <slice id="countries_gender_slice">
      <dimension concept="country"/>
      <dimension concept="gender"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <table ref="countries_gender_slice_table"/>
    </slice>

  </slices>

  <tables>
    <table id="countries_table">
      <column id="country" type="string"/>
      <column id="name" type="string"/>
      <column id="latitude" type="float"/>
      <column id="longitude" type="float"/>
      <data>
        <file format="csv" encoding="utf-8">countries.csv</file>
      </data>
    </table>

    <table id="countries_slice_table">
      <column id="country" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
      <data>
        <file format="csv" encoding="utf-8">country_slice.csv</file>
      </data>
    </table>

    <table id="states_table">
      <column id="state" type="string"/>
      <column id="name" type="string"/>
      <column id="country" type="string">
        <value>US</value>
      </column>
      <column id="latitude" type="float"/>
      <column id="longitude" type="float"/>
      <data>
        <file format="csv" encoding="utf-8">states.csv</file>
      </data>
    </table>

    <table id="states_slice_table">
      <column id="state" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
      <column id="unemployment_rate" type="float"/>
      <data>
        <file format="csv" encoding="utf-8">state_slice.csv</file>
      </data>
    </table>

    <table id="genders_table">
      <column id="gender" type="string"/>
      <column id="name" type="string"/>
      <data>
        <file format="csv" encoding="utf-8">genders.csv</file>
      </data>
    </table>

    <table id="countries_gender_slice_table">
      <column id="country" type="string"/>
      <column id="gender" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
      <data>
        <file format="csv" encoding="utf-8">gender_country_slice.csv</file>
      </data>
    </table>
  </tables>

</dspl>
1305763609000000 1339774517000000
#1145 enhancement timmcnamara ckan-backlog new Support the Handle System

The Handle System is an initiative to provide persistent references for resources. That is, it's basically a proxy system for preventing link rot.

Its documentation is here: http://www.handle.net/. Servers running CKAN could host a "Local Handle Service", which redirects a hash of a resource to an actual URL.

Some suggested use cases:

  • Researcher would like to cite where data came from
  • Agencies would like to have a way to prevent vendor lock-in from CKAN if they decide to move to another platform
1305764775000000 1339774502000000
#1146 enhancement kindly kindly ckan-v1.5-sprint-1 closed fixed make logic layer control its own state

Logic layer should not use any vdm defined state and should manage it itself.

1305829117000000 1307957527000000
#1147 enhancement kindly kindly ckan-v1.5-sprint-1 closed fixed Add expired_id to all revision tables.

Revision tables need expired_id to make querying history AND pending changes more efficient.

This involves making a session extension and a large table migration.

1305839833000000 1307957556000000
#1149 enhancement kindly kindly ckan-v1.5-sprint-1 closed fixed Change domain object modification plugin to use Session extension.

This should make it more efficient as it currently does a lot of repeating work. i.e if you change a package and a resource in the same commit it sends out 2 notifications and should only really send out 1.

1305969863000000 1306090663000000
#1151 enhancement rgrp timmcnamara ckan-sprint-2012-03-05 closed wontfix Preview for geographic data should be a map

Data viewer / previewer for a resource that has a KML file, or others, such as GeoRSS and GeoJSON, we should provide a map.

  • Support for KML files: #1458

This is non-trivial for external files as we need a way to jsonify. For files stored locally this is more of a recline issue (and will require a bit of work to either guess columns or allow user to specify them).

1306043217000000 1330908188000000
#1152 enhancement amercader amercader ckan-backlog new True support for generic CSW servers

The CSW harvesters implemented at the moment were developed with the DGU project in mind, and they assume all remote CSW servers to implement the Gemini 2 specification. Gemini 2 is the profile defined in the UK for INSPIRE complying metadata, so obviously catalogs from other countries or non-INSPIRE complying ones won't be able to be harvested.

The changes needed to support generic CSW servers (i.e. those implementing the ISO 19139 profile) are:

  • Handling the validators (right now are hardcoded in the harvester

code). This probably involves issues discussed in the CREP 3 (ticket #1134)

  • Changes in the model to adapt the specification to ISO 19139
  • Renaming objects and classes which are now Gemini-centric

List of CSW servers tested:

https://spreadsheets.google.com/spreadsheet/ccc?key=0Atp3cZFjuIOAdDBVQWRINnlfN1d0b2lleHVEdjBSb2c&hl=en_US&authkey=CNu4hsEB#gid=0

1306141334000000 1313411822000000
#1153 enhancement lucychambers lucychambers ckan-backlog closed fixed Update CKAN wiki front page

Update CKAN wiki front page - a la OpenSpending?: http://wiki.openspending.org/Main_Page

Sections should relate to different types of people using the site:

Developers, Users etc..

1306155211000000 1306941386000000
#1154 enhancement johnglover nils.toedtmann ckan-sprint-2011-10-28 closed fixed Make ckan robust against solr failure

According to pudo, a ckan with activated solr extension throws a 5xx when solr is unreachable. Instead, it should behave more like a ckan without ckanext-solr when this happens.

1306254472000000 1314287519000000
#1155 enhancement pudo pudo pdeu-1 closed fixed Harvester for data.london.gov.uk

Write a harvester for data.london.gov.uk to import catalogue metadata into PDEU. API (or at least documentation) is available at: http://sourceforge.net/projects/londondatastore/files/

1306337318000000 1306773174000000
#1156 enhancement pudo pudo pdeu-1 closed fixed Scraping harvesters for Paris and Vienna Catalogues

Import metadata from both sources into PDEU via the Harvesting framework but by scraping their respective catalogue pages.

1306337428000000 1306855111000000
#1157 enhancement pudo pudo pdeu-1 closed fixed A simple theme for publicdata.eu, CKAN

Change the visual style of CKAN to be more like these sites:

This does not include major UX work.

1306337573000000 1309804029000000
#1158 enhancement pudo pudo pdeu-1 closed fixed Put PDEU into read-only mode and remove unneeded functions

PDEU should be in a mode where:

  • Users cannot edit or create packages
  • Users cannot sign up on their own
  • AuthorizationGroups? and RevisionHistory? is completely gone from the UI
  • Groups have been repurposed as Subjects
1306337786000000 1309804060000000
#1159 enhancement pudo pudo pdeu-1 closed fixed Add RDFa to CKAN package pages

RDFa can be used as a simple way to expose linked data or at least sameAs the API version of the data but we need to make sure we do not expose anything different from the "official" representation.

1306337907000000 1307615133000000
#1160 enhancement amercader amercader pdeu-1 closed wontfix Havest CSW servers on PDEU

It could be interesting for the publicdata.eu project to add some CSW datasources. Thanks to the INSPIRE directive, there are quite a few national catalogs of geodata that could be worth harvesting. Work on this should include:

  • Compose Endpoint List (1d)
  • Harvest on staging system and evaluate which sources have useful material (2d)

First point is being performed in this GDoc: https://spreadsheets.google.com/spreadsheet/ccc?key=0Atp3cZFjuIOAdDBVQWRINnlfN1d0b2lleHVEdjBSb2c&authkey=CNu4hsEB&hl=en_US#gid=0

1306407271000000 1309804038000000
#1162 enhancement amercader amercader pdeu-1 closed fixed Define a common extra for storing the geographic extent

We need a common extra across the different harvested sources that defines the extent of the data. We will have mainly national level catalogues, but also at a lower level (region, city).

Relevant discussion: http://lists.okfn.org/pipermail/ckan-discuss/2011-May/001245.html

1306408704000000 1308647187000000
#1163 enhancement rgrp rgrp ckan-backlog new Improvements to Storage Extension

Storage is now working but there are

  • Integrate with Resources (e.g. create a resource for each file upload and give option to associate with a package)
    • Should we introduce rule that files *not* associated with a Resource are periodically deleted?
  • Allow setting of a file name/path before upload
  • Allow for file overwriting/deleting etc (how should this work -- do we want to allow this sort of thing)
  • Integrate local file upload stuff in api/auth/*

Different Backend Issues

Local file store is rather different from 'remote' storage in various ways:

  • For remote you don't want to use many buckets as there are bucket limits while for local you want to. Should we there have a single path that users provide which we then partition differently for different backends.
1306408778000000 1310133808000000
#1164 enhancement amercader amercader pdeu-1 closed fixed Cloropleth Map of European Data Availability for PDEU

A nice map in the homepage showing the availability of data across Europe

1306408824000000 1308647224000000
#1165 enhancement nils.toedtmann ckan-future new Add multi-site support to ckan

Currently, each ckan site needs its own ckan wsgi process. That eats a lot of resources where many ckan sites are served from one machine (e.g. eu3).

That would dramatically change if a ckan process could behave like multiple ckans (e.g. like Apache's "<VirtualHost?>", or tracd). Depending on the "Host:" header in the HTTP1.1 request, it would choose which local ckan ini file to obey.

I see two ways to constitute the map hostname-to-ini-file map:

  • ckan reads a set of ini files, and each ini file declares which servers names it is responsible for
  • In a global ini file, there are directives mapping servernames to ini files.

In either case there should be a global ckan ini having the default settings for all local ckan sites. Each site ini could be very short then, just having e.g. title, name, database credentials, active plugins etc.

1306413667000000 1339774466000000
#1166 enhancement amercader amercader ckan-sprint-2011-10-10 closed fixed Less stringent filters for displaying the WMS preview

Right now, for the "View available WMS layers" link to be displayed, the package needs to have an "INSPIRE=True" extra, a "resource-type=service" extra and a resource of type "WMS".

This is due to DGU (the project this was developed for) related reasons, but for a generic support, the resource type should probably be enough.

(The repository for this is ckanext-spatial, but it's not on the list)

1306422646000000 1317381932000000
#1168 enhancement thejimmyg dread ckan-backlog assigned Test system for deb packaging

Get buildbot to:

  • build the deb packages
  • install them into a fresh virtual machine
  • run smoke tests on the installed ckan
1306441994000000 1330990423000000
#1169 enhancement dread dread ckan-v1.5-sprint-2 closed fixed Link to csv/json dumps on ckan.net 1306749768000000 1306754833000000
#1171 enhancement mark.wainwright dread ckan 2.0 assigned Citation instructions on dataset and resource view pages

Some sort of citation helper. Something small on the dataset and resource page that would show how to cite.

wwaites: Some related thoughts on this from opb: http://homepages.inf.ed.ac.uk/opb/papers/ssdbm2006.pdf

timclicks: I'm looking at Dataverse for the first time[0]. It seems very popular in the social sciences. I noticed that there is a recommended citation for each dataset. For example, [1] is has this one: "Targeted Input Programme (TIP) 2000-01", http://hdl.handle.net/1902.1/SSC-MWI-TIP2000-01-M1 V1 [Version]"

Implementation

Add a small box at bottom of dataset / resource page (or in sidebar on dataset page) with title "Cite this" with contents like:

%title. %author. Retrieved %date. %site_title.

For resource: %title = %dataset_title. %resource_name.

Could also add export to ref managers (e.g. to bibtex) but that is for later.

1306920799000000 1347358705000000
#1173 enhancement amercader amercader pdeu-1 closed fixed Offer a DCat representation for packages in the API

/api/rest/package/foo.rdf should return a DCat representation of the package. To create it, we will use the functions in ckanext-rdf.

1306939639000000 1307615271000000
#1174 enhancement pudo pudo pdeu-1 closed wontfix API Representation Registry

As CKAN grows, more and more optional representations of packages will become used. Besides RDF (which is the motivation for this ticket), support for DSPL, home-grown XML, or CSV listings is imaginable.

To properly support this CKAN should have an extensible representation registry that can be extended when new output types become available (and without changing the API in the process). This needs to integrate in two places:

  • We need to add support for file format extensions to the package handler and REST API.
  • We need to add HTTP content type negotiation (Accept headers).

To support this we should have a registry with two registers:

  • to map format extensions to mime types (e.g. "json" -> "application/json")
  • to map mime types and entity types to converter functions (e.g. 8"application/json", ckan.model.Package) -> func(obj, mime_type) )

This should be set up on load_environment so that IConfigurer plugins can feed into it.

The registry should then first be added to _finish in the REST API (which needs to be refactored to be passed the {format} part of the URL if one is given. Based on the format part and HTTP headers, an appropriate representation can be generated by the registry and then be returned to the user.

Forwarding of requests to the regular WUI controllers with Accept headers set or a format specified can be implemented in a separate effort.

1307050992000000 1307615200000000
#1179 enhancement timmcnamara ckan-backlog new Support tag aliases

A small number of tags are near-duplicates of each other.

Perhaps we could support word stemming from NLTK and/or manual tag aliases:

statistics statistik ... survey surveying surveys

1307429221000000 1339774332000000
#1184 enhancement timmcnamara ckan-backlog new Support Wuala as CKAN storage option

Most of CKANs storage options are tied to the USA. This brings concerns of data security for some organisations who may wish to adopt the system. Wuala is a distributed file system that stores data in a peer-to-peer manner. The company behind it, LaCie? sells storage for a fee. However, they also enable clients to have 'free' storage space when machines act as a storage node.

In order to be a storage node, a machine needs to be online for more than 14% of the time - roughly 4h per day. Most CKAN servers are likely to have a far greater uptime than this.

Supporting Wuala would go some way to enabling CKAN to be used in a secure manner. That is, CKAN could be promoted for organisational use where there is lots of data to be stored and large geographic distances to be managed. There is a Python client available and a fairly long Google Tech Talk that overviews the system.

1308034751000000 1339774306000000
#1186 enhancement dread closed fixed Password reset facility

You can register a user with password and (optional) email address. But if you forget the password you can't then log in again. We need a password reset facility that sends and email with a new password.

1308142766000000 1310556519000000
#1187 enhancement dread dread ckan-v1.5-sprint-3 closed fixed Stop spam users with links in the 'about' field

When you edit a user, you should be stopped from putting links in the 'about' field, because this is a common tactic by spammers.

1308142999000000 1308143605000000
#1188 enhancement nickstenning ckan-backlog new Allow diffing against initial (blank) package version

Currently the history page only allows diffing between different versions of a package, but there doesn't appear to be any easy way to see the changes introduced by the first version of a package.

I'm requesting the ability to diff against a "blank slate" initial state of a project, so I can see the content of the first project commit.

Not sure if this is a vdm feature, so I'm putting this ticket in against ckan.

1308153160000000 1339774275000000
#1189 enhancement timmcnamara closed invalid Spideroak support in CKAN Storage

Spideroak is a bulk storage platform by a company that releases quite a lot of free software, has an extensive developer API with pricing of $10/100GB/month.

The data store is optimised for bulk data storage and retrieval. This is the kind of use case that CKAN packages require. We don't need low latency. We are after low cost, high reliability solution.

1308187104000000 1323173227000000
#1190 enhancement rgrp rgrp ckan-v1.5 closed fixed [super] CREP 0004 Data API and Data Processing System

For some time (e.g. 1y+!) we have known that we want to integrate some kind of datastore / data processing system with CKAN. We've had a CREP in progress on this for some months (may copy that here at some point):

http://wiki.ckan.org/CEP0004

We can distinguish 3 modules that are needed:

  1. "Webstore": A datastore with dataapi - #1208

Suggestion is this would be sqlite based with a simple sql based API. http://ckan.net/api/data/{user|org}/{datastore_name}?q={some-read-sql-query}

  1. Automated conversion of suitable resources into datastore upon resource creation so that e.g. they are accessible via the API. #1398
  1. A data processing system which utilizes this datastore. One could

get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules) - #1432

More info

1308227611000000 1323270522000000
#1193 enhancement kindly kindly closed fixed bug with new formalchemy causing all resouces to be deleted

The new formalchemy stores dict in a new format.

1308525655000000 1309768960000000
#1197 enhancement timmcnamara ckan-backlog closed wontfix Add JavaScript guide for CKAN

A new library, guiders.js, has been open sourced that seems to be great at unobtrusively introducing new users to features of websites.

The library drives Optimizely's website. The GitHub repo is here: https://github.com/jeff-optimizely/Guiders-JS

Some examples:

http://posterous.com/getfile/files.posterous.com/temp-2011-06-22/BnlgHDvhjpFfqbwvprsIaklqrpdgJyebfuhxajbDhmvnFgzdzzCJdvluJFki/Screen_shot_2011-06-22_at_6.42.21_PM.png.scaled500.png

http://s3.amazonaws.com/files.posterous.com/temp-2011-06-22/CpvlcinkrxhtnAHfGtEHhxyGkraqlFmCjcnztFuugagBGxmBjCdwuquHpvEB/promo_gmail.png

1308801032000000 1340020357000000
#1198 enhancement dread ckan-backlog new Publisher hierarchy

'Publisher' entities in the model. They are hierarchical.

'User-Publisher' connections with one or more roles (e.g. drafter, moderator).

Authorization settings can control who can set what values in a 'published by' type field.

Publishers and User-Publishers available to read in the API.

Future tickets will provide:

  • API to write Publishers and User-Publishers
  • UI to edit Publishers and User-Publishers

(This feature deprecates authorization groups)

1308820592000000 1339774200000000
#1199 enhancement pudo pudo closed fixed Implement an email function for users.

We need an email function in CKAN to accept messages sent to users. The basic signature will be:

  • mail_user(user_obj, subject, body, mime_type='text/plain', headers={})

This has a number of use cases:

  • Retrieval of lost passwords
  • E-Mail confirmation

Finally, the mail function should be exposed in the API for sysadmin clients. This way we can have scripts traverse CKAN for 404s, invalid data or missing fields and ping users about that automatically (requires traversal by revision, not package, to get the associated users).

Implementation

Note we have already written code like this (*and* tested it) in isitopen:

1308821986000000 1310555589000000
#1200 enhancement pudo closed duplicate Port PDEU theme to CKAN.net

PDEU has a newer layout which could be adapted to also be used on ckan.net. For this, things should be cleaned up, copied to the ckanext-ckan.net repo as needed and re-colored the core CKAN color scheme.

Maybe we could have this coincide with a CKAN rename?

1308824017000000 1311180218000000
#1201 enhancement kindly ckan-backlog new seperate out logic in atom feeds to logic layer.

Simplify the logic in the atom feed an make all feeds use logic layer to return lists.

1308928892000000 1310124297000000
#1202 enhancement rgrp rolf closed invalid Links to datapkg utility don't lead to info about it

In the presentation of data packages, the link in "The information on this page and the downloads / resources are also available using the datapkg command line utility." (http://knowledgeforge.net/ckan/doc/datapkg/install.html) ends up at the home page of ckan.org

Perhaps link it to https://bitbucket.org/okfn/datapkg/overview for now, to save some searching for people who want to find out more?

1309263613000000 1315821804000000
#1204 enhancement timmcnamara closed invalid renaming packages seems to result in a server error

When renaming a package, and doing other changes at the same time an error is reported. However, the changes seem to go through fine.

ENV:

  • datacatalogs.org as of 29 June 2011
1309303816000000 1311179980000000
#1207 enhancement dread dread closed fixed ckanclient.package_entity_get should raise more specific exception

When package does not exist in ckan catalogue, ckanclient.package_entity_get should raise more specific exception, such as CkanNotFoundError? instead of generic CkanApiError?.

1309515582000000 1311325343000000
#1208 enhancement rgrp rgrp ckan-sprint-2011-10-28 closed fixed Webstore: a datastore with web API
  • SQLite based
  • Use Cyclone for async http server on top with auth
  • Designed to not be specific to CKAN or anything else

Current code is here: https://github.com/okfn/webstore.

API Spec

Read

Two basic ways to query::

    GET: /{owner}/{db-name}/?sql=...
    GET: /{owner}/{db-name}/?table=...&attr=value&attr=value&limit=...

Returns::

  {
      u'keys': [u'id', u'name'],
      u'data': [
          [1, u'jones'],
          [u'aaa', u'jones']
          ]
  }

Write

POST to::

/{owner/{database}/{table}

Payload is json data structured as follows::

{

unique_keys: [list of key attributes] data: {dict of values}

}

Authentication and Authorization

Authentication: use basic auth header.

Authorization:

  • Default: all read, owner can write
  • Restricted: owner can read and write, everyone can do nothing

Possible future: config file can specify a python method (TODO: method signature)

Integration with Other Systems

TODO: Specify how to delegate authenatication to user database in some other system.

1309804460000000 1312191646000000
#1211 enhancement kindly kindly ckan-v1.5 closed fixed [super] Drupal integration

Sub-tasks / tickets:

  • #1335 Action API - access to any logic layer function
  • Standardize logic functions on ordering: (context, data_dict) and put parameters into data_dict.
  • Add package_create_validate & package_update_validate logic functions
  • Allow extensions to provide logic layer functions (IActions)
1309958442000000 1315948703000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Note: See TracReports for help on using and creating reports.