CKAN: Ticket Query

#1: Visitor visits service

rgrp — Mon, 10 Jul 2006 16:36:57 GMT

Title

Visitor visits service

As a

Visitor (general web user)

I want to

Visit the website of the service (frontpage)

So that

See the front page and discover information about the service
Perform various activities related to the service such as registering and finding packages, registering as a user and logging in

#2: Register as user on the service

johnbywater — Mon, 10 Jul 2006 16:40:35 GMT

As a

Visitor

I want to

Be able to register as a user on the service

So that

One can login and be an authenticated user and perform the actions that only authenticated users can perform

Notes

Registration involves the provisional of a minimal set of user details, namely:
1. username (unique in lifetime of the service)
2. fullname
3. email
4. password

#3: Login to the service

johnbywater — Mon, 10 Jul 2006 16:41:57 GMT

As a

Visitor

I want to

So that

I am now an authenticated user and may perform the actions that only authenticated users may perform

Notes

Login by providing username and password
Login screen should also include a link or information about registration
If username do not match (or the username does not exist) a login failure is noted and the visitor is re-presented with the login screen as well as information on how to register

#4: Find a package

johnbywater — Mon, 10 Jul 2006 16:43:40 GMT

As a

Visitor

I want to

Locate packages that I am, or might be, interested in

So that

I know what packages exist and can read the information associated with those packages

Notes

Find a package by searching by (or some combination thereof):

tag
string contained in title or description
license

#5: Browse list of packages and select one to view

johnbywater — Mon, 10 Jul 2006 16:44:44 GMT

As a

Visitor

I want to

Browse a list of packages resulting from a search or browse request (see other use cases)

So that

I can select one of the packages to view in more detail (-> viewing an individual package ticket:6)

Notes

When browsing a list of packages you should be able to see summary information about the package such as title (though this may be shortened in order to conveniently fit the list
The list should be broken up into pages so that the number of packages per page should be kept to a reasonable number (<= 50). Response time should be kept reasonable

#6: View a package

johnbywater — Mon, 10 Jul 2006 16:46:00 GMT

As a

Visitor

I want to

View the details of an individual package

So that

I know more about the package. Attributes available are detailed in updating a package (../UseCase12)

#9: Register new package

johnbywater — Mon, 10 Jul 2006 16:49:31 GMT

As a

Authenticated User

I want to

Create/register a package

So that

The package will be listed in the system and visible to users

Notes

Full package registration process may be broken down into:
1. Creating a package by provision of minimal attributes:
  1. name: unique and web-usable
  2. auto-generation of default values such as:
    1. date-registered (today's date)
    2. first owner (one owner should be created corresponding to the authenticated user performing this action)
  3. (?) auto-generate a default release
2. updating the created package -> ../UseCase12

#11: Delete package

johnbywater — Mon, 10 Jul 2006 16:52:08 GMT

As a

Authenticated user

I want to

Delete a package

So that

So that the package is no longer displayed for visitors of the site

Notes

The permissions for this action are the same as for updating the package details (see ticket:10)
When a package is deleted it is put in an inactive state and is invisible to users and vistors of the service. However its name remains reserved and it may be undeleted or purged by the service administrator

#20: Purge a package

johnbywater — Mon, 10 Jul 2006 17:01:50 GMT

As A

Administrator

I want to

Purge a deleted package

Notes

this need only be possible via the admin interface

#21: Undelete a package

johnbywater — Mon, 10 Jul 2006 17:02:16 GMT

#24: View list of most recently registered or updated packages on front page

johnbywater — Mon, 10 Jul 2006 17:08:11 GMT

#25: Autocompletion (and suggestion) of package tags when adding tags to a package

johnbywater — Mon, 10 Jul 2006 17:08:33 GMT

Notes

Currently have a text box where users can add space separated tags
Tags should auto-complete in a manner similar to delicious
would also be good to have a list of suggested tags. This could either be:
- [simpler] a list below the tag box chosen by existing popularity or
- [more complex] a list of suggestions that updates as you add tags (based again on popularity but 'filtered/reranked' to reflect choices so far).
- For first attempt to should probably just go for the simpler option.

#27: Register/Update a package via a web-api

johnbywater — Mon, 10 Jul 2006 17:09:39 GMT

As A

User

I want To

Create/register and update a package via a web api.

#29: Improve installation process

rgrp — Mon, 10 Jul 2006 17:47:18 GMT

install documentation (currently source:trunk/INSTALL is empty)
improve ckan-admin as necessary
create default data (such as licenses)

#30: Tags are added to a package by package owner

rgrp — Fri, 18 Aug 2006 12:55:53 GMT

As A

Site User

I Want To

Create/Edit/Update/Delete? tags associated with my package

Notes

Tags are service-wide and not package specific
Tags should not allow spaces (but allow everything else e.g. ':')
web interface details
- should provide a text box where users can add space separated tags (a la delicious).
- tag creation should be seemless (so if a tag does not already tag exists it should automatically be created in the background)
- would like to provide a list of relevant existing tags. For time being could just list all existing tags below entry box but in long run would like auto-updating/auto-suggesting listing a la delicious

#32: View a tag

rgrp — Mon, 04 Sep 2006 13:19:28 GMT

As a

Visitor

I want to

View a tag's page which should list all packages associated with that tag (or link to a page containing that list).

Notes

The tags should have a nice primary url such as /tag/<tag-name> (could also use the plural i.e. /tag/<tag-name>)

#33: List and search tags

rgrp — Mon, 04 Sep 2006 13:19:43 GMT

As a

Visitor

I want to

View a list of the tags on the system and search for a tag by name

Notes

As for a search of packages should search for tags that contains a given search string (or could use a glob approach allowing wildcards).

#35: IP Address for non-logged in users is always 127.0.0.1 when deployed behind a proxy (e.g. on ckan.net)

rgrp — Thu, 19 Apr 2007 18:17:13 GMT

When IP address is used to identify non-logged in users (for example on create and editing of packages) it is always 127.0.0.1 when using a reverse proxy (standard deployment configuration on a production machine and used e.g. on http://www.ckan.net).

This could be corrected by using HTTP_X_FORWARDED_FOR when available.

#36: Edit package page should support previews

rgrp — Thu, 19 Apr 2007 18:25:12 GMT

When editing a package one would like to be able to preview one's changes before saving.

#37: Purge a Revision (i.e. purge all changes associated with that revision)

rgrp — Wed, 17 Oct 2007 19:51:00 GMT

Spam entries have started to occur on ckan.net in the last few months. It would be useful to be able to purge these revisions -- that is permanently delete all changes to domain objects associated with those revision (one might also permanently delete that revision -- or alternatively simply mark it as purged).

In addition to providing this facility from the shell it would also be useful to be able to do this from the web interface (with associated restrictions on usage via an authorization controller of some sort).

#38: Make data available in machine-usable form

rgrp — Thu, 08 Nov 2007 11:33:34 GMT

Currently all data is only available via the web interface which makes it difficult to obtain in bulk. Data should be provided in other more convenient, machine usable forms:

DB Dump
JSON for each package page (maybe Atom as well)

#39: When listing packages show more information than just name.

rgrp — Fri, 09 Nov 2007 11:48:40 GMT

When a package listing is shown (as on http://ckan.net/package/list) more information than just name should be shown. For example package titles should be shown as well as names (could also list tags and other stuff but simplest is just title).

Cost: 1

#40: Reserved html characters (such as &) in urls mean package does not render for read view

rgrp — Tue, 20 Nov 2007 13:27:08 GMT

A url such as: http://someurl.com/xyz?x=1&VERSION=1.1&Service=WFS when set as url or download_url breaks the rendering of the package with an error like:

There was an error rendering the package: not well-formed (invalid token): line 1, column 181

Have checked that removing the & stuff makes the error go away so this looks like an issue with escaping urls when displaying them ...

#41: Upgrade to Latest Version of Pylons (0.9.6.*) and Deploy on Production Machine

rgrp — Tue, 08 Jan 2008 10:07:34 GMT

Should do this sooner rather than later to avoid 'clear water' opening up and need to deploy then asap onto production machine to avoid 'clear water' between trunk and production (which then prevents pushing from trunk to production).

#42: In WUI where List of Items to Display is Large Results should be Paged

rgrp — Tue, 08 Jan 2008 10:12:42 GMT

As A

Visitor

I Want To

View lists of items but not have too many on a page (as this leads to slow page loads and difficulty navigating the list). Where there are more items in the results of my action that can be fit on a single page the results should be 'paged' in some manner.

Details

Main place this issue arises:

List of tags
List of packages
Repository history (/revision)

Should probably have no more than 50 (or so) items on a page. Obvious solution is for actions that return lists to have some kind of pageno parameter and then use this in controller to select what to display.

#43: Generic Attributes for Packages

rgrp — Tue, 08 Jan 2008 10:28:29 GMT

As A

User

I Want To

Add arbitrary named attributes to packages (an attribute being a name, type, value triple).

Details

We will do this using a dedicated (versioned) table associated to Package
Do we allow multiple attributes of the same name?
- For the present: No (since we will key by attribute name)
- Could allow for single attribute but with multiple values using json list ...
What types do we allow or do we just rely on JSON to take care of this?

Questions (Original)

How complex is this to implement?
What would an arbitrary user be able to edit? Possibilities:
1. 'create new attribute' and setting the value (so name and type would be chosen from predefined list).
2. 'create', setting of name and value (but not type -- type already set in predefined list)
Could just use (machine) tags -- though this could be seen as a bit of a hack.
Would solve having to create special file/url attributes (though I think that perhaps file stuff is important enough to merit first class support in the domain model -- though, that said, since one won't want to have a file limit adding unlimited file support is very similar to unlimited attributes of arbitrary type).

#44: Provide RSS/Atom Feed of Repository History

rgrp — Tue, 08 Jan 2008 10:34:17 GMT

As A

Visitor

I Want To

Get an RSS/Atom Feed of the Repository History to use in my feed reader (or elsewhere).

Details

Preference for Atom.
should just add parameter to /revision/list/ (or /revision/) to select atom format e.g. ?format=atom.
should have a 'days' attribute specifying number of days back to go e.g. &days=30

Cost

Low

#45: Immunity to SQL injection attacks

johnbywater — Wed, 23 Jan 2008 17:54:54 GMT

#46: Return error documents that look and feel like normal CKAN pages

johnbywater — Wed, 23 Jan 2008 17:56:58 GMT

#47: Return OpenID signin pages that look and feel like normal pages

johnbywater — Thu, 24 Jan 2008 14:12:00 GMT

#48: Create favicon

johnbywater — Thu, 24 Jan 2008 19:26:29 GMT

Because CKAN's worth it!

#50: Investigate apt-get in more detail

rgrp — Mon, 07 Jul 2008 18:40:52 GMT

Research apt-get especially wajig and python wrappers for apt-get. Post results on the wiki.

#51: Upgrade CKAN to use sqlalchemy and vdm v0.2

rgrp — Mon, 08 Sep 2008 19:00:38 GMT

Move CKAN codebase to use sqlalchemy (rather than sqlobject) along with vdm v0.2.

NB: the upgrade itself should not be too hard, the work will be in providing a satisfactory migration script to convert existing data on ckan.net service.

#53: View Information about Package history (i.e. package revisions)

rgrp — Thu, 09 Oct 2008 11:35:39 GMT

As A

Visitor/User?

I want to

View information about a package history, i.e. what revisions it has.

Details

History info should be available at package/history/pkg-name
Most basic info is just info from revision (when it occurred, author etc)
More sophisticated would be to have diff support to see differences between revisions

#54: Support dump and load of CKAN data to JSON

rgrp — Mon, 13 Oct 2008 14:24:43 GMT

As a

Sysadmin

I want to

Dump (serialize) CKAN data to a simple transport format (e.g. JSON) and be able to load it again.

Details

Already have simple db dump. However dumping to JSON has various advantages, particularly where changes to the data need to be made upon reloading (e.g. during a migration).
Dump should include *all* CKAN data (i.e. all the data in the CKAN db tables)

#55: Code to migrate data from v0.6 to v0.7 using dump and load

rgrp — Mon, 13 Oct 2008 14:30:40 GMT

Associated to ticket:51 (upgrade CKAN to new vdm) and ticket:54 (dump/load) need to convert v0.6 data for v0.7.

Obvious way to do this is via alteration to data load method.

#56: Show icons indicating package openness

rgrp — Thu, 25 Dec 2008 13:18:43 GMT

As A

User or Visitor

I Want To

See simple icons (e.g. tick/cross) next to packages (in lists or on main page) indicating openness status of package.

So That

It is easy to see the openness status of a package.

Details

Suggested by Liz Turner
Openness = open license at present (no easy way to determine whether accessible)
Have separate tick for downloadability (i.e. existence of download url)
Could do more stuff in future (e.g. money where tag price- exists, etc etc)

#57: Basic datapkg integration with CKAN

rgrp — Fri, 03 Apr 2009 09:27:06 GMT

Using datapkg a user can:

list packages on CKAN
get info about package on CKAN
register and update a package on CKAN

#58: Make bookmarklet to enable easy package registration on CKAN

rgrp — Mon, 06 Apr 2009 11:48:37 GMT

Make bookmarklet to enable easy package registration on CKAN

Details

This requires redoing web interface to allow 1-step package creation/editing (as opposed to name then everything else approach at the moment)

#59: Provide a basic guide to CKAN on the site

rgrp — Mon, 06 Apr 2009 11:49:43 GMT

Use contents from ckan package notes section (rgrp)
Update that section to be more guide-like (jwyg)

#60: Front page usability improvements

rgrp — Tue, 07 Apr 2009 08:51:06 GMT

List in sidebar 3 latest created/updated packages.

Put package search box on front page (might also want to move some of front page content into an about page).

Similarly might want package creation box on front page.

#61: When dumping data to json do not dump private information like API keys

rgrp — Tue, 07 Apr 2009 16:58:49 GMT

Cost: 1h

#62: Change tags to contain any character (other than space)

rgrp — Fri, 24 Apr 2009 14:58:15 GMT

Requires us to url encode the tag names when displaying them ...

#63: Purge is broken after upgrade of vdm

rgrp — Tue, 16 Jun 2009 10:37:03 GMT

Since we upgraded to new vdm (v0.4) which has built in purge support existing purge support looks to be broken. Should be reasonably easy to fix this.

Cost: 1h

#64: Switch to repoze.who for authentication

rgrp — Tue, 16 Jun 2009 16:14:13 GMT

Switch to repoze.who + openid plugin for authentication from authkit.

Already did this in microfacts so should be fairly easy.

#65: Add version attribute to package

rgrp — Wed, 17 Jun 2009 18:21:04 GMT

It is essential packages have a version attribute to assist in tracking releases etc (and to allow compatibility with other packaging systems e.g. distutils which is being used by datapkg).

cost: 2h

#66: Improve user account UI

rgrp — Wed, 17 Jun 2009 18:34:45 GMT

Link at top should say "Your Account" when you are logged in
- when not logged in should say: "Log in via openid"
- Open Id info on /account/ should move to /account/login/ or be deleted
- Going to /account/ when not logged in should redirect to /account/login/

When logged in the basic account index page should not give generic information but should reflect fact you are logged in by:

Saying something like: "You are logged in as: ..."
Showing your apikey or a link to apikey
Giving you a logout link

We can think of a bunch of other stuff that could go on their (recent edits, packages you own etc) but these will be separate tickets.

#67: List all of a user's recent edits on their home page

rgrp — Wed, 17 Jun 2009 18:35:31 GMT

Additional feature related to ticket:66.

cost: 2h

#68: Show affected packages in revision list view

rgrp — Mon, 22 Jun 2009 07:35:53 GMT

When listing revisions (at /revision/ or in atom feed) show affected packages. Minor but v. useful UI improvement.

Cost: 1h

#69: Change to text-only license field and use external license repo

rgrp — Mon, 22 Jun 2009 16:17:29 GMT

Switch from license domain object to a simple license field and use license list from new centralised license repo:

<http://knowledgeforge.net/okfn/licenses/>

This will require a migration

Cost: 4h (plus migration ...)

#70: Convert from py.test to nosetests

rgrp — Wed, 01 Jul 2009 07:52:33 GMT

Cost: 1h

#71: Upgrade to Pylons 0.9.7

rgrp — Wed, 01 Jul 2009 07:57:21 GMT

Cost: 2h

#72: Integrate new logo

rgrp — Wed, 01 Jul 2009 09:37:01 GMT

Integrate new logo from http://wiki.okfn.org/ckan/logo into site:

In title

Also can do favicon (separate ticket:48)

Cost: 1h

#74: Add Is It Open links to package pages

rgrp — Fri, 17 Jul 2009 10:56:58 GMT

On each package page in the openness field if the package is not open provide a link to is it open create enquiry page.

Details:

may want to add to the url a query parameter indicating which package this relates to e.g. ?ckan-package=...
http://isitopen.ckan.net/enquiry/create/?ckan-package=...
do we provide link even if package is open (e.g. so that people can make enquiries about e.g. adding open data/content buttons ...)

Cost: 0.5h

#76: Convert to use formalchemy for all forms

rgrp — Fri, 17 Jul 2009 11:10:41 GMT

Current form stuff is not very good (uses formencode). Switch to formalchemy would improve this, especially on validation.

As an extra we could utilize the formalchemy pylons admin interface (pretty much for free).

Cost: 12h

Details

Replace htmlfill and formencode extract in controllers/templates with formalchemy
1. Crude and simple
2. Suppress unwanted fields (revision, state, all revisions) and sort out ordering to be similar to before.
3. Sort out tag field with a special renderer
Validation - testing definitely required.
Fix up description and pretty css etc

#77: Rate packages

rgrp — Fri, 17 Jul 2009 11:13:13 GMT

As a User or Visitor

I wish to rate a package in some way. On the package view, below Openness, there is a 'Rating' heading with the star rating which is selectable.

Rating out of 5. One user or IP, one vote.

package_rating table:

#78: Extend login cookie lifetime

rgrp — Fri, 17 Jul 2009 11:15:10 GMT

When you login you get given a cookie but it is very short (life of your browser session I think).

Work out how to extend to something reasonable (30 days or perhaps forever with logout unsetting cookie).

A quick investigation to see whether this is possible
If possible do it

Cost: ?

#79: Improvements to package search

rgrp — Fri, 17 Jul 2009 11:24:24 GMT

Several things:

Default package search (used e.g. on front page) should deal well with multiple items: "abc xyz" means search for both abc and xyz (as separate searches)
Should automatically display a list of matching tags in a line at top with number of associated packages
have an option to only search for fully open material (or openly licensed material ...)

Since we should be moving to a more sophisticated search solution anyway (to allow prioritisation etc) maybe we should cautious as to what we do now.

#80: Refactor or remove modes code

rgrp — Fri, 17 Jul 2009 15:24:23 GMT

Get rid of modes in the RESTful API. Do json stuff directly in controllers.

package will have to_dict and from_dict methods (called by rest controller) which provide and consume JSON friendly dictionaries representing the object. from_dict - class_method. stuff in forms for tags as_string, maybe factor out. reuse validation stuff.

#81: Support for sqlalchemy 0.5

rgrp — Wed, 22 Jul 2009 17:57:13 GMT

#82: Support diffing of versioned objects

rgrp — Wed, 22 Jul 2009 19:04:59 GMT

Should have function/facility to get a diff of a version object between 2 revisions. Function should return a dictionary of fields with diffs.

Details:

Do we deal with m2m relationships (and m2one on the many side)?
What is diff?
- For text fields output of python diff command
- For non-text field guess just simple +/- for what was there before and now

Cost: 4h (don't think this is a huge request)

#83: Allow "ignored" fields on versioned objects

rgrp — Thu, 23 Jul 2009 08:56:24 GMT

Allow 'ignored' fields on versioned objects (i.e. attributes which are not 'versioned'). This is not hard to do as we already have most of the necessary mechanisms set up in the Revisioner object.

Cost: 2h

#85: Convert state from an object to an enumeration

rgrp — Thu, 23 Jul 2009 09:01:02 GMT

No real benefit of having State object as opposed to a simple text field with enumerated values and this second option is both simpler and more flexible.

Main hassle here is that it requires migration in all projects that use vdm.

If we're going to do this we should do it sooner rather than later.

Cost: 2h (+ 1h for a conversion method)

#86: Support for stateful dict-like collections

rgrp — Fri, 24 Jul 2009 10:19:58 GMT

Already support stateful list-like collections and should extend this to dict-like collections.

(This is prerequisite for implementing versioned "extra" (key/value) attributes on packages in CKAN and elsewhere).

#88: Download (link) metadata

rgrp — Mon, 27 Jul 2009 11:16:25 GMT

Support for download metadata (file size, file formats, version information).

See also: ticket:87 (multiple download links)

#89: Tag cloud for package tags

rgrp — Mon, 27 Jul 2009 11:25:39 GMT

Should not be hard to do (lots of existing libraries) but not sure that this is very important.

#90: Link to RDF versions of CKAN data

rgrp — Thu, 30 Jul 2009 08:49:02 GMT

We should link to representations on semantic.ckan.net

Each package page could link to its RDF representation
- "Alternative formats" near star rating. Has logo for RDF (link to RDF) and JSON (link via CKAN REST i/f).
- Some instances of ckan won't need this, so have option in ini file for this. rdf_store_url=xyz
- Include machine-readable link to the RDF:
```
<meta http-equiv=... />
```
- 303 redirect to RDF when content negotiation requires RDF. Test:
```
curl -I -H "Accept: application/rdf+xml" http://localhost:5000/package/warandpeace
```

Cost: 3h

Future

Search page to link to SPARQL interface: http://semantic.ckan.net/sparql
License/dump page could link to full RDF dump.

#91: Add author and maintainer attributes to package

rgrp — Fri, 31 Jul 2009 14:16:20 GMT

Add the following attributes to package:

author, author_email
maintainer, maintainer_email

Gives us full compatibility to: http://docs.python.org/distutils/setupscript.html#additional-meta-data

Column ordering - should come after name, title, url, download_url.

#93: Access control for packages

rgrp — Fri, 31 Jul 2009 15:44:09 GMT

Should be able to limit ability to a user's ability to do things with packages (read, edit etc). This is a big ticket (it may required splitting) and full details are in separate wiki page: AccessControl.

Cost: 8d

#94: Use sqlalchemy-migrate to handle db/model upgrades

rgrp — Sat, 01 Aug 2009 13:43:05 GMT

Details of how to go about this here <http://www.rufuspollock.org/2009/07/27/sqlalchemy-migrate-with-pylons/>

Cost: 2h

#95: Add manifest support

rgrp — Tue, 11 Aug 2009 09:10:42 GMT

Support for listing files (manifest) contained within a package.

Traditional manifests just list the files. Suggest in addition we have support for optional metadata in form of key, value pairs.

#98: Replace use of pastescript templates in PythonDistribution with of something simpler

rgrp — Tue, 11 Aug 2009 09:43:30 GMT

Replace use of pastescript templates in PythonDistribution? (write) with of something simpler (allowing us to remove dependency on pastescript).

#100: Convert existing data on disk to a datapkg distribution

rgrp — Tue, 11 Aug 2009 10:30:28 GMT

(2008-10-29) convert existing directory on disk to a datapkg distribution (PythonDistribution? by default). This would be presented in the CLI as a convert command.

Details:

Add a metadata file (setup.py), perhaps prompting for input
List all existing data in that directory

Cost: 4h

#101: (Improved) Download support

rgrp — Tue, 11 Aug 2009 11:07:46 GMT

Given a url download from that url to disk
Given a package use download url to download to disk

Once on disk should uncompress (if necessary).

Details

Should download either to tmp directory (defaulted in config) or a specified path on disk
In download would like to support as many types of target urls as possible:
1. Single files (usually compressed filesets but could be individual files)
2. Index pages listing files to download
3. Revision control repositories
4. ...

This will obviously be a significant amount of work. Suggest:

Start off just supporting the first option.
Have something like a plugin system to handle new target types
Reuse existing work (e.g. easy_install and pip handle standard vcs such as svn, hg, git etc)

Cost: 1d

#102: Switch to using UUIDs for revision ids

rgrp — Wed, 12 Aug 2009 09:09:35 GMT

This is better for long term sustainability, especially if we want to "push and pull" from unrelated repositories in the future.

Details:

May wish to retain a number field updated in the usual incremental way (what's the cost/benefit here?)

#103: View a package at a given revision

rgrp — Wed, 26 Aug 2009 12:31:37 GMT

As a user I want to view a package at a given revision:

When I visit /package/read/xyz?rev=yyy I should be shown package at revision yyy
package history page should provide links to these pages

Cost: 2h

#104: View a package at a given version

rgrp — Wed, 26 Aug 2009 13:17:52 GMT

As a user i want to see a package at "version" X (NB: not revision X).

When I visit e.g. /package/read/xyz?version=0.7 I should be shown package at version 0.7 (or a message saying no such version)
Implementation:
- Find revision for this version (search revision history for when version field was last 0.7)
- Show pacakge at that revision (as in ticket:103)
On history page also shows versions in list of revisions associated with the package

#105: Package groups (view)

rgrp — Thu, 27 Aug 2009 16:58:02 GMT

As a user I want to be able to create groups of packages.

This functionality is different from tags:

The ability to add a package to a given group is restricted whereas anyone can add a given tag to a package
Specifically groups have owners and only the owners can add a package to that group

Group properties

id (primarykey, uuid style)
name (restricted content - same rules as package)
title (no restrictions)
owners -- many:many with user object
description (markdown)
packages -- many:many with package object

Don't version groups for now.

Groups address in the WUI will be:

/group - browse list of groups (reuse list action)
/group/list - same thing for now
/group/<groupname> - display: group properties with links to packages. No links to user pages (yet).

Group editing and searching will be another ticket.

Cost: 3d

#108: Package search in the REST API

rgrp — Wed, 02 Sep 2009 18:24:05 GMT

Add package search facility in the rest api at /api/search

Queries can be provided as for the normal package search either by posting to that url or by performing a get with a query string.

E.g. .../api/search/package?q=xyz

Query parameters:

q is keyword string (searches name, title, tags by default)
- split by words and ANDed
additional parameters for specific fields
qjson is alternative to q for Searching by specific fields (in addition to keyword string). Payload to json is a JSON-encoded dict which is a dictionary with a q field for free text (keywords) and additional key/value pairs for specific fields
- if q and qjson specified ignore q and just process qjson value

Additional parameters in addition to query ("q" or "qjson") are:

limit
offset
fullinfo=0/1 - return full record for each result (default=0)
order_by=field_name
search_notes=0/1 (default 0) do we search notes field in a keyword search

Return value is json encoded dictionary with keys:

'results': list of results
'count': total number of results

Extras (for the future)

Prioritisation is not part of this ticket (requires fulltext support in DB or in external app such as Xapian)

#109: Tag search in the REST API

dread — Thu, 03 Sep 2009 11:11:37 GMT

Similar to ticket:108 but for tags:

/api/search/tag?q

For tags search is extremely simple since you can only search by name.

Possible Extras

Allow "exotic" ordering of results e.g. ordering by number of packages with that tag.
Return this number with tag list.

#110: Integrate groups into packages

dread — Mon, 07 Sep 2009 09:32:36 GMT

Package needs a groups property (backref Group.packages).
Put groups in the search api.
Display them on the package, but not editable. (Only editable from the group page.)

Follow up to ticket:105 (Groups).

Cost: 1d

#111: Create user object

dread — Mon, 07 Sep 2009 09:33:14 GMT

Object properties:

id (uuid)
apikey (uuid) -- migrated from the apikey table
name (username = openid)

/account is being renamed to /user in the wui.

#113: Simple script to test data migration on a production db

dread — Mon, 07 Sep 2009 09:42:03 GMT

Must not alter the db.

Related to ticket:112.

#114: Access Control - model

dread — Wed, 09 Sep 2009 11:08:47 GMT

Create in the model basic operation of Access Control.

roles table

name | context | action
-----------------------
admin| package | edit
admin| package | edit-permissions
admin| package | read
editor| package | update
editor| package | read
reader| package | read
This data is set-up on db init and will have no interface.

user-roles table:

username | context_type | objectid | role
rgrp     | system  | n/a | admin
visitor  | package | * | reader
bob      | package | geonames | admin
visitor  | package | geonames | editor
visitor  | package | geonames | reader
john     | group | ukgov | admin
dread    | group | ukgov | editor
visitor  | group | ukgov | reader
This data will be added when someone is given permissions for the system, a package or a group.

Pseudo code:

class Package

def is_allowed(name, action):

is_allowed(name, action, context=self)

class Group

def is_allowed(name, action):

is_allowed(name, action, context=self)

def is_allowed(name, action, context=None): name: string - a username or IP for 'visitor'

action: string - 'read', 'edit', 'delete', 'edit-permissions' context: object - a Group or a Package or None (which means system)

# look up user from name. # look up in user-roles table what roles this user has for this context. # for each roles, look up in roles table what actions are allowed. # return True if action is allowed, else False.

#115: Access Control - wui constrained by model

dread — Tue, 15 Sep 2009 17:06:34 GMT

Based on a section of AccessControl design: Reading and writing to packages in the WUI and REST API are now dependent on the authz tables.

#116: Access Control - edited in wui

dread — Tue, 15 Sep 2009 17:13:22 GMT

Based on a section of AccessControl design: WUI gives controls to user and administrator to change permissions on a package.

#117: Access Control - group core functionality

dread — Fri, 18 Sep 2009 10:55:33 GMT

Based on a section of AccessControl design: Group reads and edits are controlled by access control. WUI and REST interfaces covered.

#118: Use paginate in webhelpers

dread — Fri, 18 Sep 2009 11:34:17 GMT

Take out import of paginate in setup.py. Use paginate in webhelpers instead. Make changes to take account of any i/f changes.

#119: Ensure non-active packages don't show up

dread — Mon, 21 Sep 2009 10:36:54 GMT

Ensure pending packages don't show up in search or browse

cost: 4h

#120: Security audit

dread — Mon, 21 Sep 2009 10:37:07 GMT

Look for all places where model is accessed and check authorization is checked.

Document holes (and, as necessary, suggestions for fixes) as new tickets. Likely areas that need looking at:

search i/f
package WUI commit
Write holes are obviously much more significant to us than read holes.

#121: Add 'Group' to main menu

dread — Wed, 23 Sep 2009 08:33:47 GMT

And associated page to browse group.

#122: Add Group authz page

dread — Wed, 23 Sep 2009 08:34:02 GMT

#123: Ability to edit Group in WUI

dread — Wed, 23 Sep 2009 12:14:01 GMT

Add Group editing page.

If no permissions to change group can't edit group. Also cannot view edit page.

Editable attributes: name, title, description

No preview needed

#124: Display Generic Package Attributes in WUI

rgrp — Wed, 23 Sep 2009 12:41:42 GMT

Split out from ticket:43

#126: Change package state in the WUI (delete and undelete)

dread — Thu, 24 Sep 2009 10:52:51 GMT

As a Package Admin I want to change the state of the package. In particular I wish to delete and undelete it.

(NB: this is quite separate from "purging" objects which is the term we shall use for irrevocable removal of an object from the domain model).

Only Package Admins (and sysadmins) should be able to change state
Implementation Suggestions

'delete' action should be renamed to 'change-state' (NB: this requires a db migration ...)
Have new package formalchemy form (created via inheritance?) to incorporate state attribute. Suggest this is rendered as a dropdown (and may be simple object rendering of state, i.e. do NOT need to change it to a single name such 'active').
This form should then be used when the user satisfies is_authorized(..., model.Action.CHANGE_STATE) instead of the usual fieldset

#127: Minor form and UI improvements

dread — Thu, 24 Sep 2009 13:24:19 GMT

group form: description field for needs to be a text area
group form: make title and name field wider
package view: need to move tags and license above notes

#128: Add ckan_url attribute to REST JSON representation of a Package

rgrp — Fri, 25 Sep 2009 08:18:33 GMT

Add ckan_url attribute to REST JSON representation of a Package pointing to the (read) url of package on CKAN.

Cost: 30m

#130: Create a New Group via the WUI

rgrp — Mon, 28 Sep 2009 09:30:12 GMT

As a

User

I want to

Create a new group via the WUI

Details

locate at /group/new/
You must be logged in to create a group
Group creator is automatically given role of group admin
Edit screen is same as edit screen ...

#131: Groups REST interface

dread — Wed, 30 Sep 2009 10:52:39 GMT

Controlling Groups through a REST interface.

#132: Security hole - read package/group list (REST)

dread — Thu, 01 Oct 2009 09:31:33 GMT

Using REST interface you can list packages and groups without authorization being checked.

Can be fixed using more advanced query to check authz.