CKAN: Ticket Query

#1477: [Super] Group refactor

kindly — Tue, 22 Nov 2011 10:55:20 GMT

This is the implementation of the final draft in

http://wiki.ckan.org/Groups_Refactor

Steps to complete this will follow (somewhat) in the following order.

Change package_group table to membership. (#1478) 3d

model migration. (done)
simplify revisioning (done)
Remove any sqlalchemy releations from groups. (done)
change logic functions (done)

Single type per group (#1531)

migration add type to group table. (done) 1d

Add capacity table (#1522)

model migration. (done)
add to logic functions.

Allow other objects to be added to groups. (#1531)

change logic functions (done)

Adapt thedatahub.org to use new model.

move auth groups over to groups/members
change user_object roles to use groups? keep user object roles for time being.
make create update package/group put the correct items in the group/members/capacity tables.
work out how ui works.

--not doing the below as part of this super ticker now #1669.

Create model for dgu.

adapt auth functions to use auth.
adapt package_create/update group_create/update to fill in the correct capacties/members.

Adapt Iati, and potentially other custom instances.

Sub-tickets:

#1279 Deprecate Authorization Groups
#1198 Publisher hierarchy

#1032: [super] Resources in WUI

rgrp — Thu, 10 Mar 2011 18:33:41 GMT

Add resources into Web User Interface.

Locate at: /dataset/{dataset}/resource/{id}
CRUD
Authorization

Done:

~~#945 - Richer resources - Resource Groups, new fields, improved UI~~
~~#1445 - Resource View page in WUI~~
~~#1450 - Dataset view pages to match~~

Moved to superticket #1506:

#978 - Edit Resource Extras in Web UI

#1065: [super] Change Authorization System

johnlawrenceaspden — Wed, 30 Mar 2011 18:05:31 GMT

Child tickets

#1198 Publisher hierarchy
#1050 Authz lib improvement and refactor of ckan/lib/authztool.py
#1004 Group creation instructions missing
#1099 Strange interactions between two browsers while playing with authz groups
#1115 can have two authzgroups with the same name
#1133 command line rights manipulation doesn't work
#1138 minor navigations behave inconsistently

Old ticket description:

Change name of AuthzGroup? to UserGroup? to reflect what it is for

Get rid of Roles, and replace them with direct assignment of actions, even though there are many actions, and extensions can add arbitrary ones.
- Debatable whether we should cut the number of actions to correspond to the three roles defined by the base system.
- Have a method of finding roles (or, in future, actions) relevant to a given protection object (e.g. FILE-UPLOAD(ER) not relevant to Packages)

Change UserGroups? so that they can have a hierarchical structure,

More info on Hierarchy change

e.g. UserGroup? NHS contains the User nhsysadmin, as well as the UserGroups? SURREY and BERKS, which themselves contain users.

One user in SURREY is Simon the Sysadmin, who has permissions on the whole system. His permissions should not leak out to other users or groups, and user permissions generally should not.

Each Group has permissions over various objects.

A user has permissions in his own right, and also has the permissions of his own group, and of all the groups contained in his group, and so on recursively.

Algorithm:

possible(user, action, package):

if user has permission for action on package

or any of have that permission

or any of his groups group-children (but not user-children), and so on recursively have the permission.

#1402: Migrate repository from mercurial to git

rgrp — Mon, 17 Oct 2011 00:34:11 GMT

Plan to migrate from mercurial git

Convert from mercurial to git
- Map named branches to normal branches
- Also, if possible map usernames for RP: rgrp <http://rufuspollock.org/> (mercurial) -> rgrp <rufus.pollock@…>
Place at http://github.com/okfn/ckan (can delete current repo there as an out of date mirror)

Process:

Do trial run
Announce conversion date / time
Require everyone to have pushed all outstanding changes at that time
Do conversion
Test
Announce on list and ckan.org/

#1508: [super] Finalize and deploy some of our major (new) features as of Dec 2011

rgrp — Mon, 05 Dec 2011 12:40:17 GMT

This ticket is about finalizing, then deploying (to http://thedatahub.org/), and finally announcing some of recent major new features such as:

Deploy wordpresser extension for thedatahub (so we can start blogging) - #1533 - WONTFIX
~~(Deploy new Resource and Dataset pages (invite comments))~~
Get thedatahub geo-ified i.e. ckanext-spatial - #1601 (?)
- Any additional dev that needs to be done? (e.g. geo in search ...?)
~~(Deploy / re-enable ckanext-googleanalytics on thedatahub - #1451 (more stuff #1566))~~
- Rename the extension to ckanext-analytics?
Custom forms (and use of 'dataset types') - Ticket (??)
~~(Get QA deployed and running - #1582)~~

It would be best if this was done by someone *unfamiliar* with the extensions so feedback could be provided on quality of install docs etc etc.

#1602: [super] Data Previewer / Viewer v2

rgrp — Fri, 30 Dec 2011 17:52:24 GMT

We already have first pass of Data Preview that was done as part of #1357. This is about v2 of this. Two main components:

A new version of the separate Data Explorer library (now called Recline Data Explorer) - #1446 Est: 10-15d
- Why:
  - Cleaner, backbone based solution.
  - Provides editing support if wanted.
  - Collaboration with Max Ogden
Integrate this into CKAN (into resource view page) - see #1630 Est: 1d
Improvements to Data Viewer / Previewer system in CKAN itself - Est: 2-3d
- ~~(Update Data Preview-er (View-er) to support images ...)~~
- ~~(WONTFIX here - (?) Update Data Preview-er to support geodata - #1151)~~

#945: [super] Richer resources - Resource Groups, new fields, improved UI

kindly — Mon, 31 Jan 2011 12:01:23 GMT

Super ticket: #1032

This is a meta ticket for changes that are going to happen in resources.

New resource group table. #956
New kind field in resource. #957
UI for new kind field. #958
Resources in REST API ticket:358
Resources in WUI #1445
Make Resources first class entity. #922 (duplicate?)

Background on this change can be found at:

http://ckan.okfnpad.org/resources - discussion etherpad
- Contains commented version of James Gardner's main email: http://lists.okfn.org/pipermail/ckan-discuss/2011-February/000887.html
UseCasesResources

#1637: API call for getting the list of activity detail items for a given activty stream item

seanh — Wed, 11 Jan 2012 18:00:17 GMT

(and add test cases for it)

#890: Introduce timed actions into ckanext-queue

pudo — Mon, 03 Jan 2011 10:32:49 GMT

The ckan queuing system should provide the option to subscribe to timed re-submissions of specific resources. This could look as follows:

routing_key: Package
operation: daily
payload: pkg.as_dict()

Where operation is one of daily, weekly, monthly or any other interval.

#1435: Switch to continuous.io for buildbot (?)

rgrp — Tue, 01 Nov 2011 11:10:31 GMT

#1464: Replace RabbitMQ with Celeryd to support running multiple instances

thejimmyg — Tue, 15 Nov 2011 16:48:29 GMT

The current harvesting implementation can only have one instance per server.

We could either:

Accept this
Change the RabbitMQ code to support multiple instances
Switch the entire harvesting to use Celery
Wait for James's redis based live feedback based system

#1743: Activity streams still link to deleted things

seanh — Thu, 02 Feb 2012 19:46:56 GMT

For example, after deleting a dataset old activity stream events will still hyperlink to that dataset.

#1677: Make synchronous search the default behaviour

amercader — Tue, 17 Jan 2012 13:40:04 GMT

Right now you need to explicitly load the synchronous_search plugin in your ini file, when this is probably the behaviour that all users expect by default. We could keep a config flag to deactivate it, but synchronous search should be the default behaviour.

#1682: [super] Translatability for EC

dread — Wed, 18 Jan 2012 18:38:19 GMT

We need to be able to translate:

EC extension (templates, form)
- Currently the pot file is just for CKAN core. New pot file for CKAN core and this extension? Or separate ones for extensions?
Field values in taxonomy (e.g. country names - Eurovoc)
- #1665 Research into Eurovoc
- Display Taxonomy in different languages
Field values not in taxonomy (e.g. title & description)
- Use extra fields e.g. _i18n_title_fr = Le data.
- (If lots of fields would need translating then would consider having a new package for each language, linked together with PackageRelationships?. But I think it is just title and description (resource description etc. are so minor, not worth translating?), so using extra field better.)

Timescale to complete this - Mar/Apr?.

Related work to interface with:

Eurovoc - Sean
i18n in search index - Adria

#1327: [super] Dataset Archiving

rgrp — Mon, 12 Sep 2011 09:58:10 GMT

Split out of #852. Automated archiving of datasets (related to QA).

Automated archiving using worker process

#890 - Timed actions in ckanext-queue
#891 - Resource download worker daemon
#892 - Make stored data available in WUI

#1514: Modifying user name loses connection with revisions

dread — Mon, 05 Dec 2011 15:57:39 GMT

If you edit your user name, the number of revisions you made returns to 0. This is because in the Revision object, the user's name is stored, rather than the user's ID.

rgrp: We can reconnect that pretty easily (and have a longer term solution that involves not using the usernames but the userids in in the Revision objects so we don't hvae this problem in future!)

#1615: CKAN Should work behind a proxy server

thejimmyg — Wed, 04 Jan 2012 14:37:21 GMT

This would allow deployment via Nginx or Apache using proxy to Paster, uWSGI. At the moment CKAN isn't aware of the proxy's IP address so when you perform an action which does a redirect (such as adding a package), CKAN redirects you to the *internal IP* not the external *proxy IP*.

We would like this work to facilitate testing within VMs as part of our new build infrastructure.

It would also be nice if CKAN worked when mounted at a path other than /. That could be dealt with in another ticket because it isn't a problem at the moment.