{22} Trac tickets (2647 matches)

Results (1001 - 1100 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1008 defect rgrp pudo ckan-v1.4-sprint-2 closed fixed eval() of user display name in template head

We're currently setting the user preferences links via a javascript snippet that also evals the name. This should be removed as we're not displaying the user name any longer.

We should also have page fragment caching in Genshi, which is not currently implmented.

cf. http://api.rubyonrails.org/classes/ActionController/Caching/Fragments.html

1298632686000000 1298821826000000
#1009 enhancement pudo rgrp ckan-backlog new Improvements to user accounts sytem
  • Forgot password (email a new password)
  • Confirm email
  • Do not show register page if you are logged in (redirect to home page)
  • ticket:1010 - listing of users.
    • Do not use /user for general user account home page (either user normal user page /user/{id} or /user/myaccount)
1298635991000000 1310128574000000
#1010 enhancement rgrp rgrp ckan-v1.4-sprint-2 closed fixed List CKAN users in WUI

Should have user listing at /user/ rather than user account page.

  • list users, sorted by number of packages contributed/edited
  • Move user home page to /user/{user-id}
  • Paginated
1298649180000000 1298740889000000
#1011 enhancement sebbacon sebbacon ckan-v1.4-sprint-3 closed fixed Make pluggable AuthzGroups implementation

I have a requirement to allow an external source to define the groups of which a user is a member.

I propose to create an IAuthzUserGroups plugin interface that allows an extension to arbitrarily extend the list of AuthzGroups? that a user is in.

1298819657000000 1299245206000000
#1012 defect kindly kindly ckan-v1.4 closed fixed Add package revision history to api

Revision history information is not accessible, dgu want this.

1298887351000000 1301943113000000
#1013 defect sebbacon sebbacon ckan-v1.4-sprint-5 closed fixed Refactor & write tests for Wordpresser extension

The Wordpresser extension seems to be working well for DataGM, but needs caching and tests.

1298887392000000 1301909717000000
#1014 defect sebbacon sebbacon ckan-v1.4-sprint-3 closed fixed Decouple oAuth repoze extension from DataNO, write tests

The current oAuth extension tightly couples repoze.who, ckan, and data.no. These components should be decoupled; they also need more tests

1298887493000000 1299245293000000
#1015 defect kindly dread ckan-v1.4-sprint-3 closed fixed Editing group gives exception

Editing a group properties on ckan.net gives an exception when you submit the form. (Is this related to db migration issues?)

Example page: http://ckan.net/group/edit/civil-society


WebApp Error: <class 'sqlalchemy.exc.IntegrityError'>: (IntegrityError) duplicate key value violates unique constraint "group_revision_pkey" 'INSERT INTO group_revision (id, name, title, description, created, state, revision_id, continuity_id) VALUES (%(id)s, %(name)s, %(title)s, %(description)s, %(created)s, %(state)s, %(revision_id)s, %(continuity_id)s)' {'description': u"A group for open data related to civil society supported by members of the interest group at: http://okfn.org/groups/civil-society\r\n\r\nCivil society is composed of the totality of voluntary civic and social organizations and institutions that form the basis of a functioning society, as distinct from the force-backed structures of a state (regardless of that state's political system) and commercial institutions of the market. \r\n\r\nhttp://en.wikipedia.org/wiki/Civil_society", 'created': datetime.datetime(2011, 2, 24, 14, 30, 53, 334842), 'title': u'Civil Society', 'state': u'active', 'continuity_id': u'f4f9f09
1298898588000000 1299788821000000
#1017 defect pudo sebbacon closed fixed Problem assigning users to authz groups through web interface

Against ckan-1.3.1, when I create an authz group called "administrators" and visit /authorizationgroup/edit/administrators, I am unable to add more than one user to it.

Each time I add additional users, the existing user on the list is replaced with the new one.

1299071127000000 1299668555000000
#1018 task dread dread ckan-v1.4-sprint-3 closed fixed Remove gov form

(It has been moved into ckanext-dgu)

1299072516000000 1299073340000000
#1019 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Webhooks notification service

We propose the following push approach using individual packages:

  1. New, updated and deleted packages are pushed to a url endpoint on Wordpress (WPURL). This endpoint will be a configuration option on the CKAN side. A POST or PUT http method will get used. The payload will be a json document of the following form:


payload: The entity data as available from the REST API entity-type: 'Package', operation-type: 'create'|'update'|'delete' # one of these options


It will be sent as the body of the request with content-type set to application/json.

  1. The wordpress side will provide a 200 on success. Any other response will be taken as a failure. On failure, submission will be archived and failure logged and notified to system administrator. Submission can be resent later automatically by sysadmin after review.
  1. Pushes will happen continuously and approximately simultaneously with updates (a webhooks type model)
  1. [optional] CKAN side will support configuring authorization for basic authentication if applied on WP side.
  1. [optional] List queue status (including failures) on ckan adminstrative dashboard.
1299166784000000 1299166930000000
#1020 defect kindly closed fixed harvesting doc revision table

add revisioning to harvested document table

1299205012000000 1300196215000000
#1021 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Config option to disable OpenID

HRI don't like federation, want to login normal way only. Make this a config option and perhaps even mess with runtime repoze config

1299492920000000 1299518828000000
#1022 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Error reporting in CKAN worker API

Report on errors by any worker daemons, send them out via E-Mail

1299493047000000 1299512991000000
#1023 defect pudo pudo closed wontfix Re-queueing of worker requests with errors

Schedule worker calls to be ran several times, with certain intervals.

1299493179000000 1340626231000000
#1024 enhancement pudo pudo ckan-v1.4-sprint-3 closed duplicate Lock down all controller actions in CKAN

When a user visits the site, he/she may only see a login box.

1299493428000000 1299668648000000
#1025 enhancement dread dread ckan-v1.4-sprint-3 closed fixed Default authz can be set in config

Currently the default authz for a package is hard-coded to:

 <PackageRole user="visitor" role="editor" context="Package">,
 <PackageRole user="logged_in" role="editor" context="Package">,

This should be configurable in the config, so that you can have a more locked down instance etc.

1299596110000000 1299751045000000
#1026 enhancement dread dread ckan-v1.4-sprint-3 closed fixed cli for creating users

It's handy to be able to create users using the cli (e.g. dgu migration)

1299604652000000 1299605128000000
#1027 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Authorization checks on all controller actions

We want to have authz checks on all controller actions so that we can lock down CKAN to a login-only mode.

1299666256000000 1299682082000000
#1028 defect dread dread ckan-v1.4-sprint-3 closed fixed Open redirect in locale setting 1299759883000000 1299760360000000
#1029 defect kindly ckan-v1.4-sprint-3 closed fixed synchronous search erroring when harvesting is run.

Errors are caused when harvesting documents. This also makes the count not show up correctly on the ckan search page.

1299768337000000 1301311643000000
#1030 defect amercader thejimmyg closed fixed Move harvesting out of the rest API 1299776418000000 1303117978000000
#1031 enhancement johnlawrenceaspden rgrp ckan-v1.4-sprint-4 closed fixed User lookup API

Add an api for searching users. This is needed for any kind of ajax autocomplete (needed for anywhere we want to add users).

  • API location: /api/util/user/lookup?q=querystr&limit=10
  • Return json objects containing {id: ..., name: ..., fullname: ...}
  • Put in a module called controllers/apiv2/user.py
1299780419000000 1300101520000000
#1032 enhancement rgrp rgrp ckan-v1.6 closed fixed [super] Resources in WUI

Add resources into Web User Interface.

  • Locate at: /dataset/{dataset}/resource/{id}
  • CRUD
  • Authorization


  • #945 - Richer resources - Resource Groups, new fields, improved UI
  • #1445 - Resource View page in WUI
  • #1450 - Dataset view pages to match

Moved to superticket #1506:

  • #978 - Edit Resource Extras in Web UI
1299782021000000 1330348463000000
#1033 defect dread dread closed fixed Register user with blank password causes 500

Go to http://ckan.net/user/register and fill in all the fields apart from password. On submit you get 500 error.

1299796274000000 1308310446000000
#1034 defect dread closed duplicate Flash message cached
  1. Login (shows ckan home page with flash message "Welcome back xyz")
  2. Click away, to "Add package" say
  3. Click back to ckan "Home". It shows "Welcome back xyz" again - it shouldn't, I assume?

1299845308000000 1320174353000000
#1035 enhancement thejimmyg dread closed wontfix Form impressions given an ID

To counter Cross Site Request Forgery attacks, each form generated is assigned a random number in its url, which must be passed when you submit the form.

(Something to consider for the new form mechanism)

1299857978000000 1338206251000000
#1036 defect johnlawrenceaspden johnlawrenceaspden ckan-v1.4-sprint-4 closed fixed Add tests for three functions in ckan/model/user.py

create a new test file ckan/model/test_user.py

add tests for the following three functions in ckan/model/user.py

number_of_edits, number_administered_packages, search

merged in in changeset 0046f83aedcf

1300127840000000 1301304575000000
#1037 defect amercader thejimmyg ckan-v1.4-sprint-6 closed fixed More Robust Harvesting for DGU

CKAN's harvesting facility is now live on DGU but there are some major improvements that could be made to make it more robust and better fit the generic CKAN harvesting framework proposed in #987.

Some of the key issues:

  • Error reports do not currently contain the ID or title of the document with the error.
  • We only have "added" and "error" logging on jobs when we really need a report of "added", "updated", "not changed" and "errors" with the items in each referencing a real metadata document for which harvesting was attempted
  • We need deletion and editing of sources, without deleting the harvested documents or packages
  • We need a more robust harvesting mechanism than a cron job or we need to deal with the case of multiple cron jobs running at once.
  • We need to know the last time a list of documents was scheduled for harvest and the last time each one was fetched.
1300197602000000 1304937601000000
#1038 enhancement dread dread ckan-v1.4 closed fixed Authz tool - operate on all packages at once

Add 'package:all' to authz tool to allow mass changes of authz.

1300212788000000 1300212841000000
#1039 enhancement dread dread ckan-v1.4 closed fixed Default user roles read from config

(instead of being hard coded)

1300212822000000 1300212856000000
#1040 defect thejimmyg dread ckan-v1.4 closed fixed File system mounted

Public file path code adds a trailing , and thus adds / as a static file app in the Pylons middleware cascade.

1300213855000000 1328806824000000
#1041 enhancement thejimmyg thejimmyg ckan-backlog assigned Start Using the CKAN Wiki for Tutorial-style documentation

For example, I will document the following:

I'd love if someone else would write:

  • An authorisation tutorial covering the core model, the command line tools and examples of every possible way of using the system
  • A HOWTO guide with screenshots for adding a package
1300284715000000 1312372367000000
#1042 task dread dread closed fixed 'Ckanext' split-up

It's not good to have ckanext doing lots of different things with different dependencies. Split it off into:

  • ckanext-importlib


And then deprecate the ckanext repo itself.

1300293907000000 1300969865000000
#1043 defect kindly closed fixed stop sqlalchemy message saying at least one scoped seession already present

see summary.

1300310326000000 1300321033000000
#1044 defect pudo dread closed fixed Sysadmins locked-out of API without Right: (visitor, SITE_READ, System)

The problem is that in ckan/controllers/rest.py the BaseApiController? has this method:

    def __before__(self, action, **env):
        BaseController.__before__(self, action, **env)
        if not self.authorizer.am_authorized(c, model.Action.SITE_READ, model.System):
            abort(401, _('Not authorized to see this page'))

which works on the basis of your c.user, rather than your apikey. All API users are treated as visitors (since API users don't get a login cookie) and even a sysadmin's apikey is blocked unless there is a right for a Visitor to SITE_READ.

Also needs tests.

(Also, why is this restriction only on the API, package search, group index and tags and agroup index? I'm guessing SITE_READ is only for places where other authz don't apply, but maybe it should not be called 'SITE_READ' but 'OTHER_READ' or something?)

1300358919000000 1302096155000000
#1045 defect dread dread ckan-v1.4-sprint-4 closed fixed Group identified by ID in API

returns group IDs but I can only reach a group by name:


when I also want to get a group by ID:

1300360642000000 1300793261000000
#1046 enhancement kindly thejimmyg ckan-v1.4-sprint-5 closed fixed Dictization and the new logic layer

The stages involved with doing this.

  • Convert model objects to standard dict format (DONE)
  • Convert standard dicts to current api formats (DONE)
  • Make standard dicts savable (DONE)
  • Validate standard dict format. (DONE)
  • Authorize actions
1300364694000000 1302777668000000
#1047 defect sebbacon sebbacon ckan-v1.4-sprint-4 closed fixed Package edit form claims you're not logged in at the end when you are

At the bottom, it says something like

Author: Bob Bumgardner Since you have not signed in this will just be your IP address. Click here to sign in before saving (opens in new window).

1300384556000000 1300387309000000
#1048 enhancement dread dread ckan-v1.4-sprint-4 closed fixed Complete making groups versioned
  • Deleting a group changes state to 'deleted' rather than purging it
  • Adding authz tests for deleted groups
1300387655000000 1300702752000000
#1049 defect dread closed invalid Fix database errors on offener.datenkatalog.at etc.

We're getting these sorts of exceptions from http://offener.datenkatalog.at/ like we had from ckan.net, which was due to small errors in migration scripts. Is it worth fixing this database, and in fact all the ckans on eu3 at once?

WebApp Error: <class 'sqlalchemy.exceptions.IntegrityError'>: (IntegrityError) duplicate key value violates unique constraint "group_revision_pkey" 'INSERT INTO group_revision (id, name, title, description, created, state, revision_id, continuity_id) VALUES (%(id)s, %(name)s, %(title)s, %(description)s, %(created)s, %(state)s, %(revision_id)s, ...

Thoughts James and Rufus?

1300444912000000 1323169424000000
#1050 enhancement thejimmyg johnlawrenceaspden closed invalid Authz lib improvement and refactor of ckan/lib/authztool.py

Refactor ckan/lib/authztool.py so that the relevant methods are independent of the command line interface.

The extracted methods should live in a new file ckan/authz.py. authztool.py should probably move into cli.py and will just do command line parsing and printing and use ckan/authz.py. The updated web gui for authz will also use this code.

Tests should be made. There's already a file ckan/tests/test_authz.py, which looks like the appropriate place for new tests.

all to go on a branch feature-1050-refactor-authtoolz

Optional extras

  • Rename ckan/authz.py to ckan/lib/authz.py or even ckan/logic/authz.py
1300451937000000 1315394117000000
#1051 defect sebbacon sebbacon ckan-v1.4-sprint-4 closed fixed Ability to set custom favicon

Allow deployers to set a URL pointing to their own favicon

1300703160000000 1301305079000000
#1052 defect dread dread ckan-v1.4-sprint-4 closed fixed Authz holes

No authz on:

  • Group creation/edit/listing
  • Package relationship create/edit/delete
1300709144000000 1300895410000000
#1053 defect dread dread ckan-v1.5 closed fixed Deletion in Model API

Currently in the API if you DELETE a package/group/user (and you have the required permissions) then it purges the object, when it should probably just set the state to deleted.

There is no way to delete objects at the moment - changes to 'state' are ignored in the API.

Do we need an alternative way to purge objects in the API?

1300790039000000 1310126546000000
#1054 defect kindly dread ckan-v1.4-sprint-4 closed fixed Ordering of resources

Changing an old resource and creating a new resource on the end results in the old resource moving to the end in the ordering.

This breaks tests:

  • (ckanext-dgu) ckanext/dgu/tests/ons/test_ons_loader.py:TestOnsLoadBasic.test_fields
  • (ckanext-importlib) ckanext/importlib/tests/test_loader.py:TestLoaderInsertingResources.test_0_reload

You can make a ckan test break with this patch:

diff -r e6643cf1324c ckan/tests/models/test_resource.py
--- a/ckan/tests/models/test_resource.py        Wed Mar 23 13:25:52 2011 +0000
+++ b/ckan/tests/models/test_resource.py        Wed Mar 23 19:22:35 2011 +0000
@@ -297,6 +297,8 @@
                'url':self.urls[1], 'format':u'OTHER FORMAT',
                'description':self.description, 'hash':self.hash,
+            { #new
+                'url':'new'},

There seems to be a problem with vdm creating a replacement Resource for the old resource - because it has a duplicate position it is put to the end by the SQLAlchemy ordering_list function.

1300969236000000 1301305615000000
#1055 defect dread dread ckan-v1.4-sprint-4 closed fixed @search_related tests not running

Tests marked decorated "@search_related" should only be run against postgresql, but in fact they don't get run at all.

1300985228000000 1300992395000000
#1056 defect dread pudo ckan-v1.4-sprint-6 closed fixed User links for OpenID users are broken

Use case:

  • Login using OpenID
  • Click on 'My account' - results in 404


  • User user.id instead of their name
  • Escape the URL properly.
1301060249000000 1302882616000000
#1057 defect dread closed fixed JSONP parameter isn't escaped
$ curl "<script>jsoncallback"


<script>jsoncallback({"id": "c10ebd31-5b45-4f6f-885d-dca9b18caec4", "name": "annakarenina", "title": "A Novel By Tolstoy",

which could run script code in the client who made the call.

One idea for filtering: http://tav.espians.com/sanitising-jsonp-callback-identifiers-for-security.html Maybe just better to have a restricted whitelist of characters to be even more sure.

Same as: https://trac.dataco.coi.gov.uk/projects/datagov/ticket/906

1301078389000000 1329150236000000
#1058 defect dread dread ckan-v1.4-sprint-4 closed fixed Give 400 error (not 500) for invalid locale or package_form

Examples which prompt annoying exception emails:

Module ckan.i18n:21 in set_session_locale
           assert locale in _KNOWN_LOCALES

A bot has caused these:

Module ckan.forms.registry:32 in get_fieldset
               raise ValueError('Could not find package_form name %r in those found: \n%r' % (package_form, [en.name for en in entrypoints]))
ValueError: Could not find package_form name u'gov)' in those found: ['gov', 'standard', 'ca']
1301302303000000 1301303315000000
#1059 defect dread dread ckan-v1.4-sprint-5 closed fixed Loader coping better with poor search indexing

Loader currently checks for same name, but also should check for name_, name etc.

1301310596000000 1301312516000000
#1060 defect dread ckan-v1.4-sprint-5 closed fixed Spreadsheet importer tries to import readonly keys

e.g. we just added notes_rendered and that is read in as an extra field. Tests failing in ckanext-importlib

Also related: we are missing lost metadata_created and metadata_modified in the dumps.

1301312210000000 1301312487000000
#1061 enhancement dread dread ckan-v1.4-sprint-5 closed invalid Orphaned home/license page

No links to home/license and it contains out of date references to knowledgeforge. Remove it.

1301392968000000 1301922350000000
#1062 defect johnglover sebbacon ckan-backlog assigned Data preview encoding error

The preview of "Species Misc Turtle Download" at http://ckan.net/package/taxonconcept results in the following error:

Unable to Preview - Had an error from dataproxy: Data Transformation Error (Data transformation failed. Reason: 'utf8' codec can't decode byte 0x8b in position 1: unexpected code byte

1301396143000000 1311773731000000
#1063 defect sebbacon sebbacon closed fixed Groups listing widget on package screen shouldn't show group name by default

I've been asked if we can do something about the overflow of the Group name in the right hand column on this page:


The reason is that the list display for groups is in the form "group_tltie (group_name)", and of course group_name can't have spaces and so can't wrap nicely.

I was wondering if there's a good reason why we don't only display group_title (if it exists) and group_name only when there's not a title?

1301408459000000 1302514033000000
#1064 defect amercader closed duplicate Remove Workers from ckanext-queue

The current implementation of Workers in ckanext-queue is broken. Basically the various consume / callback functions expect three arguments (routing_key, operation, payload) when they are in fact receiving only two of them (message_data, message). This is fairly easy to fix, but the question is if Workers add an extra complexity to use the messaging library directly.

1301417891000000 1323169787000000
#1065 enhancement zephod johnlawrenceaspden ckan-v1.6 closed fixed [super] Change Authorization System

Child tickets

  • #1198 Publisher hierarchy
  • #1050 Authz lib improvement and refactor of ckan/lib/authztool.py
  • #1004 Group creation instructions missing
  • #1099 Strange interactions between two browsers while playing with authz groups
  • #1115 can have two authzgroups with the same name
  • #1133 command line rights manipulation doesn't work
  • #1138 minor navigations behave inconsistently

Old ticket description:

  1. Change name of AuthzGroup? to UserGroup? to reflect what it is for
  1. Get rid of Roles, and replace them with direct assignment of actions, even though there are many actions, and extensions can add arbitrary ones.
    • Debatable whether we should cut the number of actions to correspond to the three roles defined by the base system.
    • Have a method of finding roles (or, in future, actions) relevant to a given protection object (e.g. FILE-UPLOAD(ER) not relevant to Packages)
  1. Change UserGroups? so that they can have a hierarchical structure,

More info on Hierarchy change

e.g. UserGroup? NHS contains the User nhsysadmin, as well as the UserGroups? SURREY and BERKS, which themselves contain users.

One user in SURREY is Simon the Sysadmin, who has permissions on the whole system. His permissions should not leak out to other users or groups, and user permissions generally should not.

Each Group has permissions over various objects.

A user has permissions in his own right, and also has the permissions of his own group, and of all the groups contained in his group, and so on recursively.


possible(user, action, package):

if user has permission for action on package

or any of have that permission

or any of his groups group-children (but not user-children), and so on recursively have the permission.

1301508331000000 1324550041000000
#1066 enhancement dread dread ckan-v1.4-sprint-5 closed fixed Default reader role too permissive

The definition of the 'reader' role includes creating packages, which is too permissive for some CKAN instances (e.g. DGU). 'Reader' suggests only reading, so I think this role should avoid creating and editing.

All projects so far want all roles to be able to create users, so this stays as a Reader action for now, as a convenience.


  • Action.PACKAGE_CREATE removed from reader's default_role_actions
  • Visitor has a new default role, called 'anon_editor' which can edit packages, but not groups / auth groups - you have to log in for that.
  • Migration script not needed?
  • Code comments written, to make clear the suggested policy
1301645250000000 1301932136000000
#1067 enhancement dread dread ckan-v1.4-sprint-5 closed fixed CLI for loading/dumping complete databases

Use 'db dump' and 'db load' for 'pg_dump' and 'psql -f' of a database. Use pylons config to find out database options.

1301645463000000 1302186503000000
#1068 defect dread dread ckan-v1.4-sprint-5 closed fixed metadata_modified problem

This test has been failing since the clocks changed:

FAIL: ckan.tests.models.test_package.TestPackageRevisions.test_02_metadata_created_and_modified
Traceback (most recent call last):
  File "/home/dread/hgroot/pyenv-ckan2/lib/python2.6/site-packages/nose-0.11.3-py2.6.egg/nose/case.py", line 186, in runTest
  File "/home/dread/hgroot/ckan2/ckan/tests/models/test_package.py", line 283, in test_02_metadata_created_and_modified
    assert out == exp, (out, exp)
AssertionError: (datetime.datetime(2011, 4, 1, 10, 45, 50, 875509), datetime.datetime(2011, 4, 1, 9, 45, 50, 875509))

1301652085000000 1302109505000000
#1069 enhancement tobes rgrp assigned Stub datasets (request for datasets)

Idea is to have stubs for datasets that someone wants but don't yet exist (or haven't been discovered) in the way one has stub pages on a wiki.

We could do this within the existing model by a slight 'abuse' - create a dataset and mark it with a special tag e.g. todo.does-not-yet-exist or similar ...

(Just as we have datasets listed that exist but aren't available ...)

Alternative would be to have a request for datasets subsystem.

I prefer the stub dataset model because it's simpler, provides a simple workflow (as a dataset is found or comes into existence), and the package page provides a natural space in which to accumulate information about what is wanted and what exists.


  • Agree a new dedicated tag. e.g. todo.does-not-exist
1301666919000000 1340632215000000
#1070 enhancement rgrp rgrp ckan-v1.5 closed fixed Plan a new domain model and layer architecture for CKAN

See http://wiki.ckan.net/Domain_Model especially section on v2.

  • New domain model is planned but not yet finally agreed.
  • Layer architecture is complete and implemented
1301910940000000 1310117129000000
#1071 defect dread dread ckan-v1.4-sprint-5 closed fixed Package history API moved to /api/rest/package/revisions

api/rest/package_history is not RESTful or follow API naming conventions. Therefore move it to /api/rest/package/revisions

Also, API docs incomplete.

1301937882000000 1301943180000000
#1072 enhancement dread dread ckan-v1.4-sprint-5 closed fixed Add filters to authztool

It takes several minutes to print the 'rights' on DGU, which is annoying when you only want to grep for a few lines. Much quicker than grepping is to filter in the query.

1302106311000000 1302106474000000
#1073 enhancement dread dread ckan-v1.4-sprint-5 closed fixed Search index checker

Tool that checks which packages have not been indexed.

Required for DGU: https://trac.dataco.coi.gov.uk/projects/datagov/ticket/940

1302185444000000 1302185825000000
#1074 enhancement rgrp rgrp ckan-v1.5 closed fixed Refactor authz web user interface to have common code and templating

Currently repeat the same template and code across Package Authz, Group Authz, and Authz Group authz.

Having now implemented a new, cleaner setup in ckanext-admin we should port this back into core.

  • Common template code (checkbox template)
  • Logic code (or just common code) for wiring into authz system
  • Look for all places thoroughout the system where usernames, authzgroups or groups need to be typed into boxes, and make sure that they auto-complete appropriately.

Will also deliver a significant improvement in the form of ajax user lookup.

1302271586000000 1314303581000000
#1075 enhancement johnlawrenceaspden rgrp ckan-v1.4-sprint-6 closed fixed Administrative dashboard - Edit Authorization related to System object

Roles on System object are important because admin role on system equates to being a 'sysadmin' (i.e. able to do anything).

  • Make users sysadmin (either as separate action or as part of editing roles on system object)
  • /authz subpage for editing roles on system object
    • Add and update user roles
    • Add and update authz group roles
    • List actions associated to roles at top of page (extra points for checkbox table with editability)
  • Document on http://wiki.ckan.net/Authorization what roles on System object 'mean' esp sysadmin role on System

Related Tickets

  • super ticket: #833
  • authz lib improvement and authztool refactor #1050
1302279799000000 1303227982000000
#1076 enhancement johnlawrenceaspden rgrp ckan-v1.4 closed fixed Improve revision and package purge system

Purging Revisions

  • Delete button displayed on:
    • /revision/list
    • (/package/history)
      • /package/history is problematic because html does not allow nested forms and we already have form for doing diff/comparison.
    • /revision/{id}
  • Delete button submits to delete action on revision and changes revision state to 'deleted'.
    • undelete button now displayed and revisions are marked as deleted in some way (e.g. greyed out?)
  • Sysadmins then visit /ckan-admin/trash which lists all revisions with deleted state. There is a large button: "Empty trash" (irreversible). Click button purges all revisions with deleted state.

Purging Packages

  • Put into deleted state.
  • Listed on /ckan-admin/trash
  • Separate Empty trash button which deletes all associated revisions.
    • Should be separate from Empty trash for revisions

Current system

  • Single purge link on revision listing if a sysadmin which permanently purges the revision and all associated changes (without confirmation atm!)
1302283442000000 1303236302000000
#1077 enhancement kindly rgrp ckan-backlog new Move to simpler vdm system

Option 1: 'Changeset' Model

See ticket:1135 for vdm ticket. This would involve a) moving to changeset in vdm b) doing the migration in ckan to support this.

Have developed a new "changeset" based model for revisioning in vdm.


  • The main challenge with this change is schema and data migration

Every revisioned object has a revision_id and revision attribute.

Approximate algorithm:

Revision -> Changeset

for revtype in [PackageRevision, ...]:
    for pkgrev in package_revision:
        changeset = lookupchangeset(package_revision)
        ChangeObject(cset, (table, id), dictize(pkgrev))


  • does pkg include tags attributes or not? or we have to dictize, pkgrev, pkg2tagrev, and tag. Probably the latter.

Option 2: Simplify Revision Object Model

Just use a simpler vdm, see ticket:1136 (move to SessionExtension) and ticket:1137 (remove need for statefulness in vdm).


Advantage of Option 1 versus 2:

  • Easier support for pending state and similar behaviour
  • No need to introduce new tables (and hence migrations) when making something revisioned (or not).


  • Migration is required
  • More difficult to query revision history.
    • Could be addressed by having ChangeObject have separate cols for table name and id but would likely be more difficult.
  • Performance (?)
    • Have one big ChangeObject table to query when looking at changed objects rather than many revision tables.
      • Not sure this is a biggie as even with Revision model biggest revision object tables are probably on the order of the ChangeObject table


Implement Option 2 and leave Option 1 for present.

Option 1 includes Option 2 so it seems that that is required in either case (so we may as well with Option 2).

Option 1 requires significant effort (esp migration) so leave for present and then review the situation at some later date.

1302304464000000 1340034345000000
#1078 enhancement kindly rgrp ckan-v1.5 closed fixed Refactors WUI controllers and forms to use logic layer
  • Deserialize forms to new dict format.
  • Replace controllers/forms to use dictization.
1302509347000000 1305828973000000
#1079 enhancement kindly rgrp ckan-v1.4-sprint-5 closed fixed Refactor API to use new logic layer and dictization
  • Convert current api saves to the new standard dict format.
1302509530000000 1302777504000000
#1081 defect johnlawrenceaspden johnlawrenceaspden closed fixed can't remove user from authz group

I've found that if I make an authorization group I sometimes can't remove myself from it. I've no idea why. I can add and remove other users. I'll investigate, just making a note of it here.

1302541056000000 1303489474000000
#1082 defect johnlawrenceaspden closed fixed language changes behave strangely

Set language to Greek, flash message says 'Language set to: English', but page is now about half in Greek.

Set language back to English causes server error:

AttributeError?: 'NoneType?' object has no attribute 'path'

Module ckan.controllers.error:29 in document view

if original_request.path.startswith('/api'):

However going to a new page reveals that it's back to English

1302541989000000 1315917217000000
#1083 defect johnlawrenceaspden johnlawrenceaspden ckan-v1.5-sprint-1 closed fixed userobjectroles added twice can't be deleted

the add_user_to_role/remove_user_from_role functions are asymmetrical in that the add function is happy to add the same role twice but the remove asserts that it's only in the table once and crashes if that's not true.

an attempt has been made to guard against this, but fails, I think because the add functions rely on the caller committing the change to the db.

same problem affects corresponding authorization_group functions

I'll try to sort this out. Making a note here.

1302550660000000 1305537827000000
#1084 task wwaites wwaites ckan-v1.4-sprint-6 closed fixed ckan.net RDF links changed

need to make some changes for the links to semantic.ckan.net. it should use http://semantic.ckan.net/record/<package_id> now

append .rdf, .ttl, .nt, .dot, .json (even .html for an ugly table) to taste (or just leave off the suffix and let content negotiation take care of it)

the base url is changed, but it now uses id not name.

see for example:

1302616717000000 1304934534000000
#1085 defect dread johnlawrenceaspden closed fixed local development copy of ckan depends on existence of ckan.net

ckan.net appears to have either gone down or be running ultra slowly.

this means that ckan copies running locally on my machine run very slowly indeed.

is this behaviour desirable?

This command finds lots of http://~~~ckan.net references in python, html and javascript files:

find ~/pyenv/src \( -name "*.py" -or -name "*.html" -or -name "*.js" \) -print0 | xargs -0 -e grep --color -nH -e "http://.*ckan.net"

output for reference:

/home/okfn/pyenv/src/ckan/ckan/init__.py:5:Network (CKAN) site: http://www.ckan.net. /home/okfn/pyenv/src/ckan/ckan/lib/create_test_data.py:346:<http://ckan.net/> /home/okfn/pyenv/src/ckan/ckan/lib/rdf.py:3:DOMAIN = 'http://ckan.net' /home/okfn/pyenv/src/ckan/ckan/lib/rdf.py:4:CKAN_NAMESPACE = 'http://ckan.net/#' /home/okfn/pyenv/src/ckan/ckan/lib/talis.py:60: 'ckan':'http://ckan.net/ns#', /home/okfn/pyenv/src/ckan/ckan/public/scripts/bookmarklet.js:2: f='http://ckan.net/package/new?url='+encodeURIComponent(window.location.href)+'&title='+encodeURIComponent(document.title); /home/okfn/pyenv/src/ckan/ckan/public/scripts/test_bookmarklet.html:16: addtockan.src='http://ckan.net/scripts/bookmarklet.js'; /home/okfn/pyenv/src/ckan/ckan/public/scripts/test_bookmarklet.html:27: <p><strong>Proper bookmarklet (compressed -- need to escape &amp;):</strong> <a href="javascript:(function(){f='http://ckan.net/package/new?url='+encodeURIComponent(window.location.href)+'&amp;title='+encodeURIComponent(document.title);if((n=document.getElementsByName('description')[0])&amp;&amp;(d=n.content)){f+='&amp;notes='+encodeURIComponent(d);}a=function(){if(!window.open(f)){location.href=f;}};if(/Firefox/.test(navigator.userAgent)){setTimeout(a,0)}else{a()}})()">Add to CKAN</a> /home/okfn/pyenv/src/ckan/ckan/templates/home/license.html:31: For convenience, all material - including all package, tag and revision information - is available in bulk, in the form of a full dump of the CKAN database. This (gzipped) dump file is updated daily and can be downloaded from <a href="http://www.ckan.net/dump/">http://www.ckan.net/dump/</a>. /home/okfn/pyenv/src/ckan/ckan/tests/dictization.py:71: 'notes': u'Some test notes\n\n### A 3rd level heading\n\nSome bolded text.\n\n*Some italicized text.*\n\nForeign characters:\nu with umlaut \xfc\n66-style quote \u201c\nforeign word: th\xfcmb\n \nNeeds escaping:\nleft arrow <\n\n<http://ckan.net/>\n\n', /home/okfn/pyenv/src/ckan/ckan/tests/dictization.py:137: 'notes': u'Some test notes\n\n### A 3rd level heading\n\nSome bolded text.\n\n*Some italicized text.*\n\nForeign characters:\nu with umlaut \xfc\n66-style quote \u201c\nforeign word: th\xfcmb\n \nNeeds escaping:\nleft arrow <\n\n<http://ckan.net/>\n\n', /home/okfn/pyenv/src/ckan/ckan/tests/dictization.py:447: 'notes': u'Some test notes\n\n### A 3rd level heading\n\nSome bolded text.\n\n*Some italicized text.*\n\nForeign characters:\nu with umlaut \xfc\n66-style quote \u201c\nforeign word: th\xfcmb\n \nNeeds escaping:\nleft arrow <\n\n<http://ckan.net/>\n\n', /home/okfn/pyenv/src/ckan/ckan/tests/dictization.py:458: 'notes': u'Some test notes\n\n### A 3rd level heading\n\nSome bolded text.\n\n*Some italicized text.*\n\nForeign characters:\nu with umlaut \xfc\n66-style quote \u201c\nforeign word: th\xfcmb\n \nNeeds escaping:\nleft arrow <\n\n<http://ckan.net/>\n\n', /home/okfn/pyenv/src/ckan/ckan/tests/functional/api/base.py:178: assert '"ckan_url": "http://test.ckan.net/package/annakarenina"' in msg, msg /home/okfn/pyenv/src/ckanclient/ckanclient/init__.py:116: api e.g. http://ckan.net/api rather than http://ckan.net/api/rest) /home/okfn/pyenv/src/ckanclient/ckanclient/init__.py:261: :param base_location: default *http://www.ckan.net/api* /home/okfn/pyenv/src/ckanclient/ckanclient/init__.py:267: base_location = 'http://www.ckan.net/api'

1302620434000000 1302625314000000
#1086 defect thejimmyg johnlawrenceaspden closed wontfix no way to delete authorization groups from web interface

as title.

1302625333000000 1323346552000000
#1087 enhancement wwaites ckan-sprint-2011-11-21 closed fixed version and contact info api call

a simple api call that returns data like this:

{ "version": ckan_software_version,
  "contact": { "name": "Some Admin", "mbox": "[email protected]" },
  "description": "Site Description",
  "url": "http://canonical.name.ckan.net/"
1302628944000000 1320866159000000
#1088 defect wwaites ckan-v1.4 closed fixed content-type autonegotiation is wonky

in ckan/controllers/package.py around line 130 it does some strange things...

perhaps replace with https://github.com/wwaites/autoneg

and handle redirection of these content types:

1302630261000000 1303035487000000
#1089 enhancement dread dread ckan-v1.4-sprint-6 closed fixed Check for "--ckan" when running nosetests

(because if you forget, you get difficult to understand errors, and more than one person has tripped up on this)

1302631189000000 1302631733000000
#1090 defect dread dread ckan-v1.4-sprint-6 closed fixed Visitor can't create packages on new CKAN install

Default visitor roles in default config is reader, not anon_editor.

Problem caused by changes in #1066 (released in 1.3.3)

New installs will be affected, although simple to just increase permissions when the installer realises a visitor can't create packages.

The solution to the config getting out of sync with the code like this is to not have the default_roles in the config - refer to the code in the configuration instructions.

1302635219000000 1302635699000000
#1091 defect johnlawrenceaspden closed wontfix usernames of users logged in using open ids are strange

If I use my gmail openID to log into a CKAN instance, then my username is:


This seems a bit odd.

1302701460000000 1323102767000000
#1092 defect kindly kindly ckan-v1.4-sprint-6 closed fixed refactor logic layer to seperate out api, form logic

The logic layer is a bit too api centric. Make the reusable parts separate in preparation for the wui refactor.

1302777929000000 1305570822000000
#1093 defect dread dread ckan-v1.4-sprint-6 closed fixed 500 errors on GET to api/rest/licenses

CKAN gets its license list from a license service, which can be a local file, but is often the http://licenses.opendefinition.org/2.0/ckan_original server. This server is currently flakey, but I think we only request the list on start up. The problem is we query it much more often than required. It is queried for every request to api/rest/licenses, and we are returning lots of 500 errors when the license server is timing out.

1302862261000000 1302865470000000
#1094 enhancement thejimmyg thejimmyg ckan-v1.5 closed duplicate [super] Refactor the Auth System

Here are some proposed changes related to CKAN's authorization system - they aren't very big, but should provide for some forthcoming use cases including #787.

Two man reasons for the changes are:

  • We have a completely refactored architecture now which introduces a logic layer. These Auth changes are designed to better support the way we work with that layer.
  • Different CKAN extension apps may need radically different authentication/authorisation so we need to allow whatever we have to be override-able.

The first two changes revolve around the is_authorized method, which is called by the logic layer to ask whether a particular user (e.g. Bob) is allowed to do a certain action (e.g. edit) on a certain object (e.g. Package).

  1. The first thing the is_authorized method is a hook to a plugin

which *overrides* the current call with its own implementation (note: in previous discussions we have considered allowing a chain of plugins, no longer!)

Reason: authorization can be completely delegated to another system (or partially)

  1. is_authorized method currently takes (username, action, object)

but for action=create_package, the object supplied is System, and for action=edit the object supplied is the package. Instead action should always be the string name of a function in the logic layer and object should always be the object passed to that function. This means our auth system is based around the actual actions we are performing (rather than a model them) and with the actual data that forms the action (rather than a related object). You never need a System object in this model.

  1. Rename these two classes to better reflect what they are
  1. Rename the Editor role to PriveledgeUser? since Editors sometimes can't edit.

Although this sounds a bit radical we already have auth extensions.

Read-only CKAN Web UI

(Additional requirement from #764)

Whilse using CKAN web interface, you are not tempted to edit stuff:

  • You know at all times this CKAN is read-only
  • All editing facilities are still seen but greyed-out with an indication why it is.
1303117973000000 1311173649000000
#1095 defect kindly kindly closed fixed add way to pass in schema to logic layer.

We need a way to pass in schemas to the logic layer to deal with edge cases.

1303221854000000 1310134959000000
#1096 defect rufuspollock pudo ckan-future new [super] CKAN Hosted

Many users of CKAN want to have their own instance without much effort. Setting these up in separate places is a maintenance nightmare, we should much rather have some tenant separation in core CKAN. Some ideas:

  • introduce model.Site and c.site
    • site has: custom CSS, extra_template_path, title, languages list, package_form, group_form (all configured via web UI)
  • Subdomain detector to activate sites.
  • use site in Authorizer instead of System, have a NullSite? for global things
  • allow cross-site search
  • packages are in a list of sites, m:n rather than 1:n
    • list of sites is string-based, can contain sites not in site table to express harvested external material which is not editable locally.
1303235062000000 1339774484000000
#1097 enhancement dread dread ckan-v1.4-sprint-6 closed fixed Sidebar hideable

The web interface has a sidebar (#primary) which should be hidden in some pages. This is for QA extension and useful for package new and edit pages. Must be compatible with DGU theme.

1303293416000000 1303293476000000
#1098 task dread dread ckan-v1.4-sprint-6 closed fixed --ckan-migration tests not initialised correctly

Only tests with failing --ckan-migration fail, due to authz not being initialised.

1303377336000000 1303406017000000
#1099 defect johnlawrenceaspden closed wontfix strange interactions between two browsers while playing with authz groups

While playing with the authorization groups, trying to design tests, I found that it was necessary to log in as two different users with two different browsers. Often actions of one user would cause server errors in the other user's browser.

I don't have a reproducible test case, but it happens fairly often so it shouldn't be too difficult to get one.

1303380824000000 1324057106000000
#1100 defect dread ckan-v1.4-sprint-6 closed fixed Get buildbot running on ckan branches

Need some changes to pip-requirements files in release branches.

1303385267000000 1303406103000000
#1101 enhancement sebbacon ckan-backlog new Integrate googlanalytics into site nav

There's a stats plugin (e.g. at http://trac.ckan.org/ticket/832).

Output from the googleanalytics plugin should append to that page, if the stats plugin is present.

Possibly the stats plugin and the googleanalytics plugin should be merged?

Finally, if the stats plugin is active, then a link to the stats page should be added to the main site footer.

1303393926000000 1339774582000000
#1102 defect johnlawrenceaspden closed duplicate searching broken in development setup

With the default test data created by

paster db clean paster db init paster create-test-data

going to the front page shows two recently changed packages A Wonderful Story A Novel by Tolstoy

But none of those words "Wonderful", etc produce search hits. In fact as far as I can tell, nothing produces any search hits.

That isn't true on ckan.net, where searching seems to work.

1303491912000000 1303744552000000
#1103 defect johnlawrenceaspden closed duplicate searching broken in development setup

With the default test data created by

paster db clean paster db init paster create-test-data

going to the front page shows two recently changed packages A Wonderful Story A Novel by Tolstoy

But none of those words "Wonderful", etc produce search hits. In fact as far as I can tell, nothing produces any search hits.

That isn't true on ckan.net, where searching seems to work.

1303494538000000 1303744575000000
#1104 defect dread johnlawrenceaspden ckan-v1.4-sprint-7 closed fixed create-test-data doesn't index the packages it creates

With the default test data created by

paster db clean paster db init paster create-test-data

going to the front page shows two recently changed packages A Wonderful Story A Novel by Tolstoy

But none of those words "Wonderful", etc produce search hits. In fact as far as I can tell, nothing produces any search hits.

That isn't true on ckan.net, where searching seems to work.

1303494635000000 1303920791000000
#1105 defect nils.toedtmann closed invalid test ticket, please ignore


1303508261000000 1303508330000000
#1106 defect rgrp rgrp ckan-v1.4-sprint-6 closed fixed Bugs related to routes arising from API refactor + removal of default routes

Various bugs I've been encountering:

Latter issue was masked by existence of 'default' routes:

   map.connect('/{controller}', action='index')

Having these is, I think, bad practice as it is better to be explicit and we should therefore remove asap.

In addition I think we should be cautious about 'default' routes in core such as:

    map.connect('/api/rest/:register', controller='api', action='list',

As it makes it harder for extensions to introduce their own APIs (here one could perhaps add something at /api/rest/{my-object} but only by using before_map rather than after_map).

1303747360000000 1303834069000000
#1107 refactor tidy-up bitesize rgrp closed fixed Move package autocomplete from package controller and move to API

Currently autocomplete method on package controller. This method should be in API (like other autocomplete methods).

Will need to update client code (just forms atm I think).

1303808480000000 1340632612000000
#1108 enhancement zephod pudo ckan-sprint-2011-09-12 closed fixed Create a more modern theme for CKAN

CKAN looks a bit aged, it should be styled more modernly and some elements could be re-arranged:

  • Collect user info in top bar
  • re-add the logo to ckan.net
  • Remove tags from main menu, replace with /sitemap.xml


quora.com, github.com, Google Projects, Google Refine, etc.

CKAN.net or CKAN general theme?

To be decided. Suggest we start with ckan.net specific and then backwards integrate (?). Existing ckan.net theme repo:


1303830790000000 1315140879000000
#1109 defect kindly kindly closed fixed When extras has a value other than a string an integrity error occurs in the api.

This is a regression that happened after refactoring the api.

It was shown by


1303839943000000 1305124697000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Note: See TracReports for help on using and creating reports.