{22} Trac tickets (2647 matches)

Results (2001 - 2100 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1251 enhancement rgrp rgrp ckan-backlog closed fixed Rename wiki.ckan.net to wiki.ckan.org

Inline with general naming policy going forward.

1311965147000000 1317315159000000
#1252 enhancement rgrp rgrp ckan-v1.5 closed fixed Deprecate ckan.net in favour of thedatahub.org

Deprecate ckan.net in favour of thedatahub.org

  • Announce
  • Create thedatahub.org and have it run same data as ckan.net
  • Announce closing down of ckan.net with 302
  • Implement 302 permanent

Also redirect wiki.ckan.net to wiki.ckan.org

1311965339000000 1318160334000000
#1271 enhancement rgrp rgrp ckan-sprint-2011-10-28 closed fixed CORS support

CORS - http://www.w3.org/TR/cors/ - support.

This is what you do in Apache. Should do this in lib/base.py or similar.

    Header always set Access-Control-Allow-Origin "*"
    Header always set Access-Control-Allow-Methods "POST, PUT, GET, OPTIONS"
    Header always set Access-Control-Allow-Headers "X-CKAN-API-KEY, Content-Type"

    # Respond to all OPTIONS requests with 200 OK
    # This could be done in the webapp
    # This is need for pre-flighted requests (POSTs/PUTs)
    RewriteEngine On
    RewriteRule ^(.*)$ $1 [R=200,L]
1313241839000000 1313753663000000
#1282 enhancement rgrp rgrp ckan-v1.5 closed fixed JS and WUI tidy up

Consolidate js. Remove cruft.

  • Move stuff to vendor
    • Have all vendor libs local (?)
    • jquery, jquery-ui etc
  • Consolidate autocomplete to use jquery-ui
    • package autocomplete (groups)
    • tag autocomplete (package)
    • user autocomplete (authz group create)
      • turns out that authz group edit had autocomplete disabled so refactoring meant enabling ...
  • Move forms.css into main
  • Remove tag cloud from front page (can be example going forward)
  • Remove bookmarklet (since in wiki at: http://wiki.ckan.net/Managing_Data_Packages#How_Do_I_Install_the_CKAN_Bookmarklet.3F)
  • Move name slug generation into standard js and clean up


  1. JS unit tests for the given pages ...
  2. Option to use external site to serve js and css (e.g. CDN)

1314090389000000 1314404540000000
#1294 enhancement rgrp rgrp ckan-v1.5 closed fixed [super] Package creation and editing UX improvements

Largely an integration of work from ckanjs

  • #1295 Simplify package create form
  • Improvements to edit form
    • Split edit form into sections
    • Remove preview from edit form (introduce inline preview on description/notes field)
  • #1296 Improved resource adding/editing on dataset/page page
  • #878 File upload integrated into resource creation
  • #1297 In-place editing of notes field for existing packages
1314405811000000 1315948257000000
#1295 enhancement rgrp rgrp ckan-sprint-2011-09-12 closed fixed Simplify package create form

Sub ticket of #1294

  • Initial screen just title, name, description (notes), license (and tags?).
  • State that you can add data and more information on next screen. This takes you to Package 'view' where you can then do more stuff.
    • Flash some instructions? e.g. that resources are missing?
  • Name slug should by default look uneditable (cf wordpress way to do the slug)
1314406014000000 1315948009000000
#1300 enhancement rgrp rgrp ckan-sprint-2011-11-07 closed fixed Core changes to base theme

Core changes to theme in order to make it easier to re-theme:

  • page_heading
  • side bar menu must be switchable (left to right) (through config or css)
  • Add optional_footer to complement optional_head
    • And adopt rule for template writers that all extra js must go in optional footer
    • That way we could move optional js to the bottom of the page to improve page load times.
  • Re-add support for body-class ... (seems to be removed by #1108)
  • more divs.
  • better labelling for css (and javascript)
1314862223000000 1319794520000000
#1311 enhancement rgrp rgrp ckan-backlog new Modal user register and login form

Subticket of: #1294

Rather than having to visit a dedicated page it would be good if registration and login could be done from a modal form (separate or combined ...).


  • It could be used from dataset creation page in situations where user needs to be registered / logged in to create a dataset (so we could allow someone to start creating a dataset and only get them to login at the end ...)
  • It allows for quicker and easier logging in


  • See Friedrich's work on the datahub
1315297227000000 1315297227000000
#1319 enhancement rgrp dread ckan-sprint-2011-10-24 closed fixed Require an email address on User registration

Superticket: #1343

This avoids difficulties when they forget their password.

  • (?) Email validation (navl does not have one!)
  • Require email confirmation to be activated (?) (Answer: no, not at present -- see discussion below)
1315415932000000 1324036947000000
#1343 enhancement rgrp rgrp ckan-backlog new [super] User related improvements (login, user pages etc)
  • Disallow account creation via openid - #1386
  • Require email field - #1319
    • Require email confirmation to be activated (?)
  • Improvements to user page (e.g. show activity and more info about user) - #1396
  • Modal user login - #1311
1316017098000000 1318528138000000
#1357 enhancement rgrp rgrp ckan-sprint-2011-10-10 closed fixed Integrate new version of data preview

We have an all new and much improved data preview system developed as part of QA in July. Need to integrate this.

  • Is pure javascript
  • Will have this in CKAN core but considered to be a plugin (so need datapreview in ckan.plugins list to work -- will enable this by default)
  • Will depend on the http://wiki.ckan.org/DataExplorer
1317076044000000 1318164458000000
#1359 enhancement rgrp rgrp ckan-sprint-2012-02-20 closed fixed Radically simplify standard list of licenses

Major UX grip and incredibly simple to do. May also want to have basic autocomplete support within the list (trivial to do as well I believe).

(Talk to RP about this re how we define lists of licenses).

1317077855000000 1329132464000000
#1361 enhancement rgrp rgrp ckan-v1.5 closed fixed Simple search support

Would be nice if CKAN could work out of the box without the need for SOLR (solr is great but complex and heavyweight to install).


  • ckan.simple_search config option
  • If set:
    • query via simple query to database backend
    • do no specialized indexing



  • TagSearchQuery? from lib/search (just do a search directly)?
  • #1360: filter by downloadable and filter by open search options

Possible Extras

(Probably future improvements)

  • Re-introduce full-text search indexing where supported in e.g. postgres and use for querying
1317078276000000 1317082893000000
#1384 task rgrp shevski ckan-backlog new CKAN wiki needs updating to refer to thedatahub.org instead of ckan.net and datasets instead of packages

Most articles still refer and link to ckan.net, wiki.ckan.net and to packages

1318414077000000 1318414077000000
#1386 enhancement rgrp rgrp ckan-sprint-2011-10-24 closed fixed Disallow account creation via openid

Superticket: #1343

Creation of accounts from OpenID causes large number of problems:

  • No guarantee of other profile info
  • Poor username (just from openid)

We therefore will:

  1. Require creation of user account via register (no auto-creation of accounts via openid)
    • sidebar of register page should point to login page rather than openid signin

  1. Permit association of an openid with an account ...
    • How: paste in your openid url (requires change to edit form to have this)
      • won't work for google with their weird openid urls
    • [future] Nicer way is to have login via openid while logged in (which association then happening in background)
  1. Migration (for 1+2):
    • (?) Generate decent usernames for all existing accounts
    • As this is nontrivial suggest instead we allow editing of usernames (by account owner and sysadmins). This is useful in its own right and is a reasonable 80/20 solution.
      • This does present the problem of re-associating commits with the new username. Best solution to this would be to switch revisions to point to userid rather than username. See #1534
  1. [optional] Change display_name to always be username
    • Change My Account to Username at top right of all pages (once we know usernames are short ...)
      • could do this straight away by truncating long usernames (e.g. truncate at 20 chars ...)
      • Also should we lose the icon?
1318425493000000 1324036930000000
#1396 enhancement rgrp rgrp ckan-sprint-2012-02-06 closed fixed Improvements to user page UX - 1d

Super ticket: #1506

  • Show member since
  • Display email (in obfuscated form)
  • List all owned datasets (in standard manner)
    • Will need change to logic layer to show datasets on user dict (this is useful for the API as well ...)
  • (Re)move list of changes
    • Should have activity (once activity available)


  • ?? Allow users to provide a home page, organization (requires migration)
1318527967000000 1330086237000000
#1401 defect rgrp flofokus ckan-sprint-2012-01-23 closed fixed Some links don't take account when CKAN is mounted at a non-root URL

if you want your ckan to reside not at http://yourhost.org/ but rather e.g. at http://yourhost.org/ckan/ you can use this hint http://wiki.pylonshq.com/display/pylonsdocs/Configuration+Files and set ckan.site_url accordingly in your .ini. However, there are some instances where internal links are created without the required prefix.

1318621586000000 1328001014000000
#1407 defect rgrp dread ckan-sprint-2011-12-19 closed fixed Stats extension not working

Graphs don't show any more at http://thedatahub.org/stats

1319014605000000 1323764239000000
#1412 defect rgrp seanh ckan-sprint-2012-02-06 closed worksforme application.js crashes when viewing a dataset with no resources

Create a new dataset, don't add any resources to the dataset yet, view the dataset in ckan, if you have firebug enabled you'll see the error "resource is undefined" at line 699 of application.js.

1319450625000000 1328226385000000
#1429 enhancement rgrp rgrp ckan-backlog new Provide DOIs for datasets in a CKAN instance

DOI = digital object identifier = http://www.doi.org/

As a Publisher I want a DOI for my dataset so that it can be cited by and linked to by others in a standard and easy way.


  • Probably implement as extension rather than core
1319977305000000 1319977305000000
#1446 enhancement rgrp rgrp ckan-sprint-2012-01-09 closed fixed Data Explorer v2

Super ticket: #1602 (Data Previewer v2)

We already have first pass of Data Explorer that was released as part of #1357.

Tickets include (* indicates improvement over current explorer)

Est: 10-15d (should be broken down -- partly is in recline issues)

1320665596000000 1326281658000000
#1458 enhancement rgrp rgrp ckan-future assigned Support previewing kml files in data viewer

Super ticket: #1151 (viewing geo data)

E.g. preview of http://thedatahub.org/dataset/louisvillecrime should bring up a map

1320936488000000 1340632932000000
#1459 enhancement rgrp rgrp ckan-backlog new Featured Dataset feature

Provide way to mark a dataset as featured. Featured database show up on the front page.

TODO: detail this more.

1321113012000000 1321113012000000
#1472 enhancement rgrp rgrp ckan-sprint-2011-11-21 closed fixed Convert ckanclient to git and github from mercurial 1321624905000000 1321721874000000
#1518 defect rgrp markbrough ckan-sprint-2011-12-19 closed fixed CKAN Upload fails if filename has spaces in it

E.g. uploading a file with spaces in it: OECD Monthly Exchange Rates.zip

Gives a 404 Not Found response to the following file: http://test.ckan.org/api/storage/metadata/2011-12-05T193046/OECD%20Monthly%20Exchange%20Rates.zip

The upload wheel keeps spinning and the user is not informed that the upload has failed.

Analysis: turns out that google storage (possibly s3 but not checked) replaces with ' ' in keys with '+' on upload. This breaks things because we try and look up metadata about upload using the filename/key we put in but of course that does not exist because google has changed name.

Fix is trivial: replace ' ' in keys / filenames with '-'.

1323114236000000 1330020742000000
#1533 defect rgrp amercader ckan-backlog closed wontfix Make sure ckanext-wordpresser works on CKAN 1.5.1

Superceded by #2234

Super ticket: #1508

Etherpad: http://ckan.okfnpad.org/wordpresser

The wordpresser extension does not work with ckan 1.5.1. The wordpress page is not loaded.

Looks like this changeset may be involved https://github.com/okfn/ckanext-wordpresser/commit/0efa0649896a7339865f178a302be639ddc561e1

I think that even before that changeset, the page was shown without any margins.

In any case, the extension seems to be too sensible to changes in the ckan layout so perhaps it should take into account the CKAN version and have a fixed set of xpaths and templates that work for each version

1323277660000000 1338202676000000
#1559 enhancement rgrp jilly mathews ckan-sprint-2012-04-02 closed fixed Comments Extension / Disqus

Polish off comments extension dev and test. estimate 2 days.

1324291720000000 1332242129000000
#1571 enhancement rgrp jilly mathews ckan-backlog assigned [super] Issues Extension

"Proper" issues extension.


  • This would be an extension and improvement of existing todo extension - #842 (https://github.com/okfn/ckanext-issues)
    • Current extension does work but lacks polish
    • Should we polish or do significant improvements?
  • Functionality - see etherpad


  • (Rename todo(s) -> issue(s) in existing extension)
1324294225000000 1330863183000000
#1576 enhancement rgrp rgrp ckan-sprint-2012-01-09 closed fixed Move stats extension back into core - 0.5d

Est: 0.5d.


  • Why do this?
    • tiny extension with few dependencies - and really nice to have out of the box
    • trial for doing this on larger scale
  • Do we keep as extension (even if in core)?
    • Ans: Yes, keep as extension because:
      • Already set up that way
      • Cleaner
      • Easier to disable / enable
  • How do we integrate with main them (e.g. have stats link)
    • Ans: not sure (this is part of more general issue of how we update theme for varying changes elsewhere). Best answer is to have some widgetization in theme.
  • Hide ratings section (at least until we reinstate ratings #1598)
    • Ans: no, let's not bother (and having ratings there encourages us to do #1598 and/or find out whether people are interested in ratings)
1324317313000000 1327056070000000
#1577 defect rgrp dread ckan-backlog new Can't upload file with foreign chars in filename

Looks like uploading a file with foreign characters fails due to encoding reasons.

URL: http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf
Module weberror.errormiddleware:162 in __call__
<<              __traceback_supplement__ = Supplement, self, environ
                   sr_checker = ResponseStartChecker(start_response)
                   app_iter = self.application(environ, sr_checker)
                   return self.make_catching_iter(app_iter, environ, sr_checker)
>>  app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<<                                                     self.cache_manager)
               environ[self.environ_key] = self.cache_manager
               return self.app(environ, start_response)
>>  return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<<                          headers.append(('Set-cookie', cookie))
                   return start_response(status, headers, exc_info)
               return self.wrap_app(environ, session_start_response)
           def _get_session(self):
>>  return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<<                  environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
               response = self.app(environ, start_response)
               # Wrapped in try as in rare cases the attribute will be gone already
>>  response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
               controller = self.resolve(environ, start_response)
               response = self.dispatch(controller, environ, start_response)
               if 'paste.testing_variables' in environ and hasattr(response,
>>  response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<<          if log_debug:
                   log.debug("Calling controller class with WSGI interface")
               return controller(environ, start_response)
           def load_test_env(self, environ):
>>  return controller(environ, start_response)
Module ckan.lib.base:123 in __call__
<<          # available in environ['pylons.routes_dict']    
                   return WSGIController.__call__(self, environ, start_response)
>>  return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<<                  return response(environ, self.start_response)
               response = self._dispatch_call()
               if not start_response_called:
                   self.start_response = start_response
>>  response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<<              req.environ['pylons.action_method'] = func
                   response = self._inspect_call(func)
                   if log_debug:
>>  response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<<                        func.__name__, args)
                   result = self._perform_call(func, args)
               except HTTPException, httpe:
                   if log_debug:
>>  result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
           def _inspect_call(self, func):
>>  return func(**args)
Module ckanext.storage.controller:2 in auth_form
Module ckan.lib.jsonp:26 in jsonpify
<<      Very much modelled after pylons.decorators.jsonify .
           data = func(*args, **kwargs)
           return to_jsonp(data)
>>  data = func(*args, **kwargs)
Module ckanext.storage.controller:301 in auth_form
<<          method = 'POST'
               authorize(method, bucket, label, c.userobj, self.ofs)
               data = self._get_form_data(label)
               return data
>>  authorize(method, bucket, label, c.userobj, self.ofs)
Module ckanext.storage.controller:79 in authorize
<<      if method != 'GET':
               # do not allow overwriting
               if ofs.exists(bucket, key):
               # now check user stuff
>>  if ofs.exists(bucket, key):
Module ofs.remote.botostore:53 in exists
<<          if bucket is None: 
                   return False
               return (label is None) or (label in bucket)
           def claim_bucket(self, bucket):
>>  return (label is None) or (label in bucket)
Module boto.s3.bucket:87 in __contains__
<<      def __contains__(self, key_name):
              return not (self.get_key(key_name) is None)
           def startElement(self, name, attrs, connection):
>>  return not (self.get_key(key_name) is None)
Module boto.s3.bucket:144 in get_key
<<          response = self.connection.make_request('HEAD', self.name, key_name,
               # Allow any success status (2xx) - for example this lets us
               # support Range gets, which return status 206:
>>  query_args=query_args)
Module boto.s3.connection:388 in make_request
<<          if isinstance(key, Key):
                   key = key.name
               path = self.calling_format.build_path_base(bucket, key)
               boto.log.debug('path=%s' % path)
               auth_path = self.calling_format.build_auth_path(bucket, key)
>>  path = self.calling_format.build_path_base(bucket, key)
Module boto.s3.connection:88 in build_path_base
<<      def build_path_base(self, bucket, key=''):
               return '/%s' % urllib.quote(key)
       class SubdomainCallingFormat(_CallingFormat):
>>  return '/%s' % urllib.quote(key)
Module urllib:1222 in quote
<<              safe_map[c] = (c in safe) and c or ('%%%02X' % i)
               _safemaps[cachekey] = safe_map
           res = map(safe_map.__getitem__, s)
           return ''.join(res)
>>  res = map(safe_map.__getitem__, s)
KeyError: u'\xed'
CGI Variables
AUTH_TYPE	'cookie'
CONTENT_TYPE	'; charset=utf-8'
HTTP_ACCEPT_CHARSET	'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
HTTP_ACCEPT_ENCODING	'gzip,deflate,sdch'
HTTP_CACHE_CONTROL	'max-age=259200'
HTTP_CONNECTION	'keep-alive'
HTTP_COOKIE	'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; ckan_user=elenaibp; ckan_display_name="Elena Mondo"; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; __utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10.1324298316; __utmc=119670349; __utmz=119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)'
HTTP_HOST	'thedatahub.org'
HTTP_REFERER	'http://thedatahub.org/dataset/edit/budget-library-czeck-republic'
HTTP_USER_AGENT	'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7'
HTTP_VIA	'1.1 localhost (squid/3.0.STABLE19)'
PATH	'/usr/local/bin:/usr/bin:/bin'
PATH_INFO	'/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
PATH_TRANSLATED	'/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
REMOTE_USER	u'elenaibp'
REMOTE_USER_DATA	'userid_type:unicode'
REQUEST_URI	'/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf'
SCRIPT_FILENAME	'/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py'
SCRIPT_URI	'http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
SCRIPT_URL	'/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
SERVER_ADMIN	'[no address given]'
SERVER_NAME	'thedatahub.org'
SERVER_SIGNATURE	'<address>Apache/2.2.14 (Ubuntu) Server at thedatahub.org Port 80</address>\n'
SERVER_SOFTWARE	'Apache/2.2.14 (Ubuntu)'
WSGI Variables
application	<beaker.middleware.CacheMiddleware object at 0x7f22601c7dd0>
beaker.cache	<beaker.cache.CacheManager object at 0x7f22601c7b50>
beaker.get_session	<bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x7f22601c7a90>>
beaker.session	{'_accessed_time': 1324298703.071357, '_creation_time': 1324293077.4139669}
mod_wsgi.application_group	'ckan.net|'
mod_wsgi.callable_object	'application'
mod_wsgi.listener_host	''
mod_wsgi.listener_port	'80'
mod_wsgi.process_group	'ckan.net'
mod_wsgi.reload_mechanism	'1'
mod_wsgi.script_reloading	'1'
mod_wsgi.version	(2, 8)
paste.cookies	(<SimpleCookie: __utma='119670349.1809834699.1323782464.1324293066.1324298316.4' __utmb='119670349.3.10.1324298316' __utmc='119670349' __utmz='119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' auth_tkt='f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode' ckan_apikey='decd48b1-49ee-4250-bff4-98ccca9c02a5' ckan_display_name='Elena Mondo' ckan_user='elenaibp' hide_welcome_message='1' thedatahub_net='27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e'>, 'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; ckan_user=elenaibp; ckan_display_name="Elena Mondo"; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; _ _utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10...)|utmcmd=(none)')
paste.registry	<paste.registry.Registry object at 0x7f226194df50>
paste.throw_errors	True
pylons.action_method	<bound method StorageAPIController.auth_form of <ckanext.storage.controller.StorageAPIController object at 0x7f2261dad990>>
pylons.controller	<ckanext.storage.controller.StorageAPIController object at 0x7f2261dad990>
pylons.environ_config	{'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons	<pylons.util.PylonsContext object at 0x7f2261daddd0>
pylons.routes_dict	{'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'}
repoze.who.identity	<repoze.who identity (hidden, dict-like) at 139785645747120>
repoze.who.logger	<logging.Logger instance at 0x7f225e23c098>
repoze.who.plugins	{'openid': <OpenIdIdentificationPlugin 139785625065680>, 'friendlyform': <FriendlyFormPlugin 139785618095248>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator object at 0x7f2260874c10>, 'auth_tkt': <AuthTktCookiePlugin 139785625065808>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f2260874c90>}
routes.route	<routes.route.Route object at 0x7f22601a1090>
routes.url	<routes.util.URLGenerator object at 0x7f2261dadf50>
webob._parsed_query_vars	(GET([]), '')
webob.adhoc_attrs	{'language': 'en-us'}
wsgi process	'Multiprocess'
wsgi.file_wrapper	<built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f2261da9af8>
wsgiorg.routing_args	(<routes.util.URLGenerator object at 0x7f2261dadf50>, {'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'})
1324317659000000 1325473564000000
#1599 enhancement rgrp rgrp ckan-sprint-2012-01-09 closed fixed [contrib] Simple embeddable dataset count widget (esp for group count)

Simple embeddable widget for use on 3rd party sites showing dataset counts for a given search query. Have a specific version just for groups.

  • Simple group count widget in JS for embedding in wordpress and elsewhere
    • Requested by several people (e.g Guo Xu from Econ working group). Already have something like this in CKAN JS for doing an embeddable search box.
  • All you need to do is do a dataset query over the API e.g. http://thedatahub.org/api/search/dataset?groups=economics and then embed in some html!

Estimate: 30m (for someone who knows their jquery).

1325246358000000 1325555201000000
#1600 enhancement rgrp rgrp ckan-v1.7 closed wontfix [super] User guide with integrated help in the WUI

Provide help to users in the WUI, with this linking to a detailed user guide.

  • User guide (suggest at help.thedatahub.org (or possibly help.ckan.org)
  • Snippets in the WUI with links to full user guide.
    • Provide helper methods for creating these links with config options for setting path to user guide so that the link location can vary on install by install basis
  • General info about publishing/working with/etc data (where we don't duplicate elsewhere)

Est: (?) 5-10d


See http://ckan.okfnpad.org/datahub-help

  • Step-by-step for adding a dataset including tips
  • API (perhaps link to main docs)
  • Command line interface and dpm
  • Using groups

Why do this

We already have wiki.ckan.org and docs.ckan.org. Why do this?

Answer for docs:

  • This is administrative guide and focused on general CKAN software.
  • Hard to provide completely generic help.
  • Integratable with the DataHub? site

Answer for wiki:

  • No spam
  • General feeling that wikis are a little bit loose and untidy. We want this to be smart, clean and useful.
  • Integratable with the DataHub?
1325257824000000 1333362051000000
#1602 enhancement rgrp rgrp ckan-v1.6 closed fixed [super] Data Previewer / Viewer v2

We already have first pass of Data Preview that was done as part of #1357. This is about v2 of this. Two main components:

  • A new version of the separate Data Explorer library (now called Recline Data Explorer) - #1446 Est: 10-15d
    • Why:
      • Cleaner, backbone based solution.
      • Provides editing support if wanted.
      • Collaboration with Max Ogden
  • Integrate this into CKAN (into resource view page) - see #1630 Est: 1d
  • Improvements to Data Viewer / Previewer system in CKAN itself - Est: 2-3d
    • (Update Data Preview-er (View-er) to support images ...)
    • (WONTFIX here - (?) Update Data Preview-er to support geodata - #1151)

1325267544000000 1328175719000000
#1630 enhancement rgrp rgrp ckan-sprint-2012-02-06 closed fixed Integrate Data Explorer v2 into Data Viewer

Super ticket: #1602


  • Include (or git submodule) data explorer into vendor
  • Update existing js integration code
  • Fix css and any bugs

Est: 1d

Actual: 1d

1326279098000000 1330085600000000
#1660 defect rgrp lucychambers ckan-sprint-2012-02-06 closed wontfix CSV preview broken - OpenSpending

This CSV resource used to preview but now the format appears to be unsupported: "We are unable to preview this type of resource: x-osdata-csv"


This is used as an example by the OpenSpending? team all the time.

1326717846000000 1328013627000000
#1691 enhancement rgrp rgrp ckan-sprint-2012-01-23 closed invalid paster user create command takes password on command line

Needed to support automated deployment more easily.

Est: 15m

1327077314000000 1327314081000000
#1719 defect rgrp dread ckan-sprint-2012-02-06 closed fixed Broken links for non-Gravatar use icons - 0.25d

Super ticket: #1506

e.g. http://thedatahub.org/user The users with Gravatar have their nice user icons, but the majority of users (without Gravatars) have 'broken link' symbols. Same problem wherever users are shown, such as the dataset history pages.

Broken on 1.5.2b - currently on thedatahub.org.

1327938788000000 1328541630000000
#1797 enhancement rgrp rgrp ckan-sprint-2012-03-05 closed fixed [super] Webstore v2 - Integrated and based on ElasticSearch

Main tasks:

  • Integrated seamlessly into CKAN (authorization etc)
  • Based off ElasticSearch?

Etherpad: http://ckan.okfnpad.org/webstore


  • New API point (config + controller) [0.75d]
    • /api/data/{resource-id}
  • Authorization [0.5d]
  • Documentation [0.25d]
  • Upgrade data viewer to use ES [0.5d]
  • Convert webstorer to new setup - http://github.com/okfn/ckanext-webstorer [1d] - #1753
    • See below
  • "Create DataStore? Resource" in Resource creation (??)
    • Just creates a resource with blank url and datastore enabled (webstore_url non null)
  • Tick box to enable DataStore? in Resource edit form (for time being would just set webstore_url to any non-null/non-empty value)
    • Also need to use this info in DataStore? extension?
  • Create nice instructions on Resource page about using the DataStore?
  • Rename webstore -> datastore
  • Deploy [0.5d]


Layout on ES


index = ckan-{ckan-id}
type-id = resource-id (1)

Request path

Req => Nginx => CKAN App (authz) => req to elasticsearch and returns results

See as an example: https://github.com/okfn/elastic-proxy


  1. How do we decide what to use in the viewer e.g. Webstore data versus DataProxy?
    • Check for existence of webstore_url on Resource (should really be webstore_enabled boolean or similar, however we stick with url for the time being because it exists and we avoid migration). If exists use Webstore o/w don't.
  2. How do we decide when webstore is activated
    • Ans: see previous.
  3. Do we (and how do we) allow for resources which are just for webstores
    • Ans: we have to set webstore_url = True ... on save. Or add a big button saying: "Create Blank Resource" ...
  4. How does webstorer work ...
    • CSV import - NS: make sure we add e.g. rownum or id = rownum ...
    • Separate mini-service (webstorer.thedatahub.org ...)
  5. How do we selectively disable recline where not appropriate (e.g. query support makes no sense for DataProxy?)
    • Ans: do not bother at present. Focus on Webstore route.
  6. Where does geodata support come in (in Recline or separately)
    • This is a separate issue

User Story 1

User -> CKAN instance -> Add Dataset -> Adds a Resource via uploading a CSV file -> Save

=> Result = New Dataset + Resource + Data API to resource and nice Preview of the data


  1. Allow normal upload process
  2. Webstorer queue task which should get triggered by new resource. Check the resource and if it can convert into the associated resource webstore database.
  3. Update the resource in some way (set webstore_url)

User Stories

User Story 2

User -> CKAN instance -> Add Dataset -> Adds a Resource via linking to some remote file (CSV or otherwise) or uploading some data (CSV or otherwise)

=> Result = New Dataset + Resource + nice Preview of the data (wherever possible)

User Story 3

User -> CKAN instance -> Add Dataset -> Adds a Resource which will be webstore database in CKAN -> User saves data to this data store.

=> Result = New Dataset + Resource (Webstore resource) which can now be written to or read from by the user

Solution: just create a resource with blank url (or better, have a new option to create webstore resource ...)

1329306185000000 1331412644000000
#1828 enhancement rgrp rgrp ckan-sprint-2012-03-05 closed fixed Upgrade to Recline v0.3

Recline v0.3 has introduced some breaking changes in API and usage. Upgrade to use it.

1329943304000000 1330525412000000
#2221 enhancement rgrp zephod closed fixed Tests are broken for some of us: NotAPairTreeStoreException

Since my last pull I'm getting a strange new error which is presumably to do with my local config or assumptions about the storage extension.

Test output here: https://gist.github.com/2007985

1331318781000000 1331561095000000
#2222 enhancement rgrp zephod closed duplicate Tests are broken for some of us: NotAPairTreeStoreException

Since my last pull I'm getting a strange new error which is presumably to do with my local config or assumptions about the storage extension.

Test output here: https://gist.github.com/2007985

1331318783000000 1331318825000000
#2224 enhancement rgrp rgrp ckan-sprint-2012-04-02 closed wontfix Simplify javascript and css dependencies and add minified version
  • Get rid of jquery UI
  • Remove modernizr (what do we use it for?)
  • Convert jquery.tmpl to jquery mustache (?)
  • Remove ckanjs moving core parts of it directly into our code.

Compress everything for releases and add minified version to build (with switch to enable use of this?)

1331385623000000 1338205358000000
#2226 enhancement rgrp rgrp ckan-sprint-2012-03-19 closed fixed [super] Refactor and improve documentation (v1.7)

Documentation is key to the success of projects!

Move material into main docs from wiki

Create a User Guide

Basic step-by-step on getting started with CKAN (See start on set of slides here e.g. http://blog.thedatahub.org/2012/03/02/tutorial-publish-data-with-the-datahub/)

  • Publishing data
  • Authorization and workflows
    • Publisher Profile (Workflow)

Break into sections

Suggest something like:

# General
* Intro
* Conceptual Overview
* What CKAN helps you do (http://ckan.org/)
* FAQ ? (or point to the wiki)

# For Administrators

* Installation and Setup
  * Configuration options
* Customization - Theming etc
* Authorization and Workflows
* Storage

# For Users (Publishers, Data Wranglers, etc etc)
* Walkthrough of publishing a dataset
* Storage
* API (see Developers section)

# CKAN Developers

* Domain Model
* Extensions
* i18n

CKAN Developers
* Buildbot
1331481965000000 1332475494000000
#2236 enhancement rgrp rgrp ckan-sprint-2012-03-19 closed fixed Updates to Recline and integrate them
  • (Recline) Better graphing (multiple graph types etc)
  • (Recline) Bootstrapify
  • Integrate
1331910381000000 1332153688000000
#2278 defect rgrp rgrp ckan-sprint-2012-04-16 closed fixed DataStore enabled is disabled by default but is needed for DataStorer to run

Major UX gotcha reported by several people is that DataStore? is disabled by default and must be enabled for DataStorer? to run and import data.

Suggest switch to enable datastore by default for file and file.upload types (disable for API). People can always disable if they want.

1333441048000000 1333441177000000
#2279 enhancement rgrp rgrp ckan-v1.8 new Write tutorial for our DataStore API / Data Explorer

Unfortunately ES is a little bit complex / opaque to newcomers. Need to write a simple tutorial walking through this.

This is documentation / example only (no code!)

1333482263000000 1338205426000000
#2292 defect rgrp rgrp ckan-sprint-2012-04-30 closed fixed Group edit form no longer allows for adding of multiple datasets at once

Can only add one dataset at a time now (not sure when this broke but was working ~2-3m ago).

Assigning to seanh for review and estimation. I'd rate this as quite annoying but not absolutely critical (hence assigned priority).

1334431388000000 1335021876000000
#2874 enhancement rgrp rgrp assigned Clean up bin directory

Full of obsolete material

1345190508000000 1345190515000000
#1279 enhancement ross amercader ckan-v1.8 closed duplicate Refactor or deprecate Authorization Groups

Authorization Groups are going to be deprecated in 1.8

It is still unclear how Authorization Groups fit in the new authz model, so they have been left out of the logic and auth refactoring (Tickets #1229 and #1253)

1313415367000000 1338212028000000
#1508 enhancement ross rgrp ckan-v1.6 closed fixed [super] Finalize and deploy some of our major (new) features as of Dec 2011

This ticket is about finalizing, then deploying (to http://thedatahub.org/), and finally announcing some of recent major new features such as:

  • Deploy wordpresser extension for thedatahub (so we can start blogging) - #1533 - WONTFIX
  • (Deploy new Resource and Dataset pages (invite comments))
  • Get thedatahub geo-ified i.e. ckanext-spatial - #1601 (?)
    • Any additional dev that needs to be done? (e.g. geo in search ...?)
  • (Deploy / re-enable ckanext-googleanalytics on thedatahub - #1451 (more stuff #1566))
    • Rename the extension to ckanext-analytics?
  • Custom forms (and use of 'dataset types') - Ticket (??)
  • (Get QA deployed and running - #1582)

It would be best if this was done by someone *unfamiliar* with the extensions so feedback could be provided on quality of install docs etc etc.

1323088817000000 1328000871000000
#1512 defect ross rgrp ckan-v1.8 closed fixed Modifying user name makes you semi-log-out

If you modify your user name, it looks like you are logged out, but you are actually logged in and therefore can't login. (Reported by Pablo)

As a work around, you need to logout and log back in.


  • We should force this by auto-logging you out (DR: maybe!)
1323100214000000 1342781122000000
#1551 enhancement ross ross ckan-backlog closed fixed CKAN auth for webstore changes

Webstore should use auth api ( #1550 ) for authenticating users accessing webstore rather than talking directly to the CKAN database. We also need it to suppose /user/ urls and /userid/ for accessing databases.

1324049966000000 1346662048000000
#1574 enhancement ross ross ckan-v1.7 closed fixed [super] Storage changes including automated save to Webstore

Allow uploading of files to push the data into webstore. Initially we were going to suggest changes to ckanext-storage but after further analysis we arrived at the conclusion that this should be implemented by ckanext-archiver as it already handles archiving of data from various sources and would be the best place to 'archive' to webstore.

  1. A user wants to upload a file to CKAN, and so chooses the file upload option as they do currently with ckanext-storage. The file upload itself should be handled by this code internal to core (see #1608) but still using ofs [5d]
  1. The user is provided the link to the file as currently.
  1. The archiver's update task will check configuration to determine if it should upload some mime-types (tabular formats) to webstore. Configuration should also determine where webstore is. (see #1609) [5d] - took 6
  • It should be stored in webstore using {username}/{resource-id} where {username} is the name of the user uploading the original file.
  1. The archiver will continue to move the file into permanent storage as per current practice.


  • #1608 Merge storage into Core [5d]
  • #1609 Celery task for ckanext-archiver to write to webstore. [5d] (took 6)
  • #1687: Deploy to datahub [1d]
  • #1681: Core storage documentation [1d?]
    • Internal (perhaps in README somewhere or wiki): how does process work, where does data end up and in what order etc
    • External: here's how to configure it and here's what you get ... (goes in main sphinx docs)
1324315633000000 1338202955000000
#1579 enhancement ross ross ckan-backlog closed fixed Full text search in Webstore

We should allow full text search within databases for individual databases, although we may want to wait for any pending ports to postgres.

1324385698000000 1338212496000000
#1608 enhancement ross ross ckan-sprint-2012-01-23 closed fixed Merge storage into Core

As part of Storage Changes and http://ckan.okfnpad.org/newstorage it was decided to merge ckanext-storage back into core.

Other changes:

  • Remove /storage/upload page (we have frontend integration elsewhere now and we should also ensure that upload items relate to a resource or other domain object).
  • Provide a local path as well as remote path for form upload requests in API so that frontend can see url to CKAN path rather than cloud storage path ...
1325582044000000 1330036982000000
#1609 enhancement ross ross ckan-sprint-2012-01-23 closed fixed Celery task for ckanext-archiver to write to webstore.

From super Storage changes - #1574 - and http://ckan.okfnpad.org/newstorage we determined that ckanext-archiver should have a celery task for grabbing local file uploads and writing to webstore


When I upload a file to CKAN:

  • End up with file in permanent storage
  • IF file is ot type ... csv,xls,xlsx,sqlite,.sql
    • End up with new db in webstore
      • Where? {username}/{resource-id}/...
        • If a single table: name it after the file name (appropriately slugified)
      • A resource *always* corresponds to a 'database' in webstore ...
      • In Data Explorer have "Sheets" tab ...
  • Resource url = /dataset/{x}/resource/{y}/link -> cached_url ...
1325582253000000 1327057030000000
#1610 enhancement ross ross closed duplicate Move webstore to Postgres instead of Sqlite

The default backing store for webstore should be Postgres and not sqlite. This was agreed as part of the #1574 storage changes and on http://ckan.okfnpad.org/newstorage

1325587341000000 1325852472000000
#1611 enhancement ross ross closed duplicate Implement auth API calls for webstore/external use

As part of #1574 we decided that it would be better for webstore (and future external services) to be able to authenticate simply with CKAN-Core.

Currently webstore access the CKAN database to obtain the key for the user but it would be better if this connection was not so tightly bound and that webstore used an API as any other external service might.

Need to discuss further with dread

1325590191000000 1325846987000000
#1649 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Verify RDF Extension (ckanext-rdf) works
  • Make sure that the extension works as it should and isn't broken on current releases [3d]
  • Make it work with the EC schema, including extras [2d]
1326709096000000 1338205274000000
#1650 enhancement ross ross ckan-future closed wontfix Add new language/translation of fields for datasets.

Datasets should have a field that specifies the language used in the dataset, this should be a two character ISO 639-1 code (see http://en.wikipedia.org/wiki/List_of_ISO_639-1_codes for list) . Also make sure all tests and migrations are resolved. [2d]

Determine the best approach to having 'translation_of' references to show which dataset the current one is a translation of. This may be a translation of field, or a 'copied from' field or another form of package relationship. [2d]

1326709392000000 1346670010000000
#1657 enhancement ross ross closed wontfix Configuration for reverse proxying

Provide configuration for reverse proxying that will correctly handle the mapping of a URL to a sub-folder (using X-SCRIPT-NAME)

  • Analysis of the best solution [1d]
  • Implement reverse proxying in tandem with #1653 [2d]
  • Document and store the configuration files. [1d]
1326711580000000 1338206640000000
#1658 enhancement ross ross closed wontfix Implement proxy caching

Implement proxy caching for performance gains

  • Analysis of best solution and what we can deliver in a reasonable time [2d]
  • Checks for presence of cookie 'auth_tkt' and appropriate setup for media [2d]
  • Remove session requirements for user who isn't logged in [3d]

  • Document the configuration and setup so that it can be easily modified and replicated [1d]
1326712354000000 1338206622000000
#1669 enhancement ross ross ckan-v1.7 closed fixed [super] Publisher profile

Implement the necessary backend changes to build profile for publishers using the group refactor.

See http://ckan.okfnpad.org/publisherprofiles

Subtickets are at:

  • Publisher profile analysis #1670 [2d]
  • Publisher profile sysadmin authz #1671 [1d]
  • Publisher profile user authz #1672 [1d]
  • Test auth logic hooks for publisher profile #1674 [3d]
  • Publisher admin authz #1675 [3d]
  • Publisher editor authz #1676 [2d]
  • Add approval status field to groups #1673 [1d]
1326802473000000 1338204071000000
#1670 enhancement ross ross ckan-sprint-2012-03-05 closed fixed Publisher profile documentation

Super #1669

Analysis of all permissions for users given their publishers to ensure that we have covered all of the bases and haven't missed any use-cases.

1326802549000000 1338205180000000
#1671 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Publisher profile sysadmin authz

Super #1669

Implement authz for sysadmins in publisher profile

1326802607000000 1328526614000000
#1672 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Publisher profile user authz

Super #1669

Authz for normal users in publisher profile as per sysadmin

1326802680000000 1328526625000000
#1673 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Add approval status field to groups

Super #1669

As groups need to be authorised in some cases, particularly publisher profiles, we need a field to denote how this is managed.

We should add an 'approval_status' field that will be used to denote its status. By default this will be "approved" but this will allow us to override it with a "pending" or "denied" status. The latter should also change the main status of the Group.

1326802753000000 1328526644000000
#1674 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Test auth logic hooks for publisher profile

Super #1669

Test all of the new auth logic hooks for publisher profile to make sure they work as expected. Also make sure that we have a test to compare the naming of the core auth funcs with the profile hooks (so we can see what may have changed in one but not the other).

1326802841000000 1328526677000000
#1675 enhancement ross ross ckan-sprint-2012-02-20 closed fixed Publisher admin authz

Super #1669

Authz for the publisher admin (who has permission to administer the publisher group themselves).

1326802923000000 1328709915000000
#1676 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Publisher editor authz

Super #1669

Editor authz for publisher groups need to be implemented and tested.

1326802971000000 1328526670000000
#1680 enhancement ross ross closed wontfix Group refactor top level element

The new group refactor allows for a hierarchy of groups where each group has a type - to be able to implement a tree of groups.

It will need a flag within the group to denote that it is a top level group, to circumvent the need to determine whether the group is a child of a parent node (for a specific use case - a publisher representing the department that only contains publishers).

Suggest 'is_top_level'

1326896362000000 1326900832000000
#1681 enhancement ross ross closed fixed Core storage document

Document all of the changes and usage for #1574 so that we can mark it 'done done'.

Make sure we document any setup issues/processes and the settings.py setup required for CKAN so that Mark can announce next week and it should be easily deployable/testable.

Internal (perhaps in README somewhere or wiki): how does process work, where does data end up and in what order etc

External: here's how to configure it and here's what you get ... (goes in main sphinx docs)

1326904939000000 1338202930000000
#1685 enhancement ross ross ckan-backlog closed wontfix Move archiver download function to core

Archiver is included in some places just for access to download(), we should move this to lib in core.

1326984170000000 1346670530000000
#1687 enhancement ross ross closed invalid Deploy new storage changes from master to datahub

We need to deploy the new storage changes (and webstore) to thedatahub for further testing

1327056976000000 1329338671000000
#1690 enhancement ross ross closed fixed Rename storage settings with the ckan prefix

Missed the ckan prefix on the storage settings names so this needs to be fixed.

1327064844000000 1327066713000000
#1693 defect ross ross closed fixed register_pluggable_behaviour should not be in routing

We should try and move register_pluggable_behaviour into environment and out of routing as it is causing import issues when using simple search.

These functions already take a map and so they should be fine any time after the call to make_map

1327321474000000 1338206601000000
#1742 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Naming of a new package/group can clash with a route

It is possible to create a group or a dataset called new, or edit, which will then be inaccessible as it is blocked by the existing routes with those names.

We should maintain a list of invalid names in the verification that happens behind the scenes in the UI.

1328116317000000 1331555193000000
#1755 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Alpha paged view of publishers

The publisher index page should show an alpha paged list of publishers.

1328530634000000 1328631064000000
#1756 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Treeview of hierarchical publishers

The publisher index page should show a treeview of the publishers within the system. Probably behind a tab.

1328530689000000 1328709846000000
#1757 enhancement ross ross ckan-sprint-2012-03-05 closed fixed DGU Hide administrators from public view

On publisher view page - hide administrators from public view - only logged in admins of group should see them.

1328530775000000 1338205187000000
#1758 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Check user management auth

Ensure publisher edit page only allows admins to manage users. #1675 is related.

1328530915000000 1328709897000000
#1759 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Create publisher form access

Only sysadmin can access create publisher form.

1328530956000000 1329732744000000
#1760 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Extract current publisher hierarchy

Get access to live Drupal MySQL database to get latest publisher list for migration.

Write a one-off script for import of the data.

1328531001000000 1328709560000000
#1761 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU History visibility

History on user profile should only be visible to a user who is a member of at least one group.

1328532889000000 1328716686000000
#1762 enhancement ross ross ckan-sprint-2012-02-20 closed fixed DGU Join publisher form

A new form for create publisher -> Add publisher access to your account' New wizard Autocomplete publisher name if linked from publisher view. Email address used to publish should be activated/validated on first use. Original email still used to log in.

1328532992000000 1329729305000000
#1793 enhancement ross ross closed duplicate Publisher changes

Changes to publishers for customer

1329214299000000 1329214522000000
#1794 enhancement ross ross closed fixed Publisher changes

Changes to publishers for customer that would otherwise clutter up the master branch.

1329214301000000 1332148972000000
#1795 enhancement ross ross closed wontfix Add approval_status to Package (as for Group)

The Package model should have a approval_status as the Group model does. See migration 049.

1329234060000000 1338206587000000
#1796 enhancement ross dread ckan-future closed fixed Get rid of lxml dependency

lxml is used in CKAN core in two places:

  • ckan/lib/helpers.py:239 Parsing Markdown and ensuring it is valid XML (i.e. tags close nicely so the HTML of the rest of the CKAN page is not disruptable).
  • reading the SOLR config XML to determine what version it is.

Both of these could be done with xml.dom built into Python and would save us a sizeable and slightly problematic dependency.

lxml is needed by these extensions: ckanext-inspire, ckanext-harvest, ckanext-csw & ckanext-wordpresser, so we'd have to add this to their requirements.

1329244440000000 1340635557000000
#2204 enhancement ross rgrp ckan-sprint-2012-04-30 closed fixed [super] Related (Stuff) Extension

This is a reworking of the existing apps extension.

Initial proposal at http://wiki.ckan.org/Proposals#Apps_in_CKAN and http://wiki.ckan.org/Proposals#References.2FLinks_in_CKAN


What do we call this extension?

  • related
    • RP best IMO but perhaps too close to separate relationships concept?
  • relatedstuff
  • links
  • references


"Related Stuff": Apps as in an application (website/service/tool) that uses this dataset (as in Apps and Ideas extension)


New table named Related with following structure

|| id (int) || type || title || description (markdown) || image_url || owner_id || url || created (timestamp)
  • type = Idea | App | API | Visualization | Post | Paper | News Article
    • Suggest we make this a ckan.ini config option (comma separated ...?)
      • Do we want the possibility of different templates for different types of Related objects?
  • image: ?? Depends where we store images. Simplest option would be to change to image_url and leave it to users to have already uploaded an image somewhere. If not we need to support image uploading and storage. See #1692 (add image attribute to datasets and groups) for more discussion, once implemented the URL here can be an internal url.
  • owner_id = user_id or creating user (see authorization below)

Related2Dataset (note that related_id, dataset_id tuple should be unique). This allows for m2m connections. If a given related item is only with one dataset this could be simplified. May contain status so dataset owner can turn this on/off.

|| id || dataset_id || related_id || status

status should be used to allow for a dataset owner (for dataset_id) to de-activate the relationship between the dataset and the related.


  • /dataset/{dataset-name}/related/{related-item-id}/{related-item-title-stringified}
    • If a reference item could exist in its own right (and perhaps refer to multiple datasets then it should get its own url at e.g. /related/{id}
  • /dataset/{dataset-name}/related/add => Modal dialog on related tab so we can use API to create them.

/dataset/{id}/related <- list

  • use image_url for small icon etc, title description (shortened?)
  • Click through to full related item (optional)
  • dataset owner is shown show / hide button ... (or on /dataset/{id}/related/{id} )
  • related owner sees an edit button / icon (pops up modal)

/dataset/{id}/related/{id} (optional)

/dataset/{id} will have a Related tab (with bubble with count).

  • Drop down with Add Related -> Pop-up modal and save via API

(Not used: /related/add with dataset prefilled ... )



Addition of related item be considered orthogonal to datasets (and hence with separate authorization i.e. i can add the info that site X uses dataset Y without needing permission to edit dataset Y).

Thus any logged in user could add a Related item. We set the owner of the related item to creating user and going forward only that user or a sysadmin can update or delete.

NB: we could have a system where datasets owners have to approve related items before they show up next to their dataset. This would add substantial complexity so I propose we leave out of phase 1.

Tasks and estimates {7.5d}

[x] Model + Migration for Related table. {0.75d}

[x] Controller for Related (or relevant sections in Package controller). {0.75d}

[x] Routing setup. {0.25d}

[x] Schema for related. {0.5d}

[x] Logic layer actions. {1.0d}

[x] Auth (default + publisher). {0.5d}

[x] Templates + Dataset changes (new tab etc). {1.0d}

[x] JS Application for adding Related objects in a modal. {1.0d}

[x] Testing. {0.75d}

[x] Dataset owner disabling of Related (via M2M table). {0.5d}

  • Updated to allow author of related to delete as well

[x] Documentation. {0.25d}

1330862992000000 1337594723000000
#2209 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Object->RDF mapping

Super is [1649]

We need to provide customisable mappings between CKAN models and RDF. This needs to be easily configurable for different clients and should take into account objects related to the model being mapped.

Current contenders for impl are:

  1. ordf.org's Object Description Mapper (which uses OWL)
  1. RDFAlchemy, although this looks like it may result in fixed mappings
  1. A custom scheme with a simple json file
  1. A template based approach using Genshi templates?

Approach is to get a mapping complete of the attributes directly in the package class, and then provide mappings for related objects (tags/resources etc).

We'll then need to be able to publish the RDF to a triplestore.

See http://ckan.okfnpad.org/rdfmapping for further analysis.

1331027277000000 1336566051000000
#2210 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Add method to group for adding members

It is necessary to add a helper method to Group for adding members to that group so that

  • nobody needs to work with the Member model
  • It handles existing membership and existing membership with a different capacity
1331028470000000 1331549369000000
#2211 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Logic layer access to publisher functionality

Most of the get_group/add_member style calls should be handled, or at least provided via logic layer.

Don't need to break existing code, but do need to provide access via logic layer

1331028751000000 1331549406000000
#2228 enhancement ross ross ckan-sprint-2012-03-19 closed fixed IGroupForm should allow overriding of non-edit templates

IGroupForm should allow overriding of non-edit templates so that it can specify the index +/or search +/or view.

1331637803000000 1332149030000000
#2239 enhancement ross ross ckan-sprint-2012-04-30 closed fixed EC ODP RDF/N3 templates

Implement the EC/ODP template for RDF to match the requested schemas.

Using the previous work on allowing for templatised RDF representations we should implement the proposed schema for EC/ODP.

We also agreed to allow upload of RDF as an extra (raised to top level in package schema) which will instead be used for the RDF template. On insertion extra CKAN triples will be added... initially this is just owl:sameAs although we're awaiting EC/ODP agreement on which triples we should add to show the CKAN link.

1332169703000000 1335516499000000
#2240 enhancement ross ross ckan-sprint-2012-04-02 closed fixed Document CityData ideas

Document the ideas raised on 14/03 for CityData? and distribute to team for discussion

1332169761000000 1332408769000000
#2255 enhancement ross ross ckan-sprint-2012-04-30 closed fixed Organizations (Privacy + Ownership)

[x] The publisher_form extension and the 'publisher' component of the system will be renamed to organizations to both make its functionality clear, and to differentiate it from the publish auth profile.

Ownership of datasets is important and this ticket will enforce membership for a dataset to a single individual organization. The workflow for adding a dataset to a publisher/organization will stay the same but there will be no mechanism for adding existing datasets to an organization from within the organization forms.

[x] New datasets can be created from the organization form, in which case the organization will be set in the dataset form dropdown (with privacy set to private - see below)

[x] Datasets created from direct access to /dataset/new will have a drop down showing all of the organizations available to the current user.

[x] When editing a dataset the organization can only be changed if the current user is the admin of the current organization, and only then to one of the organizations available to the current user.

Only Organization admin can see the edit Organization form.

[x] Privacy will be enforced by changing the dataset form to have a public/private switch next to the organization drop-down which will be set as the capacity with which the dataset is added to an Organization. Private means visible only within the Organization, and should be the default.

[x] Search will be affected as the public/private status of a dataset will need to be added to the schema

[x] Only public datasets showing in site search, and private showing in organization dataset search (if user is within the organization).

[x] Make sure resources are restricted in visibility if the parent dataset is private

[x] Documentation should be updated to reflect these changes.

[x] Verify, then add, then re-add to documentation.

  • A User joins or creates an Organization
  • If creator of the Organization User becomes administrator of the Organization otherwise they become a Member.
  • Creation of new Organization must be approved by a System Administrator
  • New Members must be approved by the Organization Administrator
1332417218000000 1334663670000000
#2280 enhancement ross johnglover ckan-sprint-2012-04-30 closed fixed Move ecportal to organizations, add history template 1333552029000000 1334834302000000
#2293 defect ross rgrp ckan-sprint-2012-06-25 closed fixed Rename of Group results in disappearance of associated datasets from group page

Rename of group makes all the datasets associated with the group disappear on the group listing page. (But they are still there if you look at edit).

Suspect this is because we are using group name (rather than id!!) in the search index ...

As group rename should be rare I'm marking this as minor though actual effect is major.

Assigning to rossjones for review and cost assessment.

1334432078000000 1340266982000000
#2303 enhancement ross ross ckan-sprint-2012-04-30 closed fixed Move ECPortal rdf dump paster

Move ECPortal rdf dump paster command into core so that it can be run on thedatahub for virtuoso install.

1334582692000000 1334664098000000
Note: See TracReports for help on using and creating reports.