{22} Trac tickets (2647 matches)

Results (901 - 1000 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1195 defect rgrp dread ckan-sprint-2011-10-28 closed fixed Storage extension to handle incorrect routes better

We've seen exceptions caused by people getting the file storage link wrong, adding a slash to the end.

e.g. http://ckan.net/storage/f/file/e696d857-e997-41c8-be08-408697168ca8/

We should aim to handle all requests better than causing exception. 404 (or even 302 in this case) is better than 500.

1308576769000000 1311964374000000
#1208 enhancement rgrp rgrp ckan-sprint-2011-10-28 closed fixed Webstore: a datastore with web API
  • SQLite based
  • Use Cyclone for async http server on top with auth
  • Designed to not be specific to CKAN or anything else

Current code is here: https://github.com/okfn/webstore.

API Spec

Read

Two basic ways to query::

    GET: /{owner}/{db-name}/?sql=...
    GET: /{owner}/{db-name}/?table=...&attr=value&attr=value&limit=...

Returns::

  {
      u'keys': [u'id', u'name'],
      u'data': [
          [1, u'jones'],
          [u'aaa', u'jones']
          ]
  }

Write

POST to::

/{owner/{database}/{table}

Payload is json data structured as follows::

{

unique_keys: [list of key attributes] data: {dict of values}

}

Authentication and Authorization

Authentication: use basic auth header.

Authorization:

  • Default: all read, owner can write
  • Restricted: owner can read and write, everyone can do nothing

Possible future: config file can specify a python method (TODO: method signature)

Integration with Other Systems

TODO: Specify how to delegate authenatication to user database in some other system.

1309804460000000 1312191646000000
#1218 enhancement dread minspamboks@… ckan-sprint-2011-10-28 closed fixed Colour the History tab icon

Change the color of the "History" tab icon to yellowish, like the rest of the icons in the other tabs ("View" and "Edit", "Authorization").

Reasoning

When you view a data package, for instance http://ckan.net/package/thesaurus-w, you will see "View", "Edit", "History" tabs on the top. "History" tab has a black-and-white icon which makes it look like an inactive/disabled tab (since the text is also grayed out when the tab is not selected). This is not a major issue, but it is a little bit confusing for the users. This icon exists in v1.3.2 and also in v1.4.1a (that runs on ckan.net).

The simple solution would be to change the color of the "History" tab icon and give it the same yellowish color like the rest of the icons in the other tabs (e.g. "View" and "Edit").

1310375768000000 1310389390000000
#1221 defect dread dread ckan-sprint-2011-10-28 closed fixed Password reset tidy up
  • No tests.
  • 500 error given when user is not found.
1310475661000000 1310556607000000
#1222 defect dread dread ckan-sprint-2011-10-28 closed fixed Mailer has no tests 1310479136000000 1310556618000000
#1225 enhancement dread dread ckan-sprint-2011-10-28 closed fixed Tool for mass tag changes

Using ckanapi

1310642161000000 1310642407000000
#1228 enhancement dread dread ckan-sprint-2011-10-28 closed fixed Display OpenID login errors

When there are errors when logging into OpenID, these don't get displayed. You just get the login screen again.

1311086562000000 1311325161000000
#1230 enhancement kindly ckan-sprint-2011-10-28 closed fixed Make sure plugins tables work in tests.

Dgu tests failed as harvesting table are not created at the correct point.

1311100811000000 1311154142000000
#1234 enhancement dread dread ckan-sprint-2011-10-28 closed fixed Setting locale error conditions handled badly

Needs tests for error conditions for bad locale and redirect parameters.

1311179871000000 1311183328000000
#1238 defect kindly dread ckan-sprint-2011-10-28 closed invalid Viewing a package @date misses non-PackageRevisions

There's a problem with getting old versions of packages using @date. The query looks for the PackageRevision? related to the date, but ignores TagRevisions? etc.

So for example if you compare this table: http://ckan.net/package/history/osm with this view: http://ckan.net/package/osm%402010-11-30%2000%3A21%3A49.627830 you actually see the 2010-01-13 11:13 revision - the wrong revision.

1311351303000000 1311759784000000
#1239 defect rgrp rgrp ckan-sprint-2011-10-28 closed fixed Remove and re-add tag on package not working

See reports on list at:

Tracked this to issue in dictization/model_save whereby was ignoring case where Package Tag already there but in deleted state. Rather than describe at length see fix.

1311441020000000 1312453156000000
#1246 defect pudo adrian.pohl@… ckan-sprint-2011-10-28 closed fixed Search results on ckan.net are mistakenly all 'open'

All package search results on ckan.net are labelled as 'open' even when their license is closed or unknown: http://ckan.net/package

1311863353000000 1311892816000000
#1247 task rgrp dread ckan-sprint-2011-10-28 closed fixed Put docs on ReadTheDocs

Put docs on ReadTheDocs? instead of eu13.

1311941948000000 1311955185000000
#1253 enhancement amercader amercader ckan-sprint-2011-10-28 closed fixed Authz refactor

We agreed on a system that allows overriding the authz checks at a logic function level (e.g. package_create, user_show) and supports the old (current) authz system.

1312199491000000 1313401367000000
#1254 task dread dread ckan-sprint-2011-10-28 closed fixed Feature sheet

Compose a feature list for CKAN. Use the existing Feature Matrix. Basic explanation of terms. For use by customers.

1312211925000000 1313401629000000
#1256 defect kindly kindly ckan-sprint-2011-10-28 closed fixed fix modification extension for moderation

Solr is not working for moderation, pending changes are not being indexed.

1312280026000000 1319812556000000
#1258 enhancement kindly kindly ckan-sprint-2011-10-28 closed fixed Bring purge revision into ckan repo from vdm

In order to make purge revision work correctly with the moderated edits we need to modify purge revision in vdm. This is best modified in ckan so we will override the vdm one in the reposotory.

1312289539000000 1319812452000000
#1265 defect dread dread ckan-sprint-2011-10-28 closed fixed user/edit error conditions cause exceptions

We get exceptions on these occasions:

  • user/edit if not logged in
  • user/edit/bob if bob is not a known user

These are a problem only on 1.4.3b.

Introduced in #1229 - merge of branch feature-1229-db-out-of-controllers

1312565788000000 1312566052000000
#1266 defect dread dread ckan-sprint-2011-10-28 closed fixed Timestamps without microseconds causing exception

I'm not sure why some timestamps are missing microseconds in the ckan.net data, but plenty seem to. This causes this exception when viewing the package rss feed:

URL: http://ckan.net/package/history/sfk_publishing?format=atom&days=7
In Module ckan.lib.helpers:227 in date_str_to_datetime
WebApp Error: <type 'exceptions.ValueError'>: time data '2008-04-13T20:40:20' does not match format '%Y-%m-%dT%H:%M:%S.%f'

Only seen with ckan 1.4.3b

1312794831000000 1312797477000000
#1270 defect dread dread ckan-sprint-2011-10-28 closed fixed Default site www.ckan.net is a redirect and loses POSTs

The default site was www.ckan.net, which had an Apache 301 redirect to ckan.net, but the problem was urllib was not rePOSTing stuff. This affected search parameters:

In [1]: import ckanclient

In [2]: c = ckanclient.CkanClient()

In [3]: c2 = ckanclient.CkanClient('http://ckan.net/api')

In [4]: c.package_search('', search_options={'groups':'openspending'})
Out[4]: 
{u'count': 2102,
 u'results': <generator object _result_generator at 0x10168b1e0>}

In [5]: c2.package_search('', search_options={'groups':'openspending'})
Out[5]: {u'count': 29, u'results': <generator object _result_generator at 0x10168b410>}

Thanks to borior for finding this and raising it.

1312911847000000 1312912188000000
#1271 enhancement rgrp rgrp ckan-sprint-2011-10-28 closed fixed CORS support

CORS - http://www.w3.org/TR/cors/ - support.

This is what you do in Apache. Should do this in lib/base.py or similar.

    Header always set Access-Control-Allow-Origin "*"
    Header always set Access-Control-Allow-Methods "POST, PUT, GET, OPTIONS"
    Header always set Access-Control-Allow-Headers "X-CKAN-API-KEY, Content-Type"

    # Respond to all OPTIONS requests with 200 OK
    # This could be done in the webapp
    # This is need for pre-flighted requests (POSTs/PUTs)
    RewriteEngine On
    RewriteCond %{REQUEST_METHOD} OPTIONS
    RewriteRule ^(.*)$ $1 [R=200,L]
1313241839000000 1313753663000000
#1274 enhancement johnglover johnglover ckan-sprint-2011-10-28 closed fixed Testing solr search

Copy ckan core postgres package search tests to the ckanext-solr extension and update them so that they use the solr search backend.

1313413202000000 1313429049000000
#1281 defect pudo pudo ckan-sprint-2011-10-28 closed fixed Locale change flash message in wrong language

When you change the locale, it displays a flash message "Language has been set to: English" in the original language, not the one we have just changed to.

1314021656000000 1314021919000000
#1312 enhancement dread florian.marienfeld@… ckan-sprint-2011-10-28 closed invalid Particular characters in JSON cause exception creating package on API

From Florian:

I am having trouble with the characters "=&;".
if they occur e.g. in the title of a package, I get a 400-Bad-Request
when I try to register a package by POSTing to ckan/rest/api/package

However, when I PUT the same package as an edit to
ckan/rest/api/package/existing_package, it works and the title shows
correctly on the front end and API.

Reproduced

(pyenv-ckan)dread@dread-laptop:~/hgroot/ckan$ curl -i http://localhost:5000/api/rest/package -d '{"name": "test3", "title": "Test &"}' -H "Authorization:tester" 
HTTP/1.0 400 Bad Request
Server: PasteWSGIServer/0.5 Python/2.6.5
Date: Tue, 06 Sep 2011 14:09:31 GMT
Pragma: no-cache
Cache-Control: no-cache
Content-Type: application/json;charset=utf-8
Content-Length: 87

"Bad request - JSON Error: Unterminated string starting at: line 1 column 27 (char 27)"
1315318863000000 1315327743000000
#1318 defect dread dread ckan-sprint-2011-10-28 closed fixed 'Register user' doesn't log you in

When you register a user, it doesn't seem to be logged in properly straight away. Top-right it invites you to "log in" and "register". Also the "create package" menu option doesn't appear. Going to /user/me sends you to login page. If you then login, then all these things are corrected.

1315415783000000 1315569362000000
#1323 enhancement dread dread ckan-sprint-2011-10-28 closed fixed GET /authorizationgroup causes Exception when logged in

on thedatahub.org

1315475685000000 1315594333000000
#1329 defect dread amercader ckan-sprint-2011-10-28 closed fixed i18n poor in some templates - missing i18n tags

Just a tip for developers regarding translations:

  • When writing templates, mark <p>, <div>, etc elements with the

i18:msg attribute, otherwise the messages don't get extracted properly.

Good: https://bitbucket.org/okfn/ckan/src/1996b0439e81/ckan/templates/home/about.html#cl-40

<p i18n:msg="">More about CKAN, and the background to its creation can
   be found in <a
     href="http://m.okfn.org/files/talks/opentech_20080705/">these
     slides</a>.</p>

gives this: https://bitbucket.org/okfn/ckan/src/1996b0439e81/ckan/i18n/ca/LC_MESSAGES/ckan.po#cl-1238

Bad: https://bitbucket.org/okfn/ckan/src/1996b0439e81/ckan/templates/package/new_package_form.html#cl-211

<p class="hints">
<strong>Important:</strong> By submitting content, you agree to
release your contributions
 under the <a href="http://opendatacommons.org/licenses/odbl/1.0/">Open
Database License</a>. Please <strong>refrain</strong> from editing
this page if you are <strong>not</strong> happy to do this.
</p>

gives this: https://bitbucket.org/okfn/ckan/src/1996b0439e81/ckan/i18n/ca/LC_MESSAGES/ckan.po#cl-1384

The message is split in different parts that not necessarily match the grammar or syntax of the translated language.

1315824288000000 1317404566000000
#1333 enhancement amercader amercader ckan-sprint-2011-10-28 closed fixed Move form converters to ckan core

Functions like convert_to_extras and convert_from_extras will be widely used from extensions and should be on ckan core.

1315909268000000 1316951285000000
#1334 defect amercader amercader ckan-sprint-2011-10-28 closed fixed "AttributeError: 'NoneType' object has no attribute 'name'" exception when logging in

On same cases this exception is thrown when logging in:

File '/home/adria/dev/envs/iati/src/ckan/ckan/controllers/user.py', line 252 in logged_in
  user_dict = get_action('user_show')(context,data_dict)
File '/home/adria/dev/envs/iati/src/ckan/ckan/logic/action/get.py', line 414 in user_show
  revision_dict = revision_show(context,{'id':revision.id})
File '/home/adria/dev/envs/iati/src/ckan/ckan/logic/action/get.py', line 332 in revision_show
  ref_package_by=ref_package_by)
File '/home/adria/dev/envs/iati/src/ckan/ckan/model/__init__.py', line 317 in revision_as_dict
  for pkg in revision.packages]
AttributeError: 'NoneType' object has no attribute 'name'
1315909508000000 1319798961000000
#1337 defect amercader amercader ckan-sprint-2011-10-28 closed fixed Group extras are JSON endoded in the REST API responses

Instead of this:

extras: {
type: ""Primary source""
license_id: ""localauth-withrights""
publisher_frequency: ""2Pre\u00f1o \u20ac $ \u00e0dri\u00e0""
}

we should return this:

extras: {
type: "Primary source",
license_id: "localauth-withrights",
publisher_frequency: "2Preño € $ àdrià",
}
1315924727000000 1316951259000000
#1338 defect amercader amercader ckan-sprint-2011-10-28 closed fixed Exception when trying to use a custom form schema from an extension

Both with the package and group controllers, when overriding the default _db_to_form_schema method (which does nothing) from an extension, you get an exception because the context is not passed to the validators.

File '/home/adria/dev/envs/iati/src/ckan/ckan/controllers/group.py', line 140 in edit
  old_data, errors = validate(old_data, schema)
File '/home/adria/dev/envs/iati/src/ckan/ckan/lib/navl/dictization_functions.py', line 215 in validate
  converted_data, errors = _validate(flattened, schema, context)
File '/home/adria/dev/envs/iati/src/ckan/ckan/lib/navl/dictization_functions.py', line 270 in _validate
  convert(converter, key, converted_data, errors, context)
File '/home/adria/dev/envs/iati/src/ckan/ckan/lib/navl/dictization_functions.py', line 178 in convert
  converter(key, converted_data, errors, context)
File '/home/adria/dev/envs/iati/src/ckan/ckan/logic/validators.py', line 108 in group_name_validator
  model = context['model']
KeyError: 'model'
1315925217000000 1317117738000000
#1340 defect dread dread ckan-sprint-2011-10-28 closed fixed Editing non-existent group causes Exception

e.g. http://ckan.net/group/edit/history

Module ckan.controllers.group:135 in edit
<<          try:
                   old_data = get_action('group_show')(context, data_dict)
                   c.grouptitle = old_data.get('title')
                   c.groupname = old_data.get('name')
>>  old_data = get_action('group_show')(context, data_dict)
Module ckan.logic.action.get:347 in group_show
<<      if group is None:
               raise NotFound
       
           check_access('group_show',context, data_dict)
>>  raise NotFound
1315991374000000 1315991534000000
#1346 defect dread dread ckan-sprint-2011-10-28 closed fixed 'get_authorization_group_dict not found' exception

This stems from a feature to allow you to prefill fields in the authorizationgroup creation form.

e.g. http://ckan.net/authorizationgroup/new?name=science

It's not tested, not supported in the new logic layer and therefore broken. I don't think anyone uses it - can remove it.

1316081974000000 1316094632000000
#1347 defect dread dread ckan-sprint-2011-10-28 closed fixed Exception caused by New Group form POST with bad formed parameter

Exception caused by bots POSTing 'new group' form, but with bad (old-style) parameters.

<type 'exceptions.ValueError'>: invalid literal for int() with base 10: 'cache'
...
Module ckan.controllers.group:102 in new
<<          if context['save'] and not data:
                   return self._save_new(context)
               
               data = data or {}
>>  return self._save_new(context)
Module ckan.controllers.group:152 in _save_new
<<          try:
                   data_dict = clean_dict(unflatten(
                       tuplize_dict(parse_params(request.params))))
                   context['message'] = data_dict.get('log_message', '')
                   group = create.group_create(data_dict, context)
>>  tuplize_dict(parse_params(request.params))))
Module ckan.logic:57 in tuplize_dict
<<          for num, key in enumerate(key_list):
                   if num % 2 == 1:
                       key_list[num] = int(key)
               tuplized_dict[tuple(key_list)] = value
           return tuplized_dict
>>  key_list[num] = int(key)
ValueError: invalid literal for int() with base 10: 'cache'
1316094466000000 1316094608000000
#1418 defect dread dread ckan-sprint-2011-10-28 closed duplicate If default language is not 'en' then you can't switch language to 'en'

e.g. cz.ckan.net has lang=cz but when you click on Home page link "English" it flashes message "Language has been set to: English" but the rest of the page is still Czech.

1319539146000000 1319539691000000
#1421 defect dread dread ckan-sprint-2011-10-28 closed fixed Tag cloud recipe broken

home page doesn't have c.facets setup.

Only affects 1.5 branch - ok 1.4.3.1. Was broken in cset:05f60f2d26e8.

1319638575000000 1319639311000000
#1426 defect dread dread ckan-sprint-2011-10-28 closed invalid Validation errors cause 409 not 400 response

Validation error is incorrect arguments, so should be a "400 Bad Request" response. "409 Conflict" is for when there is a conflict creating two resources with the same identifier. Yet we get 409 for all validation errors.

1319708619000000 1319709236000000
#1427 enhancement dread dread ckan-sprint-2011-10-28 closed fixed Tags validation needs to limit length to 100 characters

The tag name is limited in length in the db table to 100 characters. If you try and create a longer one you get an exception. This should cause a ValidationError? and tell the user this sensibly.

1319708698000000 1319709925000000
#891 task johnglover pudo ckan-sprint-2011-11-07 closed fixed Resource download worker daemon

Superticket: #1397

Write a worker daemon to download all resources from a CKAN instance to a local repository.

Questions

  • Do we only want to download openly licensed information? ANS: no, we do everything (though do need to think about this re. IP issues)
  • Should we have clever ways to dump APIs? ANS: no.
  • Do we respect robots.txt even for openly licensed information? ANS: No (we're not crawling we're archiving)
  • Use HTTP/1.1 Caching headers? ANS: if not changed since we last updated don't bother to recache.
    • Complete support for ETags
    • Expires, Max-Age etc.
  • Check

Functionality

  • Download files via HTTP, HTTPS (will not do FTP)

Process:

  1. [Archiver.Update checks queue (automated as part of celery)]
  2. Open url and get any info from resource on cache / content-length etc
    1. If FAILURE status: update task_status table (could retry if not more than 3 failures so far). Report task failure in celery
    2. Check headers for content-length and content-type ...
      • IF: content-length > max_content_length: EXIT (store outcomes on task_status, and update resource with size and content-type and any other info we get?)
      • ELSE: check content-type.
        • IF: NOT data stuff (e.g. text/html) then EXIT. (store outcomes and info on resource)
        • ELSE: archive it (compute md5 hash etc)
      • IF: get content-length and content-length unchanged GOTO step 4
  3. Archive it: connect to storage system and store it. Bucket: from config, Key: /archive/{timestamp}/{resourceid}/filename.ext
    • Add cache url to resource and updated date
    • Add other relevant info to resource such as md5, content-type etc
  4. Update task_status

Optional functionality

  • If result object is HTML, search for references to "proper data" (CSV download pages etc.)
  • Download from POST forms (accepting licenses or weird proprietary systems)
  • Support running on Google Apps Engine to save traffic costs.

Existing work

1294052979000000 1320149841000000
#1217 defect dread dread ckan-sprint-2011-11-07 closed invalid Diff exception

Exception raised doing this particular diff.

http://ckan.net/revision/diff/sudocfr?diff_entity=package&oldid=a18b2520-09e0-44be-80dd-4fea94ffe7d1&diff=3934ff16-a3a7-4481-b390-72944903b945

...
Module ckan.controllers.revision:142 in diff
<<          if c.diff_entity == 'package':
                   c.pkg = model.Package.by_name(id)
                   diff = c.pkg.diff(c.revision_to, c.revision_from)
               elif c.diff_entity == 'group':
                   c.group = model.Group.by_name(id)
>>  diff = c.pkg.diff(c.revision_to, c.revision_from)
Module ckan.model.package:449 in diff
<<                              display_id = to_obj_rev.tag.name
                               elif obj_class.__name__ == 'PackageExtra':
                                   display_id = to_obj_rev.key
                               else:
                                   display_id = related_obj_id[:4]
>>  display_id = to_obj_rev.key
AttributeError: 'NoneType' object has no attribute 'key'

Seems to be to do with this particular extra key:

[<PackageExtraRevision id=ba1f3b3e-f61f-4924-bae0-ec3dd2163ae4 package_id=febe4da9-5874-4411-a7da-b497fdebf739 key=namespace value=http://www.sudoc.fr/ state=active revision_id=5ec8e730-c73d-4f1c-989b-2e015eb3d315 continuity_id=ba1f3b3e-f61f-4924-bae0-ec3dd2163ae4 expired_id=None revision_timestamp=2011-07-08 13:48:11.027260 expired_timestamp=9999-12-31 00:00:00 current=True>]

Is it because it was created between the from_revision and to_revision?

1310375498000000 1320663277000000
#1236 enhancement dread dread ckan-sprint-2011-11-07 closed fixed View a package at a given revision

As a user I want to view a package at a given revision:

  • When I visit /package/osm@yyyy-mm-dd or /package/osm@rrrrrrr I should be shown package at date yyyy-mm-dd or revision rrrrrrr.
  • viewing this page, it should say it is the old revision, just like wikipedia
  • package history page should provide links to these pages
1311269333000000 1330019788000000
#1248 defect dread dread ckan-sprint-2011-11-07 closed fixed License exception

Exception received on test.ckan.net at 10.12 on 29/7/11

http://test.ckan.net/package?__cache=2759373&page=4
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
           
           def _inspect_call(self, func):
>>  return func(**args)
Module ckan.controllers.package:153 in search
<<                          }
       
                   query = get.package_search(context,data_dict)
       
                   c.page = h.Page(
>>  query = get.package_search(context,data_dict)
Module ckan.logic.action.get:514 in package_search
<<      for package in query.results:
               result_dict = table_dictize(package, context)
               result_dict = _extend_package_dict(result_dict,context)
       
               results.append(result_dict)
>>  result_dict = _extend_package_dict(result_dict,context)
Module ckan.logic.action.get:537 in _extend_package_dict
<<      license_id = package_dict['license_id']
           if license_id:
               isopen = model.Package.get_license_register()[license_id].isopen()
               package_dict['isopen'] = isopen
           else:
>>  isopen = model.Package.get_license_register()[license_id].isopen()
Module ckan.model.license:59 in __getitem__
<<              return default
               else:
                   raise KeyError, "License not found: %s" % key
       
           def get(self, key, default=None):
>>  raise KeyError, "License not found: %s" % key
KeyError: u'License not found: None'

and a similar one:

 http://test.ckan.net/tag/barbecue
Module pylons.wsgiapp:324 in dispatch
<<          if log_debug:
                   log.debug("Calling controller class with WSGI interface")
               return controller(environ, start_response)
           
           def load_test_env(self, environ):
>>  return controller(environ, start_response)
Module ckan.lib.base:118 in __call__
<<          # available in environ['pylons.routes_dict']    
               try:
                   return WSGIController.__call__(self, environ, start_response)
               finally:
                   model.Session.remove()
>>  return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<<                  return response(environ, self.start_response)
               
               response = self._dispatch_call()
               if not start_response_called:
                   self.start_response = start_response
>>  response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<<              req.environ['pylons.action_method'] = func
                   
                   response = self._inspect_call(func)
               else:
                   if log_debug:
>>  response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<<                        func.__name__, args)
               try:
                   result = self._perform_call(func, args)
               except HTTPException, httpe:
                   if log_debug:
>>  result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
           
           def _inspect_call(self, func):
>>  return func(**args)
Module ckan.controllers.tag:2 in read
Module ckan.lib.cache:167 in wrapper
<<      log = __import__("logging").getLogger("proxy_cache")
           def wrapper(func, *args, **kwargs):
               result = func(*args, **kwargs)
       
               pylons = get_pylons(args)
>>  result = func(*args, **kwargs)
Module ckan.controllers.tag:64 in read
<<          data_dict = {'id':id}
               try:
                   c.tag = get.tag_show(context,data_dict)
               except NotFound:
                   abort(404, _('Tag not found'))
>>  c.tag = get.tag_show(context,data_dict)
Module ckan.logic.action.get:341 in tag_show
<<      extended_packages = []
           for package in tag_dict['packages']:
               extended_packages.append(_extend_package_dict(package,context))
       
           tag_dict['packages'] = extended_packages
>>  extended_packages.append(_extend_package_dict(package,context))
Module ckan.logic.action.get:537 in _extend_package_dict
<<      license_id = package_dict['license_id']
           if license_id:
               isopen = model.Package.get_license_register()[license_id].isopen()
               package_dict['isopen'] = isopen
           else:
>>  isopen = model.Package.get_license_register()[license_id].isopen()
Module ckan.model.license:59 in __getitem__
<<              return default
               else:
                   raise KeyError, "License not found: %s" % key
       
           def get(self, key, default=None):
>>  raise KeyError, "License not found: %s" % key
KeyError: u'License not found: CreativeCommonsAttributionCCBY25'
1311954136000000 1319797763000000
#1292 enhancement dread dread ckan-sprint-2011-11-07 closed fixed Internationalization improvements
  • Review process of releases and updating strings (pot) and translations (po) files
  • Get language changing thing to get home page to mostly work - push to key strings translated (or don't advertise the translation?)
  • Review long list of strings for improvement
    • Consolidate any?
    • Get rid of i18n for strings in api
    • Try and remove javascript
    • Add in extensions
  • #1374 Fix switching to en from other default language
  • #1417 Fix browser language detection
1314303314000000 1320145732000000
#1300 enhancement rgrp rgrp ckan-sprint-2011-11-07 closed fixed Core changes to base theme

Core changes to theme in order to make it easier to re-theme:

  • page_heading
  • side bar menu must be switchable (left to right) (through config or css)
  • Add optional_footer to complement optional_head
    • And adopt rule for template writers that all extra js must go in optional footer
    • That way we could move optional js to the bottom of the page to improve page load times.
  • Re-add support for body-class ... (seems to be removed by #1108)
  • more divs.
  • better labelling for css (and javascript)
1314862223000000 1319794520000000
#1315 enhancement dread dread ckan-sprint-2011-11-07 closed fixed Document /api/action

We need some documentation for the /api/action. Even though it's beta and subject to change it would be good to have a guide.

1315396722000000 1320173768000000
#1365 defect dread dread ckan-sprint-2011-11-07 closed fixed Support old search formats

Lots of things rely on the old (i.e. pre-SOLR) search param formats, including ckanclient / DGU. Need a translation layer to convert these params (in api v1&2) to SOLR syntax.

This functionality was broken by the changes in #1277 which went into 1.5a

1317233338000000 1319815052000000
#1373 defect shevski ckan-sprint-2011-11-07 closed fixed home page view does not react to logging in / out

Either: thedatahub.org does not display 'add a dataset' top navigation link or the 'create dataset' main central link to logged in user - instead showing 'sign up' link in the central box. It does however display the 'my account' and 'logout' link in the top right correctly.

Navigating to another part of the site (e.g. search or about or my account) does bring the 'add a dataset' navigation link back and the functionality works.

Works fine on test.ckan.net so only on thedatahub.

Not obviously a caching problem, since I did try clearing cache.

OR: it displays as if you're logged in even when you log out. The links doesn't work and prompt you to log in though.

1317829479000000 1320174240000000
#1379 defect zephod zephod ckan-sprint-2011-11-07 closed fixed Markdown conversion of linked references broken

eg. http://thedatahub.org/dataset/us-gov-tarp

[This Paper][my-paper] is very interesting.

[my-paper]: http://somelink.com/paper

Write a test to check this markdown feature works - if not, use external markdown lib rather than webhelpers.markdown. Probably need to add markdown lib to list of requirements.

1318247719000000 1320161439000000
#1380 enhancement zephod zephod ckan-sprint-2011-11-07 closed fixed Making customizing CSS easier

While integrating the new theme and making a large number of UX tweaks, I've allowed the CSS file to bloat out of control with some very specific rules and subtle hacks, as well as a couple of clearfix <div> objects in the markup which isn't really good practice.

In general we want a logical CSS structure which is as easy to modify as possible. You should not find yourself being overwritten by highly specific rules all the time!

Actions

  1. Clean up css
  2. Add a config option ckan.template_head_end - arbitrary string inserted at end of <head> tag so you can add custom css etc (may also want to update theming docs to reflect possibility of using this)
  3. (While we're at it): Add ckan.template_footer_end config option to deployment.ini_template. (This already exists, we just haven't told anybody)
  4. The /users index page isn't linked anywhere. Clean it up and link it in the footer next to Groups etc.
  5. package/new, package/edit, group/new and group/edit all include their forms differently. Use h.literal in all cases.
  6. Fix the padding on minornavigation.

Background

David Read sent me the following as background:

  1. basic theming is described here: http://docs.ckan.org/en/latest/theming.html

Deep customisation of the templates is done by creating an extension, with this as a template: https://bitbucket.org/okfn/ckanext-exampletheme/overview

That's what Augusto and the Brazilians have done http://alpha.dados.gov.br/dados/ and their code is here: hg clone http://dev.dados.gov.br/codigo/dev/tema-ckan <snip> Another thing of interest was us becoming compatible with Wordpress themes last December. Richard Pope worked with Rufus on this. I believe the base theme is 'twentyten' and I'm guessing that one would swap that css file with another to change theme, but I don't know. I don't know if this feature has been dropped since then. http://lists.okfn.org/pipermail/ckan-dev/2010-December/000073.html

Dave

1318263956000000 1324048324000000
#1392 enhancement johnglover johnglover ckan-sprint-2011-11-07 closed fixed Move datacatalogs.org to an OKFN server 1318497251000000 1320143150000000
#1405 defect dread jonathan.gray@… ckan-sprint-2011-11-07 closed fixed You can add a package twice to a group

To reproduce (from Jonathan Gray):

  • I go to add an item to the group
  • As I type, I get autocomplete options (cool!)
  • When I click on one, it adds this, and copies the same autocomplete option in the box
  • When I click submit, it seems to add the item twice
  • On the main group page I think it just shows up once, but when I click edit it shows two identical boxes, both ticked - and the count includes two copies of the same item

Basically the problem is allowing the same package associated multiple times with a group.

This is an issue on CKAN 1.5b and probably earlier.

1318856354000000 1320152291000000
#1409 enhancement zephod zephod ckan-sprint-2011-11-07 closed fixed Add Gravatars to user profiles

Since we now require email addresses, it's sensible to request users' gravatars to add a little flavour to their user profile (and, potentially, other places eg. comments threads?)

[Reopened: Additional] Use gravatars in the HTML generated by helpers.py to create lists of users. Eg. on user/list and dataset/history.

1319200773000000 1324047722000000
#1416 defect dread dread ckan-sprint-2011-11-07 closed fixed Registering with blank email can cause exception

This might well be a bot, but not POSTing the email field should not cause an exception - it should be 400 and then the sysadmin doesn't get these nags.

http://www.publicdata.eu/user/register
...
Module ckan.controllers.user:79 in register
<<              c.login = request.params.getone('login')
                   c.fullname = request.params.getone('fullname')
                   c.email = request.params.getone('email')
                   if not model.User.check_name_available(c.login):
                       h.flash_error(_("That username is not available."))
>>  c.email = request.params.getone('email')
Module webob.multidict:335 in getone
<<          values were found.
               """
               return self._decode_value(self.multi.getone(self._encode_key(key)))
       
           def mixed(self):
>>  return self._decode_value(self.multi.getone(self._encode_key(key)))
Module webob.multidict:99 in getone
<<          v = self.getall(key)
               if not v:
                   raise KeyError('Key not found: %r' % key)
               if len(v) > 1:
                   raise KeyError('Multiple values match %r: %r' % (key, v))
>>  raise KeyError('Key not found: %r' % key)
KeyError: "Key not found: 'email'"
HTTP_USER_AGENT	'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MRA 4.3 (build 01218); .NET CLR 1.1.4322)'
webob._parsed_post_vars	(MultiDict([('login', 'ahusernames9'), ('fullname', 'ahusernames9'), ('password1', '******'), ('password2', '******'), ('s', 'Sign up')]), <FakeCGIBody at 0xb9d7c5ec viewing MultiDict([('lo...p')])>)
1319533989000000 1320153507000000
#1419 enhancement dread ckan-sprint-2011-11-07 closed invalid Can't log in via OpenID

I couldn't log into theDataHub with OpenID today. I tried both Google ID and MyOpenID. Both times the login on the remote auth server went fine, but when it returns you to theDataHub you get error "Login failed. Bad username or password."

1319543013000000 1319796164000000
#1428 defect dread dread ckan-sprint-2011-11-07 closed fixed config options not used or documented

Need to ensure config options in the deployment.ini_tmpl match up with doc/configuration.rst. Remove any unused ones too.

1319803594000000 1319804773000000
#1430 defect amercader ckan-sprint-2011-11-07 closed fixed Documents get mixed between SOLR cores

On some occasions (apparently random), the documents indexed in a specific SOLR core get mixed with different site_ids.

E.g: We look for all documents in the testing.iatiregistry.org core, faceted by site_id. We would expect all documents to have site_id = iati_testing, but some of them have site_id = iatiregistry.org

http://okfn-solr.fry-it.com:8080/solr/testing.iatiregistry.org/select?indent=on&version=2.2&q=*:*&fq=+state:active&facet=true&facet.field=site_id

<lst name="facet_fields">
<lst name="site_id">
<int name="iati_testing">265</int>
<int name="iatiregistry.org">255</int>
</lst>
</lst>

If we compare one of the records which disappeared from the "iati_testing" site_id in both the production and testing SOLR cores of the server, the records are exactly the same, including the indexed_ts property:

http://okfn-solr.fry-it.com:8080/solr/iatiregistry.org/select?indent=on&version=2.2&q=id:97d1ab0d-b203-4757-8f4e-a0c84d2f759f&facet=true&facet.field=site_id

http://okfn-solr.fry-it.com:8080/solr/testing.iatiregistry.org/select?indent=on&version=2.2&q=id:97d1ab0d-b203-4757-8f4e-a0c84d2f759f&facet=true&facet.field=site_id

Note that the response from the URLs shown may vary, as the testing site could have been reindexed.

1320068076000000 1324033923000000
#1436 defect johnglover johnglover ckan-sprint-2011-11-07 closed fixed Packages with a 'pending' state not listed on groups page 1320155227000000 1320243278000000
#1437 defect dread dread ckan-sprint-2011-11-07 closed fixed JSONP parameter in Action API

Action API needs JSONP support - be able to return responses encapsulated in a function of a supplied name. This is important for remote sites running javascript to interact with a CKAN site.

Specifying the callback parameter is the way we've achieved JSONP with the RESTful and Search APIs. It should work like this:

curl http://test.ckan.net/api/action/package_show?callback=jsoncallback -d '{"id": "fd788e57-dce4-481c-832d-497235bf9f78"}'

Or maybe the callback should be specified in the payload in the context or data_dict?

(My understanding is that CORS is similar - when more browsers support it, can we drop JSONP?)

1320161088000000 1320173795000000
#1440 defect dread dread ckan-sprint-2011-11-07 closed fixed 'paster db load' doesn't update search index

paster db load ckan.pgdump will load in the database but not update the search index, no we're using SOLR.

implementation

We can't tack on 'rebuild' to this command, because it needs a 'db upgrade' first. I'm tempted to make 'db load' do the upgrade and rebuild all in one command.

1320232415000000 1320235084000000
#1442 defect dread dread ckan-sprint-2011-11-07 closed fixed You can't GET then POST a dataset as new

If you try to GET a dataset, purge it then POST it again, it fails checking authz for the group:

Module ckan.controllers.api:278 in create
<<                  (register, subregister))
               try:
                   response_data = action(context, data_dict)
                   location = None
                   if "id" in data_dict:
>>  response_data = action(context, data_dict)
Module ckan.logic.action.create:240 in package_create_rest
<<      api = context.get('api_version') or '1'
       
           check_access('package_create_rest', context, data_dict)
       
           dictized_package = package_api_to_dict(data_dict, context)
>>  check_access('package_create_rest', context, data_dict)
Module ckan.logic:129 in check_access
<<          #    log.debug('Valid API key needed to make changes')
               #    raise NotAuthorized
               logic_authorization = new_authz.is_authorized(action, context, data_dict)
               if not logic_authorization['success']:
                   msg = logic_authorization.get('msg','')
>>  logic_authorization = new_authz.is_authorized(action, context, data_dict)
Module ckan.new_authz:16 in is_authorized
<<      auth_function = _get_auth_function(action)
           if auth_function:
               return auth_function(context, data_dict)
           else:
               raise ValueError(_('Authorization function not found: %s' % action))
>>  return auth_function(context, data_dict)
Module ckan.logic.auth.create:115 in package_create_rest
<<          return {'success': False, 'msg': _('Valid API key needed to create a package')}
       
           return package_create(context, data_dict)
       
       def group_create_rest(context, data_dict):
>>  return package_create(context, data_dict)
Module ckan.logic.auth.create:15 in package_create
<<      else:
               
               check2 = check_group_auth(context,data_dict)
               if not check2:
                   return {'success': False, 'msg': _('User %s not authorized to edit these groups') % str(user)}
>>  check2 = check_group_auth(context,data_dict)
Module ckan.logic.auth.create:90 in check_group_auth
<<      groups = set()
           for group_dict in group_dicts:
               id = group_dict.get('id')
               if not id:
                   continue
>>  id = group_dict.get('id')
AttributeError: 'unicode' object has no attribute 'get'

(using RESTful Model API)

The reason is the POST is expecting the group to be expressed as a dict, not the list of names that GET returns.

1320263318000000 1320275623000000
#1443 defect dread dread ckan-sprint-2011-11-07 closed fixed language detection

If language detected doesn't match any of the offered CKAN languages, you get the default language, but see an error flash up for every page, saying:

Could not change language to 'None': IOError: [Errno 2] No translation file found for domain: 'ckan'

Affects 1.5b only.

1320425588000000 1320432511000000
#1087 enhancement wwaites ckan-sprint-2011-11-21 closed fixed version and contact info api call

a simple api call that returns data like this:

{ "version": ckan_software_version,
  "contact": { "name": "Some Admin", "mbox": "[email protected]" },
  "description": "Site Description",
  "url": "http://canonical.name.ckan.net/"
1302628944000000 1320866159000000
#1398 enhancement kindly rgrp ckan-sprint-2011-11-21 closed fixed Automated conversion of resource data into webstore
  • Possible sibling / requirement: #1397 (resource archiving)
  • Setup test environment. 1d
  • Make task to upload files to webstorer. 3d
    • This includes writing back webstore_url and updated date. 0.5d
    • Type guessing of file format. 1d
    • Make cron to upload all un webstored resources. 0.5d
  • Testing for errors by running cron 2d
  • Make nice way to deploy celery 1d.
  • Documentation. 1d
  • Deployment. 1d
1318529724000000 1321827970000000
#1420 enhancement johnglover johnglover ckan-sprint-2011-11-21 closed fixed Refactor ckanext-qa to work as a celery task

Currently can run as a celery task launched by paster command.

Still have to add a CKAN plugin so that QA will run when a resource is added or URL changed, and then update the extension controller/templates to look for QA data from the task_status table instead of directly on the resource.

1319626138000000 1320843789000000
#1433 enhancement kindly rgrp ckan-sprint-2011-11-21 closed fixed Support SQLAlchemy 0.7

Why: current stable version of sqlalchemy. geoalchemy stuff required 0.7 and likely that some other things will require it soon.

Probably requires work on vdm https://bitbucket.org/okfn/vdm

NB: should have discussion before making 0.7 the default required version in CKAN core.

1320143453000000 1324472583000000
#1449 enhancement johnglover johnglover ckan-sprint-2011-11-21 closed fixed Improved resource listing on dataset view page

Should list resources in the form of search results rather than as a table.

1320677693000000 1321873008000000
#1454 enhancement dread dread ckan-sprint-2011-11-21 closed fixed postgres search code/data removal

We made SOLR the main and only supported way to do CKAN search three months ago (#1275), but I notice that we still have remnants of the postgres full-text search hanging around:

  • model code (ckan/model/search_index.py)
  • table in the database (package_search)

The presence of these still create sqlalchemy warnings and add to our db, (& db backups etc), so it would be good to get rid of them if we are not using them. John says this can probably be removed

1320838999000000 1320849527000000
#1456 enhancement amercader amercader ckan-sprint-2011-11-21 closed fixed Use resource description instead of name if both are present

If a resource has both description and name the name is used. Descriptions are generally more, well, descriptive, so let's use those.

1320862619000000 1324472178000000
#1463 defect thejimmyg ckan-sprint-2011-11-21 closed wontfix QA extension no-longer works with packaged CKAN 1.5

The extension needs upgrading:

No module named webstore.database

The other option is to delay this functionality and merge with the publisher dashboards after the group refactor.

John, David Raznick, David Read, Ira, what do you think?

1321375614000000 1321874259000000
#1470 defect dread amercader ckan-sprint-2011-11-21 closed fixed Check user name in the profile form 1321446143000000 1324473955000000
#1472 enhancement rgrp rgrp ckan-sprint-2011-11-21 closed fixed Convert ckanclient to git and github from mercurial 1321624905000000 1321721874000000
#1473 defect dread dread ckan-sprint-2011-11-21 closed fixed max name length validators missing

Max length of package name and version field is 100 characters, but the validators seem to enforce this, although in the web interface the javascript limits typing in these fields, so it just affects the API really.

1321631452000000 1321872717000000
#1474 enhancement kindly ckan-sprint-2011-11-21 closed fixed fix up navl tests

navl tests are being skipped unskip them!

1321825892000000 1321826753000000
#1488 enhancement dread dread ckan-sprint-2011-11-21 closed fixed List of installed extensions - add to the status_show

Add the list of extensions install (config's ckan.plugins) to status_show logic action (See #1087)

1322130555000000 1322241966000000
#1002 enhancement dread rgrp ckan-sprint-2011-12-05 closed fixed Remove changeset model code

This code is unused and tests are disabled i belive. I suggest stripping this out (can always be re-added in an extension later). If we remove we should probably delete the relevant tables (and so a migration is needed).

1298489773000000 1322515696000000
#1124 enhancement thejimmyg nils.toedtmann ckan-sprint-2011-12-05 closed fixed push apt package python-ckanext-solr into our debian repository

python-ckanext-solr is already available in http://apt-alpha.ckan.org/datanl-dev, but not yet in http://apt-alpha.ckan.org/debian (that is why we had to [pip-install it for DataGM). Please push into main repo.

1304537793000000 1323168156000000
#1415 enhancement thejimmyg nils.toedtmann ckan-sprint-2011-12-05 closed fixed Comments on current status of ckan deb packages

This is a scratch pad ticket with some comments on the current status of our ckan deb packages. I know that some of it is the deb packaging roadmap anyway, please forgive me if i mention them here again.

Rufus and me re-deployed some community ckan instances onto s022 (see http://trac.okfn.org/ticket/926). We followed the documentation http://docs.ckan.org/en/latest/install-from-package.html

  • Deb package version number: the version of the deb package is "python-ckan 1309471251~149be76faabc+lucid-1", and it's hard to guess from there that it contains a ckan 1.4.2a
  • When is 1.4.3/1.5.x expected as deb package?
  • There was a bug in the DB upgrade script /usr/share/pyshared/ckan/migration/versions/029_version_groups.py (line 150) which looks like it was fixed 1.4.1==>1.4.2 but was nevertheless present in this deb package.
  • The current script /usr/bin/ckan-std-install
    • does not set the Apache ServerName? according to the $INSTANCE variable
    • automatically configures a ckan extension named after $INSTANCE
    • depends on local postgres
    • could be replaced with "/usr/bin/ckan-deploy --name=ckan-std --domain=ckan-std.localhost (see next point)
  • (i think this is exactly James' plan): have more generic deployment script /usr/bin/ckan-deploy as part of python-ckan which takes arguments like
    • --domain=cc.ckan.net
    • --aliases=$list-of-domains
    • --name=cc (defaults to "domain")
    • --no-db (does not configure a DB)
    • --sql-alchemy=$DB_CONFIG_STRING (also runs "paster --plugin ckan db upgrade --config")
    • --extension $list-of-extesions
    • ...
1319457069000000 1323167941000000
#1445 enhancement zephod rgrp ckan-sprint-2011-12-05 closed fixed Resource View page in WUI

Super ticket: #1032

  • Locate at: /dataset/{dataset}/resource/{id}

See: http://wiki.ckan.org/Dataset_View_Page

Implemented in branch feature-1445-resource-view.

Still to do:

  • Add inline data explorer to page
1320665049000000 1330019916000000
#1450 enhancement zephod johnglover ckan-sprint-2011-12-05 closed fixed Simplify dataset view page

See http://wiki.ckan.org/Dataset_and_Resource_Pages_Proposal

  • Create a collapsible section for the README
  • Reduce the number of things on the page
  • Tidy up groups and tags in the sidebar
  • Resource view becomes a list of search results

(additional)

  • Resource navbar item links to first resource
  • Resource navbar has a dropdown menu of all resources in the dataset
  • Edit and Authz navbar items should be in a separate logical group to the right

Very rough summary: create more info section and move stuff out (relayout page e.g. README, tags etc)

1320677788000000 1330019974000000
#1453 enhancement icmurray icmurray ckan-sprint-2011-12-05 closed fixed Flexible tag names

Allowing more flexible tag names:

  • allowing spaces
  • allow capital letters (search is case in-sensitive)
  • allow all punctuation except for commas and double-quotes '"'
  • allow unicode
  • commas delimit tag names in the package create/edit form

Effects:

  • package creation/edit form.
  • /tag/{tagname} uri
  • search action
  • api controller (search/package-create/edit)
  • web controller (search/package-create/edit)
  • search api documentation
  • autocomplete for tag names
1320771850000000 1329395697000000
#1455 defect johnglover dread ckan-sprint-2011-12-05 closed fixed Search results when 'all_fields' don't include 'extra' fields

When you do a search like this:

http://thedatahub.org/api/search/package?q=tauberer+census&all_fields=1

the "extra" fields (e.g. "triples", "shortname") get missed off the results. The docs say it should be a "full record" and I don't see any reason why this is missed off.

This is a problem because search all_fields is the only way for clients and front-ends to get packages in bulk. They end up (like lodcloud) doing thousands of requests to get packages individually.

The full record is:

http://thedatahub.org/api/rest/dataset/2000-us-census-rdf
{"count": 1, "results": [{"res_description": ["Download", "XML Sitemap", "SPARQL enpdoint", "Example (RDF/XML)"], "name": "2000-us-census-rdf", "license": "Non-OKD Compliant::Creative Commons Non-Commercial (Any)", "author": "Joshua Tauberer", "author_email": "http://razor.occams.info/", "ckan_url": "http://thedatahub.org/dataset/2000-us-census-rdf", "notes": "2000 U.S. Census converted into over a billion RDF triples.\n\nPopulation statistics at various geographic levels, from the U.S. as a whole, down through states, counties, sub-counties (roughly, cities and incorporated towns)\n\nNotes: also found in the of SPARQL Endpoints.\n\nFrom home page:\n\n> * For the detailed Census statistics, you'll have to download the raw Census data files from the Census Bureau, my Perl script and the patch file below and run it yourself because the files are too big for me to offer as a download!\n> \n> * The data and scripts can be reused under Creative Commons Attribution-NonCommercial-ShareAlike.\n", "entity_type": "package", "site_id": "www.ckan.net", "download_url": "http://www.rdfabout.com/demo/census/", "indexed_ts": "2011-11-01T12:52:36.034Z", "url": "http://www.rdfabout.com/demo/census/", "state": "active", "title": "2000 U.S. Census in RDF (rdfabout.com)", "groups": ["lod", "lodcloud"], "res_format": ["", "meta/sitemap", "api/sparql", "example/rdf+xml"], "license_id": "cc-nc", "revision_id": "fcbad0de-79ea-41bd-8e01-eb832a05b732", "res_url": ["http://www.rdfabout.com/demo/census/", "http://www.rdfabout.com/sitemap.xml", "http://www.rdfabout.com/sparql", "http://www.rdfabout.com/rdf/usgov/geo/us/ny"], "id": "551ec435-f198-4d52-9b56-ec0b0be6aec9", "tags": ["census", "data", "demographics", "deref-vocab", "format-dc", "format-geonames", "format-politico", "format-rdf", "geographic", "linkeddata", "lod", "lodcloud.nolinks", "no-license-metadata", "no-provenance-metadata", "no-vocab-mappings", "population", "published-by-third-party", "rdf", "statistics", "us"]}]}
1320858265000000 1324474466000000
#1468 defect johnglover thejimmyg ckan-sprint-2011-12-05 closed fixed Update the search index for individual packages

When I delete a package from the pylons shell I need to remove that package's entry from the Solr search index. That doesn't happen at the moment.

It would be useful if the search-index rebuild command could take a list of package IDs to rebuild.

1321376195000000 1322591997000000
#1478 enhancement kindly kindly ckan-sprint-2011-12-05 closed fixed change package_group to members table

This will comprise of the following steps.

  • model migration.
  • simplify revisioning
  • Remove any sqlalchemy relations from groups.
  • change logic functions

estimate 2d

1321959973000000 1323172654000000
#1479 defect dread dread ckan-sprint-2011-12-05 closed fixed Can't edit a user with a unicode email address
  1. Register User with an email address with a unicode char (e.g. u'\u044e')
  2. View the User in the UI (/user/) or with 'user_show' Action API

Exception:

Module ckan.controllers.user:98 in read
<<          try:
                   user_dict = get_action('user_show')(context,data_dict)
               except NotFound:
                   h.redirect_to(controller='user', action='login', id=None)
>>  user_dict = get_action('user_show')(context,data_dict)
Module ckan.logic.action.get:488 in user_show
<<      check_access('user_show',context, data_dict)
       
           user_dict = user_dictize(user_obj,context)
       
           if not (Authorizer().is_sysadmin(unicode(user)) or user == user_obj.name):
>>  user_dict = user_dictize(user_obj,context)
Module ckan.lib.dictization.model_dictize:189 in user_dictize
<<      
           result_dict['display_name'] = user.display_name
           result_dict['email_hash'] = user.email_hash
           result_dict['number_of_edits'] = user.number_of_edits()
           result_dict['number_administered_packages'] = user.number_administered_packages()
>>  result_dict['email_hash'] = user.email_hash
Module ckan.model.user:59 in email_hash
<<          if self.email:
                   e = self.email.strip().lower()
               return hashlib.md5(e).hexdigest()
               
           def get_reference_preferred_for_uri(self):
>>  return hashlib.md5(e).hexdigest()
UnicodeEncodeError: 'ascii' codec can't encode character u'\u044e' in position 17: ordinal not in range(128)
1321960486000000 1321961592000000
#1480 enhancement dread dread ckan-sprint-2011-12-05 closed fixed Munge functions in util

munge_title_to_name, munge_tag v. useful for client software importing things into CKAN. I want it for ONS.

1321965837000000 1321978546000000
#1481 defect johnglover dread ckan-sprint-2011-12-05 closed fixed License not shown if no resources

The license is not displayed any more, unless there are resources e.g. http://test.ckan.net/dataset/dtest#section-basic-information is OGL, but that's not shown.

1321983266000000 1323169343000000
#1482 defect johnglover zephod ckan-sprint-2011-12-05 closed fixed Minor JS bugs

Three JS bugs have appeared:

  • The slug editor is not updating the text field again. [Fixed; commit:fadda17383]
  • The View Dataset page will stop executing javascript if there are no resources. [Fixed; commit:e54b089f6e]
  • The Add-Dataset and Add-Group URL editor shows the wrong URL for datasets and groups. Broken in this commit:

https://github.com/okfn/ckan/commit/5a01e67 by 'map.explicit=true'. JohnG: Can you look into this? [Fixed; commit:88e88380f6]

1321991091000000 1322062576000000
#1484 defect dread dread ckan-sprint-2011-12-05 closed fixed Creating package with duplicate tags causes exception

e.g.

curl http://localhost:5000/api/action/package_create -d '{"name":"test5", "tags":[{"name": "tag6"}, {"name": "tag6"}]}' -H 'Authorization:tester'

Also broken in Web UI (adding a package).

Affects API and Web UI in 1.5 and I think all versions from 1.4.1

1322052046000000 1322059006000000
#1487 enhancement kindly ckan-sprint-2011-12-05 closed fixed Fix group ordering on homepage

ordering on homepage by name instead of group count

1322094280000000 1324474147000000
#1490 enhancement amercader amercader ckan-sprint-2011-12-05 closed fixed Standardize output from package listings coming from the logic layer

Right now, the two logic functions that return a list of packages (package_search [1] and group_packages_list [2])use custom functions to generate the output dict. That's suboptimal because:

  • The template function that renders the package listing (package_list_from_dict) could end up receiving different dicts.
  • Not all package properties are available at the template level (e.g extras, tags...). These can be really useful to customize the listing from an extension.

In general only the functions present in lib/dictization/model_dictize.py should be used to build the output of a logic function, in that case package_dictize. If necessary, they can be modified to include missing properties, like on this particular case the "isopen" property, needed by the template renderer.

[1] https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L685 [2] https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L442

1322225239000000 1324033557000000
#1493 defect dread dread ckan-sprint-2011-12-05 closed fixed 'search-index rebuild/clear' doesn't work if no ckan.site_id

You can't delete things from the SOLR search index if the ckan.site_id and ckan.site_url are blank.

Should assert that one of these are set up.

1322484422000000 1324474360000000
#1497 enhancement dread dread ckan-sprint-2011-12-05 closed fixed Suppress benign warnings

We really shouldn't have lots of benign stuff printed on stderr when we run every command. It confuses users and hides important stuff.

1322569640000000 1322569805000000
#1498 enhancement amercader amercader ckan-sprint-2011-12-05 closed fixed Support multiple SOLR schema versions

At some point the SOLR schema used by CKAN will need to be changed and the changes won't be always backward compatible, so we need a mechanism to support instances that use older versions of the schema. This process will involve:

  • Better documentation on how to deploy SOLR, as single instance or multicore
  • A convention on how to tag schema versions and SOLR endpoints to ensure that CKAN instances point to the correct SOLR URL.
  • (If possible) Checks in CKAN core that deal with different schema versions.
1322585264000000 1323165876000000
#1499 enhancement dread dread ckan-sprint-2011-12-05 closed fixed list/read/delete packages on the command-line

It would be useful to be able to manipulate packages using the paster command-line interface. In particular listing, reading and deleting packages. Seeing which packages were deleted too.

1322591656000000 1322591888000000
#1500 defect dread dread ckan-sprint-2011-12-05 closed fixed Slug API calls tidy up
  • API call documented but don't exist any more: dataset create_slug
  • API call not documented but exists: is_slug_valid
  • API call not documented, badly sited in API and using custom code in controller (not logic layer): /package/autocomplete
1322600285000000 1323166918000000
#1501 defect johnglover amercader ckan-sprint-2011-12-05 closed fixed Pagination links broken

Most of the different pagination links are broken and point to wrong locations. e.g:

  • Dataset listings in group page:

http://test.ckan.net/group/bibliographic

  • Tag letters in tag listing:

http://test.ckan.net/tag

  • User listing:

http://test.ckan.net/user

  • Revision listing:

http://test.ckan.net/revision

This is most probably related with:

https://github.com/okfn/ckan/commit/1772a5c

1322646397000000 1322684411000000
#1502 defect johnglover johnglover ckan-sprint-2011-12-05 closed fixed Group package list is ordered by revision timestamp instead of alphabetically 1322680312000000 1324480415000000
#1503 task dread markbrough ckan-sprint-2011-12-05 closed fixed Move from Bitbucket to Github

Reasons: more people familiar with Git, bigger Open Source community on Github.

Keeping track of the move here: https://docs.google.com/spreadsheet/ccc?key=0AvCDqUH8jVN8dGNVNEpXLWk3UTlJZXhFMjBPbXVtRmc&hl=en_GB#gid=0

1322745084000000 1323793662000000
#1504 defect dread dread ckan-sprint-2011-12-05 closed fixed Action API given blank data causes exception

This is what caused the exception:

curl http://localhost:5000/api/action/package_search -d='{"q": "osm"}'

The problem is the content-type is formencoding, the '=' is the first character of the request, and so internally the data_dict is set to ' ', which needs to be a dictionary.

1322756748000000 1324035928000000
#1505 defect dread dread ckan-sprint-2011-12-05 closed fixed SearchError and SearchQueryError cause exception in Action API

This query caused ckan to except because ckan/controllers/api.py doesn't catch SearchError? and SearchQueryError?:

curl http://localhost:5000/api/action/package_search -d '{"sort": "metadata_modified"}'
1322758968000000 1324474577000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Note: See TracReports for help on using and creating reports.