{22} Trac tickets (2647 matches)

Results (901 - 1000 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#902 defect rgrp dread ckan-v1.4-sprint-1 closed fixed Package buttons not clickable / Improve submenu tabs

On a package page there are three buttons to the right - View, Edit, History. Each of these buttons contain an icon and the text, but you can only activate the button by clicking on the icon. When you click on the part of the button where the text is (about 50% of the button) then nothing happens.

Using: Chrome, Firefox

1294756898000000 1297078885000000
#903 defect pudo pudo ckan-v1.3 closed fixed Proper handling of deleted packages in Solr serps

Currently, deleted packages are filtered from search results after solr returns. That means result count can drop below the requested number, down to 0. Thus we need to filter before or in Solr.

1294831054000000 1296588070000000
#904 task rgrp Stiivi ckan-v1.4-sprint-3 closed fixed Review CKAN documentation

What's bad at the moment?

  • lack of documentation e.g. config (very poorly documented)
  • too many sources of documentation
  • no common theming


Resulting meta-ticket with things to do: ticket:927

1294832610000000 1299840539000000
#905 defect thejimmyg Stiivi ckan-sprint-2012-02-06 closed worksforme Unable to search with accented characters in package names

I have a package with name: "Štatistické prehľady agendy cestovných pasov" and when I try to search for "Štatistické" there are no results.

CKAN setup:

  • psql DB with UTF-8 tables
  • psql search backend
  • ckan version: 1.3.1b



try to search for the first word in the package name.

1294939120000000 1328638536000000
#906 enhancement thejimmyg Stiivi ckan-sprint-2012-03-05 closed fixed Ability to search without accents for accented words

In Slovakia users are expecting from sites to be able to search without typing accents, for example if they do not have SK keyboard, just US. For example searching for: 'Obyvateľstvo' and 'obyvatelstvo' (population) or 'štatistika' and 'statistika' (statistics) should yield same results.

This should work the other way around as well, as some people might enter entries without accents into CKAN instance and others might search with accents.

For SK language simple ASCII transliteration for searched term and indexed words is sufficient (iconv ASCIITRANSLIT).

1294939588000000 1330990400000000
#907 enhancement rgrp ckan-v1.3-sprint-3 closed fixed Data previews: preview package resource data in the WUI

As a user I want to preview a dataset ('resource') before downloading it.


This will be implemented as an Extension

NB: This ticket requires a way to get data out of a remote url (the dataset). We do not want to put this data load/streaming into ckan core due to a) complexity b) load problems (do not want to take ckan out by accidentally pulling in a 1GB file)

  1. Need a 'data proxy' - see ticket:698 and ticket:888
  2. Preview using javascript in WUI - do this in an extension http://bitbucket.org/okfn/ckanext-dataapi
1295261173000000 1297072303000000
#908 defect pudo rgrp ckan-v1.4-sprint-1 closed fixed Issues deploying extensions with modwsgi / uwsgi

Some extensions import pylons features (such as request) at root.

Have encountered following issues:

  1. You need to import ckanext before any ckan stuff in your wsgi script (extension stats and dataapi)
  1. Running under wsgi daemon mode (normal mode is fine!) the front page (front page only) will not work (does not occur with disqus or synchronous_search but does with stats and dataapi extensions). You get errors like this in the log:
[Mon Jan 17 09:01:19 2011] [error] Exception KeyError: KeyError(-1218594160,) in <module 'threading' from '/usr/lib/python2.6/threading.pyc'> ignored
[Mon Jan 17 09:01:20 2011] [error] Exception AttributeError: "'NoneType' object has no attribute 'clearing'" in <bound method PluginEnvironment.__del__ of  Services for Environment '<default>'
1295264624000000 1296730578000000
#909 enhancement pudo ckan-backlog assigned DCat importer for CKAN

Write an importer that supports most well known variants of DCat in importing the data as CKAN packages. In particular, the following sources should be supported:

  • CKANrdf generated exports
  • opengov.se RDF (not really DCat)
  • Sunlight Nationdatacatalog as harvested by dcat-tools.
1295265958000000 1346669602000000
#910 enhancement pudo pudo ckan-v1.5 closed invalid Refactor CKANrdf to support extras

Extra fields should be represented in some manner.

1295266026000000 1310128544000000
#911 defect pudo pudo ckan-v1.3-sprint-3 closed fixed Add pager to dcat-tools/rdfsolr 1295266211000000 1296467375000000
#912 defect pudo pudo ckan-v1.5 closed wontfix Rethink result row representation in dcat-tools/rdfsolr
  • Should license go in the bottom line?
  • Formats should be styled as in CKAN
1295266299000000 1306774876000000
#913 enhancement pudo pudo ckan-v1.5 closed wontfix Handle licenses in dcat-tools
  • Create a list of all valid predicates where licenses are stored.
  • Create a license normalization registry in GSpread
  • Fill spreadsheet from OKD/OSI lists.
1295266407000000 1306774962000000
#914 defect pudo pudo ckan-v1.3-sprint-3 closed fixed Handle formats in dcat-tools
  • Create a list of valid predicates for format information (accessURL et al)
  • Execute #235 via Google Spreadsheets and include in DCat enrichment
1295266569000000 1297069958000000
#915 defect pudo pudo ckan-v1.3-sprint-3 closed fixed Deploy dcat-tools/rdfsolr to publicdata.eu as a placeholder

We should present something there, why not start with a meta-search.

1295267040000000 1296467518000000
#916 enhancement rgrp pudo vdm-0.8 closed duplicate Port new vdm to Mongodb

We should create a port of VDM 0.9 for MongoDB to support user editing in WDMMG.

1295277011000000 1297066902000000
#917 enhancement kindly kindly closed fixed Turning off Autoflush in SQLAlchemy

David Raznik: I propose we remove autoflush by default in ckan. It caused a lot of bugs on the version upgrade due to its magical behavior.

I have a patch in https://bitbucket.org/kindly/ckan/ on branch feature_switch_autoflush_off_by_default.

This change simplifies code. It only took 7 extra flushes/commits to make work and there are over 40 instances that we remove autoflush, so as not to cause error (I have not removed them in the patch yet).

Things to know about using flush:

  • Most of the time nothing will change. A commit implies flush.
  • You only need to flush if you expect a query to return an object you just saved. i.e if you session.add(obj) and you expect obj to be in a session.query.
  • A flush is handy when you want to get the primary key out before you do a commit. i.e session.add(obj); session.flush(); obj.id is now generated.
  • You can use them as a limited form of nested transaction.

It increased the performance on the tests by about 5%. There are less flushes altogether so is faster.

I do not see any downsides.

1295279694000000 1295280232000000
#918 defect memespring dread closed invalid Package preview doesn't show tags or groups
  1. Edit any package
  2. Hit 'preview'

Notice that the preview at the bottom doesn't show what tags or groups the package is in.

This is because the new design of the package view only has this info in the sidebar, which is not displayed in the preview.

One might argue that this is ok because the tag is not a central part of the metadata, only used for searching, so it is ok not to preview it. But I wanted to check that this change was a conscious decision.

1295346496000000 1315911507000000
#919 enhancement dread dread ckan-v1.4-sprint-6 closed fixed Package preview contains API & datapkg instructions
  1. Edit a package
  2. Hit 'preview'.

The preview contains the section "CKAN API / datapkg" which seems irrelevant at this point. Looking at ckan.net previews, the Comments section only appears when you view a package and not when you preview it. This seems more sensible - can this be done for the "CKAN API / datapkg" section too?

1295346730000000 1303202627000000
#920 defect thejimmyg [email protected] closed fixed empty tags are kind of confusing

http://ckan.net/tag/linguistsic doesn't have a single entry, but is still kept and displayed.

It should be purged !!!

1295355280000000 1300319140000000
#921 enhancement sebbacon thejimmyg closed fixed Refactor DGU Harvesting

Refactor harvesting code to simplify.

1295389237000000 1300197629000000
#922 enhancement kindly kindly closed duplicate Make Resources first class entity.


  • Will make it easier to give resources their own form, independent of packages.
  • Makes uploading, harvesting and previewing data cleaner as its separate from the packages.
  • Gives the option of sharing a resource between packages.
  • Leads the way to supporting storing richer semantic information against them.
1295713858000000 1320664187000000
#923 defect rgrp dread closed worksforme Search box doesn't work in leaderboard page
  1. Go to: http://ckan.net/stats/leaderboard#content
  2. In the far top-right of the browser, select the search box in 'water'.
  3. Press enter to search. Nothing happens.

Tried in: chrome, firefox

1295867328000000 1340632144000000
#924 enhancement dread ckan-backlog new Search box has no search button

The search box at the top-right of CKAN's page doesn't have a 'go' button. I feel that a larger percentage of users expect a 'go' or 'search' button on the right-hand side of the box to press to start searching. Techies tend to know the keyboard shortcut of pressing 'carriage-return' but it might be better to follow standard practise on this.

Examples with 'search' button: Internet Explorer, Firefox, Google, Amazon, trac Examples without: ?

1295867533000000 1323170436000000
#925 defect dread ckan-backlog closed fixed Change the search box icon to remove the down arrow

Is there a good reason why the search box has a 'down arrow' icon when there is no drop-down menu? Or can this be usefully removed?

1295867593000000 1323168588000000
#926 enhancement sebbacon sebbacon ckan-v1.4-sprint-1 closed fixed Pick a simpler form framework

The current formalchemy setup conflates view, controller and model code in a way that makes it hard to debug and customise.

Review existing (and potentially non-existing) frameworks with a view to porting forms over to something more explicit and lightweight.

Implement the current Package forms as an example of how this would work.

Document and circulate.

Sub-ticket of #961 (form, validation, model sync meta-ticket)

1295869056000000 1298541597000000
#927 enhancement rgrp rgrp ckan-v1.4-sprint-3 closed fixed [super] Improvements to CKAN documentation

List of improvements to CKAN documentation (result of ticket:904). Should convert most items into sub-tickets as we go along.



  • Contributors / credits page: ticket:928
  • (Data proxy documentation: move it to packages.python.org/dataproxy)
  • (List dataapi extensions on http://ckan.org/wiki/Extensions (Name, summary string + point to pypi page ...))
  • Document license configuration and license system - ticket:973
  • Refactor trac frontpage moving general feature overview to ckan.org
  • Create proper roadmap information on http://trac.ckan.org/
  • Add link to wiki.ckan.net to ckan.org
1295869492000000 1300105638000000
#928 enhancement rgrp rgrp ckan-v1.4-sprint-1 closed fixed Documentation - CKAN credits / contributors page
  1. Remove contributors list from about page (out-of-date and does not belong there)
    • A "Powered by CKAN" link in footer is enough
  2. Add contributors / team / credits page to ckan.org
    • May want to separate credits (e.g. software use), from contributors (also is team different from contributors)
    • (perhaps with photos!) so that people can see who is part of it and from which organisations. (this is medium-term, not really part of this ticket)
  3. ? CREDITS.txt file to the source repository and add a link to it on bitbucket for more detailed credits info.

Cost: 1h

1295872661000000 1297115136000000
#929 defect rgrp rene.kapusta ckan-v1.4 closed fixed Handle the case when the licenses service is down better

I'm using v 1.3.1a and get a lot of "WebApp? Error: <type 'exceptions.Exception'>: Couldn't connect to licenses service: <urlopen error (104, 'Connection reset by peer')>" error messages.

Module ckan.forms.package:87 in build_package_form
<<      # Options/settings
           builder.set_field_option('name', 'validate', package_name_validator)
           builder.set_field_option('license_id', 'dropdown', {'options':[('', None)] + model.Package.get_license_options()})
           builder.set_field_option('state', 'dropdown', {'options':model.State.all})
           builder.set_field_option('notes', 'textarea', {'size':'60x15'})
>>  builder.set_field_option('license_id', 'dropdown', {'options':[('', None)] + model.Package.get_license_options()})
Module ckan.model.package:283 in get_license_options
<<      @classmethod
           def get_license_options(self):
               register = self.get_license_register()
               return [(l.title, l.id) for l in register.values()]
>>  register = self.get_license_register()
Module ckan.model.package:278 in get_license_register
<<      def get_license_register(self):
               if not hasattr(self, '_license_register'):
                   self._license_register = LicenseRegister()
               return self._license_register
>>  self._license_register = LicenseRegister()
Module ckan.model.license:46 in __init__
<<              from licenses.service import LicensesService2
                   self.service = LicensesService2(group_url)
                   entity_list = self.service.get_licenses()
                   from licenses import Licenses
>>  entity_list = self.service.get_licenses()
Module licenses.service:44 in get_licenses
<<          except Exception, inst:
                   msg = "Couldn't connect to licenses service: %s" % inst
                   raise Exception, msg
                   licenses = loads(response_body)
>>  raise Exception, msg
Exception: Couldn't connect to licenses service: <urlopen error (104, 'Connection reset by peer')>
1295995409000000 1299840884000000
#930 defect wwaites dread closed fixed call_timing files created for every request

On a production server we produce literally millions of little files saying how long every single request took to process.

They are here:


They were added by ckan/lib/base.py in cset:da438a9085d3.

I don't believe anyone uses these stats.

1296061721000000 1297066292000000
#931 enhancement dread dread ckan-v1.4-sprint-1 closed fixed Search results generator hides paging functionality

ckanclient's search results list only packages up to the 'limit'. It would be good to return a generator instead of a list. When the limit is reached on the generator then another 'page' is loaded automatically.

1296147360000000 1298379187000000
#932 enhancement dread dread ckan-v1.3 closed fixed Move to SQLAlchemy-Migrate 0.6

Basic steps from kindly:

  • changing the setup.py dependencies for sqlalchemy-migrate and sqlalchemy to their latest versions.
  • running the migrate script on the repository (or changing the repository scripts manually).
  • checking in both these changes at once.
1296156660000000 1297076710000000
#933 enhancement kindly ckan-v1.4 closed fixed get rid of self when it should be cls in classmethods

use pylint

1296174526000000 1297348975000000
#934 enhancement kindly closed invalid Key value store for extensions to store data

We need to allow plugins to store information.

Each plug-in should have its own name space and allow any data to be set.

There are two options.

  • make a key value table in ckan that will hold random information.
  • use redis (or other key value store)

If in sql suggest schema of form:

  • (namespace, obj_id, key, value [, value_type])
  • value is json
1296228758000000 1323171047000000
#935 defect pudo pudo closed fixed "groups" field bug in IATI

An internal server error was reported when trying to create a new package (not when editing):

WebApp? Error: <class 'sqlalchemy.exceptions.IntegrityError?'>: (IntegrityError?) null value in column "name" violates not-null constraint 'INSERT INTO package (id, name, title, version, url, author, author_email, maintainer, maintainer_email, notes, license_id, state, revision_id) VALUES (%(id)s, %(name)s, %(title)s, %(version)s, %(url)s, %(author)s, %(author_email)s, %(maintainer)s, %(maintainer_email)s, %(notes)s, %(license_id)s, %(state)s, %(revision_id)s)' {'maintainer': None, 'name': None, 'author': None, 'author_email': None, 'notes': None, 'title': None, 'maintainer_email': None, 'url': None, 'state': 'active', 'version': None, 'license_id': None, 'revision_id': '0141c09c-4eb3-4dda-a6b1-e07f2c73b4c5', 'id': '7fe16a29-f183-471a-99c5-2c2e81dd86e4'}

The error could not be reproduced in a non-apache/mod_wsgi environment, even with a dump of the live database. Further investigation by manual editing of the package form left the "groups" field as the likely cause. After updating the packages installed, and temporarily editing the package controller the problem vanished after a restart of the web server.

Possible causes include:

  • Stale pyc file
  • Threading issue
  • Dependency issue

As the problem is gone for now, it cannot be further investigated atm.

1296302915000000 1306778682000000
#936 enhancement johnglover rgrp ckan-v1.4-sprint-6 closed fixed Follow / watch package extension

As a (logged-in) User I want to watch (follow) a package, that is register my interest about a package. (Similar to watch/follow features in github/bitbucket/wikis).

NB: this is as much (if not more) about showing what packages are interesting to people as giving info to 'watchers'.

Need to finalize terminology (github uses watch for repos and follow for users while bitbucket combines both in 'followers'). Decision: use follow



Become a follower:

  • Follow button on packages (if already watching say 'unfollow')

Package-related changes:

  • Show number of followers on a package
  • List followers of a package at /package/{name}/followers
    • On a separate page

a package User-related changes:

  • List followed packages
    • Either on user's page on a separate 'following' page. (NB: called 'following')
  • Does watching involve notifications (by email)
    • Probably not: you can already subscribe to RSS feed after all and email not that necessary (?)
  • [Future - don't have activity stream yet] Show what packages a user has started/stopped followed on a user's public activity stream on their user page


  • Want to do this in ajax-y manner
  • API endpoint: /api/2/follower
  • Store data in a new follower table



follow => PUT / POST

If this is submitted by a user with user.id != user_id => error (401)

unfollow => DELETE

=> list of followers
    { safe dictized user }

NB: depends on access to a 'safe' dictized user object. Dictization is in nearly done, and current example of doing this by hand is in user API autocomplete method.


Called 'follower'

user_id, table, object_id, created
xxx, package, yyy, ...
xxx, user, yyy, ... [future]

Random Extras

  • What about following users as well
1296339079000000 1303838713000000
#937 enhancement sebbacon rgrp ckan-v1.4-sprint-5 closed fixed Record download stats for resources (extension)

As a User (especially as a Package Owner/Maintainer?) I want to know how many times a resource has been downloaded (and when).

So let's record download stats (as in clicks on the link for a resource).


Old Spec (do it in CKAN)

  • Record info of form: resource id (or url?), timestamp
  • Do this via javascript capturing of onclick event talking to an api
  • API: /api/resource/{id}/download
    • POST to increment (how do we stop spamming -- could use a nonce setup with a random string set on each page load for the js)
    • GET to get data back { total: X, day_count: [ [yyyy-mm-dd, count], ... ] }


  • Do we record ip addresses (to handle de-botting etc)?
  • Do we count preview clicks as well?
1296340273000000 1302513831000000
#938 enhancement pudo rgrp ckan-v1.4-sprint-1 closed fixed Message flashing / notification in WUI

Good user interfaces provide feedback to users. We should provide more feedback about the success / failure of actions using message flashing (see http://flask.pocoo.org/docs/patterns/flashing/).

We already do some of this (slightly poorly), e.g. on adding an authorization the app says: "X authorization added".


  • Standardize this process and add section to html for this purpose.
  • Already done in other OKFN apps e.g. yourtopia.net, wheredoesmymoneygo and commentonit
  • Testing: suggest keeping this very simple


  • Cost: 0.5h (html etc) + 1h to do some integration
1296398920000000 1297077023000000
#939 enhancement rgrp closed fixed Notification bar at top of site shown to new users pointing them to simple instructions

On http://getthedata.org/ or stackoverflow there is a pop-up bar for new users that give them some simple instructions.

Talking with users it is clear that some people are not clear how ckan.net (or other sites) work and what they are and are not 'allowed to do. This could help make this better.


  • Pop-up bar (based on a cookie or just being not logged in?) - uses ticket:938 (message flashing)
  • FAQ/instructions page (use the new wiki?)



  • Cost: 1h
1296399008000000 1323171158000000
#940 defect pudo rgrp ckan-v1.3 closed wontfix OpenID login via google for same user generates 2 different openid urls

This has been noted twice in the last couple of weeks.

Checking around my guess is this is due to someone logging in to www.ckan.net and ckan.net. Google generates openid based on the RP (requesting party) which is the requesting domain url unless the openid.realm is set. See:

To fix this we should set the openid.realm (not sure how atm).

If this is not possible an alternative is to make sure we redirect all users to a single site url (perhaps using ckan config option site_url, or doing this in apache).

Cost: 1h

1296401521000000 1297071748000000
#941 enhancement thejimmyg rgrp ckan-v1.5 closed wontfix Submit apps or ideas for apps related to datasets (extension)

The basic purpose of this extension is provide a way to record 'apps' (applications) that relate to datasets in CKAN. Both existing apps and ideas for apps should be permitted.

1296403099000000 1310134339000000
#942 defect pudo pudo ckan-v1.4-sprint-1 closed fixed Fix SERP pager argument passing

It only passes "q", but the whole query must be serialized.

1296469490000000 1297074807000000
#943 enhancement pudo pudo ckan-v1.4-sprint-1 closed fixed Create CKAN Wiki

Set up a Mediawiki to contain info on CKAN.net (community) conventions etc.

1296469547000000 1297076498000000
#944 defect pudo pudo iati-4 closed fixed Upgrade IATI to CKAN 'default'
  • Move code base, fix any breakage.
  • Deploy to eu8
  • Test
  • Deploy to eu14


Later CKAN codebase has a bunch of improvements / fixes that it would be good to have.


  • 8h
1296469688000000 1306774766000000
#945 enhancement kindly kindly ckan-v1.6 closed fixed [super] Richer resources - Resource Groups, new fields, improved UI

Super ticket: #1032

This is a meta ticket for changes that are going to happen in resources.

  • New resource group table. #956
  • New kind field in resource. #957
  • UI for new kind field. #958
  • Resources in REST API ticket:358
  • Resources in WUI #1445
  • Make Resources first class entity. #922 (duplicate?)

Background on this change can be found at:

1296475283000000 1325259350000000
#946 defect pudo dread ckan-v1.3 closed fixed No default system rights

CKAN currently comes out of the box being uneditable. We'd like to encourage people to have open CKANs and therefore by default init the db with this right:

visitor -> is editor on -> System system
1296499922000000 1296833383000000
#947 enhancement rgrp closed duplicate Move the assignment of roles to a package on package creation to an extension

At the moment the assignment of user roles on package creation is done in setup_user_roles method in ckan/model/authz.py and is hard-coded.

This can be a pain to override (you can use a Plugin listening for package create events) and makes some things such as putting system in restricted mode more complex than it should be (see ticket:833).

An elegant solution would be to move this into an Extension this simplifies the code and make it easier for people override (just remove the default extension and plugin your own).

1296557478000000 1314031310000000
#948 enhancement dread ckan-future assigned Highlight (to a sysadmin) which packages are deleted

When a customer logs in as a sysadmin then he/she see all packages, including deleted and pending ones. These are hidden to the average user, but the sysadmin has no idea of this until he clicks on the package and sees at the bottom 'state: deleted'.

It should be more obvious than that on the search view - an icon, message or crossed-out name to packages are deleted.

1296646369000000 1338206280000000
#949 enhancement pudo pudo ckan-v1.4-sprint-1 closed fixed Generic username/password login

based on repoze.who-friendlyForms

1296658126000000 1297074827000000
#950 enhancement dread dread ckan-v1.3 closed fixed Move importer controller to ckanext-importer

ckan/controllers/importer.py provides a Pylons UI for importing spreadsheet data into CKAN. This would better live in an extension than core CKAN.

In the same move, ckan/tests/misc/test_spreadsheet_importer.py, which is used by importer.py as well as various dgu data importers, should move into ckanext.

1296666038000000 1297079743000000
#951 defect [email protected] closed invalid Can't add a package to group

I can't add a package (e.g. http://ckan.net/package/ub-konstanz) to a group (e.g. http://ckan.net/group/bibliographic). It's neither possible when editing a package (the only group in drop down menu is "history") nor on the group page.

1296726886000000 1314031006000000
#952 task wwaites pudo ckan-backlog closed invalid CKAN should run under nginx/uswgi

second part of #908

1296730498000000 1310125204000000
#953 enhancement rgrp rgrp ckan-v1.4-sprint-1 closed fixed Add tagline/description to template and set in configuration

Have a tagline / description are under main title and set if from ckan.site_description config variable.

1296809808000000 1296809834000000
#954 enhancement kindly rgrp ckan-v1.5 closed fixed [super] API version 3

Child tickets:

  • #1107 Move package autocomplete from package controller and move to API
  • #1087 version and contact info api call

Move to a format that has a separate responseHeader and response.

A standard package response

  responseHeader: {
    status: 0,
  response: {package-dict}

On error:

  responseHeader: {
    status: {err-code},
    error: 'message'
  response: none

A search query

Based directly on solr.

  responseHeader: {
    status: 0,
  response: {
    numFound: 5,
    start: 0
    docs: [


This is a breaking change for clients


1296811899000000 1320142744000000
#955 defect dread dread ckan-v1.3 closed fixed Tidy up tests
  • Removal of init_db from every setup etc.
  • Get them working
1296819300000000 1297342534000000
#956 enhancement kindly kindly ckan-v1.4-sprint-2 closed fixed Add a resource group table.

We need a way to group resources by various different factors. The most important of which is timeseries.

The model will look like

Package <o2m> resource_group <o2m> resource

The use cases to this are outlined in this email. http://lists.okfn.org/pipermail/ckan-discuss/2011-February/000887.html

1296821112000000 1299489084000000
#957 enhancement kindly kindly closed wontfix new "kind " field added to resources

The new field was outlined in


it says

Can be "data", "metadata", "service", NULL. There is a need to do this for UKLP where some of the data associated with a package is metadata, some is data and some could be a service.

1296821398000000 1314218701000000
#958 enhancement kindly ckan-v1.5 closed duplicate ui for new kind field

Ui for related new field in resource (#957)

1296822896000000 1320664462000000
#960 defect pudo rgrp ckan-v1.4-sprint-1 closed fixed Support non-ascii character in internal error messages

Just saw an error like the following which looks like it is to do with having a user object with non-ascii characters in it. Either get rid of 'cast' to str type here or do it in a unicode aware way.

Should do this not just here but everywhere we can find in the code base.

Module ckan.controllers.package:302 in edit
<<          am_authz = self.authorizer.am_authorized(c, model.Action.EDIT, pkg)
               if not am_authz:
                   abort(401, str(gettext('User %r not authorized to edit %s') % (c.user, id)))
               auth_for_change_state = self.authorizer.am_authorized(c, model.Action.CHANGE_STATE, pkg)
>>  abort(401, str(gettext('User %r not authorized to edit %s') % (c.user, id)))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)


Cost: 1h

1297069721000000 1297169056000000
#961 enhancement kindly rgrp ckan-v1.5 closed fixed [super] Refactoring of forms, validation and model synchronization

This is a meta-ticket to hold all of the work on refactoring forms, validation and model-synchronization in CKAN.

ckan-dev thread: http://lists.okfn.org/pipermail/ckan-dev/2011-January/000180.html

The Issue

From #926:

The current formalchemy setup conflates view, controller and model code in a way that makes it hard to debug and customise.

From http://lists.okfn.org/pipermail/ckan-dev/2011-January/000181.html:

... FormAlchemy, in retrospect, was probably a mistake as it merges too much model/validation/form generation into one thing.

At least 3 functions involved [in this area]:

  1. Generating (or just filling) a form template with 'form data' (and errors)
  2. Converting model data to form data (also happens for APIs in fact) -- let's call this 'dict-ization'
  3. Converting form data to model data (and validating) (inverse of previous step)

Related Tickets

  • #926 - Pick a simpler form framework
  • #1046 'dictization' and the logic layer - serialization / deserialization of package (and other domain objects) to standard intermediate format such as json-convertable python dict
    • #1079 Refactor API to use new logic layer and dictization
    • #1078 Refactor WUI controllers and forms to use logic layer
    • cf existing dumper and importer code
    • This will fix #662
  • [not ticketed yet] - validation layer (should work on serialized objects?)
  • #662 - Can't put entity that is returned by posting to package register (Defect)
  • #972 - Merge 'extras' into main package dict rather than having separate key
  • #1035 - Form impressions are given IDs
  • #810 - Move "add packages" field up in group form (easier to do this once forms are done)
1297069849000000 1310126100000000
#962 enhancement rgrp rgrp ckan-v1.4-sprint-5 closed fixed Improvements to data preview extension

Basic implementation done (and deployed):


However plenty to improve, e.g.

  • Support more formats (use external systems for preview?)
    • json (!)
    • html (trivial!)
    • sparql
    • ...
  • Do not display preview if no preview

Also suggest reworking to use external services rather than doing preview 'in house' (doing in house places heavy reliance on data proxy service and on converting data to a standard format).

1297072524000000 1301364987000000
#963 enhancement thejimmyg rgrp ckan-v1.4-sprint-1 closed fixed Package CKAN as a debian package

Package CKAN as a debian package so it is easier to install.

1297076364000000 1298284252000000
#964 enhancement rgrp rgrp datapkg-0.8 closed fixed Create Resource Downloader and make it pluggable

In downloading packages we need to download resources and we need a ResourceDownloader? object for this. these should be pluggable so that we can add support for different types of resources.

1297211004000000 1297211237000000
#965 enhancement kindly kindly ckan-v1.4 closed fixed make migration testing plugin to nose

In order to test migrations properly, we need to test the upgraded database against *all* the tests.

To do this a special flag to nosetests should be added. nosetests --migrations

This flag will make sure that the database is recreated from scratch each time using the migrate repository, bypassing init_db.

1297213492000000 1298280637000000
#971 enhancement sebbacon sebbacon closed duplicate Support middleware as a plugin

I need to completely munge the rendered data coming from the app. There's not currently a plugin hook for this, where I get access to the entire environment. The most generic form of plugin would be WSGI Middleware. I propose adding an IMiddleware plugin hook for inserting abritrary middleware via our plugin system.

1297277473000000 1299245064000000
#972 enhancement sebbacon rgrp ckan-v1.5 closed wontfix Merge 'extras' into main package dict rather than having separate key

Rather than have a separate 'extras' key all the extras fields should be consolidated into the main package dict when presenting the package internally or e.g. via the API.

Why? Extras are really just an artefact of our internal storage model. Clients of the system (both internal and external) should just see a set of key/values with no distinction between extras and non-extras.


  • Possible breaking change to the API (could enforce backwards compatibility by keepings extras for the time being)

Possible subticket of forms refactoring: #961

1297332282000000 1310134129000000
#973 enhancement rgrp rgrp ckan-v1.4-sprint-1 closed fixed Document license configuration and license system.

Document how license system works and specifically how licenses can be configured.

Cost: 2h

1297332948000000 1297678851000000
#974 defect dread ckan-v1.3 closed fixed Document site_description

Configuration option 'ckan.site_description' isn't documented

1297342254000000 1297342599000000
#975 defect rgrp rgrp ckan-v1.4-sprint-1 closed fixed Placeholder attribute not displayed in some browsers leading to poor UX

Currently placeholder text like 'Search ...' is not shown in some browsers (e.g. FF and IE).

This leads to poor UX, for example in top bar search where it unclear whether that box is for login or search.

We should fix this by finding a way to display placeholder attribute in all browsers.

1297343755000000 1297344448000000
#976 defect dread ckan-v1.3 closed fixed Documentation references knowledgeforge.net 1297346649000000 1297346954000000
#977 defect dread ckan-v1.3 closed fixed db_upgrade errors
  • db_upgrade (in ckan/model/init) makes call to validate_authorization_setup but forgets about rest of initialisation.
  • it uses self.metadata.bind.url when it should be self.metadata.bind it seems (confused by sqlmigrate update?)
1297420558000000 1297420742000000
#978 enhancement zephod dread ckan-sprint-2012-03-05 closed fixed Edit Resource including Extras in Web UI

Part of super-ticket #1506.

Follows on from #826.

We can configure some extra fields in resources and can edit them in the Web UI, but we can't create new columns in the Web UI. Update the WUI to handle tis.

1297429619000000 1330547181000000
#979 enhancement kindly dread ckan-backlog assigned Edit Resource extras in the API

Follows on from #826. We can now edit resource extras in the WUI (to some extent - see #978 for remaining issues) and we can view resource extras in the API, but we can't yet edit them in the API.

1297429777000000 1315222244000000
#980 defect pudo pudo ckan-v1.4-sprint-1 closed fixed Fixes to HR Infoshare

We've fixed a few things in CKAN that were discovered and tracked by HRI in their own issue tacker. This includes:

  • add authz checks to package edit links (cset:0752316cd2fe)
  • replace gettext with _ in controllers, to support unicode error msgs (cset:822340e6077e)
  • handle broken html in notes field without crashing on package read (cset:4b6be037dda0)
  • update i18n (cset:37d57dc3c492, cset:ea03173f5e77, cset:f16f4ee40fe7)
1297430918000000 1297503177000000
#981 defect kindly kindly ckan-v1.4-sprint-2 closed fixed fix authorization_group_user.id so it's added in migrate

This was missed out the migrate script. It should be to be added as a conditional script at the end of the repository so that databases can catch up.

1297434480000000 1298278654000000
#982 enhancement dread rgrp ckan-v1.4-sprint-2 closed fixed Remove various pip-requirements versions

We can just use the branch name to pull down specific versions of pip requirements so we don't need all the different versions in head.

Remove them and update any client systems.

Cost: 20m

1297516060000000 1298887980000000
#983 defect dread dread ckan-v1.3 closed fixed 'db upgrade' creates system priviledges

(should leave priviledges alone)

1297518265000000 1297773407000000
#984 defect kindly kindly ckan-v1.4-sprint-2 closed fixed Fix migration, where upgrade diverges from actual model.

The migration scripts have not been inline with the upgrade scripts. There are lots of discrepancies.

1297627447000000 1298280689000000
#985 enhancement pudo pudo pdeu-1 closed duplicate Harvest other data catalogues
  • data.london.gov.uk (1d, ping DERI)
  • catalogues from Paris and Vienna (1d)


  • digitaliser.dk (1d)
  • data-publica.com (2d)
  • data.suomi.fi (2d)
  • dati.piedmonte.it (1d)
  • portalu.de (2d)
1297679799000000 1306408134000000
#986 defect wwitzel3 thejimmyg ckan-v1.4-sprint-2 closed fixed Broken link report from the ckanext-qa code

Should have the following features:

  • A list of all packages with broken links on the resource
  • Under each packages a description of any resources with broken links to include:
    • Link
    • Description
    • Format
    • First found to be broken date

[If you can't do the last one yet, don't worry]

1297683994000000 1297812401000000
#987 defect pudo pudo closed duplicate Common harvesting framework

We are now harvesting metadata from other sources in various places around CKAN. Such harvesting can include:

  • CSW/WFS for INSPIRE/UKLII (yields CKAN packages)
  • Catalogue scraping for LOD2 experiments (yields RDF graphs)
  • Atom/DCat for LOD2 production (yields RDF graphs)
  • OAI-PMH for http://datadryad.org/ and other dspace (yields CKAN packages)

We should aim to consolidate the harvesting clients into a common system that is easy to extend when needed and can be re-used in different scenarios.

In general, such a system would have the following stages:

  • Source selection: find what to download/scrape/harvest/parse
  • Index retrieval (i.e. package index)
  • Item retrieval (i.e. package entity)
  • (Optional: Serialization)
  • Normalisation
  • Loading/Merging? into CKAN

Exisiting harvesters are at:

1297684756000000 1311177705000000
#988 task wwaites thejimmyg ckan-v1.4-sprint-2 closed fixed Get a better HTML view of a GEMINI document using XSL 1297685975000000 1299493598000000
#989 enhancement kindly pudo ckan-future new Extending the model from plugins

We need to support extending the model from plugins. This could involve:

  • Adding a plugin hook to extend the mapper
  • Adding an upgrade hook for plugin schema migrations
  • Documenting how this is to be done
  • Find a way to avoid conflicts
1297689724000000 1340034311000000
#990 defect rene.kapusta closed fixed umlauts in title / text

It's not possible to use umlauts (like "CKAN Österreich") in the title / text.

1297801036000000 1311180850000000
#991 defect dread ckan-v1.3 closed fixed Checkbox defaults to True

Form for new package has CheckboxExtraField? checked, when the value is False. (as used in ckanext-dgu package v3 form)

1298035175000000 1298037717000000
#992 defect rgrp rgrp ckan-v1.3 closed fixed Use X-forwarded-for whenever it is available to set remote IP address

At the moment we only use X-forwarded to determine remote user address when remote-addr header is However if a site is behind a cache remote_addr will always be the IP of the cache.

We should fix this by using x-forwarded-for header whenever available and only use remote-addr when it is not available.

Cost: 10m

1298058532000000 1298060474000000
#993 defect thejimmyg [email protected] ckan-v1.4-sprint-2 closed fixed Link on ckan.net footer needs changing

The link for About > 'Project Home Page' on the CKAN.net footer should be changed to point to:


The current link is broken.

Thanks, Jason

1298154006000000 1298373114000000
#994 defect kindly kindly ckan-v1.4-sprint-3 closed fixed document a new migration methodology

It is too easy to make a mistake with the migrations at the moment. A more systematic way of doing them is needed and this needs to be documented.

1298281192000000 1298912842000000
#995 defect kindly kindly closed wontfix consolidate caching

Try and minimise the amount of different caching methodologies, especially for api calls.

1298283499000000 1311179009000000
#996 task kindly kindly ckan-v1.4 closed fixed Run some basic load testing.

This will involve running a sample of real requests synchronously against real data.

1298283994000000 1300364398000000
#997 defect kindly kindly closed duplicate change create on cli to upgrade

When doing create on the cli upgrade should be run instead of create to make sure that every database is initiated in the same way.

1298287981000000 1298288665000000
#998 defect kindly kindly closed fixed change create on cli to upgrade

When doing create on the cli upgrade should be run instead of create to make sure that every database is initiated in the same way.

1298287981000000 1300364423000000
#999 enhancement rgrp dread ckan-v1.4-sprint-3 closed fixed Factor out ckan.net theme changes to a separate theme repo and apply

Rather than polluting the ckan core code base on ckan.net install with ckan.net specific changes these adaptations should be moved out into a dedicated ckan.net theme.

1298386729000000 1300707328000000
#1000 enhancement kindly rgrp ckan-v1.4-sprint-3 closed fixed Remove call timings code

Not used and extra overhead. Very useful to strip this out and low cost.

Places where it seems to be:

  • lib/base.py: c.time_call_started
  • ....
1298489643000000 1298912726000000
#1001 enhancement rgrp rgrp ckan-v1.4-sprint-4 closed fixed API should use normal user credentials if available

When using the API 'locally' i.e. from the CKAN instance (as would be the case with an ajax interface) the API, especially that allowing READ requests should use the normal user credentials if they are available prior to looking for an API key.

The key change appears to be to change _get_user_for_apikey method in lib/base.py BaseController? to check the c.user attribute (may wish to rename as the name may now be a bit misleading ...).

This is critical to incorporating any ajax editing into the frontend.

As part of this ticket we should do a general consolidation of the identification system in lib/base.py so that both api_key and normal user auth lead to the same set of auth-related objects being available (suggest c.user and c.userobj and c.author).

1298489705000000 1301310351000000
#1002 enhancement dread rgrp ckan-sprint-2011-12-05 closed fixed Remove changeset model code

This code is unused and tests are disabled i belive. I suggest stripping this out (can always be re-added in an extension later). If we remove we should probably delete the relevant tables (and so a migration is needed).

1298489773000000 1322515696000000
#1003 enhancement rgrp rgrp ckan-v1.4-sprint-3 closed fixed CKAN Javascript library and demonstration web interface

A plain javascript library for interfacing with CKAN would be very useful (why? see below!). It would also be nice to have a pure html + javascript web interface to CKAN both for its own sake and to act as a demonstrator for the library.


  • Development of bespoke interfaces -- much easier to edit html + javascript than to change ckan core
    • E.g. for specific communities e.g. geodata, science
    • Specialized tasks - multi-package editing
  • Very easy deployment and integration (e.g. can drop in to getthedata or other sites)
1298490086000000 1300100411000000
#1004 defect zephod dread ckan-backlog closed fixed Group creation instructions missing

Need instructions on the group page to tell people they need to login to create a group. Someone must have deleted this.

1298559144000000 1323195485000000
#1005 defect dread dread closed duplicate Editing in ckan missing publishers

Package edit form at hmg.ckan.net has no publishers any more.

1298561348000000 1300100085000000
#1006 enhancement kindly rgrp ckan-v1.4-sprint-4 closed fixed Deprecate stable branch

Now that we have release branches we should deprecate the stable branch (ie. make sure it is no longer a head and then do --close-branch and merge into default one last time).

Cost: 10m (giving high priority because of low cost)

(Assigning to dread as he has been managing the stable branch).

1298624996000000 1300372286000000
#1007 defect pudo pudo closed wontfix Backport WDMMG translations for extensions i18n

Needed to allow extensions to be internationalized in the code and in templates. Requires in-template i18n domain switching which is broken between Genshi and Pylons, thus needs this hackish class.

cf. https://bitbucket.org/okfn/wdmmg/src/198841e770e5/wdmmg/config/environment.py#cl-16

1298630621000000 1340626203000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Note: See TracReports for help on using and creating reports.