{22} Trac tickets (2647 matches)

Results (1901 - 2000 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#941 enhancement thejimmyg rgrp ckan-v1.5 closed wontfix Submit apps or ideas for apps related to datasets (extension)

The basic purpose of this extension is provide a way to record 'apps' (applications) that relate to datasets in CKAN. Both existing apps and ideas for apps should be permitted.

1296403099000000 1310134339000000
#492 story johnbywater ckan-v1.2 closed Submit harvest source create form response to the API 1282427042000000 1284493145000000
#791 story johnbywater johnbywater ckan-v1.3 closed duplicate Submit harvest source form with invalid CSW location 1289212751000000 1294409723000000
#547 story johnbywater johnbywater ckan-v1.2 closed Submit package create form response to the API 1283339487000000 1283458645000000
#2637 enhancement aron.carroll aron.carroll demo phase 2 closed fixed Submit search form when user changes sort order 1341835218000000 1342617226000000
#479 task dread johnbywater ckan-v1.2 closed fixed Suggest DGU presents database dumps through front end 1282324365000000 1288004211000000
#1144 enhancement timmcnamara ckan-backlog new Support DSPL

DSPL, the Dataset Publishing Language, is being promoted by Google for its "Google Public Data Explorer" system. It is an XML format with metadata.

The format is described on the developer docs ofthe Google Code site.

Google provides a Python script which reads CSV data and generates DSPL

Sample from http://code.google.com/apis/publicdata/docs/dspl_sample.html:

<?xml version="1.0" encoding="UTF-8"?>
<dspl xmlns="http://schemas.google.com/dspl/2010"

  <import namespace="http://www.google.com/publicdata/dataset/google/time"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/quantity"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/entity"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/geo"/>
      <value>My statistics</value>
      <value>Some very interesting statistics about countries</value>

      <value>Bureau of Statistics</value>

    <topic id="geography">
    <topic id="social_indicators">
        <name><value>Social indicators</value></name>
      <topic id="population_indicators">
          <name><value>Population indicators</value></name>
      <topic id="poverty_and_income">
          <name><value>Poverty & income</value></name>
      <topic id="health">

    <!-- As noted in the tutorial, this concept should extend quantity:amount.-->
    <concept id="population">
          <value>Size of the resident population.</value>
      <topic ref="population_indicators"/>
      <type ref="integer"/>

    <!-- This country concept is defined for educational purposes only. A country
    concept exists in the Google geo dataset. See:

    http://code.google.com/apis/publicdata/docs/canonical/geo.html --> 
    <concept id="country" extends="geo:location">
          <value>My list of countries</value>
      <type ref="string"/>
      <property id="name">
          <name><value xml:lang="en">Country name</value></name>
            <value xml:lang="en">The official name of the country</value>
        <type ref="string"/>
      <table ref="countries_table"/>

    <!-- This US state concept is defined for educational purposes only. A US state
      concept exists in the Google geo US dataset. See:

      http://code.google.com/apis/publicdata/docs/canonical/geo.us.html --> 
    <concept id="state" extends="geo:location">
          <value>US states</value>
      <type ref="string"/>
      <property concept="country" isParent="true"/>
      <table ref="states_table"/>

    <concept id="gender" extends="entity:entity">
          <value>Gender, Male or Female</value>
        <totalName><value>Both genders</value></totalName>
      <type ref="string"/>
      <table ref="genders_table"/>

    <concept id="unemployment_rate" extends="quantity:rate">
          <value>unemployment rate</value>
          <value>The percent of the labor force that is unemployed, not seasonally
      <topic ref="social_indicators"/>
      <type ref="float"/>
      <attribute id="is_percentage">
        <type ref="boolean"/>


    <slice id="countries_slice">
      <dimension concept="country"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <table ref="countries_slice_table"/>

    <slice id="states_slice">
      <dimension concept="state"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <metric concept="unemployment_rate"/>
      <table ref="states_slice_table"/>

    <slice id="countries_gender_slice">
      <dimension concept="country"/>
      <dimension concept="gender"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <table ref="countries_gender_slice_table"/>


    <table id="countries_table">
      <column id="country" type="string"/>
      <column id="name" type="string"/>
      <column id="latitude" type="float"/>
      <column id="longitude" type="float"/>
        <file format="csv" encoding="utf-8">countries.csv</file>

    <table id="countries_slice_table">
      <column id="country" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
        <file format="csv" encoding="utf-8">country_slice.csv</file>

    <table id="states_table">
      <column id="state" type="string"/>
      <column id="name" type="string"/>
      <column id="country" type="string">
      <column id="latitude" type="float"/>
      <column id="longitude" type="float"/>
        <file format="csv" encoding="utf-8">states.csv</file>

    <table id="states_slice_table">
      <column id="state" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
      <column id="unemployment_rate" type="float"/>
        <file format="csv" encoding="utf-8">state_slice.csv</file>

    <table id="genders_table">
      <column id="gender" type="string"/>
      <column id="name" type="string"/>
        <file format="csv" encoding="utf-8">genders.csv</file>

    <table id="countries_gender_slice_table">
      <column id="country" type="string"/>
      <column id="gender" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
        <file format="csv" encoding="utf-8">gender_country_slice.csv</file>

1305763609000000 1339774517000000
#693 requirement wwaites ckan-v1.2 closed fixed Support ETag in cache decorator

... and make cache-control header optional

1286831051000000 1286831188000000
#2889 enhancement seanh ckan-v1.9 new Support JSON values in extras, when returning JSON from API

For example, edit the warandpeace dataset and add an extra with key "comments" and value "[{u'date': u'Tue Aug 21 13:01:48 GMT 2012', u'comment': u'testing ckan', u'userId': u'11702'}, {u'date': u'Wed Aug 22 13:01:48 GMT 2012', u'comment': u'testing ckan again', u'userId': u'11702'}]", now visit API URLs such as:




The value of the extra comes out as a string in the dataset JSON, rather than having its contents unpacked into a list of JSON objects.

1345811737000000 1345811737000000
#1433 enhancement kindly rgrp ckan-sprint-2011-11-21 closed fixed Support SQLAlchemy 0.7

Why: current stable version of sqlalchemy. geoalchemy stuff required 0.7 and likely that some other things will require it soon.

Probably requires work on vdm https://bitbucket.org/okfn/vdm

NB: should have discussion before making 0.7 the default required version in CKAN core.

1320143453000000 1324472583000000
#1184 enhancement timmcnamara ckan-backlog new Support Wuala as CKAN storage option

Most of CKANs storage options are tied to the USA. This brings concerns of data security for some organisations who may wish to adopt the system. Wuala is a distributed file system that stores data in a peer-to-peer manner. The company behind it, LaCie? sells storage for a fee. However, they also enable clients to have 'free' storage space when machines act as a storage node.

In order to be a storage node, a machine needs to be online for more than 14% of the time - roughly 4h per day. Most CKAN servers are likely to have a far greater uptime than this.

Supporting Wuala would go some way to enabling CKAN to be used in a secure manner. That is, CKAN could be promoted for organisational use where there is lots of data to be stored and large geographic distances to be managed. There is a Python client available and a fairly long Google Tech Talk that overviews the system.

1308034751000000 1339774306000000
#563 requirement thejimmyg johnbywater closed duplicate Support a minimal CSW server interface or export to GeoNetwork 1284033576000000 1296592472000000
#82 enhancement rgrp rgrp closed fixed Support diffing of versioned objects

Should have function/facility to get a diff of a version object between 2 revisions. Function should return a dictionary of fields with diffs.


  • Do we deal with m2m relationships (and m2one on the many side)?
  • What is diff?
    • For text fields output of python diff command
    • For non-text field guess just simple +/- for what was there before and now

Cost: 4h (don't think this is a huge request)

1248289499000000 1256565441000000
#54 enhancement johnbywater rgrp v0.7 closed fixed Support dump and load of CKAN data to JSON

As a


I want to

Dump (serialize) CKAN data to a simple transport format (e.g. JSON) and be able to load it again.


  • Already have simple db dump. However dumping to JSON has various advantages, particularly where changes to the data need to be made upon reloading (e.g. during a migration).
  • Dump should include *all* CKAN data (i.e. all the data in the CKAN db tables)
1223907883000000 1230211256000000
#865 defect pudo pudo ckan-v1.3-sprint-1 closed fixed Support external strings in JS
  • Provide and include a "language.js" file.
1291801938000000 1295259773000000
#889 enhancement rgrp rgrp ckan-v1.3-sprint-2 closed fixed Support extra footer material in config option (e.g. for google analytics)

Add a config option 'template_footer_end' which is inserted in layout_template just before </body>.

This allows sysadmins to add extra items, especially, scripts directly into site without having to do any theming and is especially useful for things like google analytics.

Aside: going forward may want to turn this into a extension.

Cost: 1h

1293713189000000 1293715109000000
#245 enhancement rgrp rgrp closed duplicate Support for composite primary keys

Problem here is that foreign key then becomes "complicated" (composite).

  • Could also deprecate continuity_id field in favour of the basic foreign key on ie
1265882630000000 1297066620000000
#246 enhancement rgrp rgrp closed duplicate Support for primary key not named id

At the moment setting of continuity_id depends on base table pkcol being id. Should not be hard to change this -- and may get for free as part of ticket:245 (composite primary keys)

1265882862000000 1297066757000000
#81 enhancement rgrp rgrp vdm-0.7 closed fixed Support for sqlalchemy 0.5 1248285433000000 1268487327000000
#86 enhancement rgrp rgrp closed fixed Support for stateful dict-like collections

Already support stateful list-like collections and should extend this to dict-like collections.

(This is prerequisite for implementing versioned "extra" (key/value) attributes on packages in CKAN and elsewhere).

1248430798000000 1249050202000000
#3026 enhancement amercader amercader new Support icons on nav_named_link

nav_named_link won't include the nice icon even when passing the icon keyword param.

1354301516000000 1354301516000000
#971 enhancement sebbacon sebbacon closed duplicate Support middleware as a plugin

I need to completely munge the rendered data coming from the app. There's not currently a plugin hook for this, where I get access to the entire environment. The most generic form of plugin would be WSGI Middleware. I propose adding an IMiddleware plugin hook for inserting abritrary middleware via our plugin system.

1297277473000000 1299245064000000
#1498 enhancement amercader amercader ckan-sprint-2011-12-05 closed fixed Support multiple SOLR schema versions

At some point the SOLR schema used by CKAN will need to be changed and the changes won't be always backward compatible, so we need a mechanism to support instances that use older versions of the schema. This process will involve:

  • Better documentation on how to deploy SOLR, as single instance or multicore
  • A convention on how to tag schema versions and SOLR endpoints to ensure that CKAN instances point to the correct SOLR URL.
  • (If possible) Checks in CKAN core that deal with different schema versions.
1322585264000000 1323165876000000
#960 defect pudo rgrp ckan-v1.4-sprint-1 closed fixed Support non-ascii character in internal error messages

Just saw an error like the following which looks like it is to do with having a user object with non-ascii characters in it. Either get rid of 'cast' to str type here or do it in a unicode aware way.

Should do this not just here but everywhere we can find in the code base.

Module ckan.controllers.package:302 in edit
<<          am_authz = self.authorizer.am_authorized(c, model.Action.EDIT, pkg)
               if not am_authz:
                   abort(401, str(gettext('User %r not authorized to edit %s') % (c.user, id)))
               auth_for_change_state = self.authorizer.am_authorized(c, model.Action.CHANGE_STATE, pkg)
>>  abort(401, str(gettext('User %r not authorized to edit %s') % (c.user, id)))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)


Cost: 1h

1297069721000000 1297169056000000
#1365 defect dread dread ckan-sprint-2011-11-07 closed fixed Support old search formats

Lots of things rely on the old (i.e. pre-SOLR) search param formats, including ckanclient / DGU. Need a translation layer to convert these params (in api v1&2) to SOLR syntax.

This functionality was broken by the changes in #1277 which went into 1.5a

1317233338000000 1319815052000000
#1458 enhancement rgrp rgrp ckan-future assigned Support previewing kml files in data viewer

Super ticket: #1151 (viewing geo data)

E.g. preview of http://thedatahub.org/dataset/louisvillecrime should bring up a map

1320936488000000 1340632932000000
#876 enhancement sebbacon rgrp ckan-v1.3-sprint-2 closed fixed Support sqlite as a database backend for CKAN

Among other things this will allow the tests to run much quicker.

1292858711000000 1294753889000000
#1179 enhancement timmcnamara ckan-backlog new Support tag aliases

A small number of tags are near-duplicates of each other.

Perhaps we could support word stemming from NLTK and/or manual tag aliases:

statistics statistik ... survey surveying surveys

1307429221000000 1339774332000000
#1145 enhancement timmcnamara ckan-backlog new Support the Handle System

The Handle System is an initiative to provide persistent references for resources. That is, it's basically a proxy system for preventing link rot.

Its documentation is here: http://www.handle.net/. Servers running CKAN could host a "Local Handle Service", which redirects a hash of a resource to an actual URL.

Some suggested use cases:

  • Researcher would like to cite where data came from
  • Agencies would like to have a way to prevent vendor lock-in from CKAN if they decide to move to another platform
1305764775000000 1339774502000000
#1497 enhancement dread dread ckan-sprint-2011-12-05 closed fixed Suppress benign warnings

We really shouldn't have lots of benign stuff printed on stderr when we run every command. It confuses users and hides important stuff.

1322569640000000 1322569805000000
#2223 enhancement zephod rgrp ckan-sprint-2012-03-19 closed fixed Switch to bootstrap from blueprint


  • We already use quite a bit of boostrap
  • We already plan to use bootstrap JS
  • Bootstrap is excellent and pretty light
1331384974000000 1332842692000000
#1435 enhancement rgrp ckan-v1.6 closed wontfix Switch to continuous.io for buildbot (?) 1320145831000000 1323283538000000
#64 enhancement rgrp rgrp v0.10 closed fixed Switch to repoze.who for authentication

Switch to repoze.who + openid plugin for authentication from authkit.

Already did this in microfacts so should be fairly easy.

1245168853000000 1246127063000000
#102 enhancement rgrp rgrp closed fixed Switch to using UUIDs for revision ids

This is better for long term sustainability, especially if we want to "push and pull" from unrelated repositories in the future.


  • May wish to retain a number field updated in the usual incremental way (what's the cost/benefit here?)
1250068175000000 1260285104000000
#212 enhancement rgrp rgrp v0.11 closed fixed Switch to using m.okfn.org "kforge" media (css and behaviour)

We are already using standard kforge css though copied locally and with a few modifications.

We should switch to using common copy on m.okfn.org. Some work will be required to factor out any changes we have made.

Cost: 2h

1260703728000000 1260703969000000
#435 defect thejimmyg dread ckan-v1.4 closed wontfix Switching between ckan instances causes orphaned postgres processes

Analysis using pg_top reveals orphaed processes:


This followed switching ckan instances on the dgu machine. Do these hang around forever clogging up a db server? Can switching between instances be smoothed to avoid this?

1282147736000000 1298284084000000
#727 story pudo pudo iati-3 closed fixed Sysadmin can approve a publisher 1287584754000000 1289835196000000
#1044 defect pudo dread closed fixed Sysadmins locked-out of API without Right: (visitor, SITE_READ, System)

The problem is that in ckan/controllers/rest.py the BaseApiController? has this method:

    def __before__(self, action, **env):
        BaseController.__before__(self, action, **env)
        if not self.authorizer.am_authorized(c, model.Action.SITE_READ, model.System):
            abort(401, _('Not authorized to see this page'))

which works on the basis of your c.user, rather than your apikey. All API users are treated as visitors (since API users don't get a login cookie) and even a sysadmin's apikey is blocked unless there is a right for a Visitor to SITE_READ.

Also needs tests.

(Also, why is this restriction only on the API, package search, group index and tags and agroup index? I'm guessing SITE_READ is only for places where other authz don't apply, but maybe it should not be called 'SITE_READ' but 'OTHER_READ' or something?)

1300358919000000 1302096155000000
#2691 enhancement toby shevski demo phase 2 closed fixed Tag 404 error

404 Not Found The resource could not be found.

1342538490000000 1342604273000000
#1725 defect dread dread ckan-sprint-2012-02-06 closed fixed Tag autocompletion and commas don't mix

If you use tag autocompletion, it automatically adds a comma after the tag. This is fine if you are going to add another tag, but not fine if it is the last one. If you don't manually remove the final comma than you get form validation error "Tag string: Tag "" length is less than minimum 2"

Affects only version 1.5.2a.

1328006709000000 1328204489000000
#819 defect cygri ckan-backlog closed fixed Tag autocompletion widget broken

The widget for tag autocompletion is broken in various ways.

For example, if I edit a package that is tagged "music", and just tab through the form fields to get to the Author field, then the widget changes to "industrial-music" as I tab through it.

Or if the tag is "foo bar" and I hit alt-right to jump to the end of the text field (with the intent of adding a third tag), then the contents change to "foo barbecue".

1290004121000000 1323171435000000
#180 enhancement rgrp jwyg v0.11 closed duplicate Tag cloud as way to view CKAN tags

Create big tag cloud with all CKAN tags - perhaps weighting with size and colour...

1257534254000000 1265284374000000
#89 enhancement nickstenning rgrp closed fixed Tag cloud for package tags

Should not be hard to do (lots of existing libraries) but not sure that this is very important.

1248693939000000 1265892698000000
#1421 defect dread dread ckan-sprint-2011-10-28 closed fixed Tag cloud recipe broken

home page doesn't have c.facets setup.

Only affects 1.5 branch - ok Was broken in cset:05f60f2d26e8.

1319638575000000 1319639311000000
#2396 defect seanh seanh ckan-sprint-2012-05-29 closed fixed Tag list page loads very slowly 1337160153000000 1337268769000000
#309 defect pudo dread closed fixed Tag list shows unused tags

Deleted tags are visible on the /tag page and are searchable.

Delete tags are ones that were in use once and then discarded, so this includes ones which are not used on any packages any more, including misspelling, mistakes, and deprecated tags. It would makes sense to remove them.

Example tag that is viewed at /tag and searchable: http://ckan.net/tag/2rand[0

1273247648000000 1280743432000000
#161 defect dread rgrp v0.11 closed fixed Tag listing is incorrect

E.g. http://ckan.net/tag/read/postcodes this has no-postcode listed but when you look at that package 'postcodes' isn't a tag ...

I suspect this may be to do with search using deleted tags as I think no-postcode had postcodes as a tag at some point earlier ...

1256030925000000 1258573607000000
#1597 enhancement dread ckan-sprint-2012-01-23 closed fixed Tag search - filter by group

I want to browse tags, but filtered for a particular group. Currently our tag API doesn't allow for filtering by group.

This is important for improving groups as communities within a site #1521. It would be easy to do this by adding an option to filter by a group. BUT are there any other use cases that would warrant a more complete faceted tag search?


BTW I can currently draw a tag cloud for a group - I can get the top tags used in a group like this:

curl http://thedatahub.org/api/action/package_search -d '{"q":"groups:country-ca", "facet.field":"tags", "rows":"0"}'

but it only contains the top 20 tags.

1324550492000000 1326821156000000
#109 enhancement rgrp dread closed fixed Tag search in the REST API

Similar to ticket:108 but for tags:


For tags search is extremely simple since you can only search by name.

Possible Extras

  • Allow "exotic" ordering of results e.g. ordering by number of packages with that tag.
  • Return this number with tag list.
1251976297000000 1291829457000000
#1706 enhancement seanh seanh ckan-sprint-2012-02-06 closed duplicate Tag taxonomies package schema

Enable ckanext's to add new vocabularies to default_package_schema(), use the new vocabularies in templates for dataset view, create, edit pages, helper function for extensions to create a vocabulary schema for a given vocabulary name (must match a vocab already in the db) and add it to default package schema.

1327427938000000 1329131079000000
#1721 enhancement johnglover seanh ckan-sprint-2012-02-20 closed fixed Tag taxonomies table migration 1327944237000000 1329133348000000
#30 enhancement zool rgrp v0.3 closed fixed Tags are added to a package by package owner

As A

Site User

I Want To

Create/Edit/Update/Delete? tags associated with my package


  • Tags are service-wide and not package specific
  • Tags should not allow spaces (but allow everything else e.g. ':')
  • web interface details
    • should provide a text box where users can add space separated tags (a la delicious).
    • tag creation should be seemless (so if a tag does not already tag exists it should automatically be created in the background)
    • would like to provide a list of relevant existing tags. For time being could just list all existing tags below entry box but in long run would like auto-updating/auto-suggesting listing a la delicious
1155905753000000 1185472559000000
#2893 defect seanh ckan-v1.8.1 new Tags missing from form_to_db_package_schema()

With an IDatasetForm plugin active with a form_to_db_package_schema() method that returns a schema based on ckan/logic/schema.py:form_to_db_package_schema() I'm getting validation errors like this:

[ckan.controllers.api] Validation error: "{'type': 'Validation Error', 'junk': [u'The input field junk was not expected.']}"

They seem to be caused when a data dict with key 'tags' is passed to e.g. the package_update API, for example in ckan/tests/functional/api/model/test_vocabulary.py there are various tests that get this error when they call package_update (if you run the tests with a suitable IDatasetForm plugin active).

In ckan/logic/schema.py:form_to_db_package_schema() 'tags' is for some reason deliberately removed from the schema.

1346155864000000 1346155864000000
#162 enhancement dread rgrp v0.11 closed fixed Tags should be automatially lowere-cased if not already

Rather than raising a validation error automatically lower-case tags on submission.

Cost: 0.5h

1256048269000000 1256142462000000
#1427 enhancement dread dread ckan-sprint-2011-10-28 closed fixed Tags validation needs to limit length to 100 characters

The tag name is limited in length in the db table to 100 characters. If you try and create a longer one you get an exception. This should cause a ValidationError? and tell the user this sensibly.

1319708698000000 1319709925000000
#1483 defect johnglover johnglover ckan-sprint-2011-12-19 closed fixed Tasks sometimes fail to update task_status table when trying to report a failure

The error reports do not have a "value". The original task in the task status table is not made and it tries to make a new task, but can't.

Issue: why is the original task_status entry not made?

kindly: there is a chance of course that the task is quicker than the plugin to get to the point where it needs to update the table.

kindly: so we may have to find a way to update the task status before we run the task.

kindly: I think you can supply a task_id to celery. i.e we make our own guid as the task id. So we can use that one to populate the task status.

1322050605000000 1323279151000000
#424 task dread dread ckan-v1.2 closed fixed Tell JF the format spec & examples 1281436534000000 1282919429000000
#1349 enhancement zephod zephod ckan-sprint-2011-09-26 closed fixed Template tidying

Various things in templates can be tidied up after work has finished on #1294:

  • remove styling in page in group/index.html (put in css main)
    • ditto for package/search.html
    • ditto for home/index.html
  • Forms.css - to merge into style.css; remove explicit imports elsewhere
    • @import at the top of style.css @import url(forms.css)
  • Make a note: Success/failure controllers clash CSS with success/failure flash banners
1316171025000000 1317650050000000
#99 enhancement rgrp rgrp closed wontfix Test DbIndex with sqla 0.5

Test DbIndex? with sqla 0.5 (and get it working if it does not work).

Cost: ??

Priority: low because DbIndex? is not currently that important to us (also wonder whether we should try using CKAN code here to stop us reinventing the wheel).

1249985537000000 1297081088000000
#2537 enhancement seanh seanh ckanbuild accepted Test and document ckanbuild


Verify that what's there so far still works, write a README explaining how it works

1339775328000000 1340639830000000
#1674 enhancement ross ross ckan-sprint-2012-02-06 closed fixed Test auth logic hooks for publisher profile

Super #1669

Test all of the new auth logic hooks for publisher profile to make sure they work as expected. Also make sure that we have a test to compare the naming of the core auth funcs with the profile hooks (so we can see what may have changed in one but not the other).

1326802841000000 1328526677000000
#112 enhancement rgrp dread v0.11 closed wontfix Test db schema created via migration scripts

'paster db upgrade' should result in the same database as 'paster db create' on a clean system. In particular, all tests should pass.

Note: equivalent of paster db upgrade isn't being run from tests at the moment (we haven't been able to get this to work).

1252316495000000 1265294069000000
#381 defect johnbywater closed invalid Test defect

please ignore

1280345937000000 1294410466000000
#707 task pudo johnbywater iati-3 closed fixed Test for adding package to package group using the package form fieldset 1287404126000000 1289910513000000
#706 task pudo johnbywater iati-3 closed fixed Test for removing package from package group using the package form fieldset 1287404073000000 1289909028000000
#708 task pudo johnbywater iati-3 closed fixed Test for showing package groups on package read page 1287404169000000 1289910506000000
#868 enhancement sebbacon thejimmyg closed fixed Test improvements

The tests currently take 41 mins to run on most laptops. This slows down development and discourages a test-driven approach.

We'd like to see the tests pass in 5 mins or less (but anything would be an improvement!)

Some suggestions for achieving this include:

  • Upgrading the entire codebase to SQLAlchemy 0.6 so that tests could run against an in-memory SQLite database
  • Not setting up and tearing down the database so frequently.
1291987811000000 1294753596000000
#2599 enhancement johnmartin aron.carroll demo phase 4 closed fixed Test in IE

can we promote this to phase 3?

1340717662000000 1346756230000000
#2517 enhancement johnglover johnglover ckan-sprint-2012-06-25 closed fixed Test installation instructions for EC ODP on CentOS 6.2 1339581689000000 1340613194000000
#239 task dread dread v1.0 closed fixed Test migrate scripts

Integrate new test upgrade into testmigrate stuff.

New "migrate.ini" config file programmed with the db to be used for migrate tests and a param saying where the db dump is.

1264439926000000 1265286521000000
#2927 enhancement seanh ckan 2.0 new Test new package/group/organization type URLs support

IPackageForm, IGroupForm and IOrganizationForm allow you to define e.g. new package types with new top-level URLs, but the example extensions don't have tests for this yet.

1347552153000000 1347552153000000
#1168 enhancement thejimmyg dread ckan-backlog assigned Test system for deb packaging

Get buildbot to:

  • build the deb packages
  • install them into a fresh virtual machine
  • run smoke tests on the installed ckan
1306441994000000 1330990423000000
#1274 enhancement johnglover johnglover ckan-sprint-2011-10-28 closed fixed Testing solr search

Copy ckan core postgres package search tests to the ckanext-solr extension and update them so that they use the solr search backend.

1313413202000000 1313429049000000
#2221 enhancement rgrp zephod closed fixed Tests are broken for some of us: NotAPairTreeStoreException

Since my last pull I'm getting a strange new error which is presumably to do with my local config or assumptions about the storage extension.

Test output here: https://gist.github.com/2007985

1331318781000000 1331561095000000
#2222 enhancement rgrp zephod closed duplicate Tests are broken for some of us: NotAPairTreeStoreException

Since my last pull I'm getting a strange new error which is presumably to do with my local config or assumptions about the storage extension.

Test output here: https://gist.github.com/2007985

1331318783000000 1331318825000000
#2227 enhancement toby toby ckan-sprint-2012-03-19 closed wontfix Tests are slow and broken

not all tests run and they take too long :(

1331556570000000 1332843126000000
#857 defect dread closed invalid Tests for CLI

Test coverage for ckan.lib.cli is 34%. Need to improve that.

1291724131000000 1311182581000000
#859 defect pudo dread closed invalid Tests for User model

Test coverage of ckan.model.user is 53%. Notable exception - setting password. Room for improvement?

1291724348000000 1311177461000000
#2975 enhancement seanh ckan 2.0 new Tests for auth functions and new actions

From Toby: tests around the auth functions and new actions - probably needs an improved testing mechanism and things like test data creation on via actions

1350297660000000 1350297660000000
#854 defect johnlawrenceaspden dread ckan-v1.4-sprint-6 closed fixed Tests for authorization_group

The coverage tool (run by buildbot in the ckan build) reports that only 24% of lines of ckan.controllers.authorization_group are run in tests and 38% of ckan.forms.authorization_group. This suggests a need for more tests.

1291723815000000 1304351843000000
#856 defect wwaites dread closed invalid Tests for caching

Test coverage for ckan.lib.cache is only 40%. Can that be usefully improved?

1291724064000000 1311177085000000
#858 defect dread dread closed fixed Tests for diff

Test coverage for ckan.lib.diff is 24%. Need to improve

1291724171000000 1291734297000000
#769 task johnbywater johnbywater ckan-v1.3 closed duplicate Tests for harvesting metadata from ArcGIS CSW API 1288178179000000 1294409318000000
#2978 enhancement seanh ckan 2.0 new Tests for permissions for organizations and groups

From Toby: checking the logic around the who can do what from the user stories - via some tests may be a good approach

1350297806000000 1350297806000000
#1306 enhancement kindly rgrp ckan-backlog closed wontfix Tests for the logic layer

AFAICT there are no tests for the logic layer at the moment. I imagine this is an issue (if it is not please explain and close as wontfix).

1315153267000000 1319797204000000
#2610 enhancement aron.carroll shevski demo phase 1 closed duplicate Text / link changes

In the footer: Change 'Powered by CityData?' to Powered by DataSuite? (with link to ckan.org/datasuite)

Links in left hand side of footer should change to a link to ckan.org, link to OKFN, link to ckan.org/datasuite/about and link to Pricing Page (on ckan.org .. will be /datasuite/solutions)

In create dataset form, prefilled tag examples should include one example with a space, e.g. "mental health"

1340810738000000 1341501424000000
#2611 enhancement aron.carroll shevski demo phase 1 closed fixed Text / link changes

In the footer: Change 'Powered by CityData?' to Powered by DataSuite? (with link to ckan.org/datasuite)

Links in left hand side of footer should change to a link to ckan.org, link to OKFN, link to ckan.org/datasuite/about and link to Pricing Page (on ckan.org .. will be /datasuite/solutions)

In create dataset form, prefilled tag examples should include one example with a space, e.g. "mental health"

Change 'Groups' to 'Publishers' everywhere

Homepage copy/images (Ira to supply?)

1340810740000000 1342008509000000
#504 requirement dread rgrp ckan-v1.3 closed invalid The DGU fieldset shall conform to v3 of DGU metadata specification

No need for requirement - closing in favour of #505

1282660246000000 1292586686000000
#715 defect rgrp johnbywater ckan-v1.4-sprint-2 closed fixed The group list page shall show the group title *or* group name

Symptom #1: Groups without titles don't get a clickable link, because there is no text. Defaulting to the group name would solve this trouble. Symptom #2: Package groups drop-down list show's 'None' when a group doesn't have a title.

Therefore, we need to centralize presentation of group title, so it defers to group name. And we need tests.

Also please sort groups by title on group list page.

1287583089000000 1297796784000000
#789 requirement wwaites johnbywater ckan-v1.3 closed duplicate The harvest source form shall validate location field by positive identification of source type 1289212106000000 1296593257000000
#533 requirement johnbywater ckan-v1.3 closed invalid The performance of CKAN shall scale

CKAN uses a database. We can put the database on another machine and then clone the CKAN machine. Then there are two machines calling the database and converting SQL results into HTTP responses.

How many machines can you have before the database becomes the bottleneck? At that point, what QoS can be obtained with normal hardware?

Afterwards, can CKAN be made to scale further than this with a little development? Can its Postgres database be mirrored in realtime? Or can we change SQL writes (create/updates) to write to many databases?

Are there any other options for scaling the persistence mechanism?

1282910499000000 1292957374000000
#348 enhancement johnbywater johnbywater closed fixed The system shall accept package edit form submissions in an API 1276523793000000 1277820496000000
#237 enhancement johnbywater johnbywater closed fixed The system shall accept submissions from authenticated owners of activated accounts using the data openness enquiry form by presenting the enquiry summary and prompting for confirmation 1264157377000000 1264157420000000
#238 enhancement johnbywater johnbywater closed invalid The system shall accept submissions from unauthenticated users of the data openness enquiry form by creating an anonymous pending enquiry action, and by redirecting the user to login form, passing a reference to the pending action 1264157814000000 1266509806000000
#539 requirement wwaites rgrp ckan-v1.4 closed invalid The system shall improve performance 1283242447000000 1303118486000000
#697 requirement pudo johnbywater iati-3 closed fixed The system shall indicate which package groups can be removed when editing a package

This seems to require doing something else with such a group's 'delete' button, such as showing it greyed, or perhaps not showing it at all.

1287066103000000 1291644457000000
#371 requirement nils.toedtmann johnbywater ckan-v1.4 closed fixed The system shall monitor QoS against SLA

Requested by DGU.

1279885622000000 1300217820000000
#575 requirement johnbywater ckan-v1.2 closed fixed The system shall notify remote metadata harvesting errors 1284047959000000 1288012749000000
#347 enhancement johnbywater johnbywater closed fixed The system shall present the package edit form in an API 1276523692000000 1277820440000000
#236 enhancement johnbywater johnbywater closed fixed The system shall present to all users a form for starting data openness enquiries 1264157100000000 1264157324000000
Note: See TracReports for help on using and creating reports.