{22} Trac tickets (2647 matches)

Results (1901 - 2000 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#2633 enhancement toby toby demo phase 2 closed fixed for dev always look for templates

so we find newly created templates

1341828058000000 1341830278000000
#2634 enhancement toby toby ckan-v1.8 closed fixed change log - helper function limited in 1.8

make sure documented

1341828248000000 1342085656000000
#2635 enhancement dread new Non-destructive SOLR reindex

You can't run the search-index reindex on a live server because it will give us bad results for 2 to 3 hours while it runs. Can there be an option that doesn't delete the entire index at the start?

Instead it could just delete any items that don't exist any more, then delete them and regenerate them one by one. So the total number of datasets doesn't change much.

1341829394000000 1341829394000000
#2636 enhancement aron.carroll aron.carroll demo phase 2 closed duplicate Style activity stream in user profile 1341835166000000 1342090614000000
#2637 enhancement aron.carroll aron.carroll demo phase 2 closed fixed Submit search form when user changes sort order 1341835218000000 1342617226000000
#2638 enhancement toby toby ckan-v1.8 closed fixed add a get from request param helper function 1341844597000000 1342112772000000
#2639 enhancement amercader amercader ckan-v1.8 closed fixed Update recline version for 1.8 release

New recline version has significant improvements like the SlickGrid? based grid view, it'd be good to have it on 1.8.

1341845809000000 1350296205000000
#2640 enhancement amercader amercader demo phase 2 closed fixed Update recline on the demo site

Depends on #2639

1341845911000000 1342112730000000
#2641 enhancement johnmartin amercader demo phase 5 assigned Adapt spatial widgets to new theme

Dataset extent map and spatial filter need to adapted to the new theme, as they are not showing up now

1341846147000000 1352658854000000
#2642 enhancement shevski amercader demo phase 3 closed fixed Adapt harvest frontend to new theme 1341846196000000 1345113645000000
#2643 enhancement aron.carroll aron.carroll demo phase 2 closed fixed Demo homepage redesign



1341848685000000 1342617247000000
#2644 enhancement shevski toby demo phase 5 assigned user dashboard for demo theme

we now have a user dashboard that needs theming not sure if we need sam to look at it


1341910821000000 1344255836000000
#2645 enhancement johnglover johnglover ckan-ecportal closed fixed Update server setup to use local files

Server setup scripts need to have two distinct steps:

1) Download all files that aren't in RHEL repositories, create RPMs where necessary (python virtualenv).

2) Install from local files.

1341916281000000 1341996302000000
#2646 enhancement toby toby demo phase 2 closed fixed delete dataset

make it so

1341920466000000 1342014170000000
#2647 enhancement aron.carroll toby demo phase 5 closed fixed api bullshit button

make it as a snippet calling thing like aron would love

1341920530000000 1343229185000000
#2648 enhancement shevski aron.carroll demo phase 2 closed fixed Add file upload to the demo site 1341921081000000 1342714659000000
#2649 enhancement ross ross ckan-v1.8 closed fixed Description on resources not rendered as markdown

When a resource doesn't have a description, it is rendered as markdown from the description of the dataset.

If the resource does have a description it isn't rendered as markdown ..

See http://thedatahub.org/dataset/wikipedia-e3-timestamp-position-modification/resource/d883ab44-07f4-4992-800a-3e4bf5d53a96

1341923135000000 1342084280000000
#2650 enhancement ross ross ckan-v1.8 closed fixed Description on resources not rendered as markdown

When a resource doesn't have a description, it is rendered as markdown from the description of the dataset.

If the resource does have a description it isn't rendered as markdown ..

See http://thedatahub.org/dataset/wikipedia-e3-timestamp-position-modification/resource/d883ab44-07f4-4992-800a-3e4bf5d53a96

1341923137000000 1341943716000000
#2651 enhancement icmurray ross ckan-v1.8 closed fixed Check support for TSV which doesn't appear to work well.

TSV support doesn't seem to work very well, may be the mimetype ( text/tab-separated-values )

See http://thedatahub.org/dataset/wikipedia-e3-timestamp-position-modification/resource/d883ab44-07f4-4992-800a-3e4bf5d53a96

1341923318000000 1343209784000000
#2652 enhancement toby toby demo phase 3 closed fixed get qa working with the demo 1341937315000000 1345106904000000
#2653 enhancement toby aron.carroll demo phase 2 closed fixed Requesting file upload authentication fails on demo

Try: http://localhost:5000/api/storage/auth/form/photo-of-a-duck


File '<string>', line 2 in auth_form
File '/Users/Aron/.virtualenvs/ckan/src/ckan/ckan/lib/jsonp.py', line 26 in jsonpify
  data = func(*args, **kwargs)
File '/Users/Aron/.virtualenvs/ckan/src/ckan/ckan/controllers/storage.py', line 407 in auth_form
  authorize(method, bucket, label, c.userobj, self.ofs)
File '/Users/Aron/.virtualenvs/ckan/src/ckan/ckan/controllers/storage.py', line 200 in ofs
  StorageAPIController._ofs_impl = get_ofs()
File '/Users/Aron/.virtualenvs/ckan/src/ckan/ckan/controllers/storage.py', line 71 in get_ofs
  storage_backend = config['ofs.impl']
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/paste/registry.py', line 146 in __getitem__
  return self._current_obj()[key]
KeyError: 'ofs.impl'
1341938467000000 1342005167000000
#2654 enhancement ross ckan 2.0 assigned UI support for ordering groups on group_read page

The group_index page has no support in WUI for ordering the groups displayed. Should allow sorting by name

Add support for this for datahub now, and discuss for new 1.9 UI

1341943891000000 1346662156000000
#2657 enhancement toby shevski demo phase 2 closed fixed hide history button on group pages until retheme

History button in top right of http://demo.ckan.org/group/data-explorer goes to http://demo.ckan.org/group/history/data-explorer and needs hiding

1342007213000000 1342008765000000
#2658 enhancement johnmartin aron.carroll demo phase 5 closed invalid On Demo change 'Groups' to 'Publishers' everywhere

Can we do this as a localisation?

1342008495000000 1352205894000000
#2661 enhancement aron.carroll toby demo phase 2 closed fixed delete dataset/group javascript confirm

in 2375-demo-theme-development

templates/package/confirm_delete.html needs to be made to look nice note some writing in black

delete button needs nice home on dataset/edit

you can add js support add ?confirm=yes to url on dataset button to force delete

Please note delete is not yet implemented - but will be in the next few hours

1342012395000000 1342716070000000
#2663 enhancement toby toby ckan-v1.9 new h.resource_display_name needs love

This function is shit and needs cleaning up and a doc string

description is markdown and should be treated properly

either we should truncate all or leave it to the templates but work universally

url if no name / desc this is in demo-theme branch

1342017746000000 1342017746000000
#2664 enhancement toby toby demo phase 2 closed fixed tags with : cause crazy error message on add dataset in demo 1342018503000000 1342632478000000
#2665 enhancement aron.carroll toby demo phase 2 closed fixed add dataset stages need to be form buttonable

the stages snippet needs to be inside <form> so I can make some links buttons - eg stage 1 -> stage 2 I want to save the dataset not just move to add resource

if you can just move it within the form html leaving everything else the same - I can then add buttons and pass back for styling

1342023971000000 1342088650000000
#2666 enhancement aron.carroll toby closed fixed templates/package/resource_edit.html

There is a new template for editing resources outside of the add dataset logic

this is the template it just throws some crap onto the page at the moment - we need to create a form - the form will end up with the usual data/edit stuff but for now you can use the resource to populate the form and I can then change it to use data - I just want to know which fields we'll have


when done reassign to me and I'll get it functioning

1342084373000000 1343220176000000
#2667 enhancement shevski toby demo phase 2 closed fixed edit resource needs form

I thought I'd made this ticket before but can't find it. When we have a fully created dataset we need to be able to edit the resources.

template: package/resource_edit.html

when you've made a form reassign to me and I'll get it working

1342170103000000 1343058578000000
#2669 enhancement aron.carroll toby demo phase 5 closed fixed style [Draft]

I've added [Draft] to partial completed datasets in the package list and read commit 6a8e74d0



do you want to style them in some way?

1342174110000000 1343907572000000
#2671 enhancement toby toby ckan-v1.9 closed fixed fix allow_partial_update

need to be able to delete stuff explicitly

1342183618000000 1345209579000000
#2672 enhancement icmurray icmurray ckan-v1.8 closed fixed Session.is_modified should use passive=True

According to sqlalchemy docs [1], when calling Session.is_modified(), the passive argument needs to be passed in as True.

[1] http://docs.sqlalchemy.org/en/rel_0_7/orm/session.html?highlight=is_modified#sqlalchemy.orm.session.Session.is_modified

1342185523000000 1343124666000000
#2673 enhancement rgrp new simplify set of options for resources

Far too many resource options. Lets restrict back to data file and API. Visualizations etc can either get linked in description or in the Related items.

1342300559000000 1342300559000000
#2676 enhancement johnmartin shevski demo phase 5 closed fixed no way to browse through related items on dataset page

Need to add JS (presumably?) to the related app box on dataset page: http://s031.okserver.org:2375/dataset/gold-prices to allow user to scroll through when multiple related items exist

1342435678000000 1352205920000000
#2677 enhancement aron.carroll shevski demo phase 2 closed fixed related item links should open in new tab

Clicking on the blue buttons or images on related page http://s031.okserver.org:2375/dataset/gold-prices/related should take you to a new tab instead of navigating you away

1342435923000000 1342461576000000
#2678 enhancement toby shevski demo phase 2 closed fixed No UI for deleting a related item

Needs to be a way for admins (owners of dataset) to delete related items. I suggest adding a button to the edit page


1342436087000000 1342685932000000
#2679 enhancement icmurray icmurray ckan-v1.9 new Change default behaviour of TemplateController.view to 404.

The current behaviour of TemplateController?.view() (which is the fallback controller should all others fail) is to attempt to render (as a genshi template) the requested file.

Although this may be a feature that some instances want. In general, it leads to:

  • 500s when attempting to access a normal template (eg - http://datahub.io/importer/preview)
  • A way of inadvertantly serving things you may not want to serve. (Small risk, as it needs to be renderable as a genshi template).


  • Change the controller to 404
  • Ensure there's a way for existing ckan instances to override that behaviour should they need it.
1342436133000000 1342436133000000
#2680 enhancement shevski shevski demo phase 3 closed fixed review group deletion

Super admins & owners/creators of groups need a way to delete them

Is the auth model in place for all this stuff? May be worth sitting down and going through how this should work

Currently can change state from active to deleted - this is not ery user friendly (want a button with a 2 step process maybe?) & I'd like to know who can view this option? Everyone who can edit a group? Who can edit?

1342436525000000 1344509493000000
#2681 enhancement aron.carroll shevski demo phase 2 closed duplicate autofill on resource format

This is pretty key for standardization.

Also would like us to change the grey text to be: e.g. CSV, JSON, [more examples... i have no idea what they key ones are tbh]

1342440406000000 1343135874000000
#2683 enhancement seanh new Add no-cache header to _tracking API call's response to make sure it doesn't get cached 1342446577000000 1342446577000000
#2684 enhancement aron.carroll toby demo phase 2 closed fixed .js leaving forms warning

I think it would be nice to have all form pages monitor their forms - excluding search ones - and if a field has changed then warn the user that they have not saved the form data but are leaving the page.

This would help with things like changing resources on the add dataset resource form etc.

1342510477000000 1342539637000000
#2685 enhancement toby toby demo phase 2 closed fixed delete buttons should be form buttons

stop bots/browsers etc following links and auto deleting stuff

1342510565000000 1342514443000000
#2688 enhancement ross new Allow ordering of groups in WUI

Currently the group_index page just shows the entire list of groups, forcing the ordering to be by name. It would be better if it could be sortable by name (or reversed) or by package_count (or reversed)

1342520875000000 1342520875000000
#2689 enhancement aron.carroll shevski demo phase 2 closed fixed clicking on 'save & add another' doesn't act as expected

Clicking on 'save & add another' button here http://s031.okserver.org:2375/dataset/new_resource/yo takes you to step 3 instead of leaving you on step 2 when you haven't added any data.

Instead it should show an alert/ message that no data has been added & that you need to link to a file or upload something

Also, can upload be option 2 instead of 3?

1342538126000000 1342617293000000
#2690 enhancement aron.carroll shevski demo phase 2 closed fixed long URL misaligns edit button for slug

See attachment. Just need to amke sure that there's enough spacing around the edit button and that it lines up so it doesn't look too wierd

1342538301000000 1342542394000000
#2691 enhancement toby shevski demo phase 2 closed fixed Tag 404 error

404 Not Found The resource could not be found.

1342538490000000 1342604273000000
#2693 enhancement seanh toby ckan-v1.8 closed fixed License is out of date

https://github.com/okfn/ckan/blob/master/LICENSE.txt mentions requires/lucid_conflict.txt which is no longer the case for 1.8 and needs updating

not sure where requirements-testing/docs fit in

@sean I've assigned this to you as you changed the requirements and my plate is full feel free to pass on to someone else

1342605447000000 1343148356000000
#2695 enhancement aron.carroll shevski demo phase 2 closed fixed autocomplete for tags

in dataset creation / edit form

1342619891000000 1343138240000000
#2696 enhancement aron.carroll shevski demo phase 2 closed fixed autocomplete for resource format

in create/edit dataset forms

1342619938000000 1343138454000000
#2697 enhancement johnmartin shevski demo phase 5 assigned create dataset validation

Includes: missing fields, existing field checks (i.e. whether a name/dataset already exists with that name) during input (i.e. no need to submit form to check)

1342620035000000 1346235925000000
#2698 enhancement toby shevski demo phase 4 assigned markdown preview

for description / other fields with markdown support

1342620085000000 1344543252000000
#2699 enhancement shevski shevski demo phase 5 assigned workflow for associating datasets with groups

needs review & speccing out e.g. datasets created by a user who belongs to a certain publisher (group) get auto added to this group

1342620176000000 1344507133000000
#2700 enhancement aron.carroll shevski demo phase 3 closed fixed language box

lang box improvements: styling, images, submit upon selection

1342620223000000 1344505614000000
#2701 enhancement toby shevski demo phase 3 closed fixed add comments to datasets


1342620406000000 1343949913000000
#2702 enhancement shevski shevski demo phase 5 assigned Future Javascript wishlist for demo

tooltip on popular datasets with number of views facets to update automatically creating a dataset without reloading page between steps hover on licences information autocomplete on search terms group filtering social share buttons in lightboxes dataset counts on homepage

1342620475000000 1344255984000000
#2703 enhancement toby shevski demo phase 2 closed fixed add new resource to existing dataset

with add button on dataset page and in 'data & resource' section when there's no data

1342620885000000 1342687332000000
#2704 enhancement johnmartin shevski demo phase 5 closed fixed add list of resources to edit dataset page sidebar

when editing a dataset (e.g. here: http://s031.okserver.org:2375/en/dataset/edit/afghanistan-election-data) add sidebar list of resources for user to click & browse through taking you to the relevant resource edit pages

1342620986000000 1352205939000000
#2705 enhancement johnmartin shevski demo phase 5 closed fixed add link to edit dataset page from edit review page

e.g. on http://s031.okserver.org:2375/en/dataset/afghanistan-election-data/resource_edit/f6331f99-51f6-44d9-95b9-b20f3b74f360

& ensure consistant with sidebar of edit dataset page

1342621122000000 1352205954000000
#2706 enhancement aron.carroll shevski demo phase 2 closed fixed hide all prefilled fileds from resource edit form page

i.e. mimetype, size, last updated etc

1342621381000000 1342707718000000
#2707 enhancement toby shevski demo phase 3 closed fixed Tidy up additional info boxes on dataset and resource pages

This currently looks terrible. Some simple things we should do to improve this.

  • Localise and captitalise known (non custom) keys into human titles. eg. "size", "created_at", "mime_type".
  • Apply units to values of known keys where possible eg. "size in appropiate units"
  • Convert dates into a more human friendly format. We can avoid localisation hell by using an ISO format like YYYY/MM/DD hh:mm:ss which is still better than ISO8061
  • Move the non user fields to the top of the table.
1342621471000000 1344504846000000
#2708 enhancement kindly toby ckan-v1.9 new limit extra data for package/group show

contextpackage_limits? = { 'tags': 5, <- get first 5

'extras': 0, <- get all


only get what you ask for have to be explicit

contextgroup_limits? = {} only main item

start with datasets/groups expand if we like it

1342622420000000 1342622420000000
#2709 enhancement icmurray markw new Atom feeds are undocumented

There doesn't seem to be any documentation yet for Atom feeds.

1342624310000000 1342626212000000
#2710 enhancement tobes toby demo phase 3 closed fixed make the api_info.html not look like shit

you know the score

1342624385000000 1343318314000000
#2711 enhancement johnmartin toby demo phase 5 closed fixed bin/less dies for toby

it would be nice if this didn't fall over when the dir is removed maybe a linux only issue?

1342631508000000 1352205968000000
#2712 enhancement toby toby demo phase 2 closed fixed delete resource functionality needed

we need a delete_resource logic action and enable for resources

1342680517000000 1342690104000000
#2713 enhancement aron.carroll toby demo phase 3 closed fixed button colour not set properly in variables.css

to reproduce run

paster color fuchsia -c development.ini

then go to http://localhost:5000/dataset/new

Add button is blue gradient at the top this should be overridden in variables.less

when added send this ticket to me and i'll update the color command to change it

1342706428000000 1343656865000000
#2715 enhancement toby shevski demo phase 2 closed fixed inactive sidebar widgets

hide trackbacks & subscribe wboxes on dataset pages until we add disqus

1342714073000000 1342716931000000
#2716 enhancement toby shevski demo phase 2 closed fixed truncate long resource names


1342714182000000 1342716474000000
#2717 enhancement shevski toby demo phase 3 closed fixed French breaks the demo header

when language is french

a) the menu overlaps the logo

b) much of the logo becomes unclickable


a) seems like you b) is aron

1342780487000000 1344359846000000
#2718 enhancement toby shevski demo phase 4 new can't add dataset to more than one group

trying to add a dataset to another group means it's no longer part of the first group


1342780550000000 1344544203000000
#2720 enhancement aron.carroll shevski demo phase 2 closed fixed no padding around gravitars on all users page


needs some spacing between the image & user name

We could also display some stats here (e.g. number of dataset created, timestamp of last activity)

1342948111000000 1343219999000000
#2724 enhancement toby shevski demo phase 3 closed fixed Inconsistent labelling / UX on draft datasets
  1. One of the datasets here: http://s031.okserver.org:2375/en/user/shevski has [DRAFT] after it. It's not clear why this has been added -> user confusion
  2. The draft dataset is not clickable in the dataset list
  3. The ff dataset (same as the ff [DRAFT] dataset) in the Activity Steam section a) is clickable and b) does not have [DRAFT] after it
1342949402000000 1343319009000000
#2725 enhancement toby shevski demo phase 5 new Case sensitivity on tags

My feeling is that 'country-US' and 'country-us' should be the same tag. However currently tags with caps are treated differently

see http://s031.okserver.org:2375/en/dataset/test-dataset

with TEST and test - there also get indexed twice in the search page

1342949667000000 1343030773000000
#2726 enhancement toby shevski demo phase 5 new confusing logic on data preview formats
  1. If a user enters the wrong format on a file that can be previewed - it simply won't be previewed (e.g. a CSV or XML file that can be filled in with JSON in format will just not work or check this
  1. If I incorrectly edit format to one that data preview will try to preview it will work even for a format that it doesn't accept (sometimes) e.g. this PDF file I changed the metadata to HTML http://s031.okserver.org:2375/en/dataset/test-dataset/resource/9d27a9d9-36ec-460e-9edb-6dff7ba4fc28
1342949927000000 1343030906000000
#2727 enhancement toby shevski demo phase 2 closed fixed long resource name breaks sidebar navigation box

see http://s031.okserver.org:2375/en/dataset/test-dataset/resource_edit/9d27a9d9-36ec-460e-9edb-6dff7ba4fc28

& image attached

1342950093000000 1343053760000000
#2729 enhancement kindly shevski ckan-backlog new searching for tags:[tag] works but tag:[tag] doesn't

which is confusing since you can only search for one tag like this at a time. I.e. tags:economics,cvs or tags:economics, csv or tags:economics+CSV doesn't work for example; therefore tag:economics, should also work!


1342951109000000 1342951176000000
#2731 enhancement markw new Some sites permanently 'down for maintenance'

A large number of XXX.ckan.net sites give the following message:

"This Site is Down for Maintenance We apologize for the inconvenience. ~ The Open Knowledge Foundation sysadmins."

The message is unhelpful and patently false - the sites do not exist. Some of them were supposed to have been redirected to a relevant group at thedatahub.org in this ticket (now closed):


However, the redirection only seems to have worked in one case, http://si.ckan.net.

The problem still affects the following sites - the first 4 of which have supposedly been merged:

Please sort this out by redirecting, removing the sites, giving a more helpful (and accurate) failure message, etc, as appropriate.

1343045168000000 1343051608000000
#2732 enhancement ross ckan-backlog assigned New file upload functionality

We should simplify upload and storage of files, initially only to local storage with archiver eventually being fixed to archive data externally. WIP pad is http://ckan.okfnpad.org/uploads

Simplifying uploads

Currently uploads are too painful/difficult/fiddly to use and/or configure. We want to simplify uploads so that they are done directly to the CKAN server, without support for remote services (S3 etc) and/or the dependencies it introduces.

We want to fix:

  • File uploads themselves
  • Storage of uploaded files
  • Notification of the upload to other components

File uploads

Things file upload should do:

  • Allow sysadmin to disable
  • Allow auth'ed users to upload
  • Store whatever they send on disk, and store DB entry linking the file to the person
  • When creating the resource, the user should be able to choose from all of the files they have uploaded but not yet associated with a resource. This will allow for bulk upload and then a delayed association. Whenver a user creates a resource they either upload a file now, or see previously uploaded files.
Can we do the upload asynchronously and then associate the 
uploaded key with the resource before the save ? What happens 
if the user tries to submit before asymc upload finishes ? Should 
we delay them?

The upload workflow should look like...

  1. File upload should be a straightforward file upload with normal auth checks and normal processing of the posted data.
    1. When called via ajax then the ID of the newly created file should be returned,
    2. When called via WUI then it should also be given the url to redirect to after the file upload has been handled - the id will be passed as a query param.
  2. The resource save should check whether it has a file id and in that case updates the file object to point to the resource.

This should enable:

  • Separate file upload into a user's temporary store, either individually or as a batch.
  • Creating resources and simply choosing from previously uploaded, unassigned files
  • Adding files/data to a resource after the fact.

File storage

File storage should be local to the CKAN install, and not a remote service. Any archiving to remove storage providers should be outside of the main request.

File storage should:

  • allow moving data, a sysadmin should be able to move the storage root and change configuration and have the system continue running (i.e. don't store absolute paths).
  • provide maintainability, it should be easy to determine which old files are not associated with resources and thus can be cleaned up.
  • allow for collection of information (i.e. estimate of storate space used)
  • check whether there is enough space and handling the conequences cleanly
  • ensure files to be written only underneath its own root folder, checks should be made after any path generation that the file begins with the location of the file storage.
  • Have a configurable maximum accepted blob size during upload.
  • Should store what meta-data was provided with the upload, such as mimetype.

Somewhere in the DB we should store ...

idAn identifier
ownerThe owning user, who uploaded the file
pathThe path (from the 'storage root') to the file
sizeThe size in bytes of the file on disk
mimetypeThe mimetype of the file, as provided by the uploader
upload_dateWhen the data was uploaded
resourceThe ID of the resource it belongs to. A unidirectional relationship.
archived_urlThe URL where this file has been archived

Generating paths should try and separate the files, perhaps based on username of the owner, or some other mechanism to avoid a single folder full of files.


We need to make sure that it is possible to notify other components within the system that an upload has taken place, or at least make it easy for them to be notified. The primary use case for this is to notify the component that will translate/upload certain formats to the data store.

We could do this based on the post-upload update to the file model (i.e. when we record the total received size of the file).

1343058789000000 1346663270000000
#2733 enhancement johnglover johnglover ckan-v1.9 new Datastore logic functions

Where does the data go?

In a postgres database configured by the ckan.datastore_write_url config option which is a sqlalchemy url.

The user should have rights to create tables.

Whats the api like?

We will just implement it as logic functions like the rest of CKAN and will part of core. After that we may add some nicer api functions that use these but that is a secondary concern.

What are the initial logic functions?

  • datastore_create
  • datastore_delete
  • datastore_show

What is the JSON input format for datastore_create

To begin with it can have the following keys. It is fairly consistent with Max Ogdens' gut servers. Except adds resource_id.

resource_id: resource_id # the data is going to be stored against.
fields: a list of dictionaries of fields/columns and their extra metadata.
records: a list of dictionaries of the data eg  [{"dob": "2005", "some_stuff": ['a', b']}, ..]
  • The first row will be used to guess types not in the fields and the guessed types will be added to the headers permanently. Consecutive rows have to conform to the field definitions.
  • rows: can be empty so that you can just set the fields
  • fields are optional but needed if you want to do type hinting or add extra information for certain columns or to explicitly define ordering.

eg: [{"id": "dob", "type": "timestamp" }, {"id": "some_stuff", "type": "text"}, ...]. A header items values can not be changed after it has been defined nor can the ordering of them be changed. They can be extended though.

  • Any error results in total failure!! For now pass back the actual error.
  • Should be transactional

What json does datastore_delete take?

resource_id: resource_id # the data is going to be deleted.
filters: dictionary of matching conditions to delete
    e.g  {'key1': 'a. 'key2': 'b'}  this will be equivalent to "delete from table where key1 = 'a' and key2 = 'b' ".
    No filters (either not present or not defined) then delete the table. If we want truncate then add truncate: true to truncate the table.

What json does datastore_search take?

resource_id: resource_id # the data is going to be selected.
filters : dictionary of matching conditions to select
    e.g  {'key1': 'a. 'key2': 'b'}  this will be equivalent to "select * from table where key1 = 'a' and key2 = 'b' "
q: full text query
limit: limit the amount of rows to size default 100
offset: offset the amount of rows
fields:  list of fields return in that order, defaults (empty or not present) to all fields in fields order.
sort: comma separated field names with ordering e.g "fieldname1, fieldname2 desc"

Some free code: https://gist.github.com/3163864

What json does datastore_search return?

fields: same type as datastore_create accepts (i.e. with metadata)
offset: The same offset that was supplied in datastore_show
limit: The original limit
filters: The filters that were applied in data_show
total: # total matching records without size or offset
records: [same as data_create] # list of matching results

On error will return:

__error__: … sql error …

What types are allowed?

Aim to support as many postgres/postgis types that have string representations.




Each row in a table will be given an _id column which has an id generated by us which you can use in queries.

Other Features

Each row will store the _full_text index of all the data in the row. At some later point there will most likely be a way to index fields add constraints etc.

1343058886000000 1343656105000000
#2734 enhancement toby shevski demo phase 2 closed fixed once format on resource form has errored, no longer submits form

Steps to reproduce:

  1. go to edit a resource, e.g. http://s031.okserver.org:2375/dataset/ted/resource_edit/a621a6ad-6f1c-46d1-b858-03cdb7a25c07
  1. Change the format field to something it doesn't get. e.g. add a comma "HTML,"
  1. Get message: "The form contains invalid entries:

Last modified: Date format incorrect Size: Invalid integer"

  1. Format field reverts, try to re-save/submit form - i.e. click on "update resource"
  1. nothing happens (same if you even try changing the format field
1343059061000000 1343127101000000
#2735 enhancement toby shevski demo phase 5 assigned Dataset order on user page

I think the datasets on user pages http://s031.okserver.org:2375/user/me should be ordered by latest updated (with most recent at the top) instead of in alphabetical order.

What do you think?

1343062877000000 1344349245000000
#2736 enhancement ross ross ckan-v1.9 closed wontfix Archiver fixes

We need to check the archiver to make sure it works as we would expect after the changes in #2732:

  • Receive notification of a file that needs processing
  • Check the file's DB entry and upload the file to the configured remote service
  • Update the URL of the file (both in the file's DB entry and the resource)
  • Once we're sure the file is safely archived, we should mark the file as being archived

so that we know it can safely be deleted at some point.

Maybe this could/should be implemented as a service rather than a celery task?

1343121237000000 1346670097000000
#2737 enhancement toby shevski demo phase 2 closed fixed change help/prefilled text in format field

Should be basic format (which is what the datastore wants & what we display to users) e.g. "CSV, XML, JSON etc"


1343121262000000 1343136141000000
#2738 enhancement toby shevski closed invalid Data API

Why is datastore not enabled for this resource http://s031.okserver.org:2375/dataset/frere/resource/cdeddb39-b0cb-4762-aa5b-f80fc892d4dd ?

Is this going to be a site wide setting? How do new resources get added to datastore? Or are we just using the dataproxy for demo?

1343123012000000 1343125335000000
#2739 enhancement aron.carroll shevski demo phase 3 closed fixed Help text for format field

Can we please add some text next to the format field when user adding a new resource along the lines of:

"This will be generated automatically, but you can edit if you wish"

It always seems to work better when the user does not fill it themselves...

1343123175000000 1343817919000000
#2740 enhancement toby shevski demo phase 2 closed fixed misleading breadcrumbs

When adding a resource, the breadcrumb should say "add data" or "add resource" not "create dataset" as you see here: http://s031.okserver.org:2375/dataset/new_resource/frere

1343123274000000 1343137576000000
#2741 enhancement johnmartin toby demo phase 4 closed fixed admin index styling


you need to be an admin

paster sysadmin add <username> -c dev.ini

we need to get links to other admin pages etc - I think this will want to be flexible so extensions can add extra links etc - but that can come later

1343124308000000 1352205988000000
#2742 enhancement johnmartin toby demo phase 5 closed fixed admin authz styling


you need to be an admin

paster sysadmin add <username> -c dev.ini

like #2741 we need other admin links

you may want to make the inputs use your form macros - I've just done a basic port

also we will need to get all the tables done but looks like we need to get auth groups working for that

1343124423000000 1352206081000000
#2743 enhancement toby shevski demo phase 2 closed fixed add dataset button within group

Currently the 'add dataset' button on a group page http://s031.okserver.org:2375/group/another-group-in-the-wall goes to edit group. This is confusing since the interface for this is no way streamlined or helpful.

Can we get this to go to the create dataset process instead (ideally with the group pre-chosen) for now?

Further on will need to re-think & plan the workflow for publishers which will be a large part of new work

1343125202000000 1343131092000000
#2746 enhancement toby shevski demo phase 2 closed fixed help bubble on demo homepage missing

Hey, this is still missing. It's the one with the add dataset text

1343209387000000 1343214031000000
#2747 enhancement toby shevski demo phase 2 closed fixed Breadcrumb should use title not URL of dataset

For example here: http://s031.okserver.org:2375/en/dataset/new_resource/gold-prices The breadcrumb shows "gold-prices" which is not the title of the dataset

1343211742000000 1343215679000000
#2748 enhancement shevski demo phase 5 new add 'add new resource' button to sidebar

When editing a resource you see the current and any other existing resources in sidebar

(e.g. see http://s031.okserver.org:2375/en/dataset/format-error-test/resource_edit/d1eac556-c16f-44af-8148-5e3467b57cf8?inner_span=True)

Would be good to have a pretty 'add new' slightly transparent resource folder/pointer undearneath - letting you add resources from the end resource page

1343212878000000 1344503744000000
#2749 enhancement aron.carroll toby demo phase 5 closed fixed page title issue with super() super()

title broken for


we have inherited the title from the extended page not from the base

should be 'add data - ckan' not 'add data - create dataset - ckan'

any idea how we fix this?

1343215828000000 1343232304000000
#2750 enhancement toby toby demo phase 3 closed fixed javascript translations need ripping

we need to ensure that translations in .js files are ripper for translation

look at what is available as preference to creating our own

1343216246000000 1343415279000000
#2751 enhancement toby toby demo phase 5 new check translations for full demo site

need to check everything gets translated - sean did this before so will have info

1343216443000000 1344243046000000
#2752 enhancement toby ross demo phase 2 closed fixed Navigation highlight missing

When navigating to http://s031.okserver.org:2375/group on the demo server, the Group link in the navbar is not highlighted in the same was as the other links are when visiting those pages.

1343216582000000 1343222618000000
#2753 enhancement aron.carroll shevski demo phase 2 closed fixed rename breadcrumb on user page

Currently lower case 'active' is the name of this page: http://s031.okserver.org:2375/user

Should be 'All Users' or 'Active Users'

1343225088000000 1343227431000000
#2754 enhancement aron.carroll shevski demo phase 2 closed fixed Add + icon to two buttons

Specifically the 'add resource' button and the 'add item' button in related section

1343225177000000 1343227446000000
#2755 enhancement toby shevski demo phase 3 closed fixed user pages very slow to load

navigating to a user page -one's own or http://s031.okserver.org:2375/en/user/ira or http://s031.okserver.org:2375/en/user/shevski, even when signed out is noticeably slower than navigating between other parts of the site; over 2 seconds. Maybe even 3

1343225395000000 1344505304000000
Note: See TracReports for help on using and creating reports.