CKAN: Ticket #826: Resource 'extra' fields

milestone set

pudo — Mon, 03 Jan 2011 11:20:58 GMT

milestone set to iati-4

owner changed

kindly — Fri, 21 Jan 2011 09:17:51 GMT

owner changed from rgrp to kindly

milestone changed

rgrp — Mon, 07 Feb 2011 10:27:28 GMT

milestone changed from iati-4 to ckan-v1.4-sprint-1

dread — Fri, 11 Feb 2011 08:53:32 GMT

It's great that the extras are added in-line with other Resource properties (as opposed to package extras which are a dict not off 'package' but off 'package.extras').

However, the resource extra field keys are defined in config option "ckan.extra_resource_fields". This config option should be removed - extras need to be entirely flexible for our purposes. (In the next ticket we should make it possible to add/remove both keys and values from the Web UI or API.)

It would also be good to tidy things in this direction: http://wiki.okfn.org/Coding_Standards

I've merged from default to the branch enhancment_826_resource_extra_fields ready for you.

pudo — Fri, 11 Feb 2011 09:04:55 GMT

I want to strongly support david in his call for fully flexible extras: one of my use cases for them is to store the various bits of fallout from an archiving process, such as:

last-status
last-crawled
last-etag
last-expires
last-md5
failure-count
fall-back-url

These are things needed to really archive the data well, but they have nothing to do with CKAN core ops. Essentially, the archiver is a seperate concern and it need not appear in the CKAN config.

kindly — Fri, 11 Feb 2011 09:34:39 GMT

There is nothing to stop anyone from putting any extra attributes in the extra_info field dict. So any have the flexibility you need.

The config option is to add some fields that act in exactly the same way as python attributes, having the same semantics as them. i.e if you have an extra field called alturl, you can do obj.alturl = 'fdsffs'.

This is the best of both worlds as far as I can tell.

kindly — Fri, 11 Feb 2011 09:51:40 GMT

I forgot to mention that he main advantage of the fixed fields, is that we can make them properly searchable i.e the values searchable. This currently does not work for package extra values as they are jsons.

I have added this searchability for the sql backend.

dread — Fri, 11 Feb 2011 10:43:42 GMT

Ah - I understand. I'd like it if you could make all the extra fields work as attributes and be searchable - is that possible?

kindly — Fri, 11 Feb 2011 11:22:22 GMT

This would be too much of a hack. You do not want users overwriting any attributes on the object. If they called the attribute "init" it would write over the actual init.

status changed; resolution set

dread — Fri, 11 Feb 2011 13:11:50 GMT

status changed from new to closed
resolution set to fixed

Merged into default in cset:613d7bd5fc96.

Future tickets in this area include:

#978 Full edit in Web UI
#979 Edit Resource extras in the API

repo, theme set

dread — Mon, 30 May 2011 14:34:17 GMT

repo set to ckan
theme set to none

This went into ckan release 1.3.2.