{22} Trac tickets (2647 matches)

Results (101 - 200 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1644 enhancement shevski ckan-backlog new Order default dataset page by most downloaded resources on thedatahub

Instead of alphabetically as we do currently, alternatively by most viewed datasets

for http://thedatahub.org/dataset

1326393542000000 1326393542000000
#1647 enhancement shevski ckan-backlog new add links to ckan discuss & dev to thedatahub

In the footer as well as more clearly & directly on the About page

1326673852000000 1326707383000000
#1667 enhancement seanh seanh ckan-future new Add an extension point for rendering activity streams

Currently rendering of activity stream events to HTML works by looking up a rendering function in a dictionary that maps activity types ('new package', 'changed group', etc.) to rendering functions that take an activity stream event and return the rendered HTML.

There needs to be an extension point where extensions can register their own rendering functions for particular activity types.

1326796151000000 1338204295000000
#1668 defect dread ckan-backlog new repoze version discrepency

There's a discrepency in repoze.who versions between the source and package installs:

  • repoze.who - package 1.0.18 vs source 1.0.19
  • repoze.who-friendlyform - package 1.0b3 vs source 1.0.8

We get a test failure [1] with the 1.0b3 version (from the ubuntu 10.04 python-repoze.who-plugins package). But we've not noticed any problems on s057 instances (br, no, ie etc) which have the package versions of repoze.who.

The reason the package install uses the earlier packaged versions rather than the ones we'd like is that repoze uses all sorts of horrendous import hacks, making it too difficult to put into our 'ckan-conflict' source package.

James suggests we 'do something horrible like dynamically patch repoze on CKAN import'.

[1] http://buildbot.okfn.org/builders/builder-ckan/builds/1371/steps/shell/logs/stdio ERROR: ckan.tests.functional.test_user.TestUserController?.test_user_create_unicode

1326801746000000 1326801746000000
#1679 enhancement dread ckan-backlog new Default roles problem

The 'editor', 'anon_editor' and 'reader' roles are intended to have immutable actions. This was designed to prevent their names being subverted - e.g. an editor should always be able to edit! It also meant that when we add Actions (e.g. DELETE-PACKAGE) then it can be added sensibly to these roles in an upgrade just by changing the defaults table (ckan/model/authz.py).

The problem is that this immutability is only enforced on 'db upgrade'. So you can happily change the editor role using the paster command and it works, right up until you do an upgrade and realise permissions are different.

We should stop the paster commands being able to edit these roles. Or get rid of the immutability completely. Views?

1326823042000000 1339773923000000
#1697 enhancement rgrp ckan-backlog new A Configurable list of states for a Dataset

Currently have 'active' and 'deleted' suggest also:

  • 'draft'
  • 'hidden'

(Do we need both). Also write out workflows related to these.

1327400630000000 1338204189000000
#1717 enhancement shevski ckan-backlog new [super] Search UX improvements
  1. Make it possible to search by tag (e.g. by typing tag:csv into the search bar and clicking enter, it should add the 'csv' tag facet to the search)
  1. Rename and standardise the list of format tags, on search page this should also be called 'Format' instead of 'res_format' (in the right hand side bar on search page).
  1. Make it possible to view full list of tags, formats and groups by clicking on the name. From here you should be table to click on a classification and go back to a search page faceted by that classification. E.g. from search page, click on 'tags', on tag page click on 'london' or whatever, and be navigated back to search page with search within 'london' tag only. Or y'know, a better way of doing it.
  1. More standard classifications, such as 'Location' and 'Theme' - like on publicdata.eu
  1. Blue search button should be displayed in line with the search bar, not underneath
  1. Datasets should be displayed in order of most viewed or downloaded instead of alphabetically. For alphabetic search we could consider adding a way to facet by first letter of dataset name
1327603981000000 1330088539000000
#1740 enhancement seanh ckan-future new Get rid of `from module import ...`

It's really bad to do from module import * and CKAN has a lot of them. I suggest a three-pronged approach:

  1. Don't add any more of them.
  1. When you're programming if you see an easy opportunity to remove one then do so.
  1. At some point we should task someone to go through the code and remove them all (which is what this ticket is for), but this will be a big job and may break things.

We should also get rid of most or all of the from module import foo and from module import foo, bar statements.

I think the right thing to do is just import module and then use module.foo in your code. But if you find yourself doing module.foo.bar then you may have a code smell.

See: http://docs.python.org/dev/howto/doanddont.html

1328094369000000 1328094884000000
#1745 enhancement rgrp ckan-v1.9 new Dataset search UX improvements as of Jan 2012

Changes to make search both more exploratory and more satisfying to use

  • Search query build - #1603
    • Ability to add new facet fields "live"
    • That is add fields which then contain faceted options (a bit like data.hri.fi)
  • (??) Autocomplete / drop down on search (i.e. search while you type)
    • Dubious about value / cost ratio here

Probably would involve to pure JS and HTML implementation.

Implementation

Probably require

  • API changes to expose solr style API directly #1737
1328224941000000 1340033358000000
#1747 enhancement seanh ckan-backlog new Expire old activities

Currently the activity streams database tables just get longer and longer over time. Do we want to eventually delete the oldest activities, to keep the length of the table within limits?

1328446589000000 1339773859000000
#1748 enhancement seanh seanh ckan-future new Make activity streams conform to http://activitystrea.ms/ standard

At the very least we should make our JSON output conform to their spec:

http://activitystrea.ms/

1328450784000000 1338204337000000
#1750 enhancement seanh ckan-backlog new Move ckan/lib/activity.py into the model

Move ckan/lib/activity.py moved to into the model - say ckan/model/activity_extension.py, because it's so tightly knit with the model code, whereas most of the lib code is used in the controllers.

1328465888000000 1339773840000000
#1782 enhancement zephod rgrp ckan-future new Improvements to datasets page UX (Feb 2012)
  • Relationships in sidebar [WONTFIX yet without some link to documentation or datastructure, something for me to render?]
  • Icons for *types* of resources in resource list
  • List of 'Owners' (administrators) at top right of sidebar (a la group page)
    • Much prefer owner to administrator ...
1328609441000000 1338205232000000
#1787 enhancement dread ckan-future new [super] Improve RESTful API
  • Lists of entities should be full URLs, rather than just the names
  • Discoverability - /api/v3/rest should list the entity types that can be listed

This could be v3 of the RESTful interface.

1328702082000000 1328702082000000
#1789 enhancement seanh ckan-backlog new Implement a tag_update() logic action function

So users can rename a tag and/or move it between vocabularies using the API.

Currently we have create_tag() and delete_tag(), but if you were to 'update' a tag by deleting it and then recreating it all the datasets that had that tag will have lost it and you'll have to re-add it to them all.

What should happen to datasets that have the tag, if the tag gets moved between vocabularies? All the datasets just keep the tag with the new vocabulary? This will become a problem if/when we support 'radio button'-style vocabularies (where each dataset can only have one tag from the vocabulary).

1328805413000000 1339773812000000
#1790 enhancement dread ckan-future new Click to delete tags, rather than have all existing tags in the tag text box

From Pablo:

Editing the tags field is clumsy when there are too many tags. Could show existing effectively as tags (like delicious), then allow clicks to delete. New tags added via text box.

1328888674000000 1328888674000000
#1800 refactor seanh ckan-future new Tidy up *_list() and *_search() functions in ckan/logic/action/get.py

For consistency all the *_list() functions should list objects only and not accept an optional search query. There should be *_search() functions whenever search is needed.

Currently it is inconsistent, e.g. package_list() and resource_list() do not accept a search query option and there are package_search() and resource_search(), but user_list() does accept a query and there is no user_search(). tag_list() also accepts a query, and there is also a tag_search() that apparently duplicates the search functionality.

1329405129000000 1338204886000000
#1815 task amercader amercader ckan-future new Reenable Sparql endpoint on publicdata.eu

Estimate: ?

The 4store Sparql endpoint has been down for a long time. It needs to be reenabled and we need to check that the necessary informtion is pushed when creating /editing a dataset.

1329757723000000 1338204641000000
#1818 enhancement amercader amercader ckan-future new Spatial metadata editor

Estimate 2.5d

Right now users need to manually enter a geojson value in the 'spatial' extra field in order to define a geometry for a dataset.

We need a map widget on the edit form that allows users to draw the geometry on top of a map. This will be based on the draw functions offered by OpenLayers? [1], [2].

Things to take into account:

  • Integration with the form sections.
  • Allow some degree of configuration (default extent, limit geometry types...)

[1] http://openlayers.org/dev/examples/draw-feature.html [2] http://openlayers.org/dev/examples/modify-feature.html

1329763941000000 1338203195000000
#1823 enhancement rgrp ckan-backlog new Spring clean bin directory

Huge number of accumulated (and likely unnecessary) scripts in /bin directory.

1329773331000000 1338203554000000
#1824 enhancement seanh ckan-future new Add vocabulary pages

For a free tag foo you can visit the page at /tag/foo and see a list of all the datasets that have the tag foo, and when the tag appears on dataset view pages etc. it's linked to this tag page.

We should do the same thing for vocabulary tags. A tag bar in vocabulary baz should be hyperlinked to a page /tag/baz/bar, or perhaps /vocab/baz/bar.

1329845089000000 1338204958000000
#1827 enhancement dread ckan-backlog new 'Register' link should be hidden if you not allowed to register
I have just deny visitors the create-user permission:
sudo -u ckanstd paster --plugin=ckan roles deny reader create-user -c /etc/ckan/std/std.ini
sudo -u ckanstd paster --plugin=ckan roles deny anon_editor create-user -c /etc/ckan/std/std.ini

and after restarting, the register link is *not* hidden, but now when you access the register page, it shows you this message "Unauthorized to create a user" (when not logged in). But anyway that is an improvement.
1329924939000000 1339773730000000
#2200 enhancement seanh seanh ckan-future new Add vocabulary_id option to tag_show() logic action function

Currently tag_show() only works with free tags, it's not possible to get a tag_show for a vocabulary tag.

1330533678000000 1338204900000000
#2235 enhancement rgrp ckan-future new Group drop down on dataset edit should use chosen and sort groups by name 1331907357000000 1338204726000000
#2244 enhancement toby toby ckan-future new clean up auth
  • cache admins etc
  • aim to move check_access so independent of logic function
  • general spruce up
1332177771000000 1338205402000000
#2247 enhancement zephod ckan-backlog new Resource preview glitch in some browsers

From Ira: Preview for google spreadsheets are not displaying correctly for me in Firefox v.10.0.02, fine in Chrome. http://i.imgur.com/KJaqz.png

1332246614000000 1332246614000000
#2257 enhancement toby toby ckan-v1.9 new cleanup template vars

look at reducing what is pulled into templates eg ckan.lib.helpers ensure that these changes don't break existing extensions etc

1332513307000000 1340097071000000
#2258 enhancement rgrp ckan-future new Customizable contributor agreement
  • Customize text at bottom of forms
  • Also need to make clear that this does not apply to the data itself (that is covered by the license you choose on your dataset ...)
1332751549000000 1338204747000000
#2265 enhancement dread ckan-future new 'More Like This' for a dataset

When viewing a dataset, it would be nice to show a couple of 'Related Datasets'. i.e. ones that are similar.

SOLR has a feature for finding documents similar to a particular document, called 'More Like This'.

We would like this for DGU.

1332865220000000 1339771350000000
#2268 enhancement thejimmyg amercader ckan-backlog new UKLP Use same schema as form for harvested records

Datasets created via the harvesting (UKLP ones) and the ones created via the form use a different schema. That could lead to problems. We need to discuss the best approach.

1333108606000000 1333108606000000
#2277 refactor icmurray icmurray ckan-backlog new Use the new atom feeds in IATI

Atom feeds have been implemented in core in #1593. The IATI-specific implementation can now be removed when IATI move to CKAN >= 1.6.1

1333386628000000 1333386628000000
#2279 enhancement rgrp rgrp ckan-v1.8 new Write tutorial for our DataStore API / Data Explorer

Unfortunately ES is a little bit complex / opaque to newcomers. Need to write a simple tutorial walking through this.

This is documentation / example only (no code!)

1333482263000000 1338205426000000
#2282 enhancement toby toby ckan-future new log api calls for users

This was part of #2251 but is now it's own ticket.

we need clarity on this. The use cases so far have been possible limiting api calls for users.

so we want to monitor the user or ip address

what about the api calls?

do we want to record all calls or just some? do we want to record the type of call and to what level of detail?

recording the url just seems to specific to me.

1334074725000000 1338210829000000
#2287 enhancement seanh ckan-future new Consistent datetime and time period strings localization

Find all places where date, times and time periods are rendered as strings in CKAN. Make them all go through the same code path (e.g. the helper function in helpers.py) with localization support.

For example, on the user index page right now there are time period strings like "less than 1 month" that currently do not get translated (and the way they're implemented doesn't look i18n-friendly).

ISO date format everywhere might be a good idea.

Need to consider local timezone issues.

Python standard library and Babel may have useful helpers for this.

Should the local timezone displayed be a global setting per CKAN instance? Or should it change depending on the location of the user who is viewing the page?

1334254123000000 1339411445000000
#2291 enhancement kindly rgrp ckan-v1.9 new DataStorer incorrectly attempts to store JSON in DataStore

See http://thedatahub.org/dataset/ecb13156-cf11-4e91-b51c-f287d154276e/resource/5db67915-40cb-467e-9d72-194ea4973670

This was marked as json format (and is JSON) but appears to have been imported as a quasi-CSV. Either this hsould not be imported at all or it should be imported as JSON ... (which is straightforward to do).

Assigning to David for triage and review.

1334431147000000 1340636737000000
#2298 enhancement icmurray icmurray ckan-v1.9 new Add sort-by controls to the search results page

Sort-by functionality was exposed through the package controller in [1]. But no controls were added to the search-page.

  • What should the sortBy controls/widget look like?
  • Which fields should be exposed?

The above commit is in the release-1.6.1 branch, so this work is dependant upon release-1.6.1 from being merged into master.

[1] https://github.com/okfn/ckan/commit/8685c6000d1cb211928b4dbc63990fb72d884f8c

1334569162000000 1340635947000000
#2301 enhancement seanh seanh ckan-future new Replace old-style string formatting with format() method (at least in strings marked for translation)

Some strings in CKAN are passed to gettext with more than one %s in them, this is no good as translators may need to change the order of substituted words.

Strings should not use the old style % substitution, they should use the new format method in Python 2.6+, e.g.:

'This {food} is {adjective}.'.format(food='spam', adjective='absolutely horrible')

http://docs.python.org/release/2.6.8/tutorial/inputoutput.html#fancier-output-formatting

We should probably only change strings that are marked for translation and that contain multiple %s, don't want to make too much work or cause too many strings to change and need to be re-translated.

1334579264000000 1340635812000000
#2310 enhancement icmurray icmurray ckan-backlog new Refactor the search-query construction in feeds.py

The feeds controller, used to construct atom feeds, duplicates code found in the package controller's search action (in order to construct the custom feed).

Refactor this to remove duplication.

1334592091000000 1337159386000000
#2320 enhancement rgrp ckan-v1.9 new Introduce new state: archived

Some datasets become obsolete: e.g. http://datahub.io/dataset/uncdb Do not want to delete them but want them marked as archived or deprecated or something.

1335211949000000 1340624119000000
#2321 enhancement rgrp ckan-v1.9 new Use resource names/titles in URLs

Would make them much nicer but requires:

  • Ensure uniqueness (or deal with ambiguity)
  • (?) Slug/names versus titles
    • Aside: what is currently called name should probably be title
1335236412000000 1340624174000000
#2324 enhancement amercader ckan-v1.9 new Link API docs to relevant version not just latest

The link to the API docs in the footer always points to the latests docs, which may not be correct for the particular CKAN version.

Also, remove the 'API' link (to /api/1) as pointless and misleading

1335351532000000 1345199411000000
#2326 enhancement toby toby ckan-future new dataset permalinks

create permalinks

eg \dataset\~a5-DQ\nice-name

~XXXXX defines the permalink the nice-name is ignored

  • packages table needs a new field for this data
  • existing datasets need updating
  • new datasets need the identifier creating
  • routing needs updating
  • needs to work for all urls like edit

Once this is implemented we can decide when we will use permalinks vs the normal urls.

1335371946000000 1338212799000000
#2328 enhancement rgrp ckan-v1.9 new Link to Catalog API for each Dataset, Group and Tag

At the moment no way for a normal user to find JSON versions of this material. Would it be worth adding back a link somewhere on the page?

1335442501000000 1340624331000000
#2333 enhancement rgrp ckan-v1.9 new Introduce new resource type datastore

This is a resource that is *just* in the datastore. At the moment there are hacky ways to do this (create resource with meaningless URL etc) but we should do this properly:

  • New Option: DataStore? Table in add a resource form
  • URL field is disabled (or points at /api/data or something rather pointless)
  • Download button is disabled
    • Better would be to find a way to serialize an ES table or stream it ...
1335651360000000 1340624473000000
#2350 enhancement icmurray icmurray ckan-v1.9 new Hooks in package controller for validating form data

When the validation schema was moved from the package controller into the logic layer, the schema's role was changed from that of processing a form and validating data; to just validating data. Whilst is makes sense to have a schema for validating data as it comes in and leaves the logic layer; there's no longer the hook available to PackageController? subclasses to do form validation/processing.

Add a hook into _save_new() and save_edit() prior to the logic-layer call which subclasses can then override.

NB - form processing/validation *is* different to validating data. For example, many data can be inferred from a user's input; or there may be aspects of the form that need to be filled in, but do not make sense to pass as data into the logic-layer, eg. agreeing to T&Cs.

  • [ ] Add hooks
  • [ ] Use in ckanext-example
1335881761000000 1340635966000000
#2363 enhancement toby kindly ckan-v1.8 new Documentation of best caching practice.

Need better documentation on best practices in making page loads faster for non logged in users.

1335889017000000 1340099794000000
#2382 task amercader amercader ckan-future new Investigate options for basic geocoding

A simple way for geocoding place names would be very useful, e.g in the spatial search, defining a geometry for a dataset (on the form or bulk)

1337017160000000 1338205325000000
#2395 defect dread ckan-backlog new paster db clean/init don't work when spatial extension enabled

If you have a spatial enabled database then if you don't disable the spatial extension in the CKAN config temporarily then you get errors when you run paster db clean and paster db init.

Can't you just modify the clean and init functions to run without extensions enabled?

The wider problem is that extensions do their own inits every time you do load_environment, which seems crazy and inefficient to me, since this occurs every time a request comes into CKAN. But that is another problem/ticket.

1337159793000000 1339771313000000
#2406 defect kindly markw ckan-v1.9 new Errors in source install doc

There are a few errors or unclarities in the source install doc - see

http://lists.okfn.org/pipermail/ckan-dev/2012-May/002203.html http://lists.okfn.org/pipermail/ckan-dev/2012-May/002204.html

1337595093000000 1340636804000000
#2407 enhancement amercader markw ckan-v1.9 new 'Access denied' message is unhelpful

This chap tried to use the API to add a dataset to a group and got the uninformative message 'Access denied':

http://lists.okfn.org/pipermail/ckan-dev/2012-May/002229.html

The problem was he didn't have access to the group. A message like 'You don't have access to the group "LODcloud"' would have been clearer.

1337612500000000 1340633440000000
#2411 enhancement toby markw ckan-future new .ini file is a mess

The .ini file needs cleaning up in assorted ways which Toby will expand upon.

1337686900000000 1338210872000000
#2415 defect kindly markw ckan-v1.9 new Confusing message on attempting to register new account with upper case

On registering a new account, if you try to use a capital letter in your 'Login', you get the unhelpful error message:

"The form contains invalid entries: Name: Url must be purely lowercase alphanumeric (ascii) characters and these symbols: -_"

What does 'Name' or 'Url' have to do with anything? It means 'Login must be ...'

But even better would be to make the login case-insensitive and allow capitals to be entered.

1337779189000000 1341268304000000
#2426 enhancement kindly seanh ckan-v1.9 new Get rid of CKAN's formalchemy dependency 1337946462000000 1340616095000000
#2427 enhancement kindly seanh ckan-v1.9 new Reduce cruft installed into CKAN's virtualenv due to Pylons 1337946507000000 1341268323000000
#2433 defect toby toby ckan-v1.8 new API uses name not id for some version 3 calls

see https://github.com/okfn/ckan/pull/5

1337957648000000 1340099820000000
#2479 enhancement markw ckan-v1.9 new Meanings of Author and Maintainer fields are unclear

CKAN's default schema has fields for Author ('The name of the main contact, for enquiries about this particular dataset') and 'Maintainer ('another important contact person'). The distinction is not clear. Also the fields are often left blank.

The roles that seems most important are those of the original owner/publisher of the data, and the person maintaining the CKAN record/copy of it. So I suggest:

(1) Rename the fields 'Owner' and 'Maintainer'.

(2) Change the explanatory text for the relevant fields:

Owner: 'The person or organisation who create/collect/publish the data in this dataset.' Owner e-mail: 'E-mail address for enquiries to the Owner named above.' Maintainer: 'The person maintaining this dataset on [name of CKAN instance], if different from the above.' Maintainer e-mail: 'E-mail address for enquiries to the Maintainer named above.'

(3) When a logged-in user creates a new dataset, the main form should have a checkbox, checked by default, marked 'I am the maintainer of this dataset'. If checked, the Maintainer name and e-mail fields are populated from the user's profile.

1338452898000000 1339771115000000
#2480 enhancement markw ckan-v1.9 new Better message when dataset has no resources

If a dataset has no resources the resources list currently says '(none)'.

Here is a suggested improvement, provided that a maintainer is named: 'There are no data resources here yet. For information about this data, contact the dataset maintainer.'

1338453093000000 1339771086000000
#2483 enhancement markw ckan-v1.9 new Non-local resources should not have Download links

At present, a resource which is just a URL link to an external resource has a 'Download' button on the resource page. This gives the misleading impression that the resource is stored locally.

This is related to another small UI issue: I think the URL of a resource should be much more prominent, not buried in the 'Additional Information' table.

Suggested fix:

  • Put the URL prominently at the top of the resource page (above the preview) (and make it link to the resource)
  • Disable the Download button unless the resource is stored locally.
1338468734000000 1339771043000000
#2485 defect icmurray nils.toedtmann ckan-v1.9 new Encourage leak containment by limiting the number of requests a CKAN process serves

CKAN has memory leaks. They can be contained by limiting the time-to-live of a ckan process. An easy way to achieve this is to limit the number of requests a ckan server process can serve before it gets killed and replaced.

One should ...

  • research ways to limit requests-per-process with the different web servers (see below for a start);
  • explain these safeguards in the CKAN documentation and encourage users to apply them;
  • consider the helper script ckan-create-instance to generate Apache configs with WSGIDaemonProcess ... maximum-requests=XY being active instead of commented out. Or at least let it warn the user to use MaxRequestsPerChild;
  • Investigate current CKAN deployments whether they suffer from mem leaks, and if so contain them.

How to limit requests-per-process

  • Apache:
    • Use WSGIDaemonProcess with maximum-requests=50 or whatever limit is appropriate. We did this sucessfully on datahub.io and the ckan farm
    • (This need to be verified) Without WSGIDaemonProcess, MaxRequestsPerChild 50 should achieve the same.
  • nginx/supervisord: to be researched.
1338551229000000 1340636028000000
#2486 defect seanh ckan-backlog new Should be able to use . in dataset names 1338655583000000 1339770978000000
#2499 enhancement toby icmurray ckan-v1.8 new Documentation for the internal analytics

There's no docs for setting up the internal analytics, nor what it provides.

Setup

Add: ckan.tracking_enabled = true to .ini file

Run: paster tracking -c dev.ini

http://trac.ckan.org/ticket/2251

1339070297000000 1339070297000000
#2508 enhancement seanh ckan-backlog new Make it possible to run CKAN tests for each language

Mistakes in translated strings can cause CKAN to crash or otherwise not work, but it's not practical to manually test every page and function of CKAN in every language that we have new translations for before a CKAN release. It'd be great if the tests could automatically be run for each language.

This is probably a big job, we would have to get the tests to respect a language setting in the ini file, check for any individual test cases that specify the language (e.g. in the URL), and also fix test cases that look for specific English words in HTML output, etc.

In the meantime, a good stop-gap solution might be a script that tests for common mistakes in the po files.

1339411335000000 1339770771000000
#2524 enhancement kindly kindly ckan-ecportal new If there are no translation files for selected language fall back to default lang.

If a user selects a language there are no mo files for then an error is raised. Revert to default language instead.

1339609048000000 1340117608000000
#2529 enhancement rgrp ckan-v1.9 new DataHub (or CKAN) widgets

Simple widgets in pure JS. For example:

  • Count of datasets in a group (could generalise to a query but not sure how useful that is ...)
  • Embeddable list of top X (5) datasets for a given query
  • Embeddable list of *my* datasets

Either these live at: {site}/widgets and we have some kind of generator (form where I choose my group, or my query).

Or: we have this attached to areas of site where relevant.

Can combine the 2 so that the latter links to the former. Think first will be easier to do and possibly more useful long-term (e.g. can just link people to that page).

Cf. http://okfnlabs.org/ckanjs/

1339750049000000 1340624917000000
#2530 enhancement kindly rgrp ckan-v1.9 new DataHub purge fails on some revisions

See http://datahub.io/ckan-admin/trash and try to purge revisions (*not* datasets). It will fail on some of the revisions with errors like:

Problem purging revision 391db9e8-df57-4e0e-8fe6-d4e0c2318344: (IntegrityError?) update or delete on table "revision" violates foreign key constraint "group_extra_revision_revision_id_fkey" on table "group_extra_revision" DETAIL: Key (id)=(391db9e8-df57-4e0e-8fe6-d4e0c2318344) is still referenced from table "group_extra_revision". 'DELETE FROM revision WHERE revision.id = %(id)s' {'id': u'391db9e8-df57-4e0e-8fe6-d4e0c2318344'}

1339750498000000 1341268280000000
#2531 enhancement rgrp ckan-backlog new New state option: archived / deprecated

Deleted means things will get purged at some point.

Archived means they stay around but get hidden from search results and a big warning notice gets displayed saying this is archived / deprecated.

@richard cyganiak ...

1339750787000000 1339770649000000
#2543 enhancement icmurray ckan-v1.9 new facet.sort is not available in the package_search action

Not all solr facet parameters are available through the pcakage_search action. In particular, facet.sort has been asked for; but this ticket should check to see if there are other parameters that would be easy to add too.

See: http://wiki.apache.org/solr/SimpleFacetParameters#facet.sort

1340013335000000 1340633091000000
#2555 enhancement toby aron.carroll demo phase 5 new Demo site needs a breadcrumb helper

Something to make building breadcrumbs a bit nicer

1340026983000000 1342618384000000
#2572 enhancement toby toby ckan-v1.9 new clean up stats plugin

attempt to disengage the stats plugin from core as much as possible

1340105054000000 1340899524000000
#2573 enhancement icmurray ckan-future new package_search does not allow solr's per-field facet parameters

Solr allows its facet parameters to be specified on a per-field basis, eg. facet.limit applies to all facet fields, but solr allows it to be overriden for a specific field, eg. facet.tags.limit.

We don't support this at the moment because we have a whitelist of valid solr query parameters that we accept. See ckan.lib.search.query.VALID_SOLR_PARAMETERS.

1340112439000000 1340633101000000
#2579 enhancement toby toby ckan-v1.9 new move sort_by functions into lib.helpers

make these more available but keep existing functionality so not to break any users

remove_field()

drill_down_url()

etc

1340281798000000 1340899337000000
#2582 enhancement rgrp ckan-v1.9 new Do not hide notes / readme on dataset pages

Current we hide most of readme and then let users reveal it. Stop doing this and if necessary add a quick link down to resources section. (Maybe also rename resources to Data and Resources ...?)

Aside: believe I have mentioned this somewhere a month + ago but could not find the ticket.

1340312340000000 1340625009000000
#2583 enhancement toby toby demo phase 5 new make sure that we implement authentication where needed

in development many auth checks may have been lost we need to check they are still working etc

1340359478000000 1342086274000000
#2585 enhancement seanh ckan-v1.9 new Escape solr control characters in search queries, add advanced search screen

Suggestion from David Read:

We noticed that some search queries produce unexpected search results in CKAN, due to them containing special characters. For example if you were to search for "Spend over £25,000 - NHS Leeds" then it would not come up with the dataset with that exact name. It was excluding datasets with the word "NHS" due to the dash/minus sign. It works fine if you escape the minus sign: "Spend over £25,000 \- NHS Leeds".

So in data.gov.uk I've added escaping of such control characters in our plugin and this useful routine:

http://fragmentsofcode.wordpress.com/2010/03/10/escape-special-characters-for-solrlucene-query/

Perhaps you would consider providing this in CKAN core in future?

I think there is an occasional case when power users would want to use the special characters - brackets, +, -, boolean operators etc. but maybe these could be reserved for an 'advanced search' screen?

1340360773000000 1340625078000000
#2603 refactor icmurray icmurray ckan-v1.9 new Remove deprecated 'fields' parameter from resource_search

The fields parameter of resource_search was deprecated when fixing #2438. It can be removed in release 1.9, and the action tidied up as a result.

1340730601000000 1340730601000000
#2621 refactor icmurray icmurray ckan-v1.9 new Remove the deprecated 'fields' parameter from tag_search and tag_autocomplete

This was deprecated in 1.8 as it wasn't accessible via GET requests due to being a dict. See #2439

In a future release of CKAN (probably 1.9) it can be removed.

Internal uses of it were removed in #2439, but there are tests that still use it.

1340900569000000 1340900569000000
#2622 defect seanh new Login fails in Opera 12

Try to login to CKAN using Opera 12, get "Login failed. Bad username or password. (Or if using OpenID, it hasn't been associated with a user account.)"

1340902602000000 1340902602000000
#2625 enhancement seanh ckan-v1.9 new Add i18n strings from non-core but supported extensions to ckan.pot file

Have to decide which non-core extensions are going to be supported first.

1341236903000000 1341236903000000
#2635 enhancement dread new Non-destructive SOLR reindex

You can't run the search-index reindex on a live server because it will give us bad results for 2 to 3 hours while it runs. Can there be an option that doesn't delete the entire index at the start?

Instead it could just delete any items that don't exist any more, then delete them and regenerate them one by one. So the total number of datasets doesn't change much.

1341829394000000 1341829394000000
#2656 defect seanh dread new Feed with few results has bad paging link, causing exception

This page http://thedatahub.org/feeds/custom.atom?q=wombat has 0 results and contains a link to http://thedatahub.org/feeds/custom.atom?q=wombat&page=0 which the page=0 causes this exception:

ckan.lib.search.common.SearchError'>: SOLR returned an error running query
Error: "'start' parameter cannot be negative"
1342001112000000 1342001112000000
#2663 enhancement toby toby ckan-v1.9 new h.resource_display_name needs love

This function is shit and needs cleaning up and a doc string

description is markdown and should be treated properly

either we should truncate all or leave it to the templates but work universally

url if no name / desc this is in demo-theme branch

1342017746000000 1342017746000000
#2673 enhancement rgrp new simplify set of options for resources

Far too many resource options. Lets restrict back to data file and API. Visualizations etc can either get linked in description or in the Related items.

1342300559000000 1342300559000000
#2679 enhancement icmurray icmurray ckan-v1.9 new Change default behaviour of TemplateController.view to 404.

The current behaviour of TemplateController?.view() (which is the fallback controller should all others fail) is to attempt to render (as a genshi template) the requested file.

Although this may be a feature that some instances want. In general, it leads to:

  • 500s when attempting to access a normal template (eg - http://datahub.io/importer/preview)
  • A way of inadvertantly serving things you may not want to serve. (Small risk, as it needs to be renderable as a genshi template).

Solution:

  • Change the controller to 404
  • Ensure there's a way for existing ckan instances to override that behaviour should they need it.
1342436133000000 1342436133000000
#2683 enhancement seanh new Add no-cache header to _tracking API call's response to make sure it doesn't get cached 1342446577000000 1342446577000000
#2688 enhancement ross new Allow ordering of groups in WUI

Currently the group_index page just shows the entire list of groups, forcing the ordering to be by name. It would be better if it could be sortable by name (or reversed) or by package_count (or reversed)

1342520875000000 1342520875000000
#2708 enhancement kindly toby ckan-v1.9 new limit extra data for package/group show

contextpackage_limits? = { 'tags': 5, <- get first 5

'extras': 0, <- get all

}

only get what you ask for have to be explicit

contextgroup_limits? = {} only main item

start with datasets/groups expand if we like it

1342622420000000 1342622420000000
#2709 enhancement icmurray markw new Atom feeds are undocumented

There doesn't seem to be any documentation yet for Atom feeds.

1342624310000000 1342626212000000
#2718 enhancement toby shevski demo phase 4 new can't add dataset to more than one group

trying to add a dataset to another group means it's no longer part of the first group

http://demo.ckan.org/dataset/edit/afterfibre

1342780550000000 1344544203000000
#2719 defect dread new Feeds controller does not catch NotAuthorized exception

Results in bad user experience and WebApp? errors emailed out. Seen in 1.7.1

1342872863000000 1342872863000000
#2721 defect toby shevski demo phase 4 new deleted groups should not show on 'Add to Groups' dropdown

Groups previously deleted still show up in the add dataset process in step 3 'Additional info'

http://s031.okserver.org:2375/dataset/new_metadata/ff

1342948632000000 1344544214000000
#2725 enhancement toby shevski demo phase 5 new Case sensitivity on tags

My feeling is that 'country-US' and 'country-us' should be the same tag. However currently tags with caps are treated differently

see http://s031.okserver.org:2375/en/dataset/test-dataset

with TEST and test - there also get indexed twice in the search page

1342949667000000 1343030773000000
#2726 enhancement toby shevski demo phase 5 new confusing logic on data preview formats
  1. If a user enters the wrong format on a file that can be previewed - it simply won't be previewed (e.g. a CSV or XML file that can be filled in with JSON in format will just not work or check this
  1. If I incorrectly edit format to one that data preview will try to preview it will work even for a format that it doesn't accept (sometimes) e.g. this PDF file I changed the metadata to HTML http://s031.okserver.org:2375/en/dataset/test-dataset/resource/9d27a9d9-36ec-460e-9edb-6dff7ba4fc28
1342949927000000 1343030906000000
#2728 defect toby shevski demo phase 4 new deleted group shows on search index - for admins

'test-group', which has been deleted,shows up on main search page under groups - and can be filtered by - see http://s031.okserver.org:2375/dataset?groups=test-group

1342950784000000 1345023944000000
#2729 enhancement kindly shevski ckan-backlog new searching for tags:[tag] works but tag:[tag] doesn't

which is confusing since you can only search for one tag like this at a time. I.e. tags:economics,cvs or tags:economics, csv or tags:economics+CSV doesn't work for example; therefore tag:economics, should also work!

http://s031.okserver.org:2375/dataset?q=tags%3Aeconomics&sort=relevance+asc

1342951109000000 1342951176000000
#2731 enhancement markw new Some sites permanently 'down for maintenance'

A large number of XXX.ckan.net sites give the following message:

"This Site is Down for Maintenance We apologize for the inconvenience. ~ The Open Knowledge Foundation sysadmins."

The message is unhelpful and patently false - the sites do not exist. Some of them were supposed to have been redirected to a relevant group at thedatahub.org in this ticket (now closed):

http://trac.okfn.org/ticket/933

However, the redirection only seems to have worked in one case, http://si.ckan.net.

The problem still affects the following sites - the first 4 of which have supposedly been merged:

Please sort this out by redirecting, removing the sites, giving a more helpful (and accurate) failure message, etc, as appropriate.

1343045168000000 1343051608000000
#2733 enhancement johnglover johnglover ckan-v1.9 new Datastore logic functions

Where does the data go?

In a postgres database configured by the ckan.datastore_write_url config option which is a sqlalchemy url.

The user should have rights to create tables.

Whats the api like?

We will just implement it as logic functions like the rest of CKAN and will part of core. After that we may add some nicer api functions that use these but that is a secondary concern.

What are the initial logic functions?

  • datastore_create
  • datastore_delete
  • datastore_show

What is the JSON input format for datastore_create

To begin with it can have the following keys. It is fairly consistent with Max Ogdens' gut servers. Except adds resource_id.

{
resource_id: resource_id # the data is going to be stored against.
fields: a list of dictionaries of fields/columns and their extra metadata.
records: a list of dictionaries of the data eg  [{"dob": "2005", "some_stuff": ['a', b']}, ..]
}
  • The first row will be used to guess types not in the fields and the guessed types will be added to the headers permanently. Consecutive rows have to conform to the field definitions.
  • rows: can be empty so that you can just set the fields
  • fields are optional but needed if you want to do type hinting or add extra information for certain columns or to explicitly define ordering.

eg: [{"id": "dob", "type": "timestamp" }, {"id": "some_stuff", "type": "text"}, ...]. A header items values can not be changed after it has been defined nor can the ordering of them be changed. They can be extended though.

  • Any error results in total failure!! For now pass back the actual error.
  • Should be transactional

What json does datastore_delete take?

{
resource_id: resource_id # the data is going to be deleted.
filters: dictionary of matching conditions to delete
    e.g  {'key1': 'a. 'key2': 'b'}  this will be equivalent to "delete from table where key1 = 'a' and key2 = 'b' ".
    No filters (either not present or not defined) then delete the table. If we want truncate then add truncate: true to truncate the table.
}

What json does datastore_search take?

{
resource_id: resource_id # the data is going to be selected.
filters : dictionary of matching conditions to select
    e.g  {'key1': 'a. 'key2': 'b'}  this will be equivalent to "select * from table where key1 = 'a' and key2 = 'b' "
q: full text query
limit: limit the amount of rows to size default 100
offset: offset the amount of rows
fields:  list of fields return in that order, defaults (empty or not present) to all fields in fields order.
sort: comma separated field names with ordering e.g "fieldname1, fieldname2 desc"
}

Some free code: https://gist.github.com/3163864

What json does datastore_search return?

{
fields: same type as datastore_create accepts (i.e. with metadata)
offset: The same offset that was supplied in datastore_show
limit: The original limit
filters: The filters that were applied in data_show
total: # total matching records without size or offset
records: [same as data_create] # list of matching results
}

On error will return:

{
__error__: … sql error …
}

What types are allowed?

Aim to support as many postgres/postgis types that have string representations.

http://www.postgresql.org/docs/9.1/static/datatype.html

http://www.postgresql.org/docs/9.1/static/sql-createdomain.html

IDs

Each row in a table will be given an _id column which has an id generated by us which you can use in queries.

Other Features

Each row will store the _full_text index of all the data in the row. At some later point there will most likely be a way to index fields add constraints etc.

1343058886000000 1343656105000000
#2745 defect amercader ckan-v1.9 new Password reset returns an exception if the key parameter is missing

Instead of showing a notice, the password reset page throws an exception if the key parameter is missing:

Module ckan.controllers.user:329 in perform_reset
         c.reset_key = request.params.get('key')
               if not mailer.verify_reset_link(user_obj, c.reset_key):
                   h.flash_error(_('Invalid reset key. Please try again.'))
                   abort(403)
 if not mailer.verify_reset_link(user_obj, c.reset_key):
Module ckan.lib.mailer:100 in verify_reset_link
     if not user.reset_key or len(user.reset_key) < 5:
               return False
           return key.strip() == user.reset_key
 return key.strip() == user.reset_key
AttributeError: 'NoneType' object has no attribute 'strip'

Apart from the obvious fix of checking for the 'key' parameter, it seems like is quite common to get these reset urls without the key parameter, so I suspect some email clients might strip the query params when building the links. We could avoid this problem by making the key part of the url instead of a param:

http://thedatahub.org/en/user/reset/3086e91c-fe09-4a98-92e1-19de67a9ac9d/b4c2d03fa8

instead of:

http://thedatahub.org/en/user/reset/3086e91c-fe09-4a98-92e1-19de67a9ac9d?key=b4c2d03fa8

1343145931000000 1343145931000000
#2748 enhancement shevski demo phase 5 new add 'add new resource' button to sidebar

When editing a resource you see the current and any other existing resources in sidebar

(e.g. see http://s031.okserver.org:2375/en/dataset/format-error-test/resource_edit/d1eac556-c16f-44af-8148-5e3467b57cf8?inner_span=True)

Would be good to have a pretty 'add new' slightly transparent resource folder/pointer undearneath - letting you add resources from the end resource page

1343212878000000 1344503744000000
#2751 enhancement toby toby demo phase 5 new check translations for full demo site

need to check everything gets translated - sean did this before so will have info

1343216443000000 1344243046000000
#2758 enhancement toby toby ckan-v1.9 new file storage gives error if config not available but no useful user information

We get an error which should be improved the actual problem is this but not passed to user

KeyError?: 'ofs.impl'

View as: Interactive (full) | Text (full) | XML (full) Module ckan.controllers.storage:2 in auth_form view Module ckan.lib.jsonp:26 in jsonpify view

data = func(*args, kwargs)

Module ckan.controllers.storage:407 in auth_form view

authorize(method, bucket, label, c.userobj, self.ofs)

Module ckan.controllers.storage:200 in ofs view

StorageAPIController._ofs_impl = get_ofs()

Module ckan.controllers.storage:71 in get_ofs view

storage_backend = configofs.impl?

Module paste.registry:146 in getitem view

return self._current_obj()[key]

KeyError?: 'ofs.impl'

1343287709000000 1343287709000000
1 2 3 4 5 6 7 8 9 10 11 12
Note: See TracReports for help on using and creating reports.