{22} Trac tickets (2647 matches)

Wish to fold service records into package resources. Trouble is that service records don't necessarily refer to a data record, so should this be a new package?

See DistributingChanges for analysis and functional requirements.

Perhaps 80% of the users of the UKLII may be using ArcGIS.

Already have the basic capacity using *_extras paths for templates and public directory (see ​http://wiki.okfn.org/ckan/doc/theme).

However, would be nice to have a proper 'theme' structure (e.g. would allow for theme switching) that also avoids directly polluting 'public'. Could be inspired by: ​http://packages.python.org/Flask-Themes/

For consistency all the *_list() functions should list objects only and not accept an optional search query. There should be *_search() functions whenever search is needed.

Currently it is inconsistent, e.g. package_list() and resource_list() do not accept a search query option and there are package_search() and resource_search(), but user_list() does accept a query and there is no user_search(). tag_list() also accepts a query, and there is also a tag_search() that apparently duplicates the search functionality.

Because of a clash between two development branches there is some duplication of code to do with code facets (note: at the time of writing the code duplication exists only on the feature-1821-multilingual-extension branch, but this will be merged into master at some point):

The package_search() function is adding the search facets to the search results twice with two different data structures, with keys "facets" and "new_facets". It should be reduced to just the new facets (with the key changed to "facets").

Also the group and package controllers are adding both facets and new_facets to the context, should be new_facets only (but renamed to facets).

The facet_items() function in helpers.py should be removed, it uses the old facets structure and shouldn't be needed anymore with the new facets structure.

In facets.html, facet_sidebar() should be removed as it uses the old facets structure and facet_div() implements the same functionality but uses the new facets.

In facets.html, facet_list_items() will have to be updated to not use the facet_items() helper and to use the new facets structure instead.

Anywhere that "new_facets" appears it will have to be changed to "facets" (e.g. in the ckanext/multilingual/plugin.py.

This is the merge commit that introduced the duplication: ​https://github.com/okfn/ckan/commit/1153aa876f54c22289e460aeececea22d1d4d51d

This is the earlier commit where the search facets were refactored: ​https://github.com/okfn/ckan/commit/3970e52008b75933fda1be1d488bed2578d98c9c

Cost - 2 days

The time period to which a package's data applies is stored in a new field called 'Relevant Date Range'. The value of this field is up to two calendar dates: i.e. either a point in time (e.g. date of a river map is 5/10/09) or a time period (e.g. pollution measurements 1/1/09-1/4/09).

Example part of package:

Date: 5/10/09
or
Date: 1/1/09 – 1/4/09

I'm not sure why some timestamps are missing microseconds in the ckan.net data, but plenty seem to. This causes this exception when viewing the package rss feed:

URL: http://ckan.net/package/history/sfk_publishing?format=atom&days=7
In Module ckan.lib.helpers:227 in date_str_to_datetime
WebApp Error: <type 'exceptions.ValueError'>: time data '2008-04-13T20:40:20' does not match format '%Y-%m-%dT%H:%M:%S.%f'

Only seen with ckan 1.4.3b

Can we finish this ready for release on data hub and CKAN Hosted.

An extension that provides a todo list feature on CKAN so that people can register and find things to do.

Extension name: ckanext-todo

User Story

Package page

As a user I come to a package:

• Have a todo count at that top that takes you down to the todo list (which may say nothing todo)
• At the bottom is a section of the package display titled "ToDo?" where I see a list of all toDos for the package most recent at the top
  • If I am logged in
    • See a form for "Add to do" at the top of the todo section and can add one straight away
    • I see a "now resolved" button next to each which goes green when you hover.

When clicked the todo fades away.

• Not logged in: I see a button that says "login to add todo"

Todo list page

When a user comes to todo overview at /todo

At top list all todo categories with counts (or a progress bar). Click on category name or bar takes you down page to list for that category.

Category list has a list of todo items (ul with li items with class todo) - link to package relevant to the todo.

Implementation

The Todo form

• One of the fields is category -> autocomplete the category (not constrained) (lowercase, no spaces, .-_ allowed)
• Add a description
• Submit, the todo gets added via AJAX to the list at the top as the most recent todo

Model:

todo table

  id (autoincrement integer)
  package_id
  todo_category_id (required)
  description (required)
  created=NOW()
  resolved=null (unresolved) or a datetime (datetime of resolution)
  creator=user
  resolver=user

todo_category table
  id
  name

Prepopulate with: broken-resource-link, no-author, bad-format, add-description

API at /api/2/todo

• GET / POST / PUT ...

/api/2/todo?package=package_id_or_name&category=...&resolved=0/1

• support limit (?)

/api/2/todo/category -> return list of todo categories

• No GET / PUT / POST (these are auto-created by creation of todo)

Optional Extras (Will not be done atm)

• Integrate todo tags (e.g. list packages tagged with a todo.{xxx} on Todo List page ...

When people link to a package, a track-back link is auto-created. (Similar system as for blogs).

As suggested by Tim Davies:

Allowing some form of ‘track back’ against datasets When a non-technical user comes to look at a dataset it would be really useful for them to be able to see if anyone has created an interface interpretation of it already.

I found quite a few cases in research of end-users struggling to make sense of a dataset when good interfaces to that data had already been built and blogged about, but without there being any link from the dataset listing to those data uses. Accepting track backs could also make it easier for technical users to find blog posts / shared code etc. relating to a given dataset.

User names and descriptions are not translated.

Also they seem to be missing the odd _() block. Such in the new package form.

​https://github.com/okfn/ckan/blob/2375-demo-theme-development/ckan/templates/package/snippets/package_form.html#L44

The CSW harvesters implemented at the moment were developed with the DGU project in mind, and they assume all remote CSW servers to implement the Gemini 2 specification. Gemini 2 is the profile defined in the UK for INSPIRE complying metadata, so obviously catalogs from other countries or non-INSPIRE complying ones won't be able to be harvested.

The changes needed to support generic CSW servers (i.e. those implementing the ISO 19139 profile) are:

• Handling the validators (right now are hardcoded in the harvester

code). This probably involves issues discussed in the CREP 3 (ticket #1134)

• Changes in the model to adapt the specification to ISO 19139

• Renaming objects and classes which are now Gemini-centric

List of CSW servers tested:

​https://spreadsheets.google.com/spreadsheet/ccc?key=0Atp3cZFjuIOAdDBVQWRINnlfN1d0b2lleHVEdjBSb2c&hl=en_US&authkey=CNu4hsEB#gid=0

Awesome to have the captcha implementation and running on thedatahub.org! Here are some further suggestions concerning the config options, before this goes into a release:

• Document "ckan.recaptcha.publickey" and "ckan.recaptcha.privatekey" in doc/configuration.rst
• Which one config option defines whether recaptcha is enabled? There is disagreement between the template and the lib. Maybe better to have a third option for this. When setting the syntax for this, I can see us wanting to use captcha for other operations, like package edit, package create, group create etc. so perhaps we'd want to be able to turn each one on/off separately, using the name of the logic action. e.g. ckan.recaptcha.enable = user_create, group_create
• We got this exception last night from the template:

  Module ?:37 in <Expression u'g.recaptcha_publickey'>
  <<          <dd><input type="password" name="password2" value="" /></dd>
         
                 <dd py:if="g.recaptcha_publickey">
                   <script type="text/javascript"
                     src="http://www.google.com/recaptcha/api/challenge?k=${g.recaptcha_publickey}">
  >>  <dd py:if="g.recaptcha_publickey">
  Module genshi.template.eval:324 in lookup_attr
  <<                      val = obj[key]
                         except (KeyError, TypeError):
                             val = cls.undefined(key, owner=obj)
                 return val
  >>  val = cls.undefined(key, owner=obj)
  Module genshi.template.eval:410 in undefined
  <<          """Raise an ``UndefinedError`` immediately."""
                 __traceback_hide__ = True
                 raise UndefinedError(key, owner=owner)
  >>  raise UndefinedError(key, owner=owner)
  UndefinedError: <ckan.lib.app_globals.Globals object at 0xb8f1182c> has no member named "recaptcha_publickey"
  

I'm not sure why - might be something to do with the maintenance on the server, but worth double checking, since we don't want exceptions, say if one or other config variable is not defined.

Futher Information -> Further Information

• There is huge inconsistency in the titles of pages across the site: to give one example "Edit Package:" vs "Package: mypkg [not linked]" vs "History - mypkg [linked to package page]"
  • Normalize and use terminology: "Data Package" - also change in navbar (but nowhere else for the time being). So hvae Data Package - mypkg, Data Package - mypkg - Edit etc
• pagination is currently lost in amongst the results list. needs to be *much* more obvious, and should appear top and bottom, or at the very least at the bottom!

Groups landing page group

• Yeesh! What's up with the typography for "Just login..."?
• Truncated description along with group name.
• Explanation text about what groups are for (RHS)
• Another case of <li> bullets adding nothing.

• 'Login via OpenID' -> 'Login with OpenID'?
• Elements:
  • Search
  • about CKAN
  • 3 column Packages view (see ticket:219)
  • Small tag cloud (RHS)

Also, what about ticket:143 ?

Package edit page package/edit/mypackagename

• Ability to add the package to groups on this page.
• Fields could be grouped.
• Inconsistent capitalisation (Url -> URL).
• Tag editor YUI script appears to break occasionally.
• Perhaps move to two column forms to save vertical space?
• Notes field to monospace?
• Extras fields need some jQuery love -- shouldn't have a fixed number available.
• Typography is a bit of a mess.

• openness/downloadability ticks/crosses are totally inscrutable to non-techies (those who don't know about the title attribute). need better icons, and a key in a box on RHS.
• maybe highlight two-tick entries with a particular colour, but the open knowledge badge is horrendous, and breaks visual cadence.
• results data should be tabulated, with the project names and descriptions made most prominent. <li> blobs are unnecessary visual noise.
• maybe links should go on the full titles? Get rid of package names.
• Should have filter by name (à la KForge alphanum widget) and also by tag: list of most common tags in box on RHS?
• same comments as search listing (ticket:217)

• allow clicking on 'only openly licensed'/'only downloadable' to select/deselect checkboxes: make them <label>s.
• cut 'only's -- use word 'filter' instead
• doesn't appear to match substrings!? A search for 'anna' doesn't return match the 'annakarenina' package.
• autocomplete package names & tags

Tags landing page tag

• Both search for tags *and* browse the list of tags right here
• Big cloud of tags (sorted by popularity/usage?) filterable with a search box.
• Tags search page tag/search and list tag/list are both redundant and should be merged straight into tag.

Afaik the new CKAN 2.0 frontend has no functional tests

Investigate and list (on the wiki at wiki:UiReviewNotes) CKAN's UI issues, in particular:

• Form usability
  • best example: ​http://ckan.net/package/new
  • "Extras" fields
  • Adding to groups (from group page or from package page?)

• Overview pages: reduction in visual noise and unnecessary elements.
  • best example: package overview page
  • consider openness/downloadability icons
  • display common metadata fields in a more structured fashion?

Est cost: 2 hrs.

The group_index page has no support in WUI for ordering the groups displayed. Should allow sorting by name

Add support for this for datahub now, and discuss for new 1.9 UI

See trace below.

As in #2231, the schema passed with the context is no longer used on the logic layer (in this case the one used on the DGU form is used). This can be fixed allowing a schema to be passed with the context.

  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 139, in import_stage
    self.import_gemini_object(harvest_object.content)
  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 163, in import_gemini_object
    package = self.write_package_from_gemini_string(unicode_gemini_string)
  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 386, in write_package_from_gemin
    package = self._create_package_from_data(package_dict, package = package)
  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 486, in _create_package_from_dat
    package_dict = action_function(context, package_dict)
  File "/usr/lib/pymodules/python2.6/ckan/logic/action/update.py", line 169, in package_update
    data, errors = validate(data_dict, schema, context)
  File "/usr/lib/pymodules/python2.6/ckan/lib/navl/dictization_functions.py", line 216, in validate
    converted_data, errors = _validate(flattened, schema, context)
  File "/usr/lib/pymodules/python2.6/ckan/lib/navl/dictization_functions.py", line 289, in _validate
    convert(converter, key, converted_data, errors, context)
  File "/usr/lib/pymodules/python2.6/ckan/lib/navl/dictization_functions.py", line 179, in convert
    converter(key, converted_data, errors, context)
  File "/var/lib/ckan/std/pyenv/src/ckanext-dgu/ckanext/dgu/validators.py", line 81, in validate_license
    if data[('license_id',)]== '__extra__':
KeyError: ('access_constraints',)
2012-03-19 10:47:41,736 ERROR [ckanext.inspire.harvesters] Error importing Gemini document: ('access_constraints',)

Confusion between entering values in the title and name/url fields in datacatalogs.org.

The help text "Url must be purely lowercase alphanumeric (ascii) characters and these symbols: -_" applies to the name/url field, yet in Lucy's browser the text that is actually hovering over the title field in my browser.

Super ticket: #1506

• Show edit summary form again, at least on edit pages (and prefill it for create to something sensible like: Creating dataset {title} {name}) [Done #1506]
• Fix drop down for resource types in resource add/edit to limit to fixed set of options (as per ​http://wiki.ckan.org/Domain_Model/Resource)
  • See also ​http://lists.okfn.org/pipermail/ckan-discuss/2011-December/001860.html
• Allow reordering of resources (or: and this is somewhat more complex but possibly simpler: allow editing of a resource_order field and we use that to sort resources (this is more like wordpress page_order -- these two options could become the same if we just hijack the existing position field on the join table and get rid of ordereredlist extension)) [DONE #1506]
• Radically simplify set of licenses [DONE #1359]
• "Unsaved changes" warning briefly when you click "Save" #1735
• "Unsaved changed" typo on dataset edit page

From Ira: PDF of bugs and suggestions is attached.

This ticket is included in release v1.5. Post-v1.5 UX improvements are collected in #1404.

DONE ?

• [dataset/edit]: Deletions are confirmed with a dialog box
• [dataset/edit]: Long URLs break the edit resources tab. Show name (& 'no name') with a pencil icon.
• [dataset/edit]: Rearrange the delete resources mechanism;
  • Use a little minus icon on the far left, followed by a pencil icon for editing.
  • The 'delete resource' button can become 'hide'
  • Deletions are confirmed with a dialog box
• [dataset/edit]: Rearrange the 'you have unsaved changes' warning;
  • Delete the little green plus sign on the right;
  • Try a flash banner instead (more general purpose & beautiful)
• [dataset/edit]: Try a pencil icon (+'hide' button) rather than the +/- buttons which look a bit like you're adding a dataset.
• [dataset/view]: Subscribe button (& subscribe to history) doesn't work as expected. Modal dialog carrying instructions?
• [dataset/view]: Drop the 'add to a group>>' and 'add tag>>'. Needs inline tag-adder.
• [markdown editor]: Tab colours confuse the user. Inactive button appears to be greyed out.
• [dataset/edit]: The upload file UI mutates and breaks across lines; make it a little more rigid. Verify at different zoom levels.
• [groups/add]: Hide unimportant fields
• [navigation] Add 3D bevel (@see github) to current tab highlight.
• [dataset/create and groups/create]: Slug generator can be tidied. Default: Not editable. @see ​http://groupspaces.com/a/group/
• [groups/view]: Show description and owner. To match dataset section.
• [groups/authz]: Tidy to match admin/authz (pretty buttons & spacing). Add navigation bar.
• [groups]: Entire section to use pretty buttons please.
• [about]: Get the about page working on test.ckan.org. (Works locally and on live).

WONTFIX ?

• [dataset/edit]: Make 'tags' a top level form section; it is too important to go below the fold. (wontfix)
• [groups]: Clicking a column heading should sort by that column. Default should be 'size', not 'name'. (ticket #1403)
• ([groups]: (Investigate) Is it easy to implement a little search-groups feature?) (ticket #1403)
• [*/authz]: Add tooltips to explain column headers. (Not required if mechanism is about to be refactored...) (wontfix: we need to rethink this mechanism; authorization refactor is coming)
• [general]: 'Authorization' refactor to 'Permissions' on nav-bars. (wontfix, further discussion. authorization might be less techy...)
• [search]: (Investigate) Can we quickly & easily reintegrate the faceted search-by-license but present it in the sidebar like the other facets? (wontfix: high cost and maintenance overhead; need compelling user story)

I have a package with name: "Štatistické prehľady agendy cestovných pasov" and when I try to search for "Štatistické" there are no results.

CKAN setup:

• psql DB with UTF-8 tables

• psql search backend

• ckan version: 1.3.1b

Example:

​http://verejnedata.sk/package/agenda-cestovnch-pasov

try to search for the first word in the package name.

A possible bug in CKAN when I tried deleting users using "paster --plugin=ckan user delete" command.

To reproduce the bug do the following:

1. Create a user with an ID (which in my case was a user's full name)

that contains non-unicode caracters like Norwegian "æ", "ø", or "å".

2. Make sure that you can see something like the example below:

(pyenv) rm@mycomputer:$ paster --plugin=ckan user Users: name=Rustæm

3. Then try deleting the user with following command:

(pyenv) rm@mycomputer:$ paster --plugin=ckan user delete "Rustæm"

You should now get a python encoding error. I know that this is quite rare case, but in our case it caused some trouble. Could you guys have a look at this bug?

CKAN ver. 1.3.3.

You get a UnicodeDecodeError?, looks like it tries to decode the ini file as ascii. This is a problem because someone might try to set their site description to something with an umlaut in it, for example.

A test case is currently failing for me on master:

ERROR: ckan.tests.functional.test_user.TestUserController.test_user_create_unicode
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/seanh/pyenv/lib/python2.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/seanh/pyenv/src/ckan/ckan/tests/functional/test_user.py", line 342, in test_user_create_unicode
    res = res.follow()
  File "/usr/lib/pymodules/python2.6/paste/fixture.py", line 603, in follow
    return self.test_app.get(location, **kw)
  File "/usr/lib/pymodules/python2.6/paste/fixture.py", line 208, in get
    return self.do_request(req, status=status)
  File "/usr/lib/pymodules/python2.6/paste/fixture.py", line 389, in do_request
    **req.environ)
  File "/usr/lib/pymodules/python2.6/paste/wsgilib.py", line 343, in raw_interactive
    app_iter = application(basic_environ, start_response)
  File "/usr/lib/pymodules/python2.6/paste/lint.py", line 170, in lint_app
    iterator = application(environ, start_response_wrapper)
  File "/usr/lib/pymodules/python2.6/paste/cascade.py", line 130, in __call__
    return self.apps[-1](environ, start_response)
  File "/usr/lib/pymodules/python2.6/paste/registry.py", line 350, in __call__
    app_iter = self.application(environ, start_response)
  File "/usr/lib/pymodules/python2.6/repoze/who/middleware.py", line 69, in __call__
    auth_ids = self.authenticate(environ, classification, ids)
  File "/usr/lib/pymodules/python2.6/repoze/who/middleware.py", line 201, in authenticate
    userid = plugin.authenticate(environ, identity)
  File "/home/seanh/pyenv/src/ckan/ckan/lib/authenticator.py", line 29, in authenticate
    if user.validate_password(identity.get('password')):
  File "/home/seanh/pyenv/src/ckan/ckan/model/user.py", line 113, in validate_password
    hashed_pass = sha1(password_8bit + self.password[:40])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 12: ordinal not in range(128)

• Only logged in users able to create
• Creation permissions have only creators able to update a dataset
• Synchronous search is enabled (#1678)

Update CKAN wiki front page - a la OpenSpending?: ​http://wiki.openspending.org/Main_Page

Sections should relate to different types of people using the site:

Developers, Users etc..