Tasks include:

• #1813 Update ckanext-pdeu (4d)
• #1814 Update harvesters (2d)
• #1649 Verify ckanext-rdf works with latest CKAN (3d)
• #1815 Reenable Sparql endpoint (?)
• #1816 Update ckanext-apps (2-3d)

Add image attribute for Dataset and Group which would be shown in prominent place.

For dataset this could be a relevant image and for Group this would be more of a logo.

I think this has very high value from a UX point of view.

Issues

• Do we need to support uploading and storing the image (or do users store elsewhere)
  • Could use something like transloadit service
  • Use our own storage facility with a reserved directory
• Do we need to do resizing etc?
  • Maybe. Perhaps can prefer this in reasonably small size and then just size down in css for thumbnails etc.

Some changes are needed in ckanext-harvest to support the new piece of work in the UKLP harvesters, mainly related to the deletion of existing sources.

• #1726 Update harvesting model
• #1727 Add "current" field to harvesting objects
• #1728 Update UKLP harvesters
• #1733 Delete (inactivate) harvest sources

At some point we also want to support a publisher model for the harvesting:

• #1731 Publisher support for harvesting

Suggest:

%title. %author. Retrieved %date. %site_title.

For resource: %title = %dataset_title. %resource_name.

There are a couple of strings on the authorization group add/edit form that are not being translated:

"Name: Please enter a value" (error message when you try to add an auth group with no name)

"User name"

"Save" button

The form seems to be built using FormBuilder?. See ckan/forms/authorizationgroup.py, builder.set_displayed_fields()

A place to list crucial fixes for v1.7 release:

Related extension:

All not complete now moved to #2347

• Related Item has comments on index page (they should not show up here -- why is this happening!)
• Related item should be to left of history (history could in fact move to RHS)
  • What about losing icons since we duplicate for related item
• Related item plus sign is very poorly aligned within box
  • Add related items should be in drop down like for resources (?) to make it easy to add items
  • Hover should show a title that gives some information about what related are (not at all clear)
• Related item only includes App, Idea, Visualization but not API | Post | Paper | News Article options as in original ticket #2204

Search results:

• #2331 - search should add not or
• Search results should be similar across group pages and normal search results (suggest adopting group page style)
  • Search results resource link should link to resource page rather than url for that resource
  • Suggest we could tidy this a bit (e.g. have resources in mini-sidebar of search result ...?)

Data viewer:

• Merge updates to Recline (that did go in before deadline but do not seem to have made v1.7 branch) ​https://github.com/okfn/ckan/commit/146a5c1130cc77cb2ad2c6ef6b3a53074908bf07
  • Ensure still using read-only (this seems to have been lost in upgrade)
• Do not show failed preview with red-bar (esp when just empty datastore ...)

a) Data Viewer checks if datastore is empty nad falls back to ther options if emtpy b) DataStore? enabled is off by default and only turned on either explicitly by user or by DataStorer? on successful store ...

• (? not really a fix) Give out a link as well as embed instructions (I often want to link to the explorer in a given state)
  • Should this link to the resource page or to a dedicated viewer page - cf # 2322 (resource sub-urls like /download, /api ...)
  • Do #2322 ...
• Decent powered by icon for the Embed for the DataHub? (make this text customizable and allow html ...)
• Embed link uses uuid rather than dataset name (this goes to perma-url question ...)
• Data viewer embed url includes all the resource info

Dataset page:

• Put home page link somewhere more prominent (e.g. the top of the sidebar)
• Put author more prominent (at top right)
• (? - new work) List CKAN owners at top right in sidebar
• Make resources stand out more
• Rename resources tab and heading to Data & Resources

Dataset edit and create:

• Corrections to dataset creation form for v1.7 - #2317
• Better auto-complete for groups on dataset edit/new page - #2319
  • Remove groups box and replace with tags on the new page (or is this needed for organizations stuff?)
• Description / notes text area is not wide enough.
  • Should also be larger on edit page
• Problem with date instructions on add/edit resource: "Dates are in ISO Format — eg. 2012-12-25 or %c2010-05-31T14:30%d."
• Add/edit resource: new code to auto-check urls being added is great but from UX point of view has draw back that (esp on low speed connections) page just seems to hang for a long time with nothing happening. I wonder if either
  • A) this could happen after resource created on form (e.g. it creates, and then an overlay comes up saying: checking url and gathering url information or similar)
  • Or B) we still do this before creating new resource form on page but we still should show something like a spinner and some info like "checking url and gathering url info ...)

Resource view:

• Do not show no description in no description (just show dataset info)

Theme:

• Fix spacing above page title (i.e. dataset title etc)
• Bootstrap 2.0.2 (@zephod was *supposed* to have done this) - should be trivial and gives important bugfixes for data viewer
• Fix subnav width on resources pages (does not extend all the way to the right like it should)
• Fix spacing of top menu and search box (do not look right)
• Group pages:
  • view page: image / logo is really poorly locate and sized, see e.g. ​http://datahub.io/group/wikimedia
  • view page: admin heading on RHS sidebar is wrongly indented

Miscellaneous:

• Ensure that we out-of-the-box config for CKAN has (#2388):
  • Only logged in users able to create
  • Creation permissions have only creators able to update a dataset

Deployments (without deployment we cannot know these are working):

• Social extension
• Analytics

If a user enters a url with no protocol (eg: www.data.gov.uk) in the 'url' field of a dataset, a broken link is created on the view page.

CKAN should probably either put in a ​http:// if no protocol is given, or validation should fail.

Nils flagged up that Ubuntu launches another LTS in April which we should probably move to. This ticket is to get the ball rolling with what we do. We can always include in our package the exact package versions of deps that we want, but it is good to be in line with the release if possible.

Here's how to see the packages included with it: ​http://packages.ubuntu.com/search?keywords=python-pylons&searchon=names&suite=all&section=all

Pylons 0.9.7 -> 1.0-2 Requires a small amount of work SQLAlchemy 0.7.3-2 OK Babel 0.9.6-1 OK Genshi 0.6-2 Perfect

Another alternative is to move to another web framework, especially minding that Pylons is not being further developed.

Right now you need to explicitly load the synchronous_search plugin in your ini file, when this is probably the behaviour that all users expect by default. We could keep a config flag to deactivate it, but synchronous search should be the default behaviour.

Super ticket: #1745

• Convert /api/rest/dataset to be search query (i.e. take arguments in ?....)
• Directly expose solr though may want to override limit. See ​https://github.com/okfn/openspending/blob/master/openspending/ui/controllers/api.py#L48

Required for some improvements to UX (such as autocomplete and better search).

Unfortunately ES is a little bit complex / opaque to newcomers. Need to write a simple tutorial walking through this.

This is documentation / example only (no code!)

There are a couple of flash messages in ckan/controllers/user.py that show up in English, even if another language is in use, and even though the strings are marked for translation with gettext and have translations in the po and mo files:

"[USER NAME] is now logged in"

"Login failed. Bad username or password. (Or if using OpenID, it hasn't been associated with a user account.)"

User story #035 here: ​http://ckan.okfnpad.org/feature-1515-activity-streams-user-stories

Technical analysis here: ​http://ckan.okfnpad.org/27

In branch feature-1821-multilingual-extension, a new faceting datastructure was introduced. This makes the old one and related functions obsolete. These have been marked as deprecated for the 1.7 release, and should be removed for the 1.8 release.

• helpers.py:facet_items()
• facets.html:facet_sidebar()
• uses of c.facets (rather than c.new_facets)

Some additions to the related stuff functionality to make it more useful post the 1.7 release.

[x] Edit item (after checking auth)

[x] My related things on my user page

[x] Show user created related things on the users' page with links direct to the related page of that dataset. {0.5d}

[x] Related stuff usage counts { 1d }

Record the number of times the link for a related item is clicked. This is likely to mean that the link fires to call CKAN and then redirects to the actual URL.

[x] Un-attached related items {0.25d}

Allow related items to be created without being attached to datasets. This may mean that the dataset field is optional in the API.

[ ] Dashboard for related

A dashboard for related items, site-wide (not package specific) and orderable by date, views, {rating}, popularity of the dataset. Would be useful (possibly) for site-admin to mark one or more as 'featured'

[x] Dashboard + Controller (Related) {1d}

[x] Ordering at logic layer (date, views, rating, popularity of package, featured) {1d}

[x] date [x] views [x] featured

[x] 'Featured' related item with mechanism to fetch it. {0.5d}

[ ] UI within dashboard for sysadmins to check/uncheck feature status {0.5d}

[x] Changing auth for related.

Currently auth is based on deletion being possible by dataset owner and the related owner but it may be that a lot of the auth for related items should be configured to have more restrictive auth.

[x] Implement auth changes for publisher auth profile. { 0.5d }

[x] General issues

[x] Serverside validation of URLs

[x] Activity updates

[x] Make optional

[x] Related item only includes App, Idea, Visualization but not API | Post | Paper | News Article options as in original ticket #2204

[ ] UI Changes

[ ] General tidy up, fix form etc (icon for menu bar)

[ ] Highlight featured items py:if="related.featured"

[ ] Show either view count or popularity ${ related.view_count }

[ ] A user's related items should be on their home page

[ ] Related item plus sign is very poorly aligned within box

[ ] Add related items should be in drop down like for resources (?) to make it easy to add items

[ ] Hover should show a title that gives some information about what related are (not at all clear)

"Apps, Ideas, Visualization and other material related to this dataset"

[x] Related is a pretty terrible name in the UI - much better to have it called Apps, Ideas etc (perhaps title tag could even give more details e.g. "Apps, Ideas, Visualization and other material related to this dataset") -- btw i thought this was in #2332 but realize it wasn't (apologies for that!)

• Went with Apps & Ideas (for now)

[x] Documentation, documentation, documentation - AFAICT I can't see anything in v1.7 or master docs. I imagine this would be a short section like ​http://docs.ckan.org/en/ckan-1.7/commenting.html (and probably coming right after that in the ToC) which says

• What it does
• How to enable (and perhaps includes a nice-screenshot!)
• How to customize (e.g. can one customize the list of options of things one can create (e.g. can I set it to just be app and idea or ...)
• Also liase with Mark W re a blog post about this from the user perspective (a screenshot walkthrough ...)

[ ] Clearly mark the old extension as deprecated from v1.7 forward: ​https://github.com/okfn/ckanext-apps (I just met someone last week who was working on integrating this and had no idea it was replaced by something better in v1.7)

Currently in the logic function tag_search you can specify limit and offset, but no count is returned. Therefore pagination is not possible for tag results. This is desired though.

• Only logged in users able to create
• Creation permissions have only creators able to update a dataset
• Synchronous search is enabled (#1678)

root@lucid32:/usr/local/feature-2375-demo-theme/pyenv/src/ckan# sudo -u www-data /usr/local/feature-2375-demo-theme/pyenv/bin/paster --plugin=ckanext-spatial spatial initdb 4326 --config=/usr/local/feature-2375-demo-theme/pyenv/src/ckan/development.ini Traceback (most recent call last):

File "/usr/local/feature-2375-demo-theme/pyenv/bin/paster", line 9, in <module>

load_entry_point('PasteScript?==1.7.3', 'console_scripts', 'paster')()

File "/usr/local/feature-2375-demo-theme/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 84, in run

invoke(command, command_name, options, args[1:])

File "/usr/local/feature-2375-demo-theme/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 123, in invoke

exit_code = runner.run(args)

File "/usr/local/feature-2375-demo-theme/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 218, in run

result = self.command()

File "/usr/local/feature-2375-demo-theme/pyenv/src/ckanext-spatial/ckanext/spatial/commands/spatial.py", line 38, in command

self._load_config()

File "/usr/local/feature-2375-demo-theme/pyenv/src/ckan/ckan/lib/cli.py", line 53, in _load_config

load_environment(conf.global_conf, conf.local_conf)

File "/usr/local/feature-2375-demo-theme/pyenv/src/ckan/ckan/config/environment.py", line 204, in load_environment

plugin.configure(config)

File "/usr/local/feature-2375-demo-theme/pyenv/src/ckanext-spatial/ckanext/spatial/plugin.py", line 60, in configure

setup_model()

File "/usr/local/feature-2375-demo-theme/pyenv/src/ckanext-spatial/ckanext/spatial/model.py", line 20, in setup

define_spatial_tables(srid)

File "/usr/local/feature-2375-demo-theme/pyenv/src/ckanext-spatial/ckanext/spatial/model.py", line 66, in define_spatial_tables

package_extent_table = Table('package_extent', metadata,

NameError?: global name 'metadata' is not defined

see ​https://github.com/okfn/ckan/pull/5

This action is not currently usable via a GET request as it relies upon a nested dict of fields.

After moving to Bootstrap the tables on the frontend look broken.

There's no docs for setting up the internal analytics, nor what it provides.

Setup

Add: ckan.tracking_enabled = true to .ini file

Run: paster tracking -c dev.ini

Related links

​http://trac.ckan.org/ticket/2251

Looks like recent CKAN cleanups broke some imports:

​https://gist.github.com/2935639

sqlalchemy fails on postgres 9.1.4 when unicode is returned

see ​http://docs.sqlalchemy.org/en/rel_0_7/dialects/postgresql.html

If you're not logged in and you click this button, it takes you to the add group page not the login page. Anyway, the default behaviour everywhere else in CKAN is to hide buttons that you don't have permission to use (e.g. add dataset, edit dataset, etc.)

From DR at DGU

Basically get_groups() appears to cache its results, but this falls down when subsequent calls are in a different session. We get this when saving a dataset. get_groups() first gets called in the controller when it is trying to work out what permissions the user has. It later gets called, and by this time c.userobj is detached, so get_groups() fails.

I'm working around this by refreshing c.userobj (from c.user) before calling get_groups(), but I wonder if the get_groups caching could detect the detached session and bypass the cache in this case?

diff at ​https://gist.github.com/3003117

IDatasetForm's form_to_db_schema() method is not getting called by ckan

make sure documented

New recline version has significant improvements like the SlickGrid? based grid view, it'd be good to have it on 1.8.

When a resource doesn't have a description, it is rendered as markdown from the description of the dataset.

If the resource does have a description it isn't rendered as markdown ..

See ​http://thedatahub.org/dataset/wikipedia-e3-timestamp-position-modification/resource/d883ab44-07f4-4992-800a-3e4bf5d53a96

I believe it should only return active ones. On DGU we see deleted publishers popping up.

CKAN's internal tracking seems to count each page view twice, the problem appears to be with the SQL in the update_tracking() method in ckan/lib/cli.py.

The internal tracking feature needs some tests, and some of the code could maybe do with some more explanatory comments, e.g. what is the intended difference between count and running_total?

It doesn't pass extras_as_string

Add it to a 'Troubleshooting' section on the filestore page: ​http://docs.ckan.org/en/ckan-1.7.1/filestore.html

For the error messages and their solutions, see various threads on ckan-dev

Luca De Santis reported these strings not marked in 1.7.1:

• /user/login.html ("Remember me" & "Sign in")
• /package/search.html and template/package/group/read.html (Tags, Resource Formats, Groups)
• /package/resource_read.html (the "No description" message)
• the "Add related item" dialog (some strings are i18n savvy, others not)
• the stats page (labels on the graphs)
• the history page (as far as the log messages are concerned).

lets check that they are marked in 1.8

Upload the new pot file/resource on transifex, put out the call for translations. Some recent commits on master need to be merged into 1.8 first

Over-enthusiastic removal of some model items

this breaks qa extension

Run CKAN with the publisher profile and organizations plugins enabled. Make an organization. Make a dataset and add it to the organization. Edit the organization and change e.g. its description. After saving the organization, all its datasets have been removed from the organization and now belong to no organization.

Set a dataset's license to 'Other (Not Open)', the dataset will have an Open Data icon next to it

If the group dict that you post to the API does not have any 'packages' key then all the group's packages get removed. I think it would be better if you could just update e.g. the group's description without having to also post the list of packages, and apparently this is how other update API actions work.

Might be worth checking all the update API actions for this behaviour, and making sure they're all consistent

e.g. 2012_08(주요국가).xls

I tested in CKAN 2.0 and it seemed to work fine there.

• Graphs not showing (looks like a flot related file is missing)
• Wrong groups counts (e.g. Data Explorer Examples show 1800 datasets when it onlu has 8)

Just to track @tauberer patch on Github. Probably going to 1.8.1