Perhaps using deliverance?

• Move About to the right of the nav tabs and make it an about IATI page
• Make a Help / FAQ section linked of main menu
• Front page more dynamic - latest data registered and news (microblog thing)
• make the front page blurb noticable / shorter etc.
• link to iatistandard.org from the front page blurb

This would allow for a community-driven definition of field values, conventions, etc.

Plus: create wiki.ckan.net

Browsing to the admin interface /admin (even logged in as a sysadmin) gives an exception.

• Should license go in the bottom line?
• Formats should be styled as in CKAN

This has been noted twice in the last couple of weeks.

Checking around my guess is this is due to someone logging in to www.ckan.net and ckan.net. Google generates openid based on the RP (requesting party) which is the requesting domain url unless the openid.realm is set. See:

• ​http://groups.google.com/group/google-federated-login-api/web/the-most-important-technical-issue-in-using-the-google-accounts-api?pli=1
• ​http://stackoverflow.com/questions/3549953/how-to-get-unique-identifier-from-openid-google

To fix this we should set the openid.realm (not sure how atm).

If this is not possible an alternative is to make sure we redirect all users to a single site url (perhaps using ckan config option site_url, or doing this in apache).

Cost: 1h

The new field was outlined in

​http://lists.okfn.org/pipermail/ckan-discuss/2011-February/000887.html

it says

Can be "data", "metadata", "service", NULL. There is a need to do this for UKLP where some of the data associated with a package is metadata, some is data and some could be a service.

Try and minimise the amount of different caching methodologies, especially for api calls.

Schedule worker calls to be ran several times, with certain intervals.

as title.

While playing with the authorization groups, trying to design tests, I found that it was necessary to log in as two different users with two different browsers. Often actions of one user would cause server errors in the other user's browser.

I don't have a reproducible test case, but it happens fairly often so it shouldn't be too difficult to get one.

If you've got edit permission on an authzgroup, then you can change its name to be the same as another existing authzgroup.

This causes some strange UI effects at worst, and probably causes worse problems somewhere else.

Is there any reason why changing the names of existing authzgroups should be allowed? And if so, name collisions should presumably be guarded against in both the name-changing and creation functions

It is possible to use the CKAN API to insert JSON format data into package extra values, but this data is not found on searching.

Full text from Pascal:

we encountered a Problem concerning accessing Arrays/Lists.

curl -XGET 'http://ckan.net/api/rest/package/hbz_unioncatalog'

will get you amongst others:

 "extras": {"publishingInstitution":
"[u'http://lobid.org/organisation/DE-605',
u'http://lobid.org/organisation/DE-290',
u'http://lobid.org/organisation/DE-38M',
u'http://lobid.org/organisation/DE-98',
u'http://lobid.org/organisation/DE-38',
u'http://lobid.org/organisation/DE-Kn41',
u'http://lobid.org/organisation/DE-82',
u'http://lobid.org/organisation/DE-107',
u'http://lobid.org/organisation/DE-929',
u'http://lobid.org/organisation/DE-Zw1',
u'http://lobid.org/organisation/DE-832']"}

but if I try to query this:

wget
'http://ckan.net/api/search/package?q=lobid&publishingInstitution="http://lobid.org/organisation/DE-605"'

I get only two packages, among the package "hbz_unioncatalog" is
missing. (These two packages have only one value for
"publishingInstitution").

The "extra/publishingInstitution"-Array was uploaded through a "curl
-XPUT ...
 "extras": {
   "publishingInstitution":[
     "http://lobid.org/organisation/DE-605",
     "http://lobid.org/organisation/DE-290",
     "http://lobid.org/organisation/DE-38M",
     "http://lobid.org/organisation/DE-98",
     "http://lobid.org/organisation/DE-38",
     "http://lobid.org/organisation/DE-Kn41",
     "http://lobid.org/organisation/DE-82",
     "http://lobid.org/organisation/DE-107",
     "http://lobid.org/organisation/DE-929",
     "http://lobid.org/organisation/DE-Zw1",
     "http://lobid.org/organisation/DE-832"
   ]
   },
...

It could be interesting for the publicdata.eu project to add some CSW datasources. Thanks to the INSPIRE directive, there are quite a few national catalogs of geodata that could be worth harvesting. Work on this should include:

• Compose Endpoint List (1d)
• Harvest on staging system and evaluate which sources have useful material (2d)

First point is being performed in this GDoc: ​https://spreadsheets.google.com/spreadsheet/ccc?key=0Atp3cZFjuIOAdDBVQWRINnlfN1d0b2lleHVEdjBSb2c&authkey=CNu4hsEB&hl=en_US#gid=0

A new library, ​guiders.js, has been open sourced that seems to be great at unobtrusively introducing new users to features of websites.

The library drives Optimizely's website. The GitHub repo is here: ​https://github.com/jeff-optimizely/Guiders-JS

Some examples:

Produce a video explaining some of the things which can be done on the datahub.org

Abstract

Package Relationships have not taken off in the 18 months we've had them in the API. There are some issues with them and we need to spend more time improving them or consider getting rid of them.

The Problem

Original use cases are expressed here: #253 Here are comments about how we could handle these specific examples better:

1. groups of packages - maybe better with a custom tag?
2. fragment resources - soon to be covered by 'kind' resource field #957

3&5. derived resource - better to have some sort of resource relationship perhaps?

4. linked resource - again better to have some sort of resource relationship perhaps?

Outstanding issues needing serious effort to fix:

• #256 Editing them in Web UI (not done yet)
• #1288 Package edit/creation can't include 'relationships' field

Specification

Remove relationships from model, API, tests, Web UI. Data migration to remove from db.

Why do it this way

Getting frustrated having problems with the code, when it's not used much. Often asked about what it's for, but rarely used. Seems an overly complicated design.

Backwards Compatibility

n/a

Implementation plan

Deliverables

See Specification

Risks and mitigations

Risk: a customer suddenly wants this, and the new ways to relate resources are not in place yet.

Mitigation: discuss this decision thoroughly to make sure we are confident the use cases are not important. Discuss with team, ckan-discuss and specifically the LOD people who have some related packages on thedatahub.org.

Participants

David Read

Progress

Not yet.

Plugins implementing some IPackageController or IGroupController operations (i.e. read, create, edit, delete) may need the context used in the logic action. For instance, to access the Session that created a package to perform additional queries from the extension (In that case the main logic action will still perform the final commit).

There are issues with setting the site's language according to the browser detection:

• Many browsers have the language setting incorrectly set. e.g. people who download US versions of Mozilla, pirated US copies in the far east.

• Although thedatahub.org has a worldwide audience, many CKAN instances are designed for a particular country's citizens. So Open Berlin should probably be in German as default for all.

I suggest the site should have a default language saved in the config. The browser language *should* be detected, and that prompts a flash message offering to change to that language. And if you change language that is saved in the cookie (as we currently do).

Then do some testing to see if this suits people.

The current harvesting implementation can only have one instance per server.

We could either:

• Accept this
• Change the RabbitMQ code to support multiple instances
• Switch the entire harvesting to use Celery
• Wait for James's redis based live feedback based system

This could be implemented as a separate API call, or the rendered text versions of the activities could be added into the JSON returned by the existing API call.

This requires setting up templates for rendering activity streams items and detail items as nice, human-readable text.

There are some open questions, e.g.: Do we want the entire activity stream rendered as a block of plain text? As HTML? Or do we want a list of JSON objects, where each object contains its textual and/or HTML representations as fields?

Activity stream items and their related detail items are separate objects that each have their own textual representations.

For a mockup of the kind of text messages we want, see:

​http://datahub.pudo.org/pudo

but note that this ticket is just for creating the text snippets themselves, not rendering then in an HTML page or RSS feed. Also the mockup only show activity items and not their detail items.

Makes more sense to only have one comprehensive stats/analytics extension, so when people are looking to add a stats extension they won't have to add two which may be confusing (is one an old version of the other? why do I have to add two? what's the difference? etc)

User-wise we need a way to display our stats with google analytics in the same place

Allow a user to save a partially-complete and probably invalid package-create form in order to come back to it later.

We decided on blob storage over a state field on the package:

• prevents a history of the drafts being stored in the revision history. This is a particular problem if we start auto-saving, say every 20s.
• the blob can store arbitrary extra data, such as how far through the wizard we are.

CKAN is pulling a number of resources from external locations. This causes problems when connectivity is limited and you have to work locally. Maybe some of them cold be moved to CKAN source to avoid external requests.

Quick search:

./ckan/templates/layout_base.html:            <img src="http://assets.okfn.org/images/logo/okf_logo_white_and_green_tiny.png" id="footer-okf-logo" />
./ckan/templates/layout_base.html:            <a href="http://opendefinition.org/"><img alt="This Content and Data is Open" src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" style="border: none ; margin-bottom: -4px;"/></a>
./ckan/templates/package/resource_read.html:                <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" />
./ckan/templates/package/read.html:          <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" /></a>
./ckan/templates/_util.html:                    <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" />
./ckan/templates/_util.html:                  <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" />
./ckan/public/scripts/vendor/ckanjs/1.0.0/ckanjs.js:      this.$dialog.html('<h2>Loading results...</h2><img src="http://assets.okfn.org/images/icons/ajaxload-circle.gif" />');
./ckan/public/scripts/vendor/ckanjs/1.0.0/ckanjs.js:          self.setMessage('Uploading file ... <img src="http://assets.okfn.org/images/icons/ajaxload-circle.gif" class="spinner" />');
./ckan/public/scripts/vendor/ckanjs/1.0.0/ckanjs.js:      self.setMessage('Checking upload permissions ... <img src="http://assets.okfn.org/images/icons/ajaxload-circle.gif" class="spinner" />');
Binary file ./ckan/lib/app_globals.pyc matches
./ckan/lib/app_globals.py:                                  'http://assets.okfn.org/p/ckan/img/ckan.ico')
./ckan/config/deployment.ini_tmpl:ckan.favicon = http://assets.okfn.org/p/ckan/img/ckan.ico

Now that QA has been updated for Celery and CKAN 1.5.1, there are a number of improvements that we should probably implement.

• Integrate QA into frontend - 1d - #1583
• QA report improvements - 2.5d - #1584
• QA - Check license as part of 5 star rating - #1587
• QA - Give SPARQL endpoints a 4 star rating - #1588
• QA - Give 5 star rating to datasets with link metadata - #1589

Main pad: ​http://ckan.okfnpad.org/qa

Some QA discussion (relating to Celery in particular) at ​http://ckan.okfnpad.org/queue

Provide help to users in the WUI, with this linking to a detailed user guide.

• User guide (suggest at help.thedatahub.org (or possibly help.ckan.org)
• Snippets in the WUI with links to full user guide.
  • Provide helper methods for creating these links with config options for setting path to user guide so that the link location can vary on install by install basis
• General info about publishing/working with/etc data (where we don't duplicate elsewhere)

Est: (?) 5-10d

Topics

See ​http://ckan.okfnpad.org/datahub-help

• Step-by-step for adding a dataset including tips
  • Reuse: ​http://datapatterns.org/datahub.html
  • Quick overview of relevant parts of domain model (dataset and resource)
• API (perhaps link to main docs)
• Command line interface and dpm
• Using groups

Why do this

We already have wiki.ckan.org and docs.ckan.org. Why do this?

Answer for docs:

• This is administrative guide and focused on general CKAN software.
• Hard to provide completely generic help.
• Integratable with the DataHub? site

Answer for wiki:

• No spam
• General feeling that wikis are a little bit loose and untidy. We want this to be smart, clean and useful.
• Integratable with the DataHub?

johnglover said: I can confirm that even with the mapping fix, the ckanext-dgu dgu_form plugin template does not work properly with 1.5.1, so should probably not be installed at present. The edit page is ugly but should work (eg: ​http://dgu-os.okfn.org/dataset/edit/abandoned-vehicles), but the 'add a dataset' page is broken (eg: ​http://dgu-os.okfn.org/dataset/new)

Add logic functions for getting the activity stream for a dataset in JSON and rendered HTML formats, add HTML activity stream into dataset page.

Dump and load JSON explicitly instead.

Provide configuration for reverse proxying that will correctly handle the mapping of a URL to a sub-folder (using X-SCRIPT-NAME)

• Analysis of the best solution [1d]

• Implement reverse proxying in tandem with #1653 [2d]

• Document and store the configuration files. [1d]

This CSV resource used to preview but now the format appears to be unsupported: "We are unable to preview this type of resource: x-osdata-csv"

​http://thedatahub.org/dataset/lbhf-spending-2010/resource/9661abbd-2816-4d58-8b20-3cb0eb770c69

This is used as an example by the OpenSpending? team all the time.

The new group refactor allows for a hierarchy of groups where each group has a type - to be able to implement a tree of groups.

It will need a flag within the group to denote that it is a top level group, to circumvent the need to determine whether the group is a child of a parent node (for a specific use case - a publisher representing the department that only contains publishers).

Suggest 'is_top_level'

Add image attribute for Dataset and Group which would be shown in prominent place.

For dataset this could be a relevant image and for Group this would be more of a logo.

I think this has very high value from a UX point of view.

Issues

• Do we need to support uploading and storing the image (or do users store elsewhere)
  • Could use something like transloadit service
  • Use our own storage facility with a reserved directory
• Do we need to do resizing etc?
  • Maybe. Perhaps can prefer this in reasonably small size and then just size down in css for thumbnails etc.

Super ticket #1744

If you edit a dataset, simply add a tag and click on "Save" then you get the message "You have unsaved changed. Make sure to click 'Save Changes' below before leaving this page." inserted whilst the submission takes place. (This is because the click moves focus away from the tag field). This (reasonably common) edge-case could usefully be squashed.

Also note typo: "unsaved changed"

Currently user, package and group activity streams only return the most recent 15 activities, even though all activities are kept in the db. Do we want to add pagination - to both the API and the HTML pages - to support retrieving older activities?

Which is where the other API tests have now been moved.

The HTML templates for some activity stream activities link to objects (packages, resources, etc.) by their names, e.g. /dataset/foo, but if the object has since been renamed then the link will be broken.

Possible solutions are to use the object ID to find its current name at rendering time and link to that, or link to the object by ID rather than by name.

Linking by ID is both simpler to implement and better. Links by ID are ugly, but they are permanent and don't break when an object is renamed. We should be using links by ID everywhere on CKAN, so that when third-party sites copy-paste our URLs and link to us, their links don't break later.

Fixing this means fixing the dataset_link(), resource_link(), group_link() etc. functions in ckan/lib/helpers.py which will change the links everywhere in CKAN where those functions are used to link to objects (which should be everywhere where we link to an object).

If we want to have nice looking by-name URLs and what them to be permanent, then we need to change CKAN so that objects cannot be renamed.

• Tag list not be added to CKAN core, what is the best way to do this in an extension?
• Does the list need to be updated via API/WUI?

• Get rid of jquery UI
  • Requires switching to bootstrap lookahead plus a small bit of coding for ajax part (see ​https://gist.github.com/2007241)
• Remove modernizr (what do we use it for?)
• Convert jquery.tmpl to jquery mustache (?)
• Remove ckanjs moving core parts of it directly into our code.

Compress everything for releases and add minified version to build (with switch to enable use of this?)

Somewhere for CMAP stuff not in other tickets

need to create some general tickets

• template changes
• general demo server setup

There are a couple of strings on the authorization group add/edit form that are not being translated:

"Name: Please enter a value" (error message when you try to add an auth group with no name)

"User name"

"Save" button

The form seems to be built using FormBuilder?. See ckan/forms/authorizationgroup.py, builder.set_displayed_fields()

• Add home page url
• Remove group option (seen it confuse several users) - perhaps replace with tags (?)
• Check form looks ok in portugese (seems to be a problem with css)

I won't be needed for this during the current sprint, closing.