{22} Trac tickets (2647 matches)

CKAN provides the /data page page of data.gov.uk

Breakdown of tasks:

• [x] Analysis / refinement of spec.
• [x] Log-in / register as publisher

(Waiting on publisher form)

• [x] Population of "browse by publisher"

list groups ordered by most datasets (Waiting on publisher integration)

• [ ] Browse by nation
• [ ] Featured datasets

Now a possible integration point with drupal

• [X] Tag cloud
• [X] Developers section

• [x] hide black stars on the data-view page
• [x] Edit on view form (if you have the right permission)
• [x] RES_FORMAT ~> Resource Format
• [x] Map Based Search -> UK Location Map Based Search
• [x] Note at the top right expalining that these are seach filters (if they are) -> this requires "add a dataset" and "other access" to be moved to a different place (just some whitespace is fine)
• [x] Display publisher and provider (UKLP records only).
• [x] analysis of filter by OGL / non-OGL
• [x] provide /dump on the build server

Only sysadmin can access create publisher form.

Three small changes:

• Set Resource type to None instead of 'Undefined'
• Set extra 'UKLP' = True instead of 'INSPIRE' = True

These two changes will require manual update of the database.

• Allow datasets to be updated if the old source is inactive, even if the metadata date hasn't changed:

    if last_harvested_object.metadata_modified_date is None \
       or last_harvested_object.metadata_modified_date < self.obj.metadata_modified_date \
       or self.force_import \
       or last_harvested_object.metadata_modified_date == self.obj.metadata_modified_date and last_harvested_object.source.active is False:

You get an exception if you use the API to search packages and specify a non-ascii character in a field other than q.

For example:

http://catalogue.data.gov.uk/api/search/package?title=%E2%80%93

This "N-dash" (Unicode character 2013) causes this exception:

Module ckan.controllers.api:460 in search
<<                      if ver in u'12':
                               # Otherwise, put all unrecognised ones into the q parameter
                               params = convert_legacy_parameters_to_solr(params)
                           query = query_for(model.Package)
                           results = query.run(params)
>>  params = convert_legacy_parameters_to_solr(params)
Module ckan.lib.search.query:38 in convert_legacy_parameters_to_solr
<<      for search_key in non_solr_params:
               value_obj = legacy_params[search_key]
               value = str(value_obj).replace('+', ' ')
               if search_key == 'all_fields':
                   if value:
>>  value = str(value_obj).replace('+', ' ')
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 30: ordinal not in range(128)

This problem affects CKAN 1.5 and 1.5.1 only.

A new table with 3 columns should be added. term, term_tranlastion, language_code. This table will be used for all translations, including tags. The table should have indexs on both the term and (term, language_code) combination.

Basic description of multilingual features intended to be supported.

Need to test:

• API call to create new vocab
• API call to add tags with a given vocab
• API call to update tags
• Dummy plugin to test that updating tags works via WUI

Add API calls for adding one or more tags to and removing one or more tags from a vocabulary, without affecting the other tags in that vocabulary and without having to pass the full list of the vocabulary's tags.

Changes to publishers for customer

When you are on the /dataset/new page and you try and change locale then you get a 500 error.

This is because it adds the 'cache' parameter, to ensure any proxy cache in the chain does not just send the cached page.

e.g. ​http://127.0.0.1:5000/dataset/new?__cache=37713707

Major UX grip and incredibly simple to do. May also want to have basic autocomplete support within the list (trivial to do as well I believe).

(Talk to RP about this re how we define lists of licenses).

In the tag-taxonomies branch tags now (optionally) belong to vocabularies and where tags previously had unique names, a CKAN instance can now have two tags with the same name as long as they belong to different vocabularies. The tests for the tags model should probably be updated to reflect this.

Other tests might also need to be updated.

Add Vocabulary domain class, add logic functions for creating, updating, listing, getting, deleting vocabularies, add tests.

This would allow deployment via Nginx or Apache using proxy to Paster, uWSGI. At the moment CKAN isn't aware of the proxy's IP address so when you perform an action which does a redirect (such as adding a package), CKAN redirects you to the *internal IP* not the external *proxy IP*.

We would like this work to facilitate testing within VMs as part of our new build infrastructure.

It would also be nice if CKAN worked when mounted at a path other than /. That could be dealt with in another ticket because it isn't a problem at the moment.

An install of CKAN should be configurable without changing any of the installed files. This makes it clear to upgrade CKAN. Complete the changes in this wiki page to allow static files to be served from outside CKAN paths over CKAN versions and additional CSS file to be pulled in.

​http://wiki.okfn.org/ckan/doc/theme

CKAN 1.5 installations need to be upgrade to 1.5.1.

Working methodically through the CKANs on ​http://datacatalogs.org/group/ckan :

At 1.5:

• ckan-brazil ​http://br.ckan.net/ s057 - DONE
• ckan_ireland ​http://ie.ckan.net/ s057 - DONE
• cz-ckan-net ​http://cz.ckan.net/ s057 - DONE
• datakilder-no ​http://no.ckan.net/ s057 - DONE
• data_gov_uk ​http://catalogue.data.gov.uk/
• datacatalogs-org ​http://datacatalogs.org/ s054 - DONE

Other versions:

• berlin-open-data ​http://daten.berlin.de/ CKAN hidden
• ckan-italia ​http://it.ckan.net/ 1.3.3b
• ckan-poland ​http://pl.ckan.net/ Down
• data-norge-no ​http://data.norge.no/ CKAN hidden
• datadotgc-ca ​http://datadotgc.ca/ ​http://ca.ckan.net/ 1.4.2a
• datagm ​http://www.datagm.org.uk/ 1.3.3
• datagov-ru ​http://datagov.ru/ CKAN hidden
• datospublicos-org ​http://datospublicos.org/ 1.4.2
• iati-registry ​http://iatiregistry.org/ 1.5.1c
• katalog-offene-daten-oesterreich ​http://offener.datenkatalog.at/ 1.3.3b
• nosdonnees_fr ​http://www.nosdonnees.fr/ 1.3.1a
• opengov-fi ​http://opengov.fi/ CKAN hidden
• opengov_es ​http://opengov.es/ 1.2
• publicdata-eu ​http://publicdata.eu/ 1.4.2a
• the-data-hub ​http://thedatahub.org/ 1.5.1
• helsinki_finland ​http://hri.fi/ CKAN hidden

Now deleted from datacatalogs.org:

• ckan-belgium ​http://be.ckan.net/ Migrated
• si-ckan-net ​http://si.ckan.net/ migrated
• gr-ckan-net ​http://gr.ckan.net/ Migrated
• hu-ckan-net ​http://hu.ckan.net/ Migrated
• ckan-lithuania ​http://lt.ckan.net/ Migrated

History on user profile should only be visible to a user who is a member of at least one group.

Super #1669

Authz for the publisher admin (who has permission to administer the publisher group themselves).

The publisher index page should show a treeview of the publishers within the system. Probably behind a tab.

• Lists of entities should be full URLs, rather than just the names
• Discoverability - /api/v3/rest should list the entity types that can be listed

This could be v3 of the RESTful interface.

I have a package with name: "Štatistické prehľady agendy cestovných pasov" and when I try to search for "Štatistické" there are no results.

CKAN setup:

• psql DB with UTF-8 tables

• psql search backend

• ckan version: 1.3.1b

Example:

​http://verejnedata.sk/package/agenda-cestovnch-pasov

try to search for the first word in the package name.

Part of #1698

Super ticket #1744

If you edit a dataset, simply add a tag and click on "Save" then you get the message "You have unsaved changed. Make sure to click 'Save Changes' below before leaving this page." inserted whilst the submission takes place. (This is because the click moves focus away from the tag field). This (reasonably common) edge-case could usefully be squashed.

Also note typo: "unsaved changed"

Super ticket: #1506

e.g. ​http://thedatahub.org/user The users with Gravatar have their nice user icons, but the majority of users (without Gravatars) have 'broken link' symbols. Same problem wherever users are shown, such as the dataset history pages.

Broken on 1.5.2b - currently on thedatahub.org.

To allow navigation between sections to still work, also potentially rephrase to avoid confusion that you have to save every time to add a tag or resource and can't say add description, tag and resource in one go

The current harvesting implementation can only have one instance per server.

We could either:

• Accept this
• Change the RabbitMQ code to support multiple instances
• Switch the entire harvesting to use Celery
• Wait for James's redis based live feedback based system

For example, after deleting a dataset old activity stream events will still hyperlink to that dataset.

The HTML and CSS for the activity streams templates needs to be tidied up. Currently the CSS is inlined in the templates themselves, and there is much unnecessary repetition of CSS.

Perhaps put this one in hold until there has been some design discussion about how the activity streams and the user, dataset and group pages with activity streams should look?

Super #1669

Editor authz for publisher groups need to be implemented and tested.

Super #1669

Authz for normal users in publisher profile as per sysadmin

Right now is quite difficult to get the most current version of a package via the harvest objects, and the queries involved are prone to errors. A new "current" field will be added which will identify unequivocally the most recent harvested object. Unfortunately this will involve some major changes:

• Model update (See #1726)
• Update queries for getting harvest sources' datasets
• Update ckanext-csw queries for getting the objects
• Update inspire and ckan harvesters
• Document new field

Test failures in master due to tom and rufus not running tests. 0.3d.

We need an API for changing the authorization (permissions) on a model object.

This was neglected in #1253 because of various changes to authz were looming. But six months later it still hasn't been done.

Suggest this is just moving _add_user_object_role from ckan/lib/base.py to the logic layer, but no doubt it will be more complicated.

Estimate: 3 days

Richard Cyganiak: I often see a problem where half the system thinks that I'm logged in, while the other half insists I'm not. Logout then login again may fix it.

David Read: I think I've seen it several times that the "My Account" link is given (suggesting you're logged in) but when you click it it asks you to log in, which means you weren't.

We already have first pass of Data Preview that was done as part of #1357. This is about v2 of this. Two main components:

• A new version of the separate Data Explorer library (now called Recline Data Explorer) - #1446 Est: 10-15d
  • Why:
    • Cleaner, backbone based solution.
    • Provides editing support if wanted.
    • Collaboration with Max Ogden
• Integrate this into CKAN (into resource view page) - see #1630 Est: 1d
• Improvements to Data Viewer / Previewer system in CKAN itself - Est: 2-3d
  • (Update Data Preview-er (View-er) to support images ...)
  • (WONTFIX here - (?) Update Data Preview-er to support geodata - #1151)

From Mark on the ckan-dev list:

In the table of broken links at <​http://thedatahub.org/qa/dataset/broken_resource_links>, the links as printed should be abridged to a fixed length (but link to the full URL). Otherwise the 'reason' column vanishes off the screen. Also a table would probably be better since as things stand the 'reasons' are not lined up

This CSV resource used to preview but now the format appears to be unsupported: "We are unable to preview this type of resource: x-osdata-csv"

​http://thedatahub.org/dataset/lbhf-spending-2010/resource/9661abbd-2816-4d58-8b20-3cb0eb770c69

This is used as an example by the OpenSpending? team all the time.

Eurostat datasets mainly.

This ticket is about finalizing, then deploying (to ​http://thedatahub.org/), and finally announcing some of recent major new features such as:

• Deploy wordpresser extension for thedatahub (so we can start blogging) - #1533 - WONTFIX
• (Deploy new Resource and Dataset pages (invite comments))
• Get thedatahub geo-ified i.e. ckanext-spatial - #1601 (?)
  • Any additional dev that needs to be done? (e.g. geo in search ...?)
• (Deploy / re-enable ckanext-googleanalytics on thedatahub - #1451 (more stuff #1566))
  • Rename the extension to ckanext-analytics?
• Custom forms (and use of 'dataset types') - Ticket (??)
• (Get QA deployed and running - #1582)

It would be best if this was done by someone *unfamiliar* with the extensions so feedback could be provided on quality of install docs etc etc.

Search results page of DGU.

Breakdown:

[X] ensure faceting by group (publisher) works [X] allow faceting by UKLP dataset type [X] allow faceting by INSPIRE

[Child of #1526]

Can't change a dataset from containing time-series resources to one containing individual resources or visa-versa.

We want to make email announcements to users of thedatahub.org.

The most simple way to do this is to get a dump of the email addresses and subscribe/invite them to a mailman list.

Different ways to do it:

• 'paster dump-users-csv' creates a CSV file. Apache serves up the directory using password protection. Is this secure enough?
• CKAN admin pages provides a button to produce the CSV and download it. A config option may be necessary to enable this (disabled by default).

Something more advanced in the future could be:

• CKAN tells mailman about new subscribers using the mailman web interface. Could be brittle.
• Mailman doesn't have an Internet API, but it does have a python interface. So CKAN could tell mailman about new subscribers using a python proxy. There may be one out there, or maybe easy to do. But annoying to have to maintain another process running on machine.
• CKAN sends out the emails itself. A sysadmin is given a form. Users have an opt in/out option in their account. But receiving emails is a sensitive thing so it would need to be right first time, and there are several risks to this approach.

The SOLR url, user and password defined in the CKAN config file are ignored and the default SOLR url is used.

This causes:

• "0 datasets" displayed on the home page
• Dataset searches result in 0 results and a small message "There was an error while searching". (Nothing about it in the logs)

To reproduce

This bug is only visible if your SOLR instance is not at the default place. To quickly reproduce this problem, setup your machine as a SOLR multicore instance and run: "paster db clean && paster create-test-data && paster serve development.ini". It quits with the error: "solr.core.SolrException?: HTTP code=400, reason=Missing solr core name in path"

Code affected

• Does not affect any released versions (1.5.1 is clean).
• Introduced to master: ​https://github.com/okfn/ckan/commit/627382e6ff575146da221efc50321a41152d8b0f by Ross

An API call that returns a user's public activity stream in JSON format.

Document deployment on tenforce test server.