CKAN: Ticket Query

#1708: SOLR configuration lost

dread — Wed, 25 Jan 2012 12:10:28 GMT

The SOLR url, user and password defined in the CKAN config file are ignored and the default SOLR url is used.

This causes:

"0 datasets" displayed on the home page
Dataset searches result in 0 results and a small message "There was an error while searching". (Nothing about it in the logs)

To reproduce

This bug is only visible if your SOLR instance is not at the default place. To quickly reproduce this problem, setup your machine as a SOLR multicore instance and run: "paster db clean && paster create-test-data && paster serve development.ini". It quits with the error: "solr.core.SolrException?: HTTP code=400, reason=Missing solr core name in path"

Code affected

Does not affect any released versions (1.5.1 is clean).
Introduced to master: https://github.com/okfn/ckan/commit/627382e6ff575146da221efc50321a41152d8b0f by Ross

#1396: Improvements to user page UX - 1d

rgrp — Thu, 13 Oct 2011 17:46:07 GMT

Super ticket: #1506

Show member since
Display email (in obfuscated form)
List all owned datasets (in standard manner)
- Will need change to logic layer to show datasets on user dict (this is useful for the API as well ...)
(Re)move list of changes
- Should have activity (once activity available)

WONTFIX:

?? Allow users to provide a home page, organization (requires migration)

#1688: API for changing the permissions on an object

dread — Fri, 20 Jan 2012 12:16:49 GMT

We need an API for changing the authorization (permissions) on a model object.

This was neglected in #1253 because of various changes to authz were looming. But six months later it still hasn't been done.

Suggest this is just moving _add_user_object_role from ckan/lib/base.py to the logic layer, but no doubt it will be more complicated.

Estimate: 3 days

#1699: Setup Ckan on tenforce test server

kindly — Tue, 24 Jan 2012 15:10:59 GMT

Need to setup server in tenforce test server.

#1513: Logged in/out problems

dread — Mon, 05 Dec 2011 15:52:33 GMT

Richard Cyganiak: I often see a problem where half the system thinks that I'm logged in, while the other half insists I'm not. Logout then login again may fix it.

David Read: I think I've seen it several times that the "My Account" link is given (suggesting you're logged in) but when you click it it asks you to log in, which means you weren't.

#1583: Integrate QA into frontend - 1d

johnglover — Wed, 21 Dec 2011 09:22:07 GMT

Tasks:

~~On dataset view page, add star rating after each resource view url. Probably similar to current format label after url.~~
~~Add star rating to resource view page, along with a way of displaying the reason for the rating and tips to improve. Should be unobtrusive.~~
~~Fix the table at http://thedatahub.org/qa/dataset/broken_resource_links - URLs should be truncated to a fixed not-too-long length and the other fields line up (eg by using a table)~~~
~~Handle 404s by showing that 'we think this resource is 404ing'.~~
~~Handle the absence of QA data on a resource (currently nothing is displayed. Maybe that's fine.~~
~~Do we need to integrate the 'reason for rating' string into the UI?~~

Next Steps:

~~Use db dump of thedatahub.org for final testing.~~

#1620: Update and test existing DGU package form : don't allow package-type selection when editing a package

icmurray — Mon, 09 Jan 2012 10:10:37 GMT

[Child of #1526]

Can't change a dataset from containing time-series resources to one containing individual resources or visa-versa.

#1630: Integrate Data Explorer v2 into Data Viewer

rgrp — Wed, 11 Jan 2012 10:51:38 GMT

Super ticket: #1602

Steps:

Include (or git submodule) data explorer into vendor
Update existing js integration code
Fix css and any bugs

Est: 1d

Actual: 1d

#1696: Maintain backwards compatibility with older way of creating custom forms

johnglover — Mon, 23 Jan 2012 13:44:03 GMT

To maintain backwards compatibility, the package controller (new/edit) should check to see if the controller has a package_form variable defined, and if so render the form pointed to by this variable before calling the new self._package_form() function.

This behaviour is now deprecated however.

#1710: 'Announce' email list for thedatahub

dread — Thu, 26 Jan 2012 12:00:59 GMT

We want to make email announcements to users of thedatahub.org.

The most simple way to do this is to get a dump of the email addresses and subscribe/invite them to a mailman list.

Different ways to do it:

'paster dump-users-csv' creates a CSV file. Apache serves up the directory using password protection. Is this secure enough?
CKAN admin pages provides a button to produce the CSV and download it. A config option may be necessary to enable this (disabled by default).

Something more advanced in the future could be:

CKAN tells mailman about new subscribers using the mailman web interface. Could be brittle.
Mailman doesn't have an Internet API, but it does have a python interface. So CKAN could tell mailman about new subscribers using a python proxy. There may be one out there, or maybe easy to do. But annoying to have to maintain another process running on machine.
CKAN sends out the emails itself. A sysadmin is given a form. Users have an opt in/out option in their account. But receiving emails is a sensitive thing so it would need to be right first time, and there are several risks to this approach.

#1712: DGU /data page

icmurray — Thu, 26 Jan 2012 15:08:47 GMT

CKAN provides the /data page page of data.gov.uk

Breakdown of tasks:

[x] Analysis / refinement of spec.
[x] Log-in / register as publisher

(Waiting on publisher form)

[x] Population of "browse by publisher"

list groups ordered by most datasets (Waiting on publisher integration)

[ ] ~~Browse by nation~~
[ ] ~~Featured datasets~~

Now a possible integration point with drupal

[X] Tag cloud
[X] Developers section

#1713: DGU dataset search page

icmurray — Thu, 26 Jan 2012 15:11:40 GMT

Search results page of DGU.

Breakdown:

[X] ensure faceting by group (publisher) works [X] allow faceting by UKLP dataset type [X] allow faceting by INSPIRE

#1714: DGU dataset view page

icmurray — Thu, 26 Jan 2012 15:18:10 GMT

Dataset read page for DGU

Breakdown:

[X] Fix missing title
[X] Links to the social media stuff
- "Share your app"
- "Share your idea"
- "Request new data"
[ ] ~~Presentation of additional information should be broken into sections~~

Content is there. Don't do anything until theming/design work is undertaken.

[ ] ~~"More like this": links to similar datasets.~~

Dropping

[N] ~~Stars~~ (I think these are being removed)
[ ] ~~Comments~~

Possible integration point with Drupal. Needs it own ticket.

[X] Give feedback to department
[X] Tidy up the QA stars

#1719: Broken links for non-Gravatar use icons - 0.25d

dread — Mon, 30 Jan 2012 15:53:08 GMT

Super ticket: #1506

e.g. http://thedatahub.org/user The users with Gravatar have their nice user icons, but the majority of users (without Gravatars) have 'broken link' symbols. Same problem wherever users are shown, such as the dataset history pages.

Broken on 1.5.2b - currently on thedatahub.org.

#1725: Tag autocompletion and commas don't mix

dread — Tue, 31 Jan 2012 10:45:09 GMT

If you use tag autocompletion, it automatically adds a comma after the tag. This is fine if you are going to add another tag, but not fine if it is the last one. If you don't manually remove the final comma than you get form validation error "Tag string: Tag "" length is less than minimum 2"

Affects only version 1.5.2a.

#1726: Update harvesting model

amercader — Tue, 31 Jan 2012 10:52:07 GMT

We need:

"current" column in harvest_object
"harvest_source_id" column in harvest_object, foreign key to harvest_source
"title" column in harvest_source

The extension should check on startup if the db needs updating and run the necessary scripts.

#1727: Add "current" property to harvest objects

amercader — Tue, 31 Jan 2012 10:54:14 GMT

Right now is quite difficult to get the most current version of a package via the harvest objects, and the queries involved are prone to errors. A new "current" field will be added which will identify unequivocally the most recent harvested object. Unfortunately this will involve some major changes:

Model update (See #1726)
Update queries for getting harvest sources' datasets
Update ckanext-csw queries for getting the objects
Update inspire and ckan harvesters
Document new field

#1704: Import test datasets from prototype into ckan.

kindly — Tue, 24 Jan 2012 15:52:35 GMT

Eurostat datasets mainly.

#1716: move or change save warning on edit dataset page [a:0.1d]

shevski — Thu, 26 Jan 2012 17:44:53 GMT

To allow navigation between sections to still work, also potentially rephrase to avoid confusion that you have to save every time to add a tag or resource and can't say add description, tag and resource in one go

#1639: Move activity streams CSS into external file

seanh — Wed, 11 Jan 2012 18:03:39 GMT

The HTML and CSS for the activity streams templates needs to be tidied up. Currently the CSS is inlined in the templates themselves, and there is much unnecessary repetition of CSS.

Perhaps put this one in hold until there has been some design discussion about how the activity streams and the user, dataset and group pages with activity streams should look?

#1663: API call for creating an activity streams event

seanh — Tue, 17 Jan 2012 10:07:24 GMT

The create_activity() logic function already exists, but validation and authorization need to be added before it's hooked up to an API call.

#1666: Add activity stream events for tags

seanh — Tue, 17 Jan 2012 10:26:25 GMT

...and show them on the tag pages.

This means adding tag_activity_list() and tag_activity_list_html() logic functions, adding the templating to render tags as HTML, and adding the tag HTML into the tag pages.

What exactly should go into a tag's activity stream? As with datasets we may want a more complex SQL query, where events from objects related to the tag (datasets, users, etc.) appear in the tag's stream.

#1671: Publisher profile sysadmin authz

ross — Tue, 17 Jan 2012 12:16:47 GMT

Super #1669

Implement authz for sysadmins in publisher profile

#1672: Publisher profile user authz

ross — Tue, 17 Jan 2012 12:18:00 GMT

Super #1669

Authz for normal users in publisher profile as per sysadmin

#1673: Add approval status field to groups

ross — Tue, 17 Jan 2012 12:19:13 GMT

Super #1669

As groups need to be authorised in some cases, particularly publisher profiles, we need a field to denote how this is managed.

We should add an 'approval_status' field that will be used to denote its status. By default this will be "approved" but this will allow us to override it with a "pending" or "denied" status. The latter should also change the main status of the Group.

#1674: Test auth logic hooks for publisher profile

ross — Tue, 17 Jan 2012 12:20:41 GMT

Super #1669

Test all of the new auth logic hooks for publisher profile to make sure they work as expected. Also make sure that we have a test to compare the naming of the core auth funcs with the profile hooks (so we can see what may have changed in one but not the other).

#1676: Publisher editor authz

ross — Tue, 17 Jan 2012 12:22:51 GMT

Super #1669

Editor authz for publisher groups need to be implemented and tested.

#1686: Add activity events for when a dataset is added to or removed from a group

seanh — Thu, 19 Jan 2012 17:21:32 GMT

#1694: Add quick functional test(s) for activity streams HTML rendering

seanh — Mon, 23 Jan 2012 12:37:28 GMT

The activity streams feature is already thoroughly tested via the logic layer and API calls, but there needs to be a quick test of the rendering code, e.g. fetch a user's page (thus testing that ckan doesn't crash while rendering it) and then assert that the expected activity streams HTML is in the fetched HTML.

#1700: Document server setup for ckan server.

kindly — Tue, 24 Jan 2012 15:12:32 GMT

Document deployment on tenforce test server.

#1701: Normalize character encoding for ckan search.

kindly — Tue, 24 Jan 2012 15:30:50 GMT

Make sure accented characters are normalized when indexed and when searched for.

#1738: Add extension point to search index, so can modify dataset dict before index.

kindly — Tue, 31 Jan 2012 17:39:32 GMT

It is useful for extensions to add or change items that go into the search index. Add an extension point for this.

#1739: Fix issues where tests where not run.

kindly — Tue, 31 Jan 2012 21:58:49 GMT

Test failures in master due to tom and rufus not running tests. 0.3d.

#1754: Add activities for extras

seanh — Mon, 06 Feb 2012 11:38:38 GMT

When an extra is added to or removed from a dataset or is updated, an activity should go in the dataset's and user's activity streams.

#1769: 6. Publisher Registration Improvements MUST PHASE 1

icmurray — Mon, 06 Feb 2012 14:49:49 GMT

#1772: 13. Provider labelling MUST PHASE 1 [UKLP #14 MUST]

icmurray — Mon, 06 Feb 2012 14:51:42 GMT

#1628: get ckanext-dgu working with ckan 1.5.1

dread — Tue, 10 Jan 2012 16:59:22 GMT

johnglover said: I can confirm that even with the mapping fix, the ckanext-dgu dgu_form plugin template does not work properly with 1.5.1, so should probably not be installed at present. The edit page is ugly but should work (eg: http://dgu-os.okfn.org/dataset/edit/abandoned-vehicles), but the 'add a dataset' page is broken (eg: http://dgu-os.okfn.org/dataset/new)

#1660: CSV preview broken - OpenSpending

lucychambers — Mon, 16 Jan 2012 12:44:06 GMT

This CSV resource used to preview but now the format appears to be unsupported: "We are unable to preview this type of resource: x-osdata-csv"

http://thedatahub.org/dataset/lbhf-spending-2010/resource/9661abbd-2816-4d58-8b20-3cb0eb770c69

This is used as an example by the OpenSpending? team all the time.

#1770: 3. Themes/Taxonomy as well as tags WONT PHASE 2

icmurray — Mon, 06 Feb 2012 14:50:34 GMT

#1707: Over-long URLs in QA ext broken links list

johnglover — Wed, 25 Jan 2012 10:53:13 GMT

From Mark on the ckan-dev list:

In the table of broken links at <http://thedatahub.org/qa/dataset/broken_resource_links>, the links as printed should be abridged to a fixed length (but link to the full URL). Otherwise the 'reason' column vanishes off the screen. Also a table would probably be better since as things stand the 'reasons' are not lined up

#1702: Normalize character encoding for ckan search.

kindly — Tue, 24 Jan 2012 15:36:09 GMT

Make sure accented characters are normalized when indexed and when searched for.

#1706: Tag taxonomies package schema

seanh — Tue, 24 Jan 2012 17:58:58 GMT

Enable ckanext's to add new vocabularies to default_package_schema(), use the new vocabularies in templates for dataset view, create, edit pages, helper function for extensions to create a vocabulary schema for a given vocabulary name (must match a vocab already in the db) and add it to default package schema.

#1412: application.js crashes when viewing a dataset with no resources

seanh — Mon, 24 Oct 2011 10:03:45 GMT

Create a new dataset, don't add any resources to the dataset yet, view the dataset in ckan, if you have firebug enabled you'll see the error "resource is undefined" at line 699 of application.js.

#905: Unable to search with accented characters in package names

Stiivi — Thu, 13 Jan 2011 17:18:40 GMT

I have a package with name: "Štatistické prehľady agendy cestovných pasov" and when I try to search for "Štatistické" there are no results.

CKAN setup:

psql DB with UTF-8 tables

psql search backend

ckan version: 1.3.1b

Example:

http://verejnedata.sk/package/agenda-cestovnch-pasov

try to search for the first word in the package name.