{22} Trac tickets (2647 matches)

Results (1301 - 1400 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#811 defect cygri ckan-backlog new Extra field editing form layout breaks when there are long field names

The layout of the editing section for extra fields breaks when a field name is slightly too long. Field names jump over to the right. See http://ckan.net/package/edit/dbpedia for examples.

1289994812000000 1323170289000000
#924 enhancement dread ckan-backlog new Search box has no search button

The search box at the top-right of CKAN's page doesn't have a 'go' button. I feel that a larger percentage of users expect a 'go' or 'search' button on the right-hand side of the box to press to start searching. Techies tend to know the keyboard shortcut of pressing 'carriage-return' but it might be better to follow standard practise on this.

Examples with 'search' button: Internet Explorer, Firefox, Google, Amazon, trac Examples without: ?

1295867533000000 1323170436000000
#934 enhancement kindly closed invalid Key value store for extensions to store data

We need to allow plugins to store information.

Each plug-in should have its own name space and allow any data to be set.

There are two options.

  • make a key value table in ckan that will hold random information.
  • use redis (or other key value store)

If in sql suggest schema of form:

  • (namespace, obj_id, key, value [, value_type])
  • value is json
1296228758000000 1323171047000000
#939 enhancement rgrp closed fixed Notification bar at top of site shown to new users pointing them to simple instructions

On http://getthedata.org/ or stackoverflow there is a pop-up bar for new users that give them some simple instructions.

Talking with users it is clear that some people are not clear how ckan.net (or other sites) work and what they are and are not 'allowed to do. This could help make this better.

Implementation

  • Pop-up bar (based on a cookie or just being not logged in?) - uses ticket:938 (message flashing)
  • FAQ/instructions page (use the new wiki?)

Depends

Estimate

  • Cost: 1h
1296399008000000 1323171158000000
#819 defect cygri ckan-backlog closed fixed Tag autocompletion widget broken

The widget for tag autocompletion is broken in various ways.

For example, if I edit a package that is tagged "music", and just tab through the form fields to get to the Author field, then the widget changes to "industrial-music" as I tab through it.

Or if the tag is "foo bar" and I hit alt-right to jump to the end of the text field (with the intent of adding a third tag), then the contents change to "foo barbecue".

1290004121000000 1323171435000000
#1486 defect seanh seanh closed fixed "allow_partial_update" option does not work with resource lists of packages
  1. Call ckan.logic.action.update:package_update()
  2. Pass in a context dict with "allow_partial_update":True
  3. Pass in a partial data dict containing only the package's ID and those fields that you want to update (e.g. title).

=> All of the package's resource will have their statuses changed to "deleted".

The "allow_partial_update" option should apply to the resource lists of packages, but it does not.

1322061318000000 1323172027000000
#1478 enhancement kindly kindly ckan-sprint-2011-12-05 closed fixed change package_group to members table

This will comprise of the following steps.

  • model migration.
  • simplify revisioning
  • Remove any sqlalchemy relations from groups.
  • change logic functions

estimate 2d

1321959973000000 1323172654000000
#1523 enhancement icmurray icmurray closed invalid [super]

Form refactor.

New create/edit package form wizard for DGU. See attached spec.

1323172661000000 1323172904000000
#1336 defect johnglover dread ckan-backlog new License fudge

cset:4b59ab34137d ckan/logic/action/get.py:

-            isopen = model.Package.get_license_register()[license_id].isopen()
-            result_dict['isopen'] = isopen
+            try:
+                isopen = model.Package.get_license_register()[license_id].isopen()
+                result_dict['isopen'] = isopen
+            except KeyError:
+                # TODO: create a log message this error?
+                result_dict['isopen'] = False 

This change hides problems with the license server and returns potentially incorrect values for openness.

This has been noted as 'temporary fix' but seems to be forgotten about, since it has been merged to default and gone into release 1.4.3.

I suggest the licenses are cached (I thought this was already the case when CKAN first requests them after start-up?). I suggest failure would return 503.

1315912057000000 1323173073000000
#1189 enhancement timmcnamara closed invalid Spideroak support in CKAN Storage

Spideroak is a bulk storage platform by a company that releases quite a lot of free software, has an extensive developer API with pricing of $10/100GB/month.

The data store is optimised for bulk data storage and retrieval. This is the kind of use case that CKAN packages require. We don't need low latency. We are after low cost, high reliability solution.

1308187104000000 1323173227000000
#1242 enhancement timmcnamara closed fixed thedatahub.org's login page

The right hand side of thedatahub.org login screen is a little confusing:

    Not a member?

    Join CKAN to contribute packages under your own name.

     - Register
     - Reset your password
     - Privacy Policy

Suggested improvements to make:

  • replace CKAN with thedatahub.org
  • in general, edit the copy to be more inviting
  • remove "Reset your password" from that bubble
  • move the privacy policy link to somewhere less prominent
1311569218000000 1323173292000000
#1308 defect shevski dread closed fixed 'create package' user case failure

On the CKAN home page there is no suggestion that you can add or register a dataset. The 'Register a dataset' option has been removed at some point! (only visible to users that have registered - why would they do that when the benefits aren't described?)

1315219500000000 1323174312000000
#1517 defect zephod zephod ckan-sprint-2011-12-05 closed fixed Accessing resource view sometimes auto-downloads the resource

Clicking any of the links to a resource page in the dataset viewer seems to load the page and immediately start downloading the resource file.

For example, being here: http://test.ckan.net/dataset/europeana-lod and clicking on any of the links to the first resource.

1323109647000000 1323175957000000
#1524 enhancement thejimmyg thejimmyg ckan-sprint-2011-12-19 closed fixed New Release of Latest MI Reports 1323172688000000 1323191733000000
#1004 defect zephod dread ckan-backlog closed fixed Group creation instructions missing

Need instructions on the group page to tell people they need to login to create a group. Someone must have deleted this.

1298559144000000 1323195485000000
#1190 enhancement rgrp rgrp ckan-v1.5 closed fixed [super] CREP 0004 Data API and Data Processing System

For some time (e.g. 1y+!) we have known that we want to integrate some kind of datastore / data processing system with CKAN. We've had a CREP in progress on this for some months (may copy that here at some point):

http://wiki.ckan.org/CEP0004

We can distinguish 3 modules that are needed:

  1. "Webstore": A datastore with dataapi - #1208

Suggestion is this would be sqlite based with a simple sql based API. http://ckan.net/api/data/{user|org}/{datastore_name}?q={some-read-sql-query}

  1. Automated conversion of suitable resources into datastore upon resource creation so that e.g. they are accessible via the API. #1398
  1. A data processing system which utilizes this datastore. One could

get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules) - #1432

More info

1308227611000000 1323270522000000
#1514 defect dread ckan-v1.6 closed duplicate Modifying user name loses connection with revisions

If you edit your user name, the number of revisions you made returns to 0. This is because in the Revision object, the user's name is stored, rather than the user's ID.

rgrp: We can reconnect that pretty easily (and have a longer term solution that involves not using the usernames but the userids in in the Revision objects so we don't hvae this problem in future!)

1323100659000000 1323279018000000
#1483 defect johnglover johnglover ckan-sprint-2011-12-19 closed fixed Tasks sometimes fail to update task_status table when trying to report a failure

The error reports do not have a "value". The original task in the task status table is not made and it tries to make a new task, but can't.

Issue: why is the original task_status entry not made?

kindly: there is a chance of course that the task is quicker than the plugin to get to the point where it needs to update the table.

kindly: so we may have to find a way to update the task status before we run the task.

kindly: I think you can supply a task_id to celery. i.e we make our own guid as the task id. So we can use that one to populate the task status.

1322050605000000 1323279151000000
#1520 task dread ckan-sprint-2011-12-19 closed fixed Disable name changing

Because of #1514 we should just disable name changing, until #1514 is done.

1323169663000000 1323280999000000
#1435 enhancement rgrp ckan-v1.6 closed wontfix Switch to continuous.io for buildbot (?) 1320145831000000 1323283538000000
#1086 defect thejimmyg johnlawrenceaspden closed wontfix no way to delete authorization groups from web interface

as title.

1302625333000000 1323346552000000
#1516 task amercader amercader ckan-sprint-2011-12-19 closed fixed Update SOLR schema after 1.5.1 release and set up multicore SOLR instances if needed

CKAN 1.5.1 will introduce changes in the SOLR schema and functions to support multiple schema versions. Unfortunately the changes in the schema will be backwards incompatible so either both CKAN and SOLR are upgraded, or SOLR is configured as multicore, with one core for each schema. The solr cores would look like:

http://<solr-server>/solr/schema-<version>

Different scenarios:

  • Single SOLR instance used by only one CKAN site (e.g. SOLR running on the same machine):
    • Update CKAN
    • Update SOLR schema (symlink to suitable version in CKAN source)
    • Rebuild search index
  • Single SOLR instance used by multiple CKAN sites.
    • Update CKAN source
    • Configure SOLR as multicore, one core for version 1.2 and another for 1.3 of the schema (symlink to appropriate version in CKAN source)
    • Update solr_url on each of the CKAN sites with the suitable core
    • Rebuild search index

These are the SOLR and CKAN instances that need to be updated (Please add any missing ones):

  • Same machine as the CKAN site:
    • test.ckan.net
    • iati.test.ckan.net
    • data.gov.uk/ catalogue.data.gov.uk (confirm)
  • s046.okserver.org
    • iatiregistry.org
  • s052.okserver.org
    • datacatalogs.org
  • s004.okserver.org / eu4.okfn.org / solr.okfn.org
    • thedatahub.org (=www.ckan.net)
    • datagm.org.uk
    • publicdata.eu
    • hri.fi (dev.fvh.fi?)
    • it.ckan.net
    • ie.ckan.net
    • cz.ckan.net
    • register.data.overheid.nl (nl.ckan.net?)
    • no.ckan.net / datakilder.no
    • br.ckan.net
    • colorado.ckan.net
    • at.ckan.net

TODO: which SOLR server are using these instances?

  • data.norge.no
  • nederland.ckan.net
  • lt.ckan.net
  • pl.ckan.net
  • datadotmontreal.ca/
  • ca.ckan.net / datadotgc.com
1323108431000000 1323362689000000
#1544 task dread ckan-backlog new delete old git branches

We have 150 odd branches (git branch -a) - most of them old - we should prune them. At very least, branches that have been merged in should be deleted. Look at old branches that haven't been merged in and wonder why.

May be of some use:

git branch --merged
git branch --no-merged
1323702610000000 1323702610000000
#1407 defect rgrp dread ckan-sprint-2011-12-19 closed fixed Stats extension not working

Graphs don't show any more at http://thedatahub.org/stats

1319014605000000 1323764239000000
#1503 task dread markbrough ckan-sprint-2011-12-05 closed fixed Move from Bitbucket to Github

Reasons: more people familiar with Git, bigger Open Source community on Github.

Keeping track of the move here: https://docs.google.com/spreadsheet/ccc?key=0AvCDqUH8jVN8dGNVNEpXLWk3UTlJZXhFMjBPbXVtRmc&hl=en_GB#gid=0

1322745084000000 1323793662000000
#1548 enhancement dread dread ckan-sprint-2011-12-19 closed fixed Simplfy inheritance in functional tests

It's hard to understand what's going on with the functional tests because there is so much going on in inherited classes. Would be better to be explicit in constructors, even if it involves cut & paste.

1323859687000000 1323888271000000
#1532 defect dread ckan-sprint-2011-12-19 closed fixed Registration with OpenID has misleading error message

The log-in page says "Login using Open ID" and gives instructions for signing up. YET this is only available to users who've already added openid to their account. If you have not done this and then sign-in via OpenID (which is successful from the OpenID end) then you are told "Login failed. Bad username or password." in a flash message.

Proposed solution (i don't know if this is possible):

  • When you log-in for the first time via OpenID, it doesn't actually log you in in CKAN. It just sends you to the 'Create User' page with the OpenID field pre-filled, and puts up a flash message "This OpenID account is not yet registered on thedatahub. Please complete your details.". This allows you to complete the registration and logs you in, and allows you to log-in directly with OpenID in the future.

Alternatives:

  • Just change the error message to be 'You need to register in CKAN first. Quote your OpenID in the registration form to use it in future.'
  • Remove OpenID altogether
1323276392000000 1323956027000000
#1490 enhancement amercader amercader ckan-sprint-2011-12-05 closed fixed Standardize output from package listings coming from the logic layer

Right now, the two logic functions that return a list of packages (package_search [1] and group_packages_list [2])use custom functions to generate the output dict. That's suboptimal because:

  • The template function that renders the package listing (package_list_from_dict) could end up receiving different dicts.
  • Not all package properties are available at the template level (e.g extras, tags...). These can be really useful to customize the listing from an extension.

In general only the functions present in lib/dictization/model_dictize.py should be used to build the output of a logic function, in that case package_dictize. If necessary, they can be modified to include missing properties, like on this particular case the "isopen" property, needed by the template renderer.

[1] https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L685 [2] https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L442

1322225239000000 1324033557000000
#1430 defect amercader ckan-sprint-2011-11-07 closed fixed Documents get mixed between SOLR cores

On some occasions (apparently random), the documents indexed in a specific SOLR core get mixed with different site_ids.

E.g: We look for all documents in the testing.iatiregistry.org core, faceted by site_id. We would expect all documents to have site_id = iati_testing, but some of them have site_id = iatiregistry.org

http://okfn-solr.fry-it.com:8080/solr/testing.iatiregistry.org/select?indent=on&version=2.2&q=*:*&fq=+state:active&facet=true&facet.field=site_id

<lst name="facet_fields">
<lst name="site_id">
<int name="iati_testing">265</int>
<int name="iatiregistry.org">255</int>
</lst>
</lst>

If we compare one of the records which disappeared from the "iati_testing" site_id in both the production and testing SOLR cores of the server, the records are exactly the same, including the indexed_ts property:

http://okfn-solr.fry-it.com:8080/solr/iatiregistry.org/select?indent=on&version=2.2&q=id:97d1ab0d-b203-4757-8f4e-a0c84d2f759f&facet=true&facet.field=site_id

http://okfn-solr.fry-it.com:8080/solr/testing.iatiregistry.org/select?indent=on&version=2.2&q=id:97d1ab0d-b203-4757-8f4e-a0c84d2f759f&facet=true&facet.field=site_id

Note that the response from the URLs shown may vary, as the testing site could have been reindexed.

1320068076000000 1324033923000000
#806 enhancement rgrp rgrp ckan-v1.3 closed fixed Metadata created and last modified timestamps for packages

Packages should provide create and last modified attributes which are included in package dict exposed via api.

These attributes will not be in db but are computed (depend not just on package but related objects -- package tags, package resources, package extras etc)

1289854894000000 1324034356000000
#1546 defect johnglover johnglover ckan-sprint-2011-12-19 closed fixed Package metadata_created returns the wrong date

metadata_created seems to be returning the date of the earliest revision (for all packages), rather than the earliest revision for a specific package.

1323707327000000 1324034396000000
#1504 defect dread dread ckan-sprint-2011-12-05 closed fixed Action API given blank data causes exception

This is what caused the exception:

curl http://localhost:5000/api/action/package_search -d='{"q": "osm"}'

The problem is the content-type is formencoding, the '=' is the first character of the request, and so internally the data_dict is set to ' ', which needs to be a dictionary.

1322756748000000 1324035928000000
#1386 enhancement rgrp rgrp ckan-sprint-2011-10-24 closed fixed Disallow account creation via openid

Superticket: #1343

Creation of accounts from OpenID causes large number of problems:

  • No guarantee of other profile info
  • Poor username (just from openid)

We therefore will:

  1. Require creation of user account via register (no auto-creation of accounts via openid)
    • sidebar of register page should point to login page rather than openid signin

  1. Permit association of an openid with an account ...
    • How: paste in your openid url (requires change to edit form to have this)
      • won't work for google with their weird openid urls
    • [future] Nicer way is to have login via openid while logged in (which association then happening in background)
  1. Migration (for 1+2):
    • (?) Generate decent usernames for all existing accounts
    • As this is nontrivial suggest instead we allow editing of usernames (by account owner and sysadmins). This is useful in its own right and is a reasonable 80/20 solution.
      • This does present the problem of re-associating commits with the new username. Best solution to this would be to switch revisions to point to userid rather than username. See #1534
  1. [optional] Change display_name to always be username
    • Change My Account to Username at top right of all pages (once we know usernames are short ...)
      • could do this straight away by truncating long usernames (e.g. truncate at 20 chars ...)
      • Also should we lose the icon?
1318425493000000 1324036930000000
#1319 enhancement rgrp dread ckan-sprint-2011-10-24 closed fixed Require an email address on User registration

Superticket: #1343

This avoids difficulties when they forget their password.

  • (?) Email validation (navl does not have one!)
  • Require email confirmation to be activated (?) (Answer: no, not at present -- see discussion below)
1315415932000000 1324036947000000
#1381 enhancement amercader pudo ckan-sprint-2011-10-24 closed fixed Managing groups via package entity API

It is not currently possible to assign or remove groups by adding or removing their group name from the "groups" list on the REST API. This should be allowed, as it is easier than first adding a package and then editing each group it must belong to.

1318266706000000 1324037815000000
#1371 enhancement johnglover johnglover ckan-sprint-2011-10-10 closed fixed Make logic layer functions for working with task_status table

The background tasks will use the logic API to access the task_status table so new logic layer functions are needed.

1317808776000000 1324037905000000
#1409 enhancement zephod zephod ckan-sprint-2011-11-07 closed fixed Add Gravatars to user profiles

Since we now require email addresses, it's sensible to request users' gravatars to add a little flavour to their user profile (and, potentially, other places eg. comments threads?)

[Reopened: Additional] Use gravatars in the HTML generated by helpers.py to create lists of users. Eg. on user/list and dataset/history.

1319200773000000 1324047722000000
#1378 enhancement zephod zephod ckan-sprint-2011-10-24 closed fixed New to TheDataHub? - StackOverflow style banner

Implement a banner across the top of the screen giving newbies a link to the about page, and making them feel welcome. See stackoverflow for reference?

1318247329000000 1324047835000000
#1380 enhancement zephod zephod ckan-sprint-2011-11-07 closed fixed Making customizing CSS easier

While integrating the new theme and making a large number of UX tweaks, I've allowed the CSS file to bloat out of control with some very specific rules and subtle hacks, as well as a couple of clearfix <div> objects in the markup which isn't really good practice.

In general we want a logical CSS structure which is as easy to modify as possible. You should not find yourself being overwritten by highly specific rules all the time!

Actions

  1. Clean up css
  2. Add a config option ckan.template_head_end - arbitrary string inserted at end of <head> tag so you can add custom css etc (may also want to update theming docs to reflect possibility of using this)
  3. (While we're at it): Add ckan.template_footer_end config option to deployment.ini_template. (This already exists, we just haven't told anybody)
  4. The /users index page isn't linked anywhere. Clean it up and link it in the footer next to Groups etc.
  5. package/new, package/edit, group/new and group/edit all include their forms differently. Use h.literal in all cases.
  6. Fix the padding on minornavigation.

Background

David Read sent me the following as background:

  1. basic theming is described here: http://docs.ckan.org/en/latest/theming.html

Deep customisation of the templates is done by creating an extension, with this as a template: https://bitbucket.org/okfn/ckanext-exampletheme/overview

That's what Augusto and the Brazilians have done http://alpha.dados.gov.br/dados/ and their code is here: hg clone http://dev.dados.gov.br/codigo/dev/tema-ckan <snip> Another thing of interest was us becoming compatible with Wordpress themes last December. Richard Pope worked with Rufus on this. I believe the base theme is 'twentyten' and I'm guessing that one would swap that css file with another to change theme, but I don't know. I don't know if this feature has been dropped since then. http://lists.okfn.org/pipermail/ckan-dev/2010-December/000073.html

Dave

1318263956000000 1324048324000000
#1115 defect johnlawrenceaspden closed wontfix can have two authzgroups with the same name

If you've got edit permission on an authzgroup, then you can change its name to be the same as another existing authzgroup.

This causes some strange UI effects at worst, and probably causes worse problems somewhere else.

Is there any reason why changing the names of existing authzgroups should be allowed? And if so, name collisions should presumably be guarded against in both the name-changing and creation functions

1304085120000000 1324054704000000
#1552 enhancement dread dread ckan-sprint-2011-12-19 closed fixed Be able to set password on the command line when creating users

(instead of using the prompt) - needed for deployment

1324056178000000 1324056240000000
#1133 defect johnlawrenceaspden closed worksforme command line rights manipulation doesn't work

It appears that the command

$ paster rights add russianfan admin warandpeace

has no effect, even though

$ paster rights remove russianfan admin warandpeace

works fine. This may be specific to something I've done, could someone confirm?

If it's the case more generally, then I'm assuming this behaviour is untested? Tests should probably be added.

1305054948000000 1324057072000000
#1099 defect johnlawrenceaspden closed wontfix strange interactions between two browsers while playing with authz groups

While playing with the authorization groups, trying to design tests, I found that it was necessary to log in as two different users with two different browsers. Often actions of one user would cause server errors in the other user's browser.

I don't have a reproducible test case, but it happens fairly often so it shouldn't be too difficult to get one.

1303380824000000 1324057106000000
#1557 enhancement David Rasnik jilly mathews ckan-future new Complete Webstore Preview Extension

Finish any work out standing on web store preview extension to be able to package and release.

Ref James and I going through existing features and trying to mention any polishing that needed doing to get exiting features ready for release with projects such as CKAN hosted.

1324291253000000 1324291253000000
#1558 enhancement David Raznik jilly mathews ckan-future new Publisher Tools

Summarise final set of requirements for this and finish development and test. Estimated 10 working days.

1324291573000000 1324291573000000
#1560 enhancement David Raznik jilly mathews ckan-future new Follow extension

Estimate 2 days to finish dev and test.

David can you add any info needed here?

1324291879000000 1324291879000000
#1561 enhancement David Raznik jilly mathews ckan-future new To do extension

Can we finish this ready for release on data hub and CKAN Hosted.

1324291972000000 1324291972000000
#1562 enhancement Adria jilly mathews ckan-future new Finish Geo Spatial

Estimated 4 weeks of Adria's time. I guess this will need to be broken down into more tickets. This feature is being requested by a number of potential customers and we have some ideas of requirements between Rufus and Jilly for this. This is the most popular new feature we talk about to new clients.

1324292193000000 1324292193000000
#1489 enhancement dread ckan-backlog assigned Updating example theme/extension

ckanext-example needs updating for CKAN 1.5:

  • theme changes
  • new forms

About: 'ckanext-exampletheme' was created in Spring 2011 as an example CKAN extension that showed how to customise the look & operation of CKAN. This moved to github and renamed 'ckanext-example'.

1322137920000000 1324292384000000
#1564 enhancement David Raznik jilly mathews ckan-future new Structured Data (Data API)

Basic websotre exists but this may be not what is described yet.

CKAN provides a rich API for the data itself, allowing users to query retrieve and use data instantly from datasets in CKAN without needing to download or process it first.

1324292834000000 1324292834000000
#1565 enhancement Rufus Pollock jilly mathews ckan-future new Admin dashboard finished?

Is testing complete and ready for release?

1324293092000000 1324293092000000
#1567 enhancement David Raznik jilly mathews ckan-future new Finish QA extension

Requires change to celeryd. Estimated 4 weeks.

1324293599000000 1324293599000000
#1569 enhancement David Raznik jilly mathews ckan-future new Wordpressser

How much effort will this be to be ready to use?

1324294056000000 1324294056000000
#1572 enhancement David Raznik jilly mathews ckan-future new Meta data Harvester

Need to write custom harvesters for each client. Is it worth having one for data hub?

1324294509000000 1324294509000000
#1573 enhancement David Raznik jilly mathews ckan-future new Apps and Ideas

Estimate 2 weeks for someone to finish and test.

1324294593000000 1324294593000000
#1570 enhancement David Raznik jilly mathews ckan-future closed invalid Integrated file Storage

Is this ready for release? What needs to be done?

1324294142000000 1324314741000000
#1563 enhancement David Raznik jilly mathews ckan-future closed invalid Finish Data Storage

Unsure what needs to happen here. Need to list outstanding tasks and implement.

1324292346000000 1324314806000000
#1519 enhancement johnglover shevski ckan-sprint-2012-01-09 closed wontfix combine stats and analytics extensions into one in UI as well as deployment

Makes more sense to only have one comprehensive stats/analytics extension, so when people are looking to add a stats extension they won't have to add two which may be confusing (is one an old version of the other? why do I have to add two? what's the difference? etc)

User-wise we need a way to display our stats with google analytics in the same place

1323169033000000 1324317373000000
#1529 enhancement dread dread ckan-sprint-2011-12-19 closed fixed Display user name when logged in

Currently when you log in it says "logged-in". Most sites show your user-name and this is helpful when you have more than one account or human using the computer.

1323252086000000 1324318628000000
#1522 enhancement kindly kindly ckan-sprint-2011-12-19 closed fixed Add capacity to member table.

Need to add capacities to member tables.

1323172610000000 1324333827000000
#1402 enhancement kindly rgrp ckan-v1.6 closed fixed Migrate repository from mercurial to git

Plan to migrate from mercurial git

Process:

  • Do trial run
  • Announce conversion date / time
  • Require everyone to have pushed all outstanding changes at that time
  • Do conversion
  • Test
  • Announce on list and ckan.org/
1318811651000000 1324334011000000
#1451 enhancement johnglover johnglover ckan-sprint-2012-01-09 closed fixed Reintegrate download stats on dataset and resource view page - 0.5d
  • css class: resource-url-analytics
  • Assign to the link tag (a) everywhere we want to count (dataset, resource view) - 0.25d
  • Display counts in same place ...

And deploy on http://thedatahub.org/ - 0.25d

Possible: Also move analytics extension into core (decided not to).

1320677859000000 1324401792000000
#892 enhancement johnglover pudo ckan-sprint-2012-01-09 closed fixed Make stored data available in WUI - 0.5d

Once we have storage, make the data available in the following ways:

  • Now have a cached_url field can show in the frontend ...
  • Add a [<a href="${cached_url}">cached</a>] link to right of real url on resource listing on dataset page.
  • On resource page: will not add it yet.
    • At the moment no clear place to pu this given nice big download button (could put in list of items on left but that does not seem right and note that it will turn up in big list of info at bottom)
  • Add test (?)
  • Deploy
1294053293000000 1324402480000000
#1456 enhancement amercader amercader ckan-sprint-2011-11-21 closed fixed Use resource description instead of name if both are present

If a resource has both description and name the name is used. Descriptions are generally more, well, descriptive, so let's use those.

1320862619000000 1324472178000000
#1433 enhancement kindly rgrp ckan-sprint-2011-11-21 closed fixed Support SQLAlchemy 0.7

Why: current stable version of sqlalchemy. geoalchemy stuff required 0.7 and likely that some other things will require it soon.

Probably requires work on vdm https://bitbucket.org/okfn/vdm

NB: should have discussion before making 0.7 the default required version in CKAN core.

1320143453000000 1324472583000000
#1470 defect dread amercader ckan-sprint-2011-11-21 closed fixed Check user name in the profile form 1321446143000000 1324473955000000
#1487 enhancement kindly ckan-sprint-2011-12-05 closed fixed Fix group ordering on homepage

ordering on homepage by name instead of group count

1322094280000000 1324474147000000
#1493 defect dread dread ckan-sprint-2011-12-05 closed fixed 'search-index rebuild/clear' doesn't work if no ckan.site_id

You can't delete things from the SOLR search index if the ckan.site_id and ckan.site_url are blank.

Should assert that one of these are set up.

1322484422000000 1324474360000000
#1455 defect johnglover dread ckan-sprint-2011-12-05 closed fixed Search results when 'all_fields' don't include 'extra' fields

When you do a search like this:

http://thedatahub.org/api/search/package?q=tauberer+census&all_fields=1

the "extra" fields (e.g. "triples", "shortname") get missed off the results. The docs say it should be a "full record" and I don't see any reason why this is missed off.

This is a problem because search all_fields is the only way for clients and front-ends to get packages in bulk. They end up (like lodcloud) doing thousands of requests to get packages individually.

The full record is:

http://thedatahub.org/api/rest/dataset/2000-us-census-rdf
{"count": 1, "results": [{"res_description": ["Download", "XML Sitemap", "SPARQL enpdoint", "Example (RDF/XML)"], "name": "2000-us-census-rdf", "license": "Non-OKD Compliant::Creative Commons Non-Commercial (Any)", "author": "Joshua Tauberer", "author_email": "http://razor.occams.info/", "ckan_url": "http://thedatahub.org/dataset/2000-us-census-rdf", "notes": "2000 U.S. Census converted into over a billion RDF triples.\n\nPopulation statistics at various geographic levels, from the U.S. as a whole, down through states, counties, sub-counties (roughly, cities and incorporated towns)\n\nNotes: also found in the of SPARQL Endpoints.\n\nFrom home page:\n\n> * For the detailed Census statistics, you'll have to download the raw Census data files from the Census Bureau, my Perl script and the patch file below and run it yourself because the files are too big for me to offer as a download!\n> \n> * The data and scripts can be reused under Creative Commons Attribution-NonCommercial-ShareAlike.\n", "entity_type": "package", "site_id": "www.ckan.net", "download_url": "http://www.rdfabout.com/demo/census/", "indexed_ts": "2011-11-01T12:52:36.034Z", "url": "http://www.rdfabout.com/demo/census/", "state": "active", "title": "2000 U.S. Census in RDF (rdfabout.com)", "groups": ["lod", "lodcloud"], "res_format": ["", "meta/sitemap", "api/sparql", "example/rdf+xml"], "license_id": "cc-nc", "revision_id": "fcbad0de-79ea-41bd-8e01-eb832a05b732", "res_url": ["http://www.rdfabout.com/demo/census/", "http://www.rdfabout.com/sitemap.xml", "http://www.rdfabout.com/sparql", "http://www.rdfabout.com/rdf/usgov/geo/us/ny"], "id": "551ec435-f198-4d52-9b56-ec0b0be6aec9", "tags": ["census", "data", "demographics", "deref-vocab", "format-dc", "format-geonames", "format-politico", "format-rdf", "geographic", "linkeddata", "lod", "lodcloud.nolinks", "no-license-metadata", "no-provenance-metadata", "no-vocab-mappings", "population", "published-by-third-party", "rdf", "statistics", "us"]}]}
1320858265000000 1324474466000000
#1505 defect dread dread ckan-sprint-2011-12-05 closed fixed SearchError and SearchQueryError cause exception in Action API

This query caused ckan to except because ckan/controllers/api.py doesn't catch SearchError? and SearchQueryError?:

curl http://localhost:5000/api/action/package_search -d '{"sort": "metadata_modified"}'
1322758968000000 1324474577000000
#1580 enhancement johnglover johnglover ckan-sprint-2012-01-09 closed fixed Documenting TaskStatus table and QA changes - 0.5d 1324399664000000 1324478635000000
#1502 defect johnglover johnglover ckan-sprint-2011-12-05 closed fixed Group package list is ordered by revision timestamp instead of alphabetically 1322680312000000 1324480415000000
#1543 defect johnglover amercader ckan-sprint-2012-01-09 closed fixed Pagination links in the dataset listings don't keep the current filters

E.g. Pagination links on this page don't include groups=lodcloud http://thedatahub.org/dataset?groups=lodcloud

Not sure if related to #1501 (probably not)

1323442623000000 1324483367000000
#1065 enhancement zephod johnlawrenceaspden ckan-v1.6 closed fixed [super] Change Authorization System

Child tickets

  • #1198 Publisher hierarchy
  • #1050 Authz lib improvement and refactor of ckan/lib/authztool.py
  • #1004 Group creation instructions missing
  • #1099 Strange interactions between two browsers while playing with authz groups
  • #1115 can have two authzgroups with the same name
  • #1133 command line rights manipulation doesn't work
  • #1138 minor navigations behave inconsistently

Old ticket description:

  1. Change name of AuthzGroup? to UserGroup? to reflect what it is for
  1. Get rid of Roles, and replace them with direct assignment of actions, even though there are many actions, and extensions can add arbitrary ones.
    • Debatable whether we should cut the number of actions to correspond to the three roles defined by the base system.
    • Have a method of finding roles (or, in future, actions) relevant to a given protection object (e.g. FILE-UPLOAD(ER) not relevant to Packages)
  1. Change UserGroups? so that they can have a hierarchical structure,

More info on Hierarchy change

e.g. UserGroup? NHS contains the User nhsysadmin, as well as the UserGroups? SURREY and BERKS, which themselves contain users.

One user in SURREY is Simon the Sysadmin, who has permissions on the whole system. His permissions should not leak out to other users or groups, and user permissions generally should not.

Each Group has permissions over various objects.

A user has permissions in his own right, and also has the permissions of his own group, and of all the groups contained in his group, and so on recursively.

Algorithm:

possible(user, action, package):

if user has permission for action on package

or any of have that permission

or any of his groups group-children (but not user-children), and so on recursively have the permission.

1301508331000000 1324550041000000
#945 enhancement kindly kindly ckan-v1.6 closed fixed [super] Richer resources - Resource Groups, new fields, improved UI

Super ticket: #1032

This is a meta ticket for changes that are going to happen in resources.

  • New resource group table. #956
  • New kind field in resource. #957
  • UI for new kind field. #958
  • Resources in REST API ticket:358
  • Resources in WUI #1445
  • Make Resources first class entity. #922 (duplicate?)

Background on this change can be found at:

1296475283000000 1325259350000000
#1545 enhancement amercader ckan-sprint-2012-01-09 closed wontfix Remove external asset dependencies

CKAN is pulling a number of resources from external locations. This causes problems when connectivity is limited and you have to work locally. Maybe some of them cold be moved to CKAN source to avoid external requests.

Quick search:

./ckan/templates/layout_base.html:            <img src="http://assets.okfn.org/images/logo/okf_logo_white_and_green_tiny.png" id="footer-okf-logo" />
./ckan/templates/layout_base.html:            <a href="http://opendefinition.org/"><img alt="This Content and Data is Open" src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" style="border: none ; margin-bottom: -4px;"/></a>
./ckan/templates/package/resource_read.html:                <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" />
./ckan/templates/package/read.html:          <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" /></a>
./ckan/templates/_util.html:                    <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" />
./ckan/templates/_util.html:                  <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" />
./ckan/public/scripts/vendor/ckanjs/1.0.0/ckanjs.js:      this.$dialog.html('<h2>Loading results...</h2><img src="http://assets.okfn.org/images/icons/ajaxload-circle.gif" />');
./ckan/public/scripts/vendor/ckanjs/1.0.0/ckanjs.js:          self.setMessage('Uploading file ... <img src="http://assets.okfn.org/images/icons/ajaxload-circle.gif" class="spinner" />');
./ckan/public/scripts/vendor/ckanjs/1.0.0/ckanjs.js:      self.setMessage('Checking upload permissions ... <img src="http://assets.okfn.org/images/icons/ajaxload-circle.gif" class="spinner" />');
Binary file ./ckan/lib/app_globals.pyc matches
./ckan/lib/app_globals.py:                                  'http://assets.okfn.org/p/ckan/img/ckan.ico')
./ckan/config/deployment.ini_tmpl:ckan.favicon = http://assets.okfn.org/p/ckan/img/ckan.ico
1323702635000000 1325260051000000
#1568 enhancement David Raznik jilly mathews ckan-future closed duplicate Moderated Edits

Can this be released as a standard CKAN feature?

1324293776000000 1325267998000000
#1129 CREP kindly ckan-v1.5 closed fixed CREP0002: Moderated Edits

Proposer: David Raznick

Abstract.

We are trying to achieve these goals.

  • To get people involved with making edits to CKAN metadata.
  • To have an ownership model as to who can moderate and validate these changes
  • To not put too huge a burden on these owners.

In order to achieve this, a feature which lets anyone edit a package but only let the moderator/owner accept it. The moderator should be able to look at a list of changes and accept the ones that

This cep is not about 'if' we need such a feature, it is about 'how' we go about implementing it. Another cep may needed for the 'if' case.

The Problem

We need the following to be possible.

  • Storing revision of objects that are not the current active one.
  • A way of the user viewing past revisions.
  • Accessing not only the history of a particular object but also of related objects at that time. i.e If a resource related to a package changes we need a way to see this when looking at the package.
  • A robust way of doing this in the face of database schema changes.
  • Make sure database queries are quick.

Solutions.

  1. Store the whole dictization of the package and all its related objects every time you change anything in its dictized representation and only save to the database proper if accepted.

Pros

  • Easy to implement, we already have a preview which makes the dictized form of a package without actually saving it. This will just need to be persisted in some way.
  • Fast retrieval.
  • Potential to store a branching revision tree of changes.

Cons

  • No easy way to remake the dictized packages historically or if there is an there a change in the way we represent packages, i.e schema changes.
  • Will only work for the particular objects we decide to store these changes for.
  • Stores a lot of repeated information
  1. Write specialized queries for every read of the database looking only at the revision tables.

This method requires there to be a change in the way we use VDM, so that we manage statefulness ourselves. We will need to add other states such as 'waiting for approval'.

Pros

  • No specialized storage required
  • Only need to change queries when schema changes
  • Can be made to work easily for other objects

Cons

  • Slower query time on read, as even looking at the last active package will need to do a fairly complicated query.

Implementation details.

1.

A new table with columns id, user, package_id, timestamp, revision_id, parent_id, dictized_package. revision_id should be null unless it is actually persisted to the database. parent_id is the id that this package_dict was changed from.

We could store only the diffs of the dictized_package as long as we assure that everything inside the json is stably sorted, this will make getting the historical data out slower.

Getting out the history of the dictized packages is an intensive task, as it will require replaying the whole history of all the changes and creating the dict for each change. This re-caching will need to be redone for every change we make to dictized representation of a package.

2.

Every normal packages read needs to look at the revision table to see the last accepted change in the dictized representation of the package. We also need to way to get what the dictized representation of the package was like at any point of its revision history. This querying is non-trivial in sql.

Participants

David Raznick to do it.

Progress.

Decided to go with option 2. However we will change the revisioning system to be like the schema attached. This gets rid of difficult querying problems caused by querying the revision tables by adding an end date, meaning you can do range queries.

The better and more normalized version of a revisioning system is outlined https://docs.google.com/drawings/d/1Y7nMgVsrs081Pame2RdbZHlCAlV33ddTZ8VAsab1j-0/edit?hl=en_GB&authkey=CJfd8vsB. We will be a step closer to that, with this change, but we will keep the current vdm more or less, intact.

1304851498000000 1325268100000000
#1604 enhancement dread ckan-backlog new Get ckanext-moderatededits working with CKAN 1.5+ templates

ckanext-moderatededits requires an old and possibly development version of CKAN. It would be good to update it for later CKAN versions.

According to the README, you need CKAN from branch feature-1141-moderated-edits-ajax but the changelog suggests this branch went into version 1.4.2. So it possibly works with 1.4.2 and 1.4.3(.1). But CKAN 1.5 has revamped templates, so the genshi stream filters definitely don't work.

BTW history_ajax/read_ajax calls have been deprecated in CKAN since 1.5.2a and will need fixing up to use the Action API too as part of this.

1325352429000000 1325352429000000
#1141 CREP johnglover ckan-backlog closed fixed [super] Moderated Edits User Interface

Proposer: John Glover
Seconder: James Gardner

Abstract

We are trying to achieve these goals:

  • To get people involved with making edits to CKAN metadata.
  • To have an ownership model as to who can moderate and validate these changes
  • To not put too huge a burden on these owners.

This feature allows anyone to edit a package and create a new revision, but requires an owner/moderator to approve a revision before it is are made "official".

There have been a lot of discussions around the revisioning system side of this ticket (CREP 0002) and I think these are now largely resolved. We now want to discuss the user interface.

The Problem

We require the following functionality:

  • Allow a group of changes to be stored as a new revision.
  • Allow a linear stack of "community" revisions.
  • Provide a way for the editor and moderator to compare previous revisions to the current one.
  • When a moderator approves a change it creates a new revision flagged "moderated" (this is analogous to a merge commit)
  • Provide a way for the editor and moderator comment on revisions if necessary.

Extra features:

  • Need a way to summarise the changes (as part of the preview perhaps)
  • Sysadmin needs to purge a revision completely

Specification

UI/UX

UI Mockup:

Revisions:

  • Revisions are per package rather than per field.
  • Internally CKAN has separate revisions for resources, extras and package metadata. From a user's point of view this could be confusing to expose, so everything that they see on a package form when they hit save is a single revision.

On the Edit page:

  • We have a panel on the right, listing all the revisions with the current moderated one selected. Moderated revisions are highligted in some way (red and bold?).
  • The values displayed in the form are by default populated from the latest revision (whether community or moderated)
  • Under each field is a "shadow", showing the value of the field in the revision selected in the panel, if it is different from the value in the field. By default the shadow values are populated from the latest moderated revision which is the one selected in the revision panel by default too.
  • When you change the value of a field, a shadow may appear or disappear accordingly. If they disappear a box saying that they are the same replaces it
  • If you want to edit values from a previous revision, you first select that revision to get the shadows populated. There is a button named "Replace fields with values from this revision" under the revision list. You click this, a warning pops up and then you say "Yes". You then select the moderated revision again.
  • We also allow package comments the same way as the todo extension works at the moment. Additionally, we need to be able to differentiate between what the moderator wrote and what a community member wrote, and so we may need to make a small change to the todo extension to facilitate this.
  • In addition to package comments, each revision will have a revision log (analogous to a commit message).

Technical Details

  • This CREP will result in a new CKAN extension.
  • It depends heavily on the new revisioning system (CREP0002), some of the details of which are yet to be finalised.
  • This CREP therefore requires working closely with David Raznick to come up with an API that the UI AJAX calls can use.
  • We will then use suitable test data to mimic these API calls until CREP0002 is ready.

Why do it this way

This hopefully provides a clear and consistent mechanism allowing both a community member to make new revisions and a moderator to view and approve revisions, with largely the same UI/UX.

Implementation plan

Deliverables

A new CKAN extension, consisting of:

  • Code: Python, HTML, CSS, Javascript
  • Unit tests
  • Localization
  • Documentation

Participants

John Glover to do it.

Progress

John has implemented the bulk of this UI. Just some things to tidy up before it is complete:

  • Genshi stream filters to be updated with CKAN 1.5 / 1.5.1 templates
  • history_ajax / read_ajax to be replaced with calls to Action API (or Util REST API)

I've split these two off into a new ticket #1604.

Related Progress

The Todo extension is written and available at: https://bitbucket.org/johnglover/ckanext-todo.

In the section 'The Problem', under extra features, we mention a need for the sysadmin to be able to purge a revision already. This is already done.

See also

#1129 Backend work

1305721003000000 1325352507000000
#1175 defect dread fccoelho@… closed invalid Stats extension not working

Hi, I get a 500, Internal server error when I enable ckanext-stats. Flavio

1307350823000000 1325355170000000
#1331 defect dread dread closed worksforme Setting a tag twice causes exception

To reproduce:

  1. Create a package with two tags the same: "bulk bulk"
  2. Click 'save'
  3. 500 ERROR - 'Server Error'
1315905959000000 1325355631000000
#1578 enhancement rgrp ckan-backlog new [super] Re-enable and refactor ratings 1324322443000000 1325473015000000
#1240 enhancement kindly rgrp ckan-backlog assigned [super] API v4

(Just creating this ticket as somewhere to keep notes)

  • Decide on REST api versus action API
    • Do we want to support both?
  • Tidying
    • Unify on /api/v{version num}/... structure (do we want a default option that points to current default? e.g. /api/default/ ...)
    • extras merged into normal field list in package
    • Get rid of /rest/ so just have api/v1/package
    • Get rid of separation of search api from 'rest' api
      • Propose that GET on REST index is search e.g. /package/?q=...
        • This is also resolves issue whereby GET at root returns whole package set (a *bad* idea) as this would now become the matchall search query (with a default limit on items returned)
  • Resource read/write in API (separate from package)
    • Does this need authorization work?
  • user/account API - read/write
  • Remove autocomplete -- can just use search
    • Do not worry about backwards compat as should only be used in our js (if others using it too bad!)
1311525660000000 1325473312000000
#1577 defect rgrp dread ckan-backlog new Can't upload file with foreign chars in filename

Looks like uploading a file with foreign characters fails due to encoding reasons.

URL: http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf
Module weberror.errormiddleware:162 in __call__
<<              __traceback_supplement__ = Supplement, self, environ
                   sr_checker = ResponseStartChecker(start_response)
                   app_iter = self.application(environ, sr_checker)
                   return self.make_catching_iter(app_iter, environ, sr_checker)
               except:
>>  app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<<                                                     self.cache_manager)
               environ[self.environ_key] = self.cache_manager
               return self.app(environ, start_response)
>>  return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<<                          headers.append(('Set-cookie', cookie))
                   return start_response(status, headers, exc_info)
               return self.wrap_app(environ, session_start_response)
           
           def _get_session(self):
>>  return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<<                  environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
               
               response = self.app(environ, start_response)
               
               # Wrapped in try as in rare cases the attribute will be gone already
>>  response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<          
               controller = self.resolve(environ, start_response)
               response = self.dispatch(controller, environ, start_response)
               
               if 'paste.testing_variables' in environ and hasattr(response,
>>  response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<<          if log_debug:
                   log.debug("Calling controller class with WSGI interface")
               return controller(environ, start_response)
           
           def load_test_env(self, environ):
>>  return controller(environ, start_response)
Module ckan.lib.base:123 in __call__
<<          # available in environ['pylons.routes_dict']    
               try:
                   return WSGIController.__call__(self, environ, start_response)
               finally:
                   model.Session.remove()
>>  return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<<                  return response(environ, self.start_response)
               
               response = self._dispatch_call()
               if not start_response_called:
                   self.start_response = start_response
>>  response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<<              req.environ['pylons.action_method'] = func
                   
                   response = self._inspect_call(func)
               else:
                   if log_debug:
>>  response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<<                        func.__name__, args)
               try:
                   result = self._perform_call(func, args)
               except HTTPException, httpe:
                   if log_debug:
>>  result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
           
           def _inspect_call(self, func):
>>  return func(**args)
Module ckanext.storage.controller:2 in auth_form
Module ckan.lib.jsonp:26 in jsonpify
<<      Very much modelled after pylons.decorators.jsonify .
           """
           data = func(*args, **kwargs)
           return to_jsonp(data)
>>  data = func(*args, **kwargs)
Module ckanext.storage.controller:301 in auth_form
<<          method = 'POST'
               authorize(method, bucket, label, c.userobj, self.ofs)
               data = self._get_form_data(label)
               return data
>>  authorize(method, bucket, label, c.userobj, self.ofs)
Module ckanext.storage.controller:79 in authorize
<<      if method != 'GET':
               # do not allow overwriting
               if ofs.exists(bucket, key):
                   abort(409)
               # now check user stuff
>>  if ofs.exists(bucket, key):
Module ofs.remote.botostore:53 in exists
<<          if bucket is None: 
                   return False
               return (label is None) or (label in bucket)
           
           def claim_bucket(self, bucket):
>>  return (label is None) or (label in bucket)
Module boto.s3.bucket:87 in __contains__
<<      def __contains__(self, key_name):
              return not (self.get_key(key_name) is None)
       
           def startElement(self, name, attrs, connection):
>>  return not (self.get_key(key_name) is None)
Module boto.s3.bucket:144 in get_key
<<          response = self.connection.make_request('HEAD', self.name, key_name,
                                                       headers=headers,
                                                       query_args=query_args)
               # Allow any success status (2xx) - for example this lets us
               # support Range gets, which return status 206:
>>  query_args=query_args)
Module boto.s3.connection:388 in make_request
<<          if isinstance(key, Key):
                   key = key.name
               path = self.calling_format.build_path_base(bucket, key)
               boto.log.debug('path=%s' % path)
               auth_path = self.calling_format.build_auth_path(bucket, key)
>>  path = self.calling_format.build_path_base(bucket, key)
Module boto.s3.connection:88 in build_path_base
<<      def build_path_base(self, bucket, key=''):
               return '/%s' % urllib.quote(key)
       
       class SubdomainCallingFormat(_CallingFormat):
>>  return '/%s' % urllib.quote(key)
Module urllib:1222 in quote
<<              safe_map[c] = (c in safe) and c or ('%%%02X' % i)
               _safemaps[cachekey] = safe_map
           res = map(safe_map.__getitem__, s)
           return ''.join(res)
>>  res = map(safe_map.__getitem__, s)
KeyError: u'\xed'
CGI Variables
AUTH_TYPE	'cookie'
CONTENT_TYPE	'; charset=utf-8'
DOCUMENT_ROOT	'/htdocs'
GATEWAY_INTERFACE	'CGI/1.1'
HTTP_ACCEPT	'*/*'
HTTP_ACCEPT_CHARSET	'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
HTTP_ACCEPT_ENCODING	'gzip,deflate,sdch'
HTTP_ACCEPT_LANGUAGE	'en-US,en;q=0.8'
HTTP_CACHE_CONTROL	'max-age=259200'
HTTP_CONNECTION	'keep-alive'
HTTP_COOKIE	'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; ckan_user=elenaibp; ckan_display_name="Elena Mondo"; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; __utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10.1324298316; __utmc=119670349; __utmz=119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)'
HTTP_HOST	'thedatahub.org'
HTTP_REFERER	'http://thedatahub.org/dataset/edit/budget-library-czeck-republic'
HTTP_USER_AGENT	'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7'
HTTP_VIA	'1.1 localhost (squid/3.0.STABLE19)'
HTTP_X_FORWARDED_FOR	'87.114.74.190'
HTTP_X_REQUESTED_WITH	'XMLHttpRequest'
PATH	'/usr/local/bin:/usr/bin:/bin'
PATH_INFO	'/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
PATH_TRANSLATED	'/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
REMOTE_ADDR	'193.34.146.142'
REMOTE_PORT	'55419'
REMOTE_USER	u'elenaibp'
REMOTE_USER_DATA	'userid_type:unicode'
REMOTE_USER_TOKENS	['']
REQUEST_METHOD	'GET'
REQUEST_URI	'/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf'
SCRIPT_FILENAME	'/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py'
SCRIPT_URI	'http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
SCRIPT_URL	'/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf'
SERVER_ADDR	'193.34.146.146'
SERVER_ADMIN	'[no address given]'
SERVER_NAME	'thedatahub.org'
SERVER_PORT	'80'
SERVER_PROTOCOL	'HTTP/1.0'
SERVER_SIGNATURE	'<address>Apache/2.2.14 (Ubuntu) Server at thedatahub.org Port 80</address>\n'
SERVER_SOFTWARE	'Apache/2.2.14 (Ubuntu)'
WSGI Variables
application	<beaker.middleware.CacheMiddleware object at 0x7f22601c7dd0>
beaker.cache	<beaker.cache.CacheManager object at 0x7f22601c7b50>
beaker.get_session	<bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x7f22601c7a90>>
beaker.session	{'_accessed_time': 1324298703.071357, '_creation_time': 1324293077.4139669}
mod_wsgi.application_group	'ckan.net|'
mod_wsgi.callable_object	'application'
mod_wsgi.listener_host	''
mod_wsgi.listener_port	'80'
mod_wsgi.process_group	'ckan.net'
mod_wsgi.reload_mechanism	'1'
mod_wsgi.script_reloading	'1'
mod_wsgi.version	(2, 8)
paste.cookies	(<SimpleCookie: __utma='119670349.1809834699.1323782464.1324293066.1324298316.4' __utmb='119670349.3.10.1324298316' __utmc='119670349' __utmz='119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' auth_tkt='f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode' ckan_apikey='decd48b1-49ee-4250-bff4-98ccca9c02a5' ckan_display_name='Elena Mondo' ckan_user='elenaibp' hide_welcome_message='1' thedatahub_net='27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e'>, 'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; ckan_user=elenaibp; ckan_display_name="Elena Mondo"; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; _ _utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10...)|utmcmd=(none)')
paste.registry	<paste.registry.Registry object at 0x7f226194df50>
paste.throw_errors	True
pylons.action_method	<bound method StorageAPIController.auth_form of <ckanext.storage.controller.StorageAPIController object at 0x7f2261dad990>>
pylons.controller	<ckanext.storage.controller.StorageAPIController object at 0x7f2261dad990>
pylons.environ_config	{'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons	<pylons.util.PylonsContext object at 0x7f2261daddd0>
pylons.routes_dict	{'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'}
repoze.who.identity	<repoze.who identity (hidden, dict-like) at 139785645747120>
repoze.who.logger	<logging.Logger instance at 0x7f225e23c098>
repoze.who.plugins	{'openid': <OpenIdIdentificationPlugin 139785625065680>, 'friendlyform': <FriendlyFormPlugin 139785618095248>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator object at 0x7f2260874c10>, 'auth_tkt': <AuthTktCookiePlugin 139785625065808>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f2260874c90>}
routes.route	<routes.route.Route object at 0x7f22601a1090>
routes.url	<routes.util.URLGenerator object at 0x7f2261dadf50>
webob._parsed_query_vars	(GET([]), '')
webob.adhoc_attrs	{'language': 'en-us'}
wsgi process	'Multiprocess'
wsgi.file_wrapper	<built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f2261da9af8>
wsgiorg.routing_args	(<routes.util.URLGenerator object at 0x7f2261dadf50>, {'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'})
1324317659000000 1325473564000000
#1549 enhancement ross ckan-backlog closed wontfix [super] Short link tool

It would be great to have a CKAN extension that allowed users (or CKAN itself) to generate short links to other URIs (both internal and external). Once created, shortlinks made by CKAN should be changeable. This would allow uploaded content to be moved without the user's link changing at all. The tool itself might also be of use as a general link-shortener to users other than the CKAN system itself.

Another useful feature would be for this to also collect some simple analytics such as the referrer and client IP for future reference. I'm not yet sure what we would do with the analytics other than some sort of popularity metric.

Questions:

  • Core, or Extension, or Self-hosted?
1324036998000000 1325474219000000
#1581 enhancement mark.wainwright@… johnglover ckan-future new Blog post about Google Analytics extension for CKAN

The CKAN Google Analytics extension has been updated to work with the latest version of CKAN, could make for a nice blog post.

Can ping John Glover in January for any details required.

Key link is: http://thedatahub.org/analytics/dataset/top though this should probably move to be under stats (e.g. http://thedatahub.org/stats/usage)

1324402800000000 1325474274000000
#1231 requirement kindly thejimmyg ckan-backlog closed wontfix [super] Management Information Reporting

Child tickets:

  • #1101 Integrate stats and googlanalytics into site nav

We have a spreadsheet from UKLP of statistics we'd like to generate

1311173919000000 1325474447000000
#1598 enhancement rgrp ckan-backlog new Reinstate Ratings

Ratings were disabled approximately a year ago because:

  • Unclear purpose and UX. What did ratings tell you? How useful were they?
  • Spamming (esp by bots: you could submit an anonymous rating via a GET request which caused problems)

Both problems are solvable and it would be nice to have this feature reinstated.

  • Purpose: can make this more purposable by limiting to logged in users (or at least distinguishing logged in from non-logged in users)
    • Even better we could allow ratings to be made public (I'm interested in what someone else I respect finds important)
  • Spamming: limit to logged in users and / or use AJAX over an API to submit ...
1325177524000000 1325474818000000
#1439 enhancement dread ckan-backlog new Action API discoverablility

A good service API needs to be discoverable, so you are not always having to refer to the documentation html.

Maybe /api/action should return a list of actions available? (Currently this returns a 404.)

  • It would be nice to sort these into get/create/update/delete.
  • #1438 Parameters for each of the actions must be discoverable too

/api/action/{action_name} should also return the help text / parameters allowable. (Currently this returns 400 error)

1320161970000000 1325474974000000
#1589 enhancement johnglover johnglover ckan-backlog new QA - Give 5 star rating to datasets with link metadata

Super: #1594

From Richard Cyganiak on the CKAN Discuss list:

Regarding the fifth star (is the dataset linked to others?). This cannot be automatically determined just by looking at the format. It either requires inspection of the actual data, or information about links in the metadata. As you're probably aware, we've established conventions for recording information on data links in CKAN [1], as part of the work of the lodcloud group on the Data Hub. Link information is captured for hundreds of datasets. I would claim that we have the majority of four-star datasets covered there, and hence you can determine if they should get the fifth star by checking for the presence of a links:xxx field.

1324480600000000 1325475095000000
#1588 enhancement johnglover johnglover ckan-backlog new QA - Give SPARQL endpoints a 4 star rating

Super: #1594

From Richard Cyganiak on the CKAN Discuss list:

Besides considering the media type of resources, it would also make sense to check for the presence of a SPARQL endpoint. SPARQL endpoints are recorded for more than 300 datasets on the Data Hub using the pseudo-type "api/sparql". A few more are recorded with the format "SPARQL". I suggest that datasets with such resources should also be considered for the fourth star.

1324480405000000 1325475178000000
#1599 enhancement rgrp rgrp ckan-sprint-2012-01-09 closed fixed [contrib] Simple embeddable dataset count widget (esp for group count)

Simple embeddable widget for use on 3rd party sites showing dataset counts for a given search query. Have a specific version just for groups.

  • Simple group count widget in JS for embedding in wordpress and elsewhere
    • Requested by several people (e.g Guo Xu from Econ working group). Already have something like this in CKAN JS for doing an embeddable search box.
  • All you need to do is do a dataset query over the API e.g. http://thedatahub.org/api/search/dataset?groups=economics and then embed in some html!

Estimate: 30m (for someone who knows their jquery).

1325246358000000 1325555201000000
#1298 enhancement kindly kindly ckan-sprint-2012-01-09 closed fixed Generate activites to be put into activities table.

This should be done from the logic layer or automatically from a session extension.

1314696442000000 1325591582000000
#1394 defect dread dread ckan-sprint-2012-01-09 closed fixed Resource validation error messages misleading

(Editing a dataset) If the second resource contains any validation error then it says "Resources: Package resource(s) incomplete" and "Resource 1:".

1318515262000000 1325604784000000
#1612 enhancement kindly kindly ckan-sprint-2012-01-09 closed fixed Group view page slow

Group show that lists packages is slow due to not using query in pagination.

1325633737000000 1325688886000000
#1614 enhancement kindly kindly ckan-sprint-2012-01-09 closed fixed remove po files from git diff

Its a pain to see the difference between branches as there are normally a lot of po file transaction. Make the default be see that they have changed without actually show the diffs themselves.

1325686639000000 1325689136000000
#1448 enhancement kindly kindly closed fixed Set up nice way to do celery deployment.

Celery is awkward to deply, need to find a way to do it more simply. i.e using celery-pylons and supervisor. A modified version of celery-pylons may be the best solution. 1d

1320666977000000 1325774155000000
#1611 enhancement ross ross closed duplicate Implement auth API calls for webstore/external use

As part of #1574 we decided that it would be better for webstore (and future external services) to be able to authenticate simply with CKAN-Core.

Currently webstore access the CKAN database to obtain the key for the user but it would be better if this connection was not so tightly bound and that webstore used an API as any other external service might.

Need to discuss further with dread

1325590191000000 1325846987000000
#1610 enhancement ross ross closed duplicate Move webstore to Postgres instead of Sqlite

The default backing store for webstore should be Postgres and not sqlite. This was agreed as part of the #1574 storage changes and on http://ckan.okfnpad.org/newstorage

1325587341000000 1325852472000000
Note: See TracReports for help on using and creating reports.