{22} Trac tickets (2647 matches)

Results (1701 - 1800 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#908 defect pudo rgrp ckan-v1.4-sprint-1 closed fixed Issues deploying extensions with modwsgi / uwsgi

Some extensions import pylons features (such as request) at root.

Have encountered following issues:

  1. You need to import ckanext before any ckan stuff in your wsgi script (extension stats and dataapi)
  1. Running under wsgi daemon mode (normal mode is fine!) the front page (front page only) will not work (does not occur with disqus or synchronous_search but does with stats and dataapi extensions). You get errors like this in the log:
[Mon Jan 17 09:01:19 2011] [error] Exception KeyError: KeyError(-1218594160,) in <module 'threading' from '/usr/lib/python2.6/threading.pyc'> ignored
[Mon Jan 17 09:01:20 2011] [error] Exception AttributeError: "'NoneType' object has no attribute 'clearing'" in <bound method PluginEnvironment.__del__ of  Services for Environment '<default>'
1295264624000000 1296730578000000
#926 enhancement sebbacon sebbacon ckan-v1.4-sprint-1 closed fixed Pick a simpler form framework

The current formalchemy setup conflates view, controller and model code in a way that makes it hard to debug and customise.

Review existing (and potentially non-existing) frameworks with a view to porting forms over to something more explicit and lightweight.

Implement the current Package forms as an example of how this would work.

Document and circulate.

Sub-ticket of #961 (form, validation, model sync meta-ticket)

1295869056000000 1298541597000000
#928 enhancement rgrp rgrp ckan-v1.4-sprint-1 closed fixed Documentation - CKAN credits / contributors page
  1. Remove contributors list from about page (out-of-date and does not belong there)
    • A "Powered by CKAN" link in footer is enough
  2. Add contributors / team / credits page to ckan.org
    • May want to separate credits (e.g. software use), from contributors (also is team different from contributors)
    • (perhaps with photos!) so that people can see who is part of it and from which organisations. (this is medium-term, not really part of this ticket)
  3. ? CREDITS.txt file to the source repository and add a link to it on bitbucket for more detailed credits info.

Cost: 1h

1295872661000000 1297115136000000
#931 enhancement dread dread ckan-v1.4-sprint-1 closed fixed Search results generator hides paging functionality

ckanclient's search results list only packages up to the 'limit'. It would be good to return a generator instead of a list. When the limit is reached on the generator then another 'page' is loaded automatically.

1296147360000000 1298379187000000
#938 enhancement pudo rgrp ckan-v1.4-sprint-1 closed fixed Message flashing / notification in WUI

Good user interfaces provide feedback to users. We should provide more feedback about the success / failure of actions using message flashing (see http://flask.pocoo.org/docs/patterns/flashing/).

We already do some of this (slightly poorly), e.g. on adding an authorization the app says: "X authorization added".

Implementation

  • Standardize this process and add section to html for this purpose.
  • Already done in other OKFN apps e.g. yourtopia.net, wheredoesmymoneygo and commentonit
  • Testing: suggest keeping this very simple

Estimate

  • Cost: 0.5h (html etc) + 1h to do some integration
1296398920000000 1297077023000000
#942 defect pudo pudo ckan-v1.4-sprint-1 closed fixed Fix SERP pager argument passing

It only passes "q", but the whole query must be serialized.

1296469490000000 1297074807000000
#943 enhancement pudo pudo ckan-v1.4-sprint-1 closed fixed Create CKAN Wiki

Set up a Mediawiki to contain info on CKAN.net (community) conventions etc.

1296469547000000 1297076498000000
#949 enhancement pudo pudo ckan-v1.4-sprint-1 closed fixed Generic username/password login

based on repoze.who-friendlyForms

1296658126000000 1297074827000000
#953 enhancement rgrp rgrp ckan-v1.4-sprint-1 closed fixed Add tagline/description to template and set in configuration

Have a tagline / description are under main title and set if from ckan.site_description config variable.

1296809808000000 1296809834000000
#960 defect pudo rgrp ckan-v1.4-sprint-1 closed fixed Support non-ascii character in internal error messages

Just saw an error like the following which looks like it is to do with having a user object with non-ascii characters in it. Either get rid of 'cast' to str type here or do it in a unicode aware way.

Should do this not just here but everywhere we can find in the code base.

Module ckan.controllers.package:302 in edit
<<          am_authz = self.authorizer.am_authorized(c, model.Action.EDIT, pkg)
               if not am_authz:
                   abort(401, str(gettext('User %r not authorized to edit %s') % (c.user, id)))
       
               auth_for_change_state = self.authorizer.am_authorized(c, model.Action.CHANGE_STATE, pkg)
>>  abort(401, str(gettext('User %r not authorized to edit %s') % (c.user, id)))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

Estimate

Cost: 1h

1297069721000000 1297169056000000
#963 enhancement thejimmyg rgrp ckan-v1.4-sprint-1 closed fixed Package CKAN as a debian package

Package CKAN as a debian package so it is easier to install.

1297076364000000 1298284252000000
#973 enhancement rgrp rgrp ckan-v1.4-sprint-1 closed fixed Document license configuration and license system.

Document how license system works and specifically how licenses can be configured.

Cost: 2h

1297332948000000 1297678851000000
#975 defect rgrp rgrp ckan-v1.4-sprint-1 closed fixed Placeholder attribute not displayed in some browsers leading to poor UX

Currently placeholder text like 'Search ...' is not shown in some browsers (e.g. FF and IE).

This leads to poor UX, for example in top bar search where it unclear whether that box is for login or search.

We should fix this by finding a way to display placeholder attribute in all browsers.

1297343755000000 1297344448000000
#980 defect pudo pudo ckan-v1.4-sprint-1 closed fixed Fixes to HR Infoshare

We've fixed a few things in CKAN that were discovered and tracked by HRI in their own issue tacker. This includes:

  • add authz checks to package edit links (cset:0752316cd2fe)
  • replace gettext with _ in controllers, to support unicode error msgs (cset:822340e6077e)
  • handle broken html in notes field without crashing on package read (cset:4b6be037dda0)
  • update i18n (cset:37d57dc3c492, cset:ea03173f5e77, cset:f16f4ee40fe7)
1297430918000000 1297503177000000
#496 requirement wwaites johnbywater ckan-v1.4-sprint-2 closed fixed Provide a mechanism for exporting GEMINII metadata via a CSW interface 1282427307000000 1299164106000000
#715 defect rgrp johnbywater ckan-v1.4-sprint-2 closed fixed The group list page shall show the group title *or* group name

Symptom #1: Groups without titles don't get a clickable link, because there is no text. Defaulting to the group name would solve this trouble. Symptom #2: Package groups drop-down list show's 'None' when a group doesn't have a title.

Therefore, we need to centralize presentation of group title, so it defers to group name. And we need tests.

Also please sort groups by title on group list page.

1287583089000000 1297796784000000
#805 task dread dread ckan-v1.4-sprint-2 closed fixed Migration scripts on buildbot 1289816755000000 1298379084000000
#808 enhancement pudo rgrp ckan-v1.4-sprint-2 closed fixed Allow i18n language choice on site (and possibly set automatically)

Allow choice of language on ckan instance.

  • Configuration variable listing language choices (could try doing this automatically but recommend against this)
  • Switcher option in UI (perhaps on user profile page)
  • Default it automatically based on browser settings (?)

Think this is high value (and cost is low)

Cost: low (4h)

1289944394000000 1297783658000000
#821 defect dread ckan-v1.4-sprint-2 closed worksforme Some resources have disappeared in CKAN

Simon has spotted 31 packages whose resources have disappeared in CKAN, but still appear in the Drupal front-end.

Here are the details:

  • in CKAN's package view (in web interface, API and dump) no resources display
  • but they WERE created in CKAN, as shown by the revision diffs.
  • they are in the Drupal front-end

So these resources must have been in the CKAN API at some point and then disappeared without trace/revision to alert Drupal.

Packages affected: anti-social-behaviour-orders-1999-2007 asylum-applications-jan-mar-2009 control-of-immigration-quarterly-statistical-summary-united-kingdom-2009-october-december coroners-statistics-england-and-wales courts-statistics-user-survey-england-and-wales court-statistics-company-insolvency-and-bankruptcy-england-and-wales court-statistics-england-and-wales court-statistics-mortages-and-landlord-possession-england-and-wales crime-in-england-and-wales crime-statistics-local-reoffending-england-and-wales crime-statistics-prison-and-probation-england-and-wales crime-statistics-reoffending-of-adults-england-and-wales crime-statistics-reoffending-of-juvenilles-england-and-wales data_gov_uk-datasets digest-uk-energy-statistics-2008 directgov-central-hottest-pages-monthly directgov-central-internal-search-terms-monthly directgov-section-visits-monthly electricity-consumption-2007 electricity-gas-consumption-2007 energy-consumption-uk-2008 final-energy-consumption-2007 foi-statistics-uk-central-government fuel-poverty-statistics-2007 gas-consumption-2007 gb-reported-bicycling-accidents gb-road-traffic-counts gb-traffic-matrix greenhouse-gas-emissions-2008 high-level-indicators-energy-use-2006 judicial-and-court-statistics-england-and-wales laboratory-tests-and-prices local-authority-carbon-dioxide-emissions-2007 magistrates-courts-statistics-survey-england-and-wales monthly-energy-prices monthly-energy-trends ni_012_refused_and_deferred_houses_in_multiple_occupation_hmos_licence_applications_leading_to_immig ni_013_migrants_english_language_skills_and_knowledge ni_023_perceptions_that_people_in_the_area_treat_one_another_with_respect_and_consideration ni_024_satisfaction_with_the_way_the_police_and_local_council_dealt_with_anti-social_behaviour ni_025_satisfaction_of_different_groups_with_the_way_the_police_and_local_council_dealt_with_anti-so ni_026_specialist_support_to_victims_of_a_serious_sexual_offence ni_029_gun_crime_rate ni_031_re-offending_rate_of_registered_sex_offenders ni_032_repeat_incidents_of_domestic_violence ni_034_domestic_violence_-_murder ni_036_protection_against_terrorist_attack ni_038_drug_related_class_a_offending_rate ni_078_reduction_in_number_of_schools_where_fewer_than_30_of_pupils_achieve_5_or_more_a-_c_grades_at ni_101_looked_after_children_achieving_5_a-c_gcses_or_equivalent_at_key_stage_4_including_english_an ni_109_delivery_of_sure_start_childrens_centres ni_126_early_access_for_women_to_maternity_services ni_127_self_reported_experience_of_social_care_users ni_128_user_reported_measure_of_respect_and_dignity_in_their_treatment ni_181_time_taken_to_process_housing_benefit-council_tax_benefit_new_claims_and_change_events ni_184_food_establishments_in_the_area_which_are_broadly_compliant_with_food_hygiene_law ni_185_co2_reduction_from_local_authority_operations ni_190_achievement_in_meeting_standards_for_the_control_system_for_animal_health ni_194_air_quality_-_reduction_in_nox_and_primary_pm10_emissions_through_local_authorities_estate_an other-fuels-consumption-2006 police-use-firearms-england-wales-2007-2008 prison-end-of-custody-licence-releases-and-recalls-england-and-wales prison-population-england-and-wales probation-offender-management-caseload-statistics-england-and-wales probation-statistics-quarterly-brief-england-and-wales quality-indicators-energy-data-2007 quarterly-energy-prices quarterly-energy-trends road-transport-energy-consumption-2007 sentencing-statistics-england-and-wales statistics-terrorism-arrests-outcomes-2001-2008 ukba-control-of-immigration-statistics-2008 ukba-control-of-immigration-statistics-2008-supplementary-tables uk-energy-in-brief-2008 uk-energy-sector-indicators-background-2008 uk-energy-sector-indicators-key-supporting-2008 uk-exportcontrollists uk-exportcontrol-sanctions uk-export-control-statistics uk-glossary-exportcontrol uk-ipo-offences weekly-fuel-prices

1290015776000000 1298486642000000
#877 enhancement rgrp rgrp ckan-v1.4-sprint-2 closed fixed File upload in WUI (extension)

Support uploading files to a storage system from WUI with a simple upload form.

Implementation

  • Implement as an Extension
  • Locate at /upload
  • Add new system action/permission (FILE_UPLOAD) and restrict by default to logged in users
  • Use OFS + boto -> generate form key (and create the bucket if it doesn't exist? Or do we assume bucket already exists)
  • This wants to be tied in to general package workflow. See parent ticket:852.
1292868146000000 1298624165000000
#956 enhancement kindly kindly ckan-v1.4-sprint-2 closed fixed Add a resource group table.

We need a way to group resources by various different factors. The most important of which is timeseries.

The model will look like

Package <o2m> resource_group <o2m> resource

The use cases to this are outlined in this email. http://lists.okfn.org/pipermail/ckan-discuss/2011-February/000887.html

1296821112000000 1299489084000000
#981 defect kindly kindly ckan-v1.4-sprint-2 closed fixed fix authorization_group_user.id so it's added in migrate

This was missed out the migrate script. It should be to be added as a conditional script at the end of the repository so that databases can catch up.

1297434480000000 1298278654000000
#982 enhancement dread rgrp ckan-v1.4-sprint-2 closed fixed Remove various pip-requirements versions

We can just use the branch name to pull down specific versions of pip requirements so we don't need all the different versions in head.

Remove them and update any client systems.

Cost: 20m

1297516060000000 1298887980000000
#984 defect kindly kindly ckan-v1.4-sprint-2 closed fixed Fix migration, where upgrade diverges from actual model.

The migration scripts have not been inline with the upgrade scripts. There are lots of discrepancies.

1297627447000000 1298280689000000
#986 defect wwitzel3 thejimmyg ckan-v1.4-sprint-2 closed fixed Broken link report from the ckanext-qa code

Should have the following features:

  • A list of all packages with broken links on the resource
  • Under each packages a description of any resources with broken links to include:
    • Link
    • Description
    • Format
    • First found to be broken date

[If you can't do the last one yet, don't worry]

1297683994000000 1297812401000000
#988 task wwaites thejimmyg ckan-v1.4-sprint-2 closed fixed Get a better HTML view of a GEMINI document using XSL 1297685975000000 1299493598000000
#993 defect thejimmyg jason.kitcat@… ckan-v1.4-sprint-2 closed fixed Link on ckan.net footer needs changing

The link for About > 'Project Home Page' on the CKAN.net footer should be changed to point to:

http://ckan.org/

The current link is broken.

Thanks, Jason

1298154006000000 1298373114000000
#1008 defect rgrp pudo ckan-v1.4-sprint-2 closed fixed eval() of user display name in template head

We're currently setting the user preferences links via a javascript snippet that also evals the name. This should be removed as we're not displaying the user name any longer.

We should also have page fragment caching in Genshi, which is not currently implmented.

cf. http://api.rubyonrails.org/classes/ActionController/Caching/Fragments.html

1298632686000000 1298821826000000
#1010 enhancement rgrp rgrp ckan-v1.4-sprint-2 closed fixed List CKAN users in WUI

Should have user listing at /user/ rather than user account page.

  • list users, sorted by number of packages contributed/edited
  • Move user home page to /user/{user-id}
  • Paginated
1298649180000000 1298740889000000
#427 task thejimmyg dread ckan-v1.4-sprint-3 closed fixed Match licenses from GEMINI document
  • examples of posting a new package
  • example license_id - explain
1281436611000000 1299164063000000
#801 enhancement wwaites johnbywater ckan-v1.4-sprint-3 closed duplicate Get/Set last harvested time for a given harvest source

Implies that last harvested time must be stored on source by harvesting job (see #802).

1289483546000000 1300196714000000
#904 task rgrp Stiivi ckan-v1.4-sprint-3 closed fixed Review CKAN documentation

What's bad at the moment?

  • lack of documentation e.g. config (very poorly documented)
  • too many sources of documentation
  • no common theming

Sources:

Resulting meta-ticket with things to do: ticket:927

1294832610000000 1299840539000000
#927 enhancement rgrp rgrp ckan-v1.4-sprint-3 closed fixed [super] Improvements to CKAN documentation

List of improvements to CKAN documentation (result of ticket:904). Should convert most items into sub-tickets as we go along.

Major

Miscellaneous

  • Contributors / credits page: ticket:928
  • (Data proxy documentation: move it to packages.python.org/dataproxy)
  • (List dataapi extensions on http://ckan.org/wiki/Extensions (Name, summary string + point to pypi page ...))
  • Document license configuration and license system - ticket:973
  • Refactor trac frontpage moving general feature overview to ckan.org
  • Create proper roadmap information on http://trac.ckan.org/
  • Add link to wiki.ckan.net to ckan.org
1295869492000000 1300105638000000
#994 defect kindly kindly ckan-v1.4-sprint-3 closed fixed document a new migration methodology

It is too easy to make a mistake with the migrations at the moment. A more systematic way of doing them is needed and this needs to be documented.

1298281192000000 1298912842000000
#999 enhancement rgrp dread ckan-v1.4-sprint-3 closed fixed Factor out ckan.net theme changes to a separate theme repo and apply

Rather than polluting the ckan core code base on ckan.net install with ckan.net specific changes these adaptations should be moved out into a dedicated ckan.net theme.

1298386729000000 1300707328000000
#1000 enhancement kindly rgrp ckan-v1.4-sprint-3 closed fixed Remove call timings code

Not used and extra overhead. Very useful to strip this out and low cost.

Places where it seems to be:

  • lib/base.py: c.time_call_started
  • ....
1298489643000000 1298912726000000
#1003 enhancement rgrp rgrp ckan-v1.4-sprint-3 closed fixed CKAN Javascript library and demonstration web interface

A plain javascript library for interfacing with CKAN would be very useful (why? see below!). It would also be nice to have a pure html + javascript web interface to CKAN both for its own sake and to act as a demonstrator for the library.

Why?

  • Development of bespoke interfaces -- much easier to edit html + javascript than to change ckan core
    • E.g. for specific communities e.g. geodata, science
    • Specialized tasks - multi-package editing
  • Very easy deployment and integration (e.g. can drop in to getthedata or other sites)
1298490086000000 1300100411000000
#1011 enhancement sebbacon sebbacon ckan-v1.4-sprint-3 closed fixed Make pluggable AuthzGroups implementation

I have a requirement to allow an external source to define the groups of which a user is a member.

I propose to create an IAuthzUserGroups plugin interface that allows an extension to arbitrarily extend the list of AuthzGroups? that a user is in.

1298819657000000 1299245206000000
#1014 defect sebbacon sebbacon ckan-v1.4-sprint-3 closed fixed Decouple oAuth repoze extension from DataNO, write tests

The current oAuth extension tightly couples repoze.who, ckan, and data.no. These components should be decoupled; they also need more tests

1298887493000000 1299245293000000
#1015 defect kindly dread ckan-v1.4-sprint-3 closed fixed Editing group gives exception

Editing a group properties on ckan.net gives an exception when you submit the form. (Is this related to db migration issues?)

Example page: http://ckan.net/group/edit/civil-society

Exception:

WebApp Error: <class 'sqlalchemy.exc.IntegrityError'>: (IntegrityError) duplicate key value violates unique constraint "group_revision_pkey" 'INSERT INTO group_revision (id, name, title, description, created, state, revision_id, continuity_id) VALUES (%(id)s, %(name)s, %(title)s, %(description)s, %(created)s, %(state)s, %(revision_id)s, %(continuity_id)s)' {'description': u"A group for open data related to civil society supported by members of the interest group at: http://okfn.org/groups/civil-society\r\n\r\nCivil society is composed of the totality of voluntary civic and social organizations and institutions that form the basis of a functioning society, as distinct from the force-backed structures of a state (regardless of that state's political system) and commercial institutions of the market. \r\n\r\nhttp://en.wikipedia.org/wiki/Civil_society", 'created': datetime.datetime(2011, 2, 24, 14, 30, 53, 334842), 'title': u'Civil Society', 'state': u'active', 'continuity_id': u'f4f9f09
1298898588000000 1299788821000000
#1018 task dread dread ckan-v1.4-sprint-3 closed fixed Remove gov form

(It has been moved into ckanext-dgu)

1299072516000000 1299073340000000
#1019 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Webhooks notification service

We propose the following push approach using individual packages:

  1. New, updated and deleted packages are pushed to a url endpoint on Wordpress (WPURL). This endpoint will be a configuration option on the CKAN side. A POST or PUT http method will get used. The payload will be a json document of the following form:

{

payload: The entity data as available from the REST API entity-type: 'Package', operation-type: 'create'|'update'|'delete' # one of these options

}

It will be sent as the body of the request with content-type set to application/json.

  1. The wordpress side will provide a 200 on success. Any other response will be taken as a failure. On failure, submission will be archived and failure logged and notified to system administrator. Submission can be resent later automatically by sysadmin after review.
  1. Pushes will happen continuously and approximately simultaneously with updates (a webhooks type model)
  1. [optional] CKAN side will support configuring authorization for basic authentication if applied on WP side.
  1. [optional] List queue status (including failures) on ckan adminstrative dashboard.
1299166784000000 1299166930000000
#1021 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Config option to disable OpenID

HRI don't like federation, want to login normal way only. Make this a config option and perhaps even mess with runtime repoze config

1299492920000000 1299518828000000
#1022 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Error reporting in CKAN worker API

Report on errors by any worker daemons, send them out via E-Mail

1299493047000000 1299512991000000
#1024 enhancement pudo pudo ckan-v1.4-sprint-3 closed duplicate Lock down all controller actions in CKAN

When a user visits the site, he/she may only see a login box.

1299493428000000 1299668648000000
#1025 enhancement dread dread ckan-v1.4-sprint-3 closed fixed Default authz can be set in config

Currently the default authz for a package is hard-coded to:

 <PackageRole user="visitor" role="editor" context="Package">,
 <PackageRole user="logged_in" role="editor" context="Package">,

This should be configurable in the config, so that you can have a more locked down instance etc.

1299596110000000 1299751045000000
#1026 enhancement dread dread ckan-v1.4-sprint-3 closed fixed cli for creating users

It's handy to be able to create users using the cli (e.g. dgu migration)

1299604652000000 1299605128000000
#1027 enhancement pudo pudo ckan-v1.4-sprint-3 closed fixed Authorization checks on all controller actions

We want to have authz checks on all controller actions so that we can lock down CKAN to a login-only mode.

1299666256000000 1299682082000000
#1028 defect dread dread ckan-v1.4-sprint-3 closed fixed Open redirect in locale setting 1299759883000000 1299760360000000
#1029 defect kindly ckan-v1.4-sprint-3 closed fixed synchronous search erroring when harvesting is run.

Errors are caused when harvesting documents. This also makes the count not show up correctly on the ckan search page.

1299768337000000 1301311643000000
#738 requirement kindly johnbywater ckan-v1.4-sprint-4 closed fixed Store revisions of harvested documents 1287775892000000 1299761436000000
#841 enhancement kindly dread ckan-v1.4-sprint-4 closed duplicate Caching docs (as a whole)

Documentation article on caching / improving performance. (To complement configuration docs.)

  • Different sorts of cache - beaker style, etags, package_dict in search results(?)
  • How each one affects performance
  • How to turn them on/off and configure them
  • Is it possible to bypass each of them in the browser or with wget/curl?
1291308879000000 1300364333000000
#1001 enhancement rgrp rgrp ckan-v1.4-sprint-4 closed fixed API should use normal user credentials if available

When using the API 'locally' i.e. from the CKAN instance (as would be the case with an ajax interface) the API, especially that allowing READ requests should use the normal user credentials if they are available prior to looking for an API key.

The key change appears to be to change _get_user_for_apikey method in lib/base.py BaseController? to check the c.user attribute (may wish to rename as the name may now be a bit misleading ...).

This is critical to incorporating any ajax editing into the frontend.

As part of this ticket we should do a general consolidation of the identification system in lib/base.py so that both api_key and normal user auth lead to the same set of auth-related objects being available (suggest c.user and c.userobj and c.author).

1298489705000000 1301310351000000
#1006 enhancement kindly rgrp ckan-v1.4-sprint-4 closed fixed Deprecate stable branch

Now that we have release branches we should deprecate the stable branch (ie. make sure it is no longer a head and then do --close-branch and merge into default one last time).

Cost: 10m (giving high priority because of low cost)

(Assigning to dread as he has been managing the stable branch).

1298624996000000 1300372286000000
#1031 enhancement johnlawrenceaspden rgrp ckan-v1.4-sprint-4 closed fixed User lookup API

Add an api for searching users. This is needed for any kind of ajax autocomplete (needed for anywhere we want to add users).

  • API location: /api/util/user/lookup?q=querystr&limit=10
  • Return json objects containing {id: ..., name: ..., fullname: ...}
  • Put in a module called controllers/apiv2/user.py
1299780419000000 1300101520000000
#1036 defect johnlawrenceaspden johnlawrenceaspden ckan-v1.4-sprint-4 closed fixed Add tests for three functions in ckan/model/user.py

create a new test file ckan/model/test_user.py

add tests for the following three functions in ckan/model/user.py

number_of_edits, number_administered_packages, search

merged in in changeset 0046f83aedcf

1300127840000000 1301304575000000
#1045 defect dread dread ckan-v1.4-sprint-4 closed fixed Group identified by ID in API
http://ckan.net/api/2/rest/group

returns group IDs but I can only reach a group by name:

http://ckan.net/api/2/rest/group/economics

when I also want to get a group by ID:

http://ckan.net/api/2/rest/group/04fb43d2-8ddf-4485-9bf5-66d47d3672f3
1300360642000000 1300793261000000
#1047 defect sebbacon sebbacon ckan-v1.4-sprint-4 closed fixed Package edit form claims you're not logged in at the end when you are

At the bottom, it says something like

Author: Bob Bumgardner Since you have not signed in this will just be your IP address. Click here to sign in before saving (opens in new window).

1300384556000000 1300387309000000
#1048 enhancement dread dread ckan-v1.4-sprint-4 closed fixed Complete making groups versioned
  • Deleting a group changes state to 'deleted' rather than purging it
  • Adding authz tests for deleted groups
1300387655000000 1300702752000000
#1051 defect sebbacon sebbacon ckan-v1.4-sprint-4 closed fixed Ability to set custom favicon

Allow deployers to set a URL pointing to their own favicon

1300703160000000 1301305079000000
#1052 defect dread dread ckan-v1.4-sprint-4 closed fixed Authz holes

No authz on:

  • Group creation/edit/listing
  • Package relationship create/edit/delete
1300709144000000 1300895410000000
#1054 defect kindly dread ckan-v1.4-sprint-4 closed fixed Ordering of resources

Changing an old resource and creating a new resource on the end results in the old resource moving to the end in the ordering.

This breaks tests:

  • (ckanext-dgu) ckanext/dgu/tests/ons/test_ons_loader.py:TestOnsLoadBasic.test_fields
  • (ckanext-importlib) ckanext/importlib/tests/test_loader.py:TestLoaderInsertingResources.test_0_reload

You can make a ckan test break with this patch:

diff -r e6643cf1324c ckan/tests/models/test_resource.py
--- a/ckan/tests/models/test_resource.py        Wed Mar 23 13:25:52 2011 +0000
+++ b/ckan/tests/models/test_resource.py        Wed Mar 23 19:22:35 2011 +0000
@@ -297,6 +297,8 @@
                'url':self.urls[1], 'format':u'OTHER FORMAT',
                'description':self.description, 'hash':self.hash,
                'id':original_res_ids[2]},
+            { #new
+                'url':'new'},
            ]
        pkg.update_resources(res_dicts)
        model.repo.commit_and_remove()

There seems to be a problem with vdm creating a replacement Resource for the old resource - because it has a duplicate position it is put to the end by the SQLAlchemy ordering_list function.

1300969236000000 1301305615000000
#1055 defect dread dread ckan-v1.4-sprint-4 closed fixed @search_related tests not running

Tests marked decorated "@search_related" should only be run against postgresql, but in fact they don't get run at all.

1300985228000000 1300992395000000
#1058 defect dread dread ckan-v1.4-sprint-4 closed fixed Give 400 error (not 500) for invalid locale or package_form

Examples which prompt annoying exception emails:

http://ckan.net/locale?locale=ja
Module ckan.i18n:21 in set_session_locale
           assert locale in _KNOWN_LOCALES

A bot has caused these:

http://ca.ckan.net/package/new?package_form=gov
Module ckan.forms.registry:32 in get_fieldset
               raise ValueError('Could not find package_form name %r in those found: \n%r' % (package_form, [en.name for en in entrypoints]))
ValueError: Could not find package_form name u'gov)' in those found: ['gov', 'standard', 'ca']
1301302303000000 1301303315000000
#754 enhancement wwaites johnbywater ckan-v1.4-sprint-5 closed wontfix Filtering for harvesting sysem

This has once again been ruled out for UKLP for the timebeing.

1288040469000000 1303118020000000
#937 enhancement sebbacon rgrp ckan-v1.4-sprint-5 closed fixed Record download stats for resources (extension)

As a User (especially as a Package Owner/Maintainer?) I want to know how many times a resource has been downloaded (and when).

So let's record download stats (as in clicks on the link for a resource).

Implementation

Old Spec (do it in CKAN)

  • Record info of form: resource id (or url?), timestamp
  • Do this via javascript capturing of onclick event talking to an api
  • API: /api/resource/{id}/download
    • POST to increment (how do we stop spamming -- could use a nonce setup with a random string set on each page load for the js)
    • GET to get data back { total: X, day_count: [ [yyyy-mm-dd, count], ... ] }

Questions

  • Do we record ip addresses (to handle de-botting etc)?
  • Do we count preview clicks as well?
1296340273000000 1302513831000000
#962 enhancement rgrp rgrp ckan-v1.4-sprint-5 closed fixed Improvements to data preview extension

Basic implementation done (and deployed):

https://bitbucket.org/okfn/ckanext-datapreview

However plenty to improve, e.g.

  • Support more formats (use external systems for preview?)
    • json (!)
    • html (trivial!)
    • sparql
    • ...
  • Do not display preview if no preview

Also suggest reworking to use external services rather than doing preview 'in house' (doing in house places heavy reliance on data proxy service and on converting data to a standard format).

1297072524000000 1301364987000000
#1013 defect sebbacon sebbacon ckan-v1.4-sprint-5 closed fixed Refactor & write tests for Wordpresser extension

The Wordpresser extension seems to be working well for DataGM, but needs caching and tests.

1298887392000000 1301909717000000
#1046 enhancement kindly thejimmyg ckan-v1.4-sprint-5 closed fixed Dictization and the new logic layer

The stages involved with doing this.

  • Convert model objects to standard dict format (DONE)
  • Convert standard dicts to current api formats (DONE)
  • Make standard dicts savable (DONE)
  • Validate standard dict format. (DONE)
  • Authorize actions
1300364694000000 1302777668000000
#1059 defect dread dread ckan-v1.4-sprint-5 closed fixed Loader coping better with poor search indexing

Loader currently checks for same name, but also should check for name_, name etc.

1301310596000000 1301312516000000
#1060 defect dread ckan-v1.4-sprint-5 closed fixed Spreadsheet importer tries to import readonly keys

e.g. we just added notes_rendered and that is read in as an extra field. Tests failing in ckanext-importlib

Also related: we are missing lost metadata_created and metadata_modified in the dumps.

1301312210000000 1301312487000000
#1061 enhancement dread dread ckan-v1.4-sprint-5 closed invalid Orphaned home/license page

No links to home/license and it contains out of date references to knowledgeforge. Remove it.

1301392968000000 1301922350000000
#1066 enhancement dread dread ckan-v1.4-sprint-5 closed fixed Default reader role too permissive

The definition of the 'reader' role includes creating packages, which is too permissive for some CKAN instances (e.g. DGU). 'Reader' suggests only reading, so I think this role should avoid creating and editing.

All projects so far want all roles to be able to create users, so this stays as a Reader action for now, as a convenience.

Implementation:

  • Action.PACKAGE_CREATE removed from reader's default_role_actions
  • Visitor has a new default role, called 'anon_editor' which can edit packages, but not groups / auth groups - you have to log in for that.
  • Migration script not needed?
  • Code comments written, to make clear the suggested policy
1301645250000000 1301932136000000
#1067 enhancement dread dread ckan-v1.4-sprint-5 closed fixed CLI for loading/dumping complete databases

Use 'db dump' and 'db load' for 'pg_dump' and 'psql -f' of a database. Use pylons config to find out database options.

1301645463000000 1302186503000000
#1068 defect dread dread ckan-v1.4-sprint-5 closed fixed metadata_modified problem

This test has been failing since the clocks changed:

======================================================================
FAIL: ckan.tests.models.test_package.TestPackageRevisions.test_02_metadata_created_and_modified
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dread/hgroot/pyenv-ckan2/lib/python2.6/site-packages/nose-0.11.3-py2.6.egg/nose/case.py", line 186, in runTest
    self.test(*self.arg)
  File "/home/dread/hgroot/ckan2/ckan/tests/models/test_package.py", line 283, in test_02_metadata_created_and_modified
    assert out == exp, (out, exp)
AssertionError: (datetime.datetime(2011, 4, 1, 10, 45, 50, 875509), datetime.datetime(2011, 4, 1, 9, 45, 50, 875509))

----------------------------------------------------------------------
1301652085000000 1302109505000000
#1071 defect dread dread ckan-v1.4-sprint-5 closed fixed Package history API moved to /api/rest/package/revisions

api/rest/package_history is not RESTful or follow API naming conventions. Therefore move it to /api/rest/package/revisions

Also, API docs incomplete.

1301937882000000 1301943180000000
#1072 enhancement dread dread ckan-v1.4-sprint-5 closed fixed Add filters to authztool

It takes several minutes to print the 'rights' on DGU, which is annoying when you only want to grep for a few lines. Much quicker than grepping is to filter in the query.

1302106311000000 1302106474000000
#1073 enhancement dread dread ckan-v1.4-sprint-5 closed fixed Search index checker

Tool that checks which packages have not been indexed.

Required for DGU: https://trac.dataco.coi.gov.uk/projects/datagov/ticket/940

1302185444000000 1302185825000000
#1079 enhancement kindly rgrp ckan-v1.4-sprint-5 closed fixed Refactor API to use new logic layer and dictization
  • Convert current api saves to the new standard dict format.
1302509530000000 1302777504000000
#515 defect kindly dread ckan-v1.4-sprint-6 closed fixed Inconsistent use of 'location' header in API

When you create a package then the 'location' header gets set. This doesn't happen for any other domain objects. I think this should be consistent - either none or all.

I've removed the info about the header in the docs in the meantime.

1282757360000000 1302774268000000
#829 defect rgrp dread ckan-v1.4-sprint-6 closed wontfix Admin CRUD broken

Browsing to the admin interface /admin (even logged in as a sysadmin) gives an exception.

1290695043000000 1303838115000000
#833 enhancement johnlawrenceaspden rgrp ckan-v1.4-sprint-6 closed fixed [super] Administrative dashboard extension

Create an admin dashboard as /ckan-admin/ allowing for admin operations and overview.

Possible features:

  • Purge revisions (or sets of revisions) and purge objects #1076
  • Set roles for users #1075
  • Put system into particular modes e.g. wiki mode (anyone can add, edit packages by default), data portal (only sysadmins or members of a special Editor group can create and edit packages)
    • WONTFIX
  • Overview of activity
    • WONTFIX - already have revision log

Currently have an admin section using the formalchemy admin controller to provide basic editing of model objects. This can still be used but located at /admin/model/.

https://bitbucket.org/okfn/ckanext-admin

Tickets

Notes

Here's putting into restricted mode (plus creating a dedicated authz group so that others can admin sysadmin simply through that group):

# first remove permissions from roles
# this is hacky but have to do it because we hardcode assignment of 
# role permissions on package on package create (see model/authz.py)
paster roles deny editor edit
paster roles deny editor create-authorization-group
paster roles deny editor create-group
paster roles deny editor create-package
paster roles deny reader create-package
# make superuser group
# create authz group administrators / Administrators (if not exists)
paster rights make agroup:administrators admin system
1290765921000000 1303236364000000
#840 enhancement dread dread ckan-v1.4-sprint-6 closed fixed On/off switch for ETags cache
  • Use config variable to switch ETags caching on or off. Consider joining in with 'cache_enabled'.
  • Default setting for (all) caching should be off.
  • Needs documenting in configuration.rst
1291308727000000 1302694133000000
#854 defect johnlawrenceaspden dread ckan-v1.4-sprint-6 closed fixed Tests for authorization_group

The coverage tool (run by buildbot in the ckan build) reports that only 24% of lines of ckan.controllers.authorization_group are run in tests and 38% of ckan.forms.authorization_group. This suggests a need for more tests.

1291723815000000 1304351843000000
#919 enhancement dread dread ckan-v1.4-sprint-6 closed fixed Package preview contains API & datapkg instructions
  1. Edit a package
  2. Hit 'preview'.

The preview contains the section "CKAN API / datapkg" which seems irrelevant at this point. Looking at ckan.net previews, the Comments section only appears when you view a package and not when you preview it. This seems more sensible - can this be done for the "CKAN API / datapkg" section too?

1295346730000000 1303202627000000
#936 enhancement johnglover rgrp ckan-v1.4-sprint-6 closed fixed Follow / watch package extension

As a (logged-in) User I want to watch (follow) a package, that is register my interest about a package. (Similar to watch/follow features in github/bitbucket/wikis).

NB: this is as much (if not more) about showing what packages are interesting to people as giving info to 'watchers'.

Need to finalize terminology (github uses watch for repos and follow for users while bitbucket combines both in 'followers'). Decision: use follow

Implementation

Interface

Become a follower:

  • Follow button on packages (if already watching say 'unfollow')

Package-related changes:

  • Show number of followers on a package
  • List followers of a package at /package/{name}/followers
    • On a separate page

a package User-related changes:

  • List followed packages
    • Either on user's page on a separate 'following' page. (NB: called 'following')
  • Does watching involve notifications (by email)
    • Probably not: you can already subscribe to RSS feed after all and email not that necessary (?)
  • [Future - don't have activity stream yet] Show what packages a user has started/stopped followed on a user's public activity stream on their user page

Nitty-Gritty

  • Want to do this in ajax-y manner
  • API endpoint: /api/2/follower
  • Store data in a new follower table

API

/api/2/follower

follow => PUT / POST
{
   user_id
   object_type
   object_id
}

If this is submitted by a user with user.id != user_id => error (401)

unfollow => DELETE

/api/2/follower/package/{id}
=> list of followers
[
    { safe dictized user }
]

NB: depends on access to a 'safe' dictized user object. Dictization is in nearly done, and current example of doing this by hand is in user API autocomplete method.

Table

Called 'follower'

user_id, table, object_id, created
xxx, package, yyy, ...
xxx, user, yyy, ... [future]

Random Extras

  • What about following users as well
1296339079000000 1303838713000000
#1037 defect amercader thejimmyg ckan-v1.4-sprint-6 closed fixed More Robust Harvesting for DGU

CKAN's harvesting facility is now live on DGU but there are some major improvements that could be made to make it more robust and better fit the generic CKAN harvesting framework proposed in #987.

Some of the key issues:

  • Error reports do not currently contain the ID or title of the document with the error.
  • We only have "added" and "error" logging on jobs when we really need a report of "added", "updated", "not changed" and "errors" with the items in each referencing a real metadata document for which harvesting was attempted
  • We need deletion and editing of sources, without deleting the harvested documents or packages
  • We need a more robust harvesting mechanism than a cron job or we need to deal with the case of multiple cron jobs running at once.
  • We need to know the last time a list of documents was scheduled for harvest and the last time each one was fetched.
1300197602000000 1304937601000000
#1056 defect dread pudo ckan-v1.4-sprint-6 closed fixed User links for OpenID users are broken

Use case:

  • Login using OpenID
  • Click on 'My account' - results in 404

Solutions:

  • User user.id instead of their name
  • Escape the URL properly.
1301060249000000 1302882616000000
#1075 enhancement johnlawrenceaspden rgrp ckan-v1.4-sprint-6 closed fixed Administrative dashboard - Edit Authorization related to System object

Roles on System object are important because admin role on system equates to being a 'sysadmin' (i.e. able to do anything).

  • Make users sysadmin (either as separate action or as part of editing roles on system object)
  • /authz subpage for editing roles on system object
    • Add and update user roles
    • Add and update authz group roles
    • List actions associated to roles at top of page (extra points for checkbox table with editability)
  • Document on http://wiki.ckan.net/Authorization what roles on System object 'mean' esp sysadmin role on System

Related Tickets

  • super ticket: #833
  • authz lib improvement and authztool refactor #1050
1302279799000000 1303227982000000
#1084 task wwaites wwaites ckan-v1.4-sprint-6 closed fixed ckan.net RDF links changed

need to make some changes for the links to semantic.ckan.net. it should use http://semantic.ckan.net/record/<package_id> now

append .rdf, .ttl, .nt, .dot, .json (even .html for an ugly table) to taste (or just leave off the suffix and let content negotiation take care of it)

the base url is changed, but it now uses id not name.

see for example:

1302616717000000 1304934534000000
#1089 enhancement dread dread ckan-v1.4-sprint-6 closed fixed Check for "--ckan" when running nosetests

(because if you forget, you get difficult to understand errors, and more than one person has tripped up on this)

1302631189000000 1302631733000000
#1090 defect dread dread ckan-v1.4-sprint-6 closed fixed Visitor can't create packages on new CKAN install

Default visitor roles in default config is reader, not anon_editor.

Problem caused by changes in #1066 (released in 1.3.3)

New installs will be affected, although simple to just increase permissions when the installer realises a visitor can't create packages.

The solution to the config getting out of sync with the code like this is to not have the default_roles in the config - refer to the code in the configuration instructions.

1302635219000000 1302635699000000
#1092 defect kindly kindly ckan-v1.4-sprint-6 closed fixed refactor logic layer to seperate out api, form logic

The logic layer is a bit too api centric. Make the reusable parts separate in preparation for the wui refactor.

1302777929000000 1305570822000000
#1093 defect dread dread ckan-v1.4-sprint-6 closed fixed 500 errors on GET to api/rest/licenses

CKAN gets its license list from a license service, which can be a local file, but is often the http://licenses.opendefinition.org/2.0/ckan_original server. This server is currently flakey, but I think we only request the list on start up. The problem is we query it much more often than required. It is queried for every request to api/rest/licenses, and we are returning lots of 500 errors when the license server is timing out.

1302862261000000 1302865470000000
#1097 enhancement dread dread ckan-v1.4-sprint-6 closed fixed Sidebar hideable

The web interface has a sidebar (#primary) which should be hidden in some pages. This is for QA extension and useful for package new and edit pages. Must be compatible with DGU theme.

1303293416000000 1303293476000000
#1098 task dread dread ckan-v1.4-sprint-6 closed fixed --ckan-migration tests not initialised correctly

Only tests with failing --ckan-migration fail, due to authz not being initialised.

1303377336000000 1303406017000000
#1100 defect dread ckan-v1.4-sprint-6 closed fixed Get buildbot running on ckan branches

Need some changes to pip-requirements files in release branches.

1303385267000000 1303406103000000
#1106 defect rgrp rgrp ckan-v1.4-sprint-6 closed fixed Bugs related to routes arising from API refactor + removal of default routes

Various bugs I've been encountering:

Latter issue was masked by existence of 'default' routes:

   map.connect('/{controller}', action='index')
   map.connect('/:controller/{action}')
   map.connect('/{controller}/{action}/{id}')

Having these is, I think, bad practice as it is better to be explicit and we should therefore remove asap.

In addition I think we should be cautious about 'default' routes in core such as:

    map.connect('/api/rest/:register', controller='api', action='list',
        conditions=dict(method=['GET'])
        )

As it makes it harder for extensions to introduce their own APIs (here one could perhaps add something at /api/rest/{my-object} but only by using before_map rather than after_map).

1303747360000000 1303834069000000
#736 requirement amercader johnbywater ckan-v1.4-sprint-7 closed fixed URL validation for CSW and WMS links

Robust against slightly incorrect URLs (e.g. with trailing slash, or without /csw), and non-CWS service which also return <ows:ExceptionReport ... />.

Both cases would be resolved by checking capabilities (with CswClient's check_get_capabilities() method).

1287749416000000 1304963598000000
#842 enhancement johnglover rgrp ckan-v1.4-sprint-7 closed fixed Todo list CKAN extension

An extension that provides a todo list feature on CKAN so that people can register and find things to do.

Extension name: ckanext-todo

User Story

Package page

As a user I come to a package:

  • Have a todo count at that top that takes you down to the todo list (which may say nothing todo)
  • At the bottom is a section of the package display titled "ToDo?" where I see a list of all toDos for the package most recent at the top
    • If I am logged in
      • See a form for "Add to do" at the top of the todo section and can add one straight away
      • I see a "now resolved" button next to each which goes green when you hover.

When clicked the todo fades away.

  • Not logged in: I see a button that says "login to add todo"

Todo list page

When a user comes to todo overview at /todo

At top list all todo categories with counts (or a progress bar). Click on category name or bar takes you down page to list for that category.

Category list has a list of todo items (ul with li items with class todo) - link to package relevant to the todo.

Implementation

The Todo form

  • One of the fields is category -> autocomplete the category (not constrained) (lowercase, no spaces, .-_ allowed)
  • Add a description
  • Submit, the todo gets added via AJAX to the list at the top as the most recent todo

Model:

todo table

  id (autoincrement integer)
  package_id
  todo_category_id (required)
  description (required)
  created=NOW()
  resolved=null (unresolved) or a datetime (datetime of resolution)
  creator=user
  resolver=user

todo_category table
  id
  name

Prepopulate with: broken-resource-link, no-author, bad-format, add-description

API at /api/2/todo

  • GET / POST / PUT ...

/api/2/todo?package=package_id_or_name&category=...&resolved=0/1

  • support limit (?)

/api/2/todo/category -> return list of todo categories

  • No GET / PUT / POST (these are auto-created by creation of todo)

Optional Extras (Will not be done atm)

  • Integrate todo tags (e.g. list packages tagged with a todo.{xxx} on Todo List page ...
1291467708000000 1305646487000000
Note: See TracReports for help on using and creating reports.