{22} Trac tickets (2647 matches)

Results (801 - 900 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#2788 enhancement amercader amercader ckan-v1.9 new Speed improvements on creating/updating and indexing

Specially needed when importing large numbers of datasets.

Profiling the import command from the harvesting extension has shown some areas where improvements could be made.

1343832992000000 1343832992000000
#287 enhancement dread dread v1.0 closed fixed Customise Canadian package edit form

Spec - see attached, which is a copy of the Pirate Pad from Lauren: http://piratepad.net/2C2iwiLDhd

1270801086000000 1270801210000000
#1341 enhancement kindly kindly ckan-backlog reopened Delete spam users from ckan

Spam users where added to thedatahub and we need to clean them.

1315995034000000 1320141540000000
#37 enhancement rgrp rgrp v0.5 closed fixed Purge a Revision (i.e. purge all changes associated with that revision)

Spam entries have started to occur on ckan.net in the last few months. It would be useful to be able to purge these revisions -- that is permanently delete all changes to domain objects associated with those revision (one might also permanently delete that revision -- or alternatively simply mark it as purged).

In addition to providing this facility from the shell it would also be useful to be able to do this from the web interface (with associated restrictions on usage via an authorization controller of some sort).

1192650660000000 1199786536000000
#2298 enhancement icmurray icmurray ckan-v1.9 new Add sort-by controls to the search results page

Sort-by functionality was exposed through the package controller in [1]. But no controls were added to the search-page.

  • What should the sortBy controls/widget look like?
  • Which fields should be exposed?

The above commit is in the release-1.6.1 branch, so this work is dependant upon release-1.6.1 from being merged into master.

[1] https://github.com/okfn/ckan/commit/8685c6000d1cb211928b4dbc63990fb72d884f8c

1334569162000000 1340635947000000
#355 defect rgrp rgrp closed fixed Dashes versus underscores in package names

Sort out how we deal with dashes versus underscores in package names.

1277221996000000 1311177552000000
#2253 enhancement toby toby closed wontfix CMAP [super]

Somewhere for CMAP stuff not in other tickets

need to create some general tickets

  • template changes
  • general demo server setup
1332341133000000 1340038490000000
#731 bug dread dread ckan-v1.3 closed worksforme Geo coverage field losses in Form API

Sometimes editing a package via api on dgu results in some countries being lost in geo coverage.

1287738539000000 1288038226000000
#2900 defect seanh ckan-v1.8.1 new snippets/package_list.html template crashes on dataset that has no 'notes'

Sometimes a dataset dict comes through package_show() with 'notes': None and then the 'notes' key gets stripped from the dict when validation happens (validation only happens if there's an IDatasetForm plugin with a db_to_form_schema() method) and then package_list.html template crashes, e.g. on user profile pages.

Fix in the template? Or in the schema?

The template also crashes on packages with no tags (empty list gets stripped from dict) and probably other empty fields as well

1346335207000000 1346335788000000
#2555 enhancement toby aron.carroll demo phase 5 new Demo site needs a breadcrumb helper

Something to make building breadcrumbs a bit nicer

1340026983000000 1342618384000000
#682 enhancement dread dread ckan-v1.4-sprint-1 closed fixed Add ckanclient tests to buildbot

Some work done on this to make tests run independently.

1286386769000000 1297358266000000
#2301 enhancement seanh seanh ckan-future new Replace old-style string formatting with format() method (at least in strings marked for translation)

Some strings in CKAN are passed to gettext with more than one %s in them, this is no good as translators may need to change the order of substituted words.

Strings should not use the old style % substitution, they should use the new format method in Python 2.6+, e.g.:

'This {food} is {adjective}.'.format(food='spam', adjective='absolutely horrible')

http://docs.python.org/release/2.6.8/tutorial/inputoutput.html#fancier-output-formatting

We should probably only change strings that are marked for translation and that contain multiple %s, don't want to make too much work or cause too many strings to change and need to be re-translated.

1334579264000000 1340635812000000
#1171 enhancement mark.wainwright dread ckan 2.0 assigned Citation instructions on dataset and resource view pages

Some sort of citation helper. Something small on the dataset and resource page that would show how to cite.

wwaites: Some related thoughts on this from opb: http://homepages.inf.ed.ac.uk/opb/papers/ssdbm2006.pdf

timclicks: I'm looking at Dataverse for the first time[0]. It seems very popular in the social sciences. I noticed that there is a recommended citation for each dataset. For example, [1] is has this one: "Targeted Input Programme (TIP) 2000-01", http://hdl.handle.net/1902.1/SSC-MWI-TIP2000-01-M1 V1 [Version]"

Implementation

Add a small box at bottom of dataset / resource page (or in sidebar on dataset page) with title "Cite this" with contents like:

%title. %author. Retrieved %date. %site_title.

For resource: %title = %dataset_title. %resource_name.

Could also add export to ref managers (e.g. to bibtex) but that is for later.

1306920799000000 1347358705000000
#1809 enhancement johnglover johnglover ckan-sprint-2012-03-05 closed fixed Catch request exceptions in archiver link_checker task

Some request exceptions are currently not being caught (see the celery log on thedatahub for examples)

1329746267000000 1330528828000000
#2259 enhancement johnglover johnglover ckan-sprint-2012-04-02 closed fixed Update ecportal theme for master changes (bootstrap)

Some recent changes caused things to break in ckanext-ecportal:

  • change from blueprint to bootstrap
  • other style / css changes (eg: tables, resource view notes field ID, etc. Forms are also broken, but that will be addressed in #2262)
  • change to going through db_to_form_schema on package_show
1332754180000000 1332769973000000
#2378 enhancement ross ckan-future assigned Extract metadata directly from resources that contain it.

Some files that might be uploaded as resources on a dataset, such as some image formats, already contain metadata. For example some jpeg files might contain Exif ( http://en.wikipedia.org/wiki/Exchangeable_image_file_format ) data.

[ ] Obtain list of embedded metadata 'standards'

[ ] Where the metadata is available it should be added as extra fields on the resource that is uploaded.

[ ] HTML?

[ ] Image formats (exif/gif etc)

[ ] PDF

[ ] Eventually consider ...

[ ] MS Office documents

1336651088000000 1346669985000000
#908 defect pudo rgrp ckan-v1.4-sprint-1 closed fixed Issues deploying extensions with modwsgi / uwsgi

Some extensions import pylons features (such as request) at root.

Have encountered following issues:

  1. You need to import ckanext before any ckan stuff in your wsgi script (extension stats and dataapi)
  1. Running under wsgi daemon mode (normal mode is fine!) the front page (front page only) will not work (does not occur with disqus or synchronous_search but does with stats and dataapi extensions). You get errors like this in the log:
[Mon Jan 17 09:01:19 2011] [error] Exception KeyError: KeyError(-1218594160,) in <module 'threading' from '/usr/lib/python2.6/threading.pyc'> ignored
[Mon Jan 17 09:01:20 2011] [error] Exception AttributeError: "'NoneType' object has no attribute 'clearing'" in <bound method PluginEnvironment.__del__ of  Services for Environment '<default>'
1295264624000000 1296730578000000
#2320 enhancement rgrp ckan-v1.9 new Introduce new state: archived

Some datasets become obsolete: e.g. http://datahub.io/dataset/uncdb Do not want to delete them but want them marked as archived or deprecated or something.

1335211949000000 1340624119000000
#1734 enhancement amercader amercader ckan-v1.7 closed fixed [super] Updates on harvesting for UKLP

Some changes are needed in ckanext-harvest to support the new piece of work in the UKLP harvesters, mainly related to the deletion of existing sources.

  • #1726 Update harvesting model
  • #1727 Add "current" field to harvesting objects
  • #1728 Update UKLP harvesters
  • #1733 Delete (inactivate) harvest sources

At some point we also want to support a publisher model for the harvesting:

  • #1731 Publisher support for harvesting
1328008474000000 1332167315000000
#872 enhancement johnbywater dread closed fixed ONS loader assigns packages to same package even when dept name changes

Solutions: ResourceSeriesLoader? allows synonyms for values.

1292424112000000 1292957110000000
#1256 defect kindly kindly ckan-sprint-2011-10-28 closed fixed fix modification extension for moderation

Solr is not working for moderation, pending changes are not being indexed.

1312280026000000 1319812556000000
#2368 enhancement seanh seanh ckan-sprint-2012-05-15 closed fixed Fix OS X Lion install instructions on wiki

Solr install instructions install the wrong version

1335893719000000 1336477258000000
#2573 enhancement icmurray ckan-future new package_search does not allow solr's per-field facet parameters

Solr allows its facet parameters to be specified on a per-field basis, eg. facet.limit applies to all facet fields, but solr allows it to be overriden for a specific field, eg. facet.tags.limit.

We don't support this at the moment because we have a whitelist of valid solr query parameters that we accept. See ckan.lib.search.query.VALID_SOLR_PARAMETERS.

1340112439000000 1340633101000000
#1789 enhancement seanh ckan-backlog new Implement a tag_update() logic action function

So users can rename a tag and/or move it between vocabularies using the API.

Currently we have create_tag() and delete_tag(), but if you were to 'update' a tag by deleting it and then recreating it all the datasets that had that tag will have lost it and you'll have to re-add it to them all.

What should happen to datasets that have the tag, if the tag gets moved between vocabularies? All the datasets just keep the tag with the new vocabulary? This will become a problem if/when we support 'radio button'-style vocabularies (where each dataset can only have one tag from the vocabulary).

1328805413000000 1339773812000000
#2568 enhancement aron.carroll shevski ckan-sprint-2012-06-25 closed fixed Crop/shorten titles in breadcrumb headings

So that they don't interfere with buttons such as here: http://s031.okserver.org:2375/en/dataset/afterfibre/resource/5d19f9fa-c5a4-426c-84de-c624d6e8c3b3

1340039534000000 1340108132000000
#714 bug johnbywater closed fixed DGU package form shall have a read only field for ??? attribute

So that Drupal doesn't need to make any adjustments to the form, and risk data being lost on submission.

1287581408000000 1291733788000000
#106 enhancement dread rgrp closed duplicate Regularly convert CKAN data to RDF and put on Talis CC

Sister to ticket:90 (Link to RDF version of CKAN data on Talis Connected Commons).

Talis have already kindly done an initial conversion. We should repeat this process regularly and re-upload the data to Talis CC.

In the long run may wish to only re-convert packages changed since the last upload. However given relatively smaller size of full dataset this optimization is probably not yet required.

Attached is the ruby script used by Talis for conversion

Cost: ? (1d+ depending on e.g. how easy integration with Talis CC is)

1251454474000000 1256140649000000
#63 defect rgrp rgrp v0.9 closed fixed Purge is broken after upgrade of vdm

Since we upgraded to new vdm (v0.4) which has built in purge support existing purge support looks to be broken. Should be reasonably easy to fix this.

Cost: 1h

1245148623000000 1245656000000000
#1409 enhancement zephod zephod ckan-sprint-2011-11-07 closed fixed Add Gravatars to user profiles

Since we now require email addresses, it's sensible to request users' gravatars to add a little flavour to their user profile (and, potentially, other places eg. comments threads?)

[Reopened: Additional] Use gravatars in the HTML generated by helpers.py to create lists of users. Eg. on user/list and dataset/history.

1319200773000000 1324047722000000
#1410 enhancement zephod zephod ckan-v1.5 closed duplicate Add Gravatars to user profiles

Since we now require email addresses, it's sensible to request users' gravatars to add a little flavour to their user profile (and, potentially, other places eg. comments threads?)

1319200774000000 1319213205000000
#3013 defect dominik dominik new common-error-messages is unreadable

Since the update of the doc theme, the page became unreadable.

http://docs.ckan.org/en/latest/common-error-messages.html

1352553505000000 1352553505000000
#870 defect memespring ckan-v1.3-sprint-1 closed fixed Menu items not selecting correctly

Since the main menu items have changed (part of the UI redesign) the selected items are not functioning correctly.

Specifically if Home / About or Add Package / Search are selected both menu items are highlighted.

This is because the code assumes only one top-level menu item per controller.

1292243583000000 1294914243000000
#2221 enhancement rgrp zephod closed fixed Tests are broken for some of us: NotAPairTreeStoreException

Since my last pull I'm getting a strange new error which is presumably to do with my local config or assumptions about the storage extension.

Test output here: https://gist.github.com/2007985

1331318781000000 1331561095000000
#2222 enhancement rgrp zephod closed duplicate Tests are broken for some of us: NotAPairTreeStoreException

Since my last pull I'm getting a strange new error which is presumably to do with my local config or assumptions about the storage extension.

Test output here: https://gist.github.com/2007985

1331318783000000 1331318825000000
#2938 enhancement Toby dominik new Explain bundling in doc/resources.rst

Since it makes sense to bundle resources into one js file that are commonly used together, there should be some explanations on how separate js scripts are merged into one file for production.

Explanations could include how groups are handled or different resource files.

1348488835000000 1348488835000000
#2780 enhancement toby shevski demo phase 4 new way for admins to undelete datasets

Since admins can see deleted datasets - there should be a way for them a) to know they are currently deleted & not viewable by normal users - ticket: #2779 b) way to undelete such datasets - this ticket

I suggest a button on the edit form instead of the delete button i.e. remove normal delete button with 'deleted dataset, only admins can view' with undelete button next to the message?

1343737248000000 1345023811000000
#1201 enhancement kindly ckan-backlog new seperate out logic in atom feeds to logic layer.

Simplify the logic in the atom feed an make all feeds use logic layer to return lists.

1308928892000000 1310124297000000
#2529 enhancement rgrp ckan-v1.9 new DataHub (or CKAN) widgets

Simple widgets in pure JS. For example:

  • Count of datasets in a group (could generalise to a query but not sure how useful that is ...)
  • Embeddable list of top X (5) datasets for a given query
  • Embeddable list of *my* datasets

Either these live at: {site}/widgets and we have some kind of generator (form where I choose my group, or my query).

Or: we have this attached to areas of site where relevant.

Can combine the 2 so that the latter links to the former. Think first will be easier to do and possibly more useful long-term (e.g. can just link people to that page).

Cf. http://okfnlabs.org/ckanjs/

1339750049000000 1340624917000000
#1599 enhancement rgrp rgrp ckan-sprint-2012-01-09 closed fixed [contrib] Simple embeddable dataset count widget (esp for group count)

Simple embeddable widget for use on 3rd party sites showing dataset counts for a given search query. Have a specific version just for groups.

  • Simple group count widget in JS for embedding in wordpress and elsewhere
    • Requested by several people (e.g Guo Xu from Econ working group). Already have something like this in CKAN JS for doing an embeddable search box.
  • All you need to do is do a dataset query over the API e.g. http://thedatahub.org/api/search/dataset?groups=economics and then embed in some html!

Estimate: 30m (for someone who knows their jquery).

1325246358000000 1325555201000000
#821 defect dread ckan-v1.4-sprint-2 closed worksforme Some resources have disappeared in CKAN

Simon has spotted 31 packages whose resources have disappeared in CKAN, but still appear in the Drupal front-end.

Here are the details:

  • in CKAN's package view (in web interface, API and dump) no resources display
  • but they WERE created in CKAN, as shown by the revision diffs.
  • they are in the Drupal front-end

So these resources must have been in the CKAN API at some point and then disappeared without trace/revision to alert Drupal.

Packages affected: anti-social-behaviour-orders-1999-2007 asylum-applications-jan-mar-2009 control-of-immigration-quarterly-statistical-summary-united-kingdom-2009-october-december coroners-statistics-england-and-wales courts-statistics-user-survey-england-and-wales court-statistics-company-insolvency-and-bankruptcy-england-and-wales court-statistics-england-and-wales court-statistics-mortages-and-landlord-possession-england-and-wales crime-in-england-and-wales crime-statistics-local-reoffending-england-and-wales crime-statistics-prison-and-probation-england-and-wales crime-statistics-reoffending-of-adults-england-and-wales crime-statistics-reoffending-of-juvenilles-england-and-wales data_gov_uk-datasets digest-uk-energy-statistics-2008 directgov-central-hottest-pages-monthly directgov-central-internal-search-terms-monthly directgov-section-visits-monthly electricity-consumption-2007 electricity-gas-consumption-2007 energy-consumption-uk-2008 final-energy-consumption-2007 foi-statistics-uk-central-government fuel-poverty-statistics-2007 gas-consumption-2007 gb-reported-bicycling-accidents gb-road-traffic-counts gb-traffic-matrix greenhouse-gas-emissions-2008 high-level-indicators-energy-use-2006 judicial-and-court-statistics-england-and-wales laboratory-tests-and-prices local-authority-carbon-dioxide-emissions-2007 magistrates-courts-statistics-survey-england-and-wales monthly-energy-prices monthly-energy-trends ni_012_refused_and_deferred_houses_in_multiple_occupation_hmos_licence_applications_leading_to_immig ni_013_migrants_english_language_skills_and_knowledge ni_023_perceptions_that_people_in_the_area_treat_one_another_with_respect_and_consideration ni_024_satisfaction_with_the_way_the_police_and_local_council_dealt_with_anti-social_behaviour ni_025_satisfaction_of_different_groups_with_the_way_the_police_and_local_council_dealt_with_anti-so ni_026_specialist_support_to_victims_of_a_serious_sexual_offence ni_029_gun_crime_rate ni_031_re-offending_rate_of_registered_sex_offenders ni_032_repeat_incidents_of_domestic_violence ni_034_domestic_violence_-_murder ni_036_protection_against_terrorist_attack ni_038_drug_related_class_a_offending_rate ni_078_reduction_in_number_of_schools_where_fewer_than_30_of_pupils_achieve_5_or_more_a-_c_grades_at ni_101_looked_after_children_achieving_5_a-c_gcses_or_equivalent_at_key_stage_4_including_english_an ni_109_delivery_of_sure_start_childrens_centres ni_126_early_access_for_women_to_maternity_services ni_127_self_reported_experience_of_social_care_users ni_128_user_reported_measure_of_respect_and_dignity_in_their_treatment ni_181_time_taken_to_process_housing_benefit-council_tax_benefit_new_claims_and_change_events ni_184_food_establishments_in_the_area_which_are_broadly_compliant_with_food_hygiene_law ni_185_co2_reduction_from_local_authority_operations ni_190_achievement_in_meeting_standards_for_the_control_system_for_animal_health ni_194_air_quality_-_reduction_in_nox_and_primary_pm10_emissions_through_local_authorities_estate_an other-fuels-consumption-2006 police-use-firearms-england-wales-2007-2008 prison-end-of-custody-licence-releases-and-recalls-england-and-wales prison-population-england-and-wales probation-offender-management-caseload-statistics-england-and-wales probation-statistics-quarterly-brief-england-and-wales quality-indicators-energy-data-2007 quarterly-energy-prices quarterly-energy-trends road-transport-energy-consumption-2007 sentencing-statistics-england-and-wales statistics-terrorism-arrests-outcomes-2001-2008 ukba-control-of-immigration-statistics-2008 ukba-control-of-immigration-statistics-2008-supplementary-tables uk-energy-in-brief-2008 uk-energy-sector-indicators-background-2008 uk-energy-sector-indicators-key-supporting-2008 uk-exportcontrollists uk-exportcontrol-sanctions uk-export-control-statistics uk-glossary-exportcontrol uk-ipo-offences weekly-fuel-prices

1290015776000000 1298486642000000
#109 enhancement rgrp dread closed fixed Tag search in the REST API

Similar to ticket:108 but for tags:

/api/search/tag?q

For tags search is extremely simple since you can only search by name.

Possible Extras

  • Allow "exotic" ordering of results e.g. ordering by number of packages with that tag.
  • Return this number with tag list.
1251976297000000 1291829457000000
#1643 enhancement shevski ckan-backlog new Add fixed tags to thedatahub for better browsing

Similar to publicdata.eu, want to have themed areas such as finance, environment, census, etc and country tags

1326393293000000 1326393293000000
#771 story johnbywater ckan-v1.3 closed worksforme Run CLI help command without arguments

Should return something sensible (currently reports an error).

1288190318000000 1294413256000000
#772 story johnbywater ckan-v1.3 closed duplicate Run CLI harvester command without arguments

Should return help for the harvester (currently crashes).

1288190373000000 1294413313000000
#328 enhancement dread rgrp v1.1 closed fixed Mention code libraries in API documentation

Should prominently (at the top?) mention existing code libraries for working with ckan api. Have:

  • Python: ckanclient
  • Perl: luke closs wrote something
  • PHP: drupal library?
  • ...?
1274988106000000 1275318745000000
#89 enhancement nickstenning rgrp closed fixed Tag cloud for package tags

Should not be hard to do (lots of existing libraries) but not sure that this is very important.

1248693939000000 1265892698000000
#2357 enhancement aron.carroll aron.carroll demo phase 5 accepted Create build script for front-end resources

Should minify and concatenate JavaScript? and CSS as part of #2354

1335886181000000 1343220524000000
#1449 enhancement johnglover johnglover ckan-sprint-2011-11-21 closed fixed Improved resource listing on dataset view page

Should list resources in the form of search results rather than as a table.

1320677693000000 1321873008000000
#1010 enhancement rgrp rgrp ckan-v1.4-sprint-2 closed fixed List CKAN users in WUI

Should have user listing at /user/ rather than user account page.

  • list users, sorted by number of packages contributed/edited
  • Move user home page to /user/{user-id}
  • Paginated
1298649180000000 1298740889000000
#986 defect wwitzel3 thejimmyg ckan-v1.4-sprint-2 closed fixed Broken link report from the ckanext-qa code

Should have the following features:

  • A list of all packages with broken links on the resource
  • Under each packages a description of any resources with broken links to include:
    • Link
    • Description
    • Format
    • First found to be broken date

[If you can't do the last one yet, don't worry]

1297683994000000 1297812401000000
#82 enhancement rgrp rgrp closed fixed Support diffing of versioned objects

Should have function/facility to get a diff of a version object between 2 revisions. Function should return a dictionary of fields with diffs.

Details:

  • Do we deal with m2m relationships (and m2one on the many side)?
  • What is diff?
    • For text fields output of python diff command
    • For non-text field guess just simple +/- for what was there before and now

Cost: 4h (don't think this is a huge request)

1248289499000000 1256565441000000
#2596 enhancement toby aron.carroll opendatasuite 1 closed fixed Create demo extension

Should have all supported extensions enabled, those that don't work should be disabled and ticketed. Also needs a customised homepage.

1340642315000000 1345209485000000
#820 defect dread closed fixed Exception browsing /user/apikey when not logged in

Should fail gracefully (not the existing 500 error) when you go to ckan.net/user/apikey (manually typing in the url) but haven't logged in.

WebApp Error: <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'apikey' 						

URL: http://test-hmg.ckan.net/user/apikey
1290006079000000 1311325226000000
#41 task rgrp rgrp v0.5 closed fixed Upgrade to Latest Version of Pylons (0.9.6.*) and Deploy on Production Machine

Should do this sooner rather than later to avoid 'clear water' opening up and need to deploy then asap onto production machine to avoid 'clear water' between trunk and production (which then prevents pushing from trunk to production).

1199786854000000 1199787967000000
#2248 enhancement johnglover johnglover ckan-sprint-2012-04-02 closed fixed Update docs for creating custom forms

Should describe implementing IDatasetForm, subclassing controllers is deprecated

1332328630000000 1332331220000000
#2913 defect seanh ckan 2.0 new IGroupForm has package_form() method

Should be group_form() I think

1347374739000000 1350303779000000
#2737 enhancement toby shevski demo phase 2 closed fixed change help/prefilled text in format field

Should be basic format (which is what the datastore wants & what we display to users) e.g. "CSV, XML, JSON etc"

http://s031.okserver.org:2375/dataset/new_resource/

1343121262000000 1343136141000000
#93 enhancement rgrp rgrp v0.10 closed fixed Access control for packages

Should be able to limit ability to a user's ability to do things with packages (read, edit etc). This is a big ticket (it may required splitting) and full details are in separate wiki page: AccessControl.

Cost: 8d

1249055049000000 1253613274000000
#2565 enhancement aron.carroll shevski demo phase 1 closed fixed Social links to open in new tab/JS lightbox

Share this (twitter/fb) links open in new tab (no JS) or nice lightbox of some description in demo

1340039322000000 1340116053000000
#79 enhancement dread rgrp v0.10 closed fixed Improvements to package search

Several things:

  • Default package search (used e.g. on front page) should deal well with multiple items: "abc xyz" means search for both abc and xyz (as separate searches)
  • Should automatically display a list of matching tags in a line at top with number of associated packages
  • have an option to only search for fully open material (or openly licensed material ...)

Since we should be moving to a more sophisticated search solution anyway (to allow prioritisation etc) maybe we should cautious as to what we do now.

1247829864000000 1265890806000000
#474 task dread dread closed wontfix Questions related to fi.ckan.net

Several questions to answer (and see file in email). To respond to the Finnish team and copy all Q&A to the ckan-discuss list.

1 ) I have played with two different datasets to get the metadata from those imported to CKAN. The other dataset is in PCAXIS format and other one is in Excel format. Please find the scripts + sample files attached in a zip file that contains instructions too to run the scripts. I looked at import scripts written in object oriented manner and getdata samples in CKAN but it was easier for me too write these from scratch (copied the unicrap function from CKAN samples). If I run the PCAXIS import script I get Unicode decode error (see errors file). When I use the unicrap function then the error does not appear in logs. I assume this is some Python environment setting issue. Of course I would like to run the script without unicrap function to get scandic letter imported correctly to ckan(and displayed in WUI). In Excel case there is no scandic letter issue which is kind of strange but this probably relates how the Excel library handles encoding.

To be honest the whole unicode/encoding/decoding issue has been quite difficult to deal with and debug. I hope I would get proper environment configuration to handle scandic letters.

2 ) It seem in CKAN by default the localisation file contains 462 strings. The transifex link Rufus sent contains 400 strings. Is transifex the one we should use as localisation file ? Where are the rest 62 strings then fetched from or are they needed at all. I have found that as I have been using the .po file coming in CKAN source (which has API translations too) this also causes similar unicode problems when an error in API occurs (the API error messages containing scandic letters provide the similar UnicodeDecoding? errors as mentioned above). So, it seems the unicode issues appear in various cases. Maybe a proper environment settings will be able to prevent also these problems... If transifex is the one to use I can prepare it and maybe then you could upload it to fi.ckan.net as there will be also other people checking the translations. To run the test and see the API translation errors especially in pcaxis case I would recommend to install attached FI translations to your environment and include lang = fi configuration. It looks like the API errors does not come when running default lang = en.

3 ) Is it possible to change the license selection/options available in license field drop-down ? Where are these located and can these be modified locally ?

4) It looks like the values stored in package extra fields are not searchable. Is this intentional ? It looks more or less that all other fields can be searched using free-text search in CKAN.

5) How package dependencies are defined and managed ? For instance we want to import some packages in a manner that the other package is a child package of another. Does API support this and/or is this possible to do in WUI ?

Best Regards, Henri

1282312255000000 1294916760000000
#1640 enhancement amercader dread closed fixed Setup publicdata.eu harvester for Serbian CKAN datasets

Set-up publicdata.eu to harvest datasets at rs.ckan.net (Serbian community CKAN).

1326370425000000 1327340939000000
#1414 enhancement shevski ckan-backlog new track user log-ins on thedatahub.org

Set up tracking for user logins so that we have stats about how many active users of thedatahub exist want to be able to see who logged in the the last x months

1319454782000000 1319454782000000
#943 enhancement pudo pudo ckan-v1.4-sprint-1 closed fixed Create CKAN Wiki

Set up a Mediawiki to contain info on CKAN.net (community) conventions etc.

1296469547000000 1297076498000000
#2884 defect seanh ckan-v1.8.1 new Lock icon shown next to unknown licenses

Set the license of a dataset to blank (the first option) or to 'License Not Specified' or 'Other (Non-Commercial)' and a lock icon appears next to the license on the dataset read page. Shouldn't this be a question mark or something?

1345564873000000 1345564873000000
#372 bug johnbywater johnbywater ckan-v1.2 closed Fix system limits on CKAN for DGU

Set limits in /etc/security/limits.conf so that we can always ssh in at least. Requested by DGU.

1279885752000000 1281522535000000
#1082 defect johnlawrenceaspden closed fixed language changes behave strangely

Set language to Greek, flash message says 'Language set to: English', but page is now about half in Greek.

Set language back to English causes server error:

AttributeError?: 'NoneType?' object has no attribute 'path'

Module ckan.controllers.error:29 in document view

if original_request.path.startswith('/api'):

However going to a new page reveals that it's back to English

1302541989000000 1315917217000000
#2872 defect seanh ckan-v1.8 closed fixed Non-open licenses are marked as open

Set a dataset's license to 'Other (Not Open)', the dataset will have an Open Data icon next to it

1345124135000000 1350304179000000
#2645 enhancement johnglover johnglover ckan-ecportal closed fixed Update server setup to use local files

Server setup scripts need to have two distinct steps:

1) Download all files that aren't in RHEL repositories, create RPMs where necessary (python virtualenv).

2) Install from local files.

1341916281000000 1341996302000000
#1304 enhancement amercader amercader ckan-sprint-2011-09-12 closed fixed Show more descriptive error messages when sending emails

Sending emails is failing in some cases, and the error message just says 'Could not send reset link'.

1314893206000000 1314893749000000
#1130 enhancement lucychambers assigned First time users

Send users to FAQ first time on CKAN

1304938761000000 1340633514000000
#139 defect rgrp rgrp v0.11 closed worksforme Ensure you cannot add same tag twice to a package

Seems some package have ended up with same tag twice: http://www.ckan.net/package/fred2dot0

Check whether this is still possible and if it is fix it ...

Cost: 2h

1254768552000000 1255188974000000
#2492 enhancement toby aron.carroll ckan-sprint-2012-06-25 closed fixed Demo create related form has broken error output

See: http://s031.okserver.org:2375/dataset/adur_district_spending/related/new

Submit the form to see broken formatting.

eg. [u'Missing value']

Example of broken image

1338977874000000 1338982678000000
#2252 defect amercader amercader ckan-sprint-2012-04-16 closed fixed UKLP harvesters returning errors when importing due to schema changes

See trace below.

As in #2231, the schema passed with the context is no longer used on the logic layer (in this case the one used on the DGU form is used). This can be fixed allowing a schema to be passed with the context.

  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 139, in import_stage
    self.import_gemini_object(harvest_object.content)
  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 163, in import_gemini_object
    package = self.write_package_from_gemini_string(unicode_gemini_string)
  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 386, in write_package_from_gemin
    package = self._create_package_from_data(package_dict, package = package)
  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 486, in _create_package_from_dat
    package_dict = action_function(context, package_dict)
  File "/usr/lib/pymodules/python2.6/ckan/logic/action/update.py", line 169, in package_update
    data, errors = validate(data_dict, schema, context)
  File "/usr/lib/pymodules/python2.6/ckan/lib/navl/dictization_functions.py", line 216, in validate
    converted_data, errors = _validate(flattened, schema, context)
  File "/usr/lib/pymodules/python2.6/ckan/lib/navl/dictization_functions.py", line 289, in _validate
    convert(converter, key, converted_data, errors, context)
  File "/usr/lib/pymodules/python2.6/ckan/lib/navl/dictization_functions.py", line 179, in convert
    converter(key, converted_data, errors, context)
  File "/var/lib/ckan/std/pyenv/src/ckanext-dgu/ckanext/dgu/validators.py", line 81, in validate_license
    if data[('license_id',)]== '__extra__':
KeyError: ('access_constraints',)
2012-03-19 10:47:41,736 ERROR [ckanext.inspire.harvesters] Error importing Gemini document: ('access_constraints',)
1332333907000000 1334567502000000
#17 enhancement johnbywater johnbywater closed invalid Delete a file

See ticket:16 (update a file)

1152550822000000 1250619180000000
#2620 enhancement toby aron.carroll demo phase 2 closed fixed Pass a related item into the dataset tempate

See the related sidebar box

http://s031.okserver.org:2375/dataset/adur_district_spending

If a package has related items the first (or a random) related item should be passed into the snippet.

1340897809000000 1342074737000000
#1722 enhancement seanh seanh ckan-sprint-2012-02-20 closed fixed Update the package model for vocabularies

See the etherpad for notes: http://ckan.okfnpad.org/22

1327950373000000 1329303197000000
#1239 defect rgrp rgrp ckan-sprint-2011-10-28 closed fixed Remove and re-add tag on package not working

See reports on list at:

Tracked this to issue in dictization/model_save whereby was ignoring case where Package Tag already there but in deleted state. Rather than describe at length see fix.

1311441020000000 1312453156000000
#667 defect thejimmyg dread ckan-v1.3 closed wontfix API is slow

See message and script: http://lists.okfn.org/pipermail/ckan-discuss/2010-September/000569.html

1500 requests shouldn't take hours.

1285885228000000 1297069121000000
#1450 enhancement zephod johnglover ckan-sprint-2011-12-05 closed fixed Simplify dataset view page

See http://wiki.ckan.org/Dataset_and_Resource_Pages_Proposal

  • Create a collapsible section for the README
  • Reduce the number of things on the page
  • Tidy up groups and tags in the sidebar
  • Resource view becomes a list of search results

(additional)

  • Resource navbar item links to first resource
  • Resource navbar has a dropdown menu of all resources in the dataset
  • Edit and Authz navbar items should be in a separate logical group to the right

Very rough summary: create more info section and move stuff out (relayout page e.g. README, tags etc)

1320677788000000 1330019974000000
#1070 enhancement rgrp rgrp ckan-v1.5 closed fixed Plan a new domain model and layer architecture for CKAN

See http://wiki.ckan.net/Domain_Model especially section on v2.

  • New domain model is planned but not yet finally agreed.
  • Layer architecture is complete and implemented
1301910940000000 1310117129000000
#2291 enhancement kindly rgrp ckan-v1.9 new DataStorer incorrectly attempts to store JSON in DataStore

See http://thedatahub.org/dataset/ecb13156-cf11-4e91-b51c-f287d154276e/resource/5db67915-40cb-467e-9d72-194ea4973670

This was marked as json format (and is JSON) but appears to have been imported as a quasi-CSV. Either this hsould not be imported at all or it should be imported as JSON ... (which is straightforward to do).

Assigning to David for triage and review.

1334431147000000 1340636737000000
#2659 defect aron.carroll shevski demo phase 2 closed fixed HTML preview window in demo too small

See http://demo.ckan.org/dataset/example-dataset/resource/d8797e51-b497-46ca-a274-8675533d110b

David previously fixed this on some version of demo, seems to have regressed. Just want the window to be longer

1342010723000000 1342089310000000
#2530 enhancement kindly rgrp ckan-v1.9 new DataHub purge fails on some revisions

See http://datahub.io/ckan-admin/trash and try to purge revisions (*not* datasets). It will fail on some of the revisions with errors like:

Problem purging revision 391db9e8-df57-4e0e-8fe6-d4e0c2318344: (IntegrityError?) update or delete on table "revision" violates foreign key constraint "group_extra_revision_revision_id_fkey" on table "group_extra_revision" DETAIL: Key (id)=(391db9e8-df57-4e0e-8fe6-d4e0c2318344) is still referenced from table "group_extra_revision". 'DELETE FROM revision WHERE revision.id = %(id)s' {'id': u'391db9e8-df57-4e0e-8fe6-d4e0c2318344'}

1339750498000000 1341268280000000
#1277 enhancement johnglover johnglover ckan-sprint-2011-09-12 closed fixed Use solr query parser for search instead of ckan query parser

See http://ckan.okfnpad.org/search for details/discussion

1313414446000000 1315948417000000
#198 enhancement rgrp dread closed fixed Change package and tag ids to uuids

See how we did it already for other things.

Note: on ckan.net older PackageRevision?.id might not be identical to Package.id but this may need sorting at this point.

1258980613000000 1266837606000000
#1802 enhancement dread dread ckan-sprint-2012-03-05 closed fixed Investigate CKAN deployment on Windows

See how possible it is to run CKAN on a Windows server.

Plan:

  • Get CKAN code and dependencies installed on Windows
    • ideally as similar to standard package install as possible
    • virtualenv/pip should work
    • try Cygwin to help
    • SOLR and Postgres could potentially run on other servers if necessary.
  • Try to run on Apache
  • See if there are any issues
1329409512000000 1329830097000000
#2690 enhancement aron.carroll shevski demo phase 2 closed fixed long URL misaligns edit button for slug

See attachment. Just need to amke sure that there's enough spacing around the edit button and that it lines up so it doesn't look too wierd

1342538301000000 1342542394000000
#189 enhancement dread dread v0.11 closed fixed Multiple download urls

See also: PackageResources which defines a 'distribution' as a package's payload (perhaps with metadata). In our case, at least within CKAN, the representation of the distribution will just be the reference to get it (plus some minor metadata). We are expanding this functionality to be any kind of 'resource' associated with a package since it is likely in some cases people will want to link to a file containing a part, but not the whole, of a package's data.

As a

User (package editor and package user)

I want to

(package editor) Associate multiple distributions with a package.

(package user) Easily have access to multiple distributions of a package should they exist.

Implementation

There is an object called 'Resource' with the following attributes:

  • url (text) (required in UI, but not in db table)
  • format (text)
  • description (text)
  • package (association) - a resource is always associated with 1 package

A package's list of resources will have an order which is preserved. The first in the list will be known as the 'default resource' (the presumption will always be that this is a 'distribution').

How to replace download url

download_url removed and replaced as a proxy attribute. Read returns the url of the default distribution. Write changes the url of the default resource. Ensures backward compatibility.

REST API has new attribute in package dictionary 'resouces' - an ordered list.

In WUI, replace download url in package read with a table of the distributions.

In the WUI package edit, be able to add/remove/edit/reorder distributions.

Example

Example part of package:

Distributions:

http://some.com/pollution.pdf PDF Original report
http://some.com/pollution.csv csv Original data
http://some.com/pollution sparql Linked version of data

Existing data to be migrated to new structure.

Cost: 4 days

1258386120000000 1265284961000000
#7 enhancement johnbywater johnbywater closed invalid View a release associated with a package

See Update a release (ticket 14) for more information on what should be shown.

1152550022000000 1204136209000000
#271 enhancement johnbywater johnbywater v1.0 closed fixed The system shall support distributing changes.

See DistributingChanges for analysis and functional requirements.

1268672137000000 1273596144000000
#1510 enhancement seanh seanh ckan-v1.7 closed wontfix Migrate existing data into activity streams database tables

See '### Task 2' in this etherpad: http://ckan.okfnpad.org/notifications

I have no idea yet about how to do this or how long it might take me.

1323093901000000 1339677620000000
#1713 enhancement icmurray icmurray ckan-sprint-2012-02-06 closed fixed DGU dataset search page

Search results page of DGU.

Breakdown:

[X] ensure faceting by group (publisher) works [X] allow faceting by UKLP dataset type [X] allow faceting by INSPIRE

1327590700000000 1327932401000000
#2402 enhancement kindly kindly ckan-sprint-2012-05-29 closed fixed search result speedup.

Search results are slow as we do many package_dictize. Store the package_dict in the search index so that we do not have to hit the database for each of them.

1337300201000000 1337781991000000
#249 enhancement rgrp dread closed invalid Regex search

Search of package name and title (and other fields) using regular expressions.

Current example use-case: Wanting to specify packages with title beginning with 'B'.

Issues:

  • Syntax for specifying regex over natural language search - could it be contained in the q param so be available to users of the WUI, or do we simply make it alternative fields?

Implementation:

  • Postgres reg ex searching detailed here:

http://www.postgresql.org/docs/current/static/functions-matching.html#FUNCTIONS-POSIX-TABLE

1265994509000000 1311182450000000
#159 defect dread rgrp v0.11 closed fixed Searching for tags:... resulting in lots of tags being found

Search of form: tags: ... behaves differently depending on whether there is a leading space:

  • tags: postcode - tags found correctly but no packages found
  • tags:postcode - tags incorrectly found but correct packages found

Let's fix this.

Cost: 0.5h

1256030097000000 1256060264000000
#1819 enhancement kindly kindly ckan-sprint-2012-04-02 closed fixed change search index code to use package_dictize instead of obj.to_dict

Search index code needs to use package_dictize so that it can get the new vocablurary information and is more consistant with the rest of the system. This is in preperation for their translation being put in the search indexer.

1329770555000000 1338193886000000
#2308 enhancement icmurray icmurray ckan-sprint-2012-04-30 closed fixed Merge facet html snippets in the multi-lingual branch

Seanh and icmurray both touched on the html snippets in facets.html. So there's a merge issue. icmurray to merge changes in the multi-lingual branch prior to merging into master.

1334591529000000 1335440505000000
#1491 defect seanh dread closed fixed Visible strings need internationalisation

Sean spotted that some strings need internationalising, such as flash messages. Generally all strings that appear on the web front-end should be internationalised.

Particular places that need i18n:

  • ckan/controllers/admin
  • ckan/lib/base

Note: there are some exceptions, such as in i18n.py, very obscure error messages and stuff that only appears on the API.

NB: there is a cost in making a string internationalisable (all our volunteers have to translate it), so we should not be too zealous. We should also look at the i18n/ckan.pot to see if any of the existing strings can be reused.

1322481000000000 1330020630000000
#357 defect johnbywater johnbywater closed fixed Fix API documentation, to correct misleading HTTP_AUTHORIZATION statement.

Sean Burlington wrote:

I recently came across a documentation issue

http://knowledgeforge.net/ckan/doc/ckan/api.html


The key should be passed in the API request header: Header Example value HTTP_AUTHORIZATION fde34a3c-b716-4c39-8dc4-881ba115c6d4


But the header name is actually just 'AUTHORIZATION'

For example to set it in PHP

curl_setopt($ch, CURLOPT_HTTPHEADER, array('AUTHORIZATION: xxxxx-xxxx-xxxx-xxxx-xxxxxxx'));

1277459886000000 1277461466000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Note: See TracReports for help on using and creating reports.