{22} Trac tickets (2647 matches)

Results (1801 - 1900 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1820 enhancement kindly kindly ckan-sprint-2012-03-19 closed fixed Index multilingual data when mulilingual extension is added.

All translated fields and vocabularies need to be added to search index in the correct fields.

1329770903000000 1332163374000000
#1821 enhancement kindly kindly ckan-sprint-2012-03-05 closed fixed create internal multilingual extension that brings together all multilingual features

This extension point needs to combine.

  • Making sure the correct data get put in search index.
  • Make the search queries are modified to weight multilingual search correctly.
  • View pages, including search results are translated.
  • Bring together all search schema modifications.
1329771267000000 1330990111000000
#1822 enhancement kindly kindly ckan-sprint-2012-03-05 closed fixed Weight mulilingual searches correctly

Dismax query across languages. Titles and selected languages, weighted more highly.

1329772080000000 1330990127000000
#1823 enhancement rgrp ckan-backlog new Spring clean bin directory

Huge number of accumulated (and likely unnecessary) scripts in /bin directory.

1329773331000000 1338203554000000
#1824 enhancement seanh ckan-future new Add vocabulary pages

For a free tag foo you can visit the page at /tag/foo and see a list of all the datasets that have the tag foo, and when the tag appears on dataset view pages etc. it's linked to this tag page.

We should do the same thing for vocabulary tags. A tag bar in vocabulary baz should be hyperlinked to a page /tag/baz/bar, or perhaps /vocab/baz/bar.

1329845089000000 1338204958000000
#1825 enhancement seanh seanh ckan-sprint-2012-03-05 closed fixed Write docs for vocabularies API

Done on branch feature-1698-tag-taxonomies

1329845122000000 1330807330000000
#1826 enhancement kindly kindly ckan-sprint-2012-03-05 closed fixed weight queries so that title is more important than rest of body

Currently everything is considered equally when doing a normal search. Use dismax when there is not a fielded search to get round this. This is part of the work for #1822 as the dismax option is needed for multilingual search.

1329924260000000 1330990247000000
#1827 enhancement dread ckan-backlog new 'Register' link should be hidden if you not allowed to register
I have just deny visitors the create-user permission:
sudo -u ckanstd paster --plugin=ckan roles deny reader create-user -c /etc/ckan/std/std.ini
sudo -u ckanstd paster --plugin=ckan roles deny anon_editor create-user -c /etc/ckan/std/std.ini

and after restarting, the register link is *not* hidden, but now when you access the register page, it shows you this message "Unauthorized to create a user" (when not logged in). But anyway that is an improvement.
1329924939000000 1339773730000000
#1828 enhancement rgrp rgrp ckan-sprint-2012-03-05 closed fixed Upgrade to Recline v0.3

Recline v0.3 has introduced some breaking changes in API and usage. Upgrade to use it.

1329943304000000 1330525412000000
#1829 defect dread dread ckan-sprint-2012-03-05 closed fixed Changing back to English prints the flash message in the previous non-English language

On the homepage click "francais" and then "English". The flash message reads "Le langage a été fixé à: français" when it should say "The language is now: English".

1330000660000000 1330001990000000
#1830 enhancement dread dread ckan-sprint-2012-03-05 closed fixed OpenID disablable

Allow OpenID to be disabled as a login option.

Currently (CKAN 1.6) there is a config option openid_enabled which stops the login option being displayed, but:

  • In the User edit form you can still add your OpenID
  • The repoze.who.openid middleware is still required - can it be avoided?
1330002137000000 1330089912000000
#1831 enhancement ross ckan-backlog assigned Login with email address

Users should be able to log in using either their username, or their email address, both of which are unique.

Will require a change to UsernamePasswordAuthenticator? in ckan.lib.authenticator.py and possible a useful User.by_email in the user model if it doesn't already exist.

1330073906000000 1346670504000000
#1832 enhancement dread ckan-backlog assigned dataset purge API

Purging datasets (deleting them fully, not just changing the state to 'deleted') is important for users testing dataset creation over the API on a test CKAN instance.

Without this, they need to resort to more difficult methods such as:

  • cleaning and reloading the database
  • setting the test datasets to state 'deleted' and also appending a suffix '_00' and incrementing the number until there is no clash of names.

Requested for NHSIC.

Implementation

  • This could slot into the Action API.
  • Of course we would need to ensure the user's had been given the specific right to purge.
  • I suggest we log the full details of the dataset being purged.
1330077745000000 1339773706000000
#2197 defect zydio ckan-backlog assigned Storage Metadata API: add/update not working with local file storage (Pairtree)

If OFS is configured with Pairtree to use a local file storage, all POST requests to add/update metadata ( /api/storage/metadata/{label} ) will fail.

This is due to the use of BotoOFS specific private methods in StorageAPIController.set_metadata(), eg: self.ofs._require_bucket(bucket), self.ofs._get_key(b, label), self.ofs._update_key_metadata(k, metadata) ... those methods can't be found in POTFS and this causes errors. The API should use only OFSInterface methods, or should conditionally make calls based off the actual type of self.ofs.

PS: I did set "ckan" as "Component" in the ticket because storage has been integrated back into the core in CKAN 1.6

1330421377000000 1346662128000000
#2198 enhancement zydio closed fixed API documentation is missing Storage Metadata API info

Now that ckanext-storage is back into the core (v1.6), CKAN documentation should probably contain info on Storate Metadata API.

1330421743000000 1339771453000000
#2199 enhancement seanh seanh ckan-sprint-2012-03-19 closed fixed Translation of content on dataset search pages 1330440107000000 1331146832000000
#2200 enhancement seanh seanh ckan-future new Add vocabulary_id option to tag_show() logic action function

Currently tag_show() only works with free tags, it's not possible to get a tag_show for a vocabulary tag.

1330533678000000 1338204900000000
#2201 enhancement rgrp ckan-v1.7 closed duplicate Add citation info to Dataset and Resource page

Suggest:

%title. %author. Retrieved %date. %site_title.

For resource: %title = %dataset_title. %resource_name.

1330765080000000 1330765400000000
#2202 enhancement rgrp ckan-future reopened Display page view count on dataset and resource pages

Just like we display download counts we should display view counts.

1330765455000000 1338204929000000
#2203 enhancement kindly rgrp ckan-v1.9 assigned Data Viewer support for PDF
  • Boot viewer for format/mimetype: application/pdf, application/x-pdf, application/acrobat, applications/vnd.pdf, text/pdf, text/x-pdf
  • propose we use pdf.js - https://github.com/mozilla/pdf.js
  • We could also try iframing for browsers with native support but this seems more complex
1330860293000000 1348568961000000
#2204 enhancement ross rgrp ckan-sprint-2012-04-30 closed fixed [super] Related (Stuff) Extension

This is a reworking of the existing apps extension.

Initial proposal at http://wiki.ckan.org/Proposals#Apps_in_CKAN and http://wiki.ckan.org/Proposals#References.2FLinks_in_CKAN

Naming

What do we call this extension?

  • related
    • RP best IMO but perhaps too close to separate relationships concept?
  • relatedstuff
  • links
  • references

Proposal

"Related Stuff": Apps as in an application (website/service/tool) that uses this dataset (as in Apps and Ideas extension)

Implementation

New table named Related with following structure

|| id (int) || type || title || description (markdown) || image_url || owner_id || url || created (timestamp)
  • type = Idea | App | API | Visualization | Post | Paper | News Article
    • Suggest we make this a ckan.ini config option (comma separated ...?)
      • Do we want the possibility of different templates for different types of Related objects?
  • image: ?? Depends where we store images. Simplest option would be to change to image_url and leave it to users to have already uploaded an image somewhere. If not we need to support image uploading and storage. See #1692 (add image attribute to datasets and groups) for more discussion, once implemented the URL here can be an internal url.
  • owner_id = user_id or creating user (see authorization below)

Related2Dataset (note that related_id, dataset_id tuple should be unique). This allows for m2m connections. If a given related item is only with one dataset this could be simplified. May contain status so dataset owner can turn this on/off.

|| id || dataset_id || related_id || status

status should be used to allow for a dataset owner (for dataset_id) to de-activate the relationship between the dataset and the related.

Url

  • /dataset/{dataset-name}/related/{related-item-id}/{related-item-title-stringified}
    • If a reference item could exist in its own right (and perhaps refer to multiple datasets then it should get its own url at e.g. /related/{id}
  • /dataset/{dataset-name}/related/add => Modal dialog on related tab so we can use API to create them.

/dataset/{id}/related <- list

  • use image_url for small icon etc, title description (shortened?)
  • Click through to full related item (optional)
  • dataset owner is shown show / hide button ... (or on /dataset/{id}/related/{id} )
  • related owner sees an edit button / icon (pops up modal)

/dataset/{id}/related/{id} (optional)

/dataset/{id} will have a Related tab (with bubble with count).

  • Drop down with Add Related -> Pop-up modal and save via API

(Not used: /related/add with dataset prefilled ... )

/related/{id}/edit

Authorization

Addition of related item be considered orthogonal to datasets (and hence with separate authorization i.e. i can add the info that site X uses dataset Y without needing permission to edit dataset Y).

Thus any logged in user could add a Related item. We set the owner of the related item to creating user and going forward only that user or a sysadmin can update or delete.

NB: we could have a system where datasets owners have to approve related items before they show up next to their dataset. This would add substantial complexity so I propose we leave out of phase 1.

Tasks and estimates {7.5d}

[x] Model + Migration for Related table. {0.75d}

[x] Controller for Related (or relevant sections in Package controller). {0.75d}

[x] Routing setup. {0.25d}

[x] Schema for related. {0.5d}

[x] Logic layer actions. {1.0d}

[x] Auth (default + publisher). {0.5d}

[x] Templates + Dataset changes (new tab etc). {1.0d}

[x] JS Application for adding Related objects in a modal. {1.0d}

[x] Testing. {0.75d}

[x] Dataset owner disabling of Related (via M2M table). {0.5d}

  • Updated to allow author of related to delete as well

[x] Documentation. {0.25d}

1330862992000000 1337594723000000
#2205 enhancement johnglover johnglover ckan-sprint-2012-03-19 closed fixed Document Tag Vocab schema/form setup 1330941067000000 1330956840000000
#2206 enhancement johnglover johnglover ckan-sprint-2012-03-19 closed fixed Change site header to match latest ODP template 1330958095000000 1331046486000000
#2207 defect seanh seanh ckan-sprint-2012-03-19 closed invalid Fix recently changed datasets activity stream on exportal site

Apparently it's working when developers run it locally but not on the demo site.

1330961739000000 1332154835000000
#2208 enhancement seanh seanh ckan-sprint-2012-06-25 closed fixed Export strings from ckan and ckanext-ecportal to combined pot file

All strings from the ckanext-ecportal extension (and from CKAN core) need to be exported to a pot file to send to tenforce.

1330961806000000 1339596371000000
#2209 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Object->RDF mapping

Super is [1649]

We need to provide customisable mappings between CKAN models and RDF. This needs to be easily configurable for different clients and should take into account objects related to the model being mapped.

Current contenders for impl are:

  1. ordf.org's Object Description Mapper (which uses OWL)
  1. RDFAlchemy, although this looks like it may result in fixed mappings
  1. A custom scheme with a simple json file
  1. A template based approach using Genshi templates?

Approach is to get a mapping complete of the attributes directly in the package class, and then provide mappings for related objects (tags/resources etc).

We'll then need to be able to publish the RDF to a triplestore.

See http://ckan.okfnpad.org/rdfmapping for further analysis.

1331027277000000 1336566051000000
#2210 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Add method to group for adding members

It is necessary to add a helper method to Group for adding members to that group so that

  • nobody needs to work with the Member model
  • It handles existing membership and existing membership with a different capacity
1331028470000000 1331549369000000
#2211 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Logic layer access to publisher functionality

Most of the get_group/add_member style calls should be handled, or at least provided via logic layer.

Don't need to break existing code, but do need to provide access via logic layer

1331028751000000 1331549406000000
#2212 enhancement johnglover johnglover ckan-sprint-2012-03-19 closed fixed Import latest data into demo site 1331047333000000 1331720191000000
#2213 enhancement johnglover johnglover ckan-sprint-2012-04-02 closed fixed Update geographical coverage tags 1331047525000000 1332342835000000
#2214 enhancement seanh seanh ckan-sprint-2012-03-19 closed fixed Translation of content on group view pages 1331146864000000 1331147119000000
#2215 enhancement seanh seanh ckan-sprint-2012-03-19 closed fixed Translation of content on group list page 1331148579000000 1331653117000000
#2216 enhancement seanh seanh ckan-sprint-2012-04-02 closed fixed Translation of selected facets on search pages 1331148648000000 1332956791000000
#2217 enhancement seanh seanh ckan-sprint-2012-03-19 closed fixed Translation of content on tag list page 1331148704000000 1331652833000000
#2218 enhancement seanh seanh ckan-sprint-2012-03-19 closed fixed Translation of content on tag pages 1331148734000000 1331747842000000
#2219 enhancement seanh seanh ckan-sprint-2012-03-19 closed fixed Translation of datasets and tags on user pages

User names and descriptions are not translated.

1331148895000000 1331750370000000
#2220 enhancement toby toby ckan-sprint-2012-03-19 closed fixed make ckan cacheable

need to sort cookies etc

should work with nginx and varnish

1331228643000000 1338205222000000
#2221 enhancement rgrp zephod closed fixed Tests are broken for some of us: NotAPairTreeStoreException

Since my last pull I'm getting a strange new error which is presumably to do with my local config or assumptions about the storage extension.

Test output here: https://gist.github.com/2007985

1331318781000000 1331561095000000
#2222 enhancement rgrp zephod closed duplicate Tests are broken for some of us: NotAPairTreeStoreException

Since my last pull I'm getting a strange new error which is presumably to do with my local config or assumptions about the storage extension.

Test output here: https://gist.github.com/2007985

1331318783000000 1331318825000000
#2223 enhancement zephod rgrp ckan-sprint-2012-03-19 closed fixed Switch to bootstrap from blueprint

Why?

  • We already use quite a bit of boostrap
  • We already plan to use bootstrap JS
  • Bootstrap is excellent and pretty light
1331384974000000 1332842692000000
#2224 enhancement rgrp rgrp ckan-sprint-2012-04-02 closed wontfix Simplify javascript and css dependencies and add minified version
  • Get rid of jquery UI
  • Remove modernizr (what do we use it for?)
  • Convert jquery.tmpl to jquery mustache (?)
  • Remove ckanjs moving core parts of it directly into our code.

Compress everything for releases and add minified version to build (with switch to enable use of this?)

1331385623000000 1338205358000000
#2225 enhancement aron.carroll rgrp demo phase 2 closed fixed Run jshint on our javascript and clean up as needed 1331407316000000 1343220502000000
#2226 enhancement rgrp rgrp ckan-sprint-2012-03-19 closed fixed [super] Refactor and improve documentation (v1.7)

Documentation is key to the success of projects!

Move material into main docs from wiki

Create a User Guide

Basic step-by-step on getting started with CKAN (See start on set of slides here e.g. http://blog.thedatahub.org/2012/03/02/tutorial-publish-data-with-the-datahub/)

  • Publishing data
  • Authorization and workflows
    • Publisher Profile (Workflow)

Break into sections

Suggest something like:

# General
* Intro
* Conceptual Overview
* What CKAN helps you do (http://ckan.org/)
* FAQ ? (or point to the wiki)

# For Administrators

* Installation and Setup
  * Configuration options
* Customization - Theming etc
* Authorization and Workflows
* Storage

# For Users (Publishers, Data Wranglers, etc etc)
* Walkthrough of publishing a dataset
* Storage
* API (see Developers section)

# CKAN Developers

* Domain Model
* API
* Extensions
* i18n

CKAN Developers
* Buildbot
1331481965000000 1332475494000000
#2227 enhancement toby toby ckan-sprint-2012-03-19 closed wontfix Tests are slow and broken

not all tests run and they take too long :(

1331556570000000 1332843126000000
#2228 enhancement ross ross ckan-sprint-2012-03-19 closed fixed IGroupForm should allow overriding of non-edit templates

IGroupForm should allow overriding of non-edit templates so that it can specify the index +/or search +/or view.

1331637803000000 1332149030000000
#2229 enhancement kindly kindly ckan-sprint-2012-03-19 closed fixed Cleanup plugin system after some test failed to run.

The logic test did not have init. This caused lots of tests to fail because there were mock extensions that ran automatically in them. Fix plugin system so this can work.

1331721611000000 1332163408000000
#2230 refactor seanh ckan-v1.7 closed fixed Tidy up of search facets code duplication

Because of a clash between two development branches there is some duplication of code to do with code facets (note: at the time of writing the code duplication exists only on the feature-1821-multilingual-extension branch, but this will be merged into master at some point):

The package_search() function is adding the search facets to the search results twice with two different data structures, with keys "facets" and "new_facets". It should be reduced to just the new facets (with the key changed to "facets").

Also the group and package controllers are adding both facets and new_facets to the context, should be new_facets only (but renamed to facets).

The facet_items() function in helpers.py should be removed, it uses the old facets structure and shouldn't be needed anymore with the new facets structure.

In facets.html, facet_sidebar() should be removed as it uses the old facets structure and facet_div() implements the same functionality but uses the new facets.

In facets.html, facet_list_items() will have to be updated to not use the facet_items() helper and to use the new facets structure instead.

Anywhere that "new_facets" appears it will have to be changed to "facets" (e.g. in the ckanext/multilingual/plugin.py.

This is the merge commit that introduced the duplication: https://github.com/okfn/ckan/commit/1153aa876f54c22289e460aeececea22d1d4d51d

This is the earlier commit where the search facets were refactored: https://github.com/okfn/ckan/commit/3970e52008b75933fda1be1d488bed2578d98c9c

1331744085000000 1338204707000000
#2231 defect amercader amercader ckan-sprint-2012-04-02 closed fixed Base harvester broken on 1.6.1

On its current version, ckanext-harvest passed a custom schema to package_create and package_update to allow defining an id for new packages (the same as in remote ckan instances) [1]

Recent changes in the logic layer have made that the context object is no longer directly read from the context, but only get from plugins [2]. This makes the schema sent from ckanext-harvest to be ignored and the harvesting fails because the id field is not expected

ValidationError: {'Id': u'The input field id was not expected.'}

We need a way to either:

  1. Be more flexible accepting schemas on the logic functions
  2. Modify the harvester plugin to implement the necessary interface

[1] https://github.com/okfn/ckanext-harvest/blob/master/ckanext/harvest/harvesters/base.py#L115

[2] https://github.com/okfn/ckan/blob/0c07d3be846777318b35c31325215dff00e99313/ckan/logic/action/create.py#L38

1331815499000000 1333374735000000
#2232 defect amercader amercader ckan-sprint-2012-04-02 closed fixed Unicode Exception when rebuilding the search index

In some cases this exception is fired when building the search index. As explained in #1616 this makes the whole process stop.

  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/solr/core.py", line 326, in wrapper
    return self._update(content, query)
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/solr/core.py", line 550, in _update
    rsp = self._post(selector, request, self.xmlheaders)
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/solr/core.py", line 639, in _post
    return check_response_status(self.conn.getresponse())
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/solr/core.py", line 1096, in check_response_status
    raise ex
SolrException: HTTP code=400, reason=ParseError at [row,col]:[1,2354] Message: An invalid XML character (Unicode: 0x1) was found in the element content of the document.
Traceback (most recent call last):
  File "/var/lib/ckan/pdeu/pyenv/bin/paster", line 9, in <module>
    load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 84, in run
    invoke(command, command_name, options, args[1:])
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 123, in invoke
    exit_code = runner.run(args)
  File "/var/lib/ckan/pdeu/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 218, in run
    result = self.command()
  File "/var/lib/ckan/pdeu/pyenv/src/ckan/ckan/lib/cli.py", line 298, in command
    rebuild()
  File "/var/lib/ckan/pdeu/pyenv/src/ckan/ckan/lib/search/__init__.py", line 134, in rebuild
    {'id': pkg.id}
  File "/var/lib/ckan/pdeu/pyenv/src/ckan/ckan/lib/search/index.py", line 54, in insert_dict
    return self.update_dict(data)
  File "/var/lib/ckan/pdeu/pyenv/src/ckan/ckan/lib/search/index.py", line 79, in update_dict
    self.index_package(pkg_dict)
  File "/var/lib/ckan/pdeu/pyenv/src/ckan/ckan/lib/search/index.py", line 153, in index_package
    raise SearchIndexError(e)
ckan.lib.search.common.SearchIndexError: HTTP code=400, reason=ParseError at [row,col]:[1,2354] Message: An invalid XML character (Unicode: 0x1) was found in the element content of the document.
1331816582000000 1332331660000000
#2233 enhancement kindly kindly ckan-sprint-2012-03-19 closed fixed Make package show, use custom json schema

There is a disrepency between what we put in and get out when using a custom form/api. Narrow this gap by making sure package_show shows the correct serilization.

1331865349000000 1332178421000000
#2234 enhancement seanh ckan-future assigned Write a CKAN extension for pulling items from RSS/Atom feeds into CKAN templates

You configure the extension with some RSS and/or Atom feeds, it automatically reads items from these feeds and makes them available in the template context, you write a custom template to e.g. display 'news' items from a Wordpress blog on your front page.

This extension might be simpler and less fragile than ckanext-wordpresser, and also more generally useful.

Details

  • Mark Pilgrim's Universal Feed Parser might be useful for reading the feeds
  • Feed items should probably be cached somewhere

Suggested

  • The news item 'widget' should be wrapped in a known class so that it can be styled easily regardless of the format of any HTML entry.
  • For non-HTML formatted items (Atom should tell you the content type of the entry) maybe we should have a template for rendering each item along with any enclosures that it might reference
  • Caching is pretty crucial and should probably obey the ttl of the feed.
1331902755000000 1346669567000000
#2235 enhancement rgrp ckan-future new Group drop down on dataset edit should use chosen and sort groups by name 1331907357000000 1338204726000000
#2236 enhancement rgrp rgrp ckan-sprint-2012-03-19 closed fixed Updates to Recline and integrate them
  • (Recline) Better graphing (multiple graph types etc)
  • (Recline) Bootstrapify
  • Integrate
1331910381000000 1332153688000000
#2237 enhancement kindly kindly ckan-sprint-2012-03-19 closed fixed Make way to split up tests and run tests with different db. 1331938496000000 1332178352000000
#2238 enhancement seanh seanh ckan-future closed duplicate Deploy some test vocabs and publishers on test.ckan.net, check tutorial-style instructions for these 1332165311000000 1343392681000000
#2239 enhancement ross ross ckan-sprint-2012-04-30 closed fixed EC ODP RDF/N3 templates

Implement the EC/ODP template for RDF to match the requested schemas.

Using the previous work on allowing for templatised RDF representations we should implement the proposed schema for EC/ODP.

We also agreed to allow upload of RDF as an extra (raised to top level in package schema) which will instead be used for the RDF template. On insertion extra CKAN triples will be added... initially this is just owl:sameAs although we're awaiting EC/ODP agreement on which triples we should add to show the CKAN link.

1332169703000000 1335516499000000
#2240 enhancement ross ross ckan-sprint-2012-04-02 closed fixed Document CityData ideas

Document the ideas raised on 14/03 for CityData? and distribute to team for discussion

1332169761000000 1332408769000000
#2241 refactor seanh seanh ckan-sprint-2012-04-02 closed fixed Look into making test_activity.py faster

...but only if it can be done reasonably quickly.

1332172354000000 1332426423000000
#2242 enhancement seanh seanh ckan-sprint-2012-04-02 closed fixed Research buildbot possibilities for CKAN 1332172405000000 1333375436000000
#2243 enhancement seanh seanh ckan-v1.9 reopened Fix ckanext-example 1332172710000000 1340635768000000
#2244 enhancement toby toby ckan-future new clean up auth
  • cache admins etc
  • aim to move check_access so independent of logic function
  • general spruce up
1332177771000000 1338205402000000
#2245 enhancement johnglover johnglover ckan-sprint-2012-04-02 closed fixed Update documentation for writing CKAN extensions 1332178546000000 1332331196000000
#2246 enhancement johnglover johnglover ckan-sprint-2012-04-02 closed fixed Change published_by metadata field to reference group instead of a custom extra
  • probably needs a new converter, as needs to be usable via API as 'published_by'.
1332243036000000 1332864871000000
#2247 enhancement zephod ckan-backlog new Resource preview glitch in some browsers

From Ira: Preview for google spreadsheets are not displaying correctly for me in Firefox v.10.0.02, fine in Chrome. http://i.imgur.com/KJaqz.png

1332246614000000 1332246614000000
#2248 enhancement johnglover johnglover ckan-sprint-2012-04-02 closed fixed Update docs for creating custom forms

Should describe implementing IDatasetForm, subclassing controllers is deprecated

1332328630000000 1332331220000000
#2249 enhancement kindly kindly ckan-sprint-2012-04-02 closed fixed Fix google spreadsheet to work with DataStore.

Need google spreadsheet app to work with the new datastore. This should simplify it a lot.

1332330242000000 1338205316000000
#2250 enhancement kindly kindly ckan-sprint-2012-04-02 closed fixed Setup nginx and datastore on ecportal 1332330570000000 1338205361000000
#2251 enhancement toby kindly ckan-sprint-2012-04-16 closed fixed Internal analytics for ckan.

Page views and Resources clicks need to be tracked.

User Stories

US1 As a Site Admin / Visitor (?) I want to see how often a page has been viewed (every page) and how often resources have been downloaded.

US1a Next to a resource or a dataset see how often it has been downloaded / viewed

US1b I want to see datasets or resources ranked by most downloaded or viewed

US1c See a trend graph for a dataset (and resources)

Adminstrative Dashboard (?)

  • I want to see the traffic breakdown by country to my site ...
  • Ditto for browser type, language, etc etc
  • I want to see it graphed over time ...

Implementation Details

  1. How do we store this data in CKAN?
  2. How do we track (and store)?
  3. How do we display
  • Config option ckan.status.enabled = False (by default)

Storing Data

How does ckanext-googleanalytics do this? Current table:

package_id | count_recent | count_total

Move to a new stats_summary table

id | item_id | object_type | stats_type (total, month_yyyy_mm, ...) | value

Do we store this data into the search (solr) so we can search by it?

Displaying Data

  • Helper functions / dictize:
    • Helper function: h.stats_get(object_type, id, stats_type)
      • h.stats_top_ranked(object_type, number) -> returns object_dicts or just labels or ...
    • Change to dictize
  • Location in the default theme (do we show for example in search results too!)
  • Support for ranking by most popular in search?

Tracking Data

  • Our own solution (just write to site_tracking)
  • Google analytics (plus extension for retrieving data) <- would need a refactor
  • Piwiki

Own Solution

(For later: not as part of this ticket probably)

site_tracking table

id | url | timestamp | action (page_view, resource_download) |

  • Make javascript to make request to ckan to store clicks and page views.
  • Add middleware so these requests do not go through pylons and just store data quickly.
1332331029000000 1336046155000000
#2252 defect amercader amercader ckan-sprint-2012-04-16 closed fixed UKLP harvesters returning errors when importing due to schema changes

See trace below.

As in #2231, the schema passed with the context is no longer used on the logic layer (in this case the one used on the DGU form is used). This can be fixed allowing a schema to be passed with the context.

  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 139, in import_stage
    self.import_gemini_object(harvest_object.content)
  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 163, in import_gemini_object
    package = self.write_package_from_gemini_string(unicode_gemini_string)
  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 386, in write_package_from_gemin
    package = self._create_package_from_data(package_dict, package = package)
  File "/var/lib/ckan/std/pyenv/src/ckanext-inspire/ckanext/inspire/harvesters.py", line 486, in _create_package_from_dat
    package_dict = action_function(context, package_dict)
  File "/usr/lib/pymodules/python2.6/ckan/logic/action/update.py", line 169, in package_update
    data, errors = validate(data_dict, schema, context)
  File "/usr/lib/pymodules/python2.6/ckan/lib/navl/dictization_functions.py", line 216, in validate
    converted_data, errors = _validate(flattened, schema, context)
  File "/usr/lib/pymodules/python2.6/ckan/lib/navl/dictization_functions.py", line 289, in _validate
    convert(converter, key, converted_data, errors, context)
  File "/usr/lib/pymodules/python2.6/ckan/lib/navl/dictization_functions.py", line 179, in convert
    converter(key, converted_data, errors, context)
  File "/var/lib/ckan/std/pyenv/src/ckanext-dgu/ckanext/dgu/validators.py", line 81, in validate_license
    if data[('license_id',)]== '__extra__':
KeyError: ('access_constraints',)
2012-03-19 10:47:41,736 ERROR [ckanext.inspire.harvesters] Error importing Gemini document: ('access_constraints',)
1332333907000000 1334567502000000
#2253 enhancement toby toby closed wontfix CMAP [super]

Somewhere for CMAP stuff not in other tickets

need to create some general tickets

  • template changes
  • general demo server setup
1332341133000000 1340038490000000
#2254 enhancement toby toby closed fixed better hooks for templates

TD look at issues and then email ckan-dev

meeting notes from 21-3-2012 general ideas non specific

Problem:

  • We want to remove the Transformer usage due to performance and its close tie to genshi
  • Want a standard way for core code and extensions to put code into the templates where *templates request it* (templates control insertion rather than extensions ...)
  • Stacking versus overwriting on entry points

Options:

  • placeholders or helper methods
  • {{user _info}} {{sub_menu}} is this equivalent to <span class="insert-user-info"></span>
    • [xyz arguments] -> h.load_snippet('xyz', ...)
  • ${h.snipit()}
1332341333000000 1338210316000000
#2255 enhancement ross ross ckan-sprint-2012-04-30 closed fixed Organizations (Privacy + Ownership)

[x] The publisher_form extension and the 'publisher' component of the system will be renamed to organizations to both make its functionality clear, and to differentiate it from the publish auth profile.

Ownership of datasets is important and this ticket will enforce membership for a dataset to a single individual organization. The workflow for adding a dataset to a publisher/organization will stay the same but there will be no mechanism for adding existing datasets to an organization from within the organization forms.

[x] New datasets can be created from the organization form, in which case the organization will be set in the dataset form dropdown (with privacy set to private - see below)

[x] Datasets created from direct access to /dataset/new will have a drop down showing all of the organizations available to the current user.

[x] When editing a dataset the organization can only be changed if the current user is the admin of the current organization, and only then to one of the organizations available to the current user.

Only Organization admin can see the edit Organization form.

[x] Privacy will be enforced by changing the dataset form to have a public/private switch next to the organization drop-down which will be set as the capacity with which the dataset is added to an Organization. Private means visible only within the Organization, and should be the default.

[x] Search will be affected as the public/private status of a dataset will need to be added to the schema

[x] Only public datasets showing in site search, and private showing in organization dataset search (if user is within the organization).

[x] Make sure resources are restricted in visibility if the parent dataset is private

[x] Documentation should be updated to reflect these changes.

[x] Verify, then add, then re-add to documentation.

  • A User joins or creates an Organization
  • If creator of the Organization User becomes administrator of the Organization otherwise they become a Member.
  • Creation of new Organization must be approved by a System Administrator
  • New Members must be approved by the Organization Administrator
1332417218000000 1334663670000000
#2256 enhancement toby toby closed fixed remove TDH from core

make core independent of TDH and make core as minimal app

1332513199000000 1336046098000000
#2257 enhancement toby toby ckan-v1.9 new cleanup template vars

look at reducing what is pulled into templates eg ckan.lib.helpers ensure that these changes don't break existing extensions etc

1332513307000000 1340097071000000
#2258 enhancement rgrp ckan-future new Customizable contributor agreement
  • Customize text at bottom of forms
  • Also need to make clear that this does not apply to the data itself (that is covered by the license you choose on your dataset ...)
1332751549000000 1338204747000000
#2259 enhancement johnglover johnglover ckan-sprint-2012-04-02 closed fixed Update ecportal theme for master changes (bootstrap)

Some recent changes caused things to break in ckanext-ecportal:

  • change from blueprint to bootstrap
  • other style / css changes (eg: tables, resource view notes field ID, etc. Forms are also broken, but that will be addressed in #2262)
  • change to going through db_to_form_schema on package_show
1332754180000000 1332769973000000
#2260 enhancement johnglover johnglover ckan-sprint-2012-04-16 closed fixed Update ecportal theme to match latest changes to tenforce theme 1332754319000000 1333554069000000
#2261 enhancement toby kindly ckan-sprint-2012-04-02 closed fixed All links need to option to change where language is placed in url.

For the ecportal case we need links to be like the following

(%host)/open-data/en/data/(.*)
(%host)/open-data/fr/data/(.*)

1332762702000000 1332840112000000
#2262 enhancement johnglover johnglover ckan-sprint-2012-04-02 closed fixed Update ecportal form and dataset style/ux to match CKAN master 1332764002000000 1332855605000000
#2263 enhancement amercader amercader closed fixed IATI: Get "Record updated" field from the registry

Don't provide a field in the form or spreadsheet import for this, just use dataset.metadata_modified.

1332766145000000 1332846807000000
#2264 enhancement seanh seanh ckan-sprint-2012-04-16 closed fixed Do an audit of string translations in CKAN core and ckanext-ecportal

Check to see that all the strings are getting translated, i.e. that all user-visible strings are being passed to gettext. Fix any strings that are not getting translated.

1332864512000000 1339411578000000
#2265 enhancement dread ckan-future new 'More Like This' for a dataset

When viewing a dataset, it would be nice to show a couple of 'Related Datasets'. i.e. ones that are similar.

SOLR has a feature for finding documents similar to a particular document, called 'More Like This'.

We would like this for DGU.

1332865220000000 1339771350000000
#2266 defect dread dread ckan-sprint-2012-04-02 closed fixed Can't delete all of a package's resources over REST API

Nothing happens if you set resources=[] or resources=null.

1332932504000000 1332932634000000
#2267 enhancement aron.carroll rgrp demo phase 5 closed fixed Add generator=ckan tag in head

Add generator=ckan tag in head of templates to identify site as generated by CKAN

1333040018000000 1343229146000000
#2268 enhancement thejimmyg amercader ckan-backlog new UKLP Use same schema as form for harvested records

Datasets created via the harvesting (UKLP ones) and the ones created via the form use a different schema. That could lead to problems. We need to discuss the best approach.

1333108606000000 1333108606000000
#2269 enhancement johnglover johnglover ckan-sprint-2012-04-30 closed fixed EC ODP review and server setup
  • code review with D.R. for ckanext-ecportal
  • merge and test all multilingual code
  • fix server issues: routing, Git access to CKAN repository

Bugs to fix:

  • rename groups to publishers
  • update home page layout (recent updates along side publishers)
  • don't add 'en' to main menu links if english
  • resource edit page, fix missing images
  • change dataset 'settings' to 'edit'
  • translate 'published by' drop-down list
  • fix JS on 'add a dataset page'
  • prepopulate 'published_by' when clicking 'add a dataset' from group
  • chosen broken (dataset edit) if in different language
1333374177000000 1335779180000000
#2270 enhancement johnglover johnglover ckan-sprint-2012-04-16 closed fixed DataExplorer overview
  • get up to speed with DataExplorer? code (and dependencies like Backbone)
1333374339000000 1334073875000000
#2271 defect amercader amercader ckan-sprint-2012-04-16 closed fixed IATI: Some datasets not being indexed

When importing a CSV file for a certain publisher, records don't come up on the search index. After some investigation this has tunrn out to be caused by the records being deleted. We need to investigate if this was caused by the import or just made by the users.

1333375469000000 1334567582000000
#2272 enhancement seanh seanh ckan-sprint-2012-04-16 closed fixed Setup jenkins on the new dev server 1333375480000000 1334075515000000
#2273 enhancement seanh seanh ckan-v1.8 closed duplicate Review publisher organisations code with Ross 1333375723000000 1340635788000000
#2274 enhancement seanh seanh ckan-sprint-2012-04-30 closed fixed Review multilingual branch with kindly, merge into master 1333375804000000 1335644408000000
#2275 enhancement johnglover seanh ckan-sprint-2012-04-16 closed fixed Implement group logos

Groups should be able to specify a URL to an image file for a group logo, the image will be shown on the group page.

Actually uploading an image file (rather than giving a URL) and cropping and resizing the file will be done later, for now just give a URL. (later ticket)

Further customisation of group pages (CSS, templates) also for later. (not sure if we really want this, but in a later ticket if so)

1333376076000000 1334594328000000
#2276 enhancement seanh seanh ckan-sprint-2012-04-16 closed fixed Upgrade Slovakia CKAN instance 1333376435000000 1334579078000000
#2277 refactor icmurray icmurray ckan-backlog new Use the new atom feeds in IATI

Atom feeds have been implemented in core in #1593. The IATI-specific implementation can now be removed when IATI move to CKAN >= 1.6.1

1333386628000000 1333386628000000
#2278 defect rgrp rgrp ckan-sprint-2012-04-16 closed fixed DataStore enabled is disabled by default but is needed for DataStorer to run

Major UX gotcha reported by several people is that DataStore? is disabled by default and must be enabled for DataStorer? to run and import data.

Suggest switch to enable datastore by default for file and file.upload types (disable for API). People can always disable if they want.

1333441048000000 1333441177000000
#2279 enhancement rgrp rgrp ckan-v1.8 new Write tutorial for our DataStore API / Data Explorer

Unfortunately ES is a little bit complex / opaque to newcomers. Need to write a simple tutorial walking through this.

This is documentation / example only (no code!)

1333482263000000 1338205426000000
#2280 enhancement ross johnglover ckan-sprint-2012-04-30 closed fixed Move ecportal to organizations, add history template 1333552029000000 1334834302000000
#2281 enhancement dread dread ckan-sprint-2012-04-16 closed fixed Pagination styling converted to Bootstrap

Make the search results pagination styled by Bootstrap.

Needs putting into a <ul> structure. Current page and '..' need become links that are styled disabled.

Wanted by DGU.

1333967506000000 1333967862000000
#2282 enhancement toby toby ckan-future new log api calls for users

This was part of #2251 but is now it's own ticket.

we need clarity on this. The use cases so far have been possible limiting api calls for users.

so we want to monitor the user or ip address

what about the api calls?

do we want to record all calls or just some? do we want to record the type of call and to what level of detail?

recording the url just seems to specific to me.

1334074725000000 1338210829000000
#2283 enhancement rgrp ckan-v1.8 closed wontfix [super] Data Previewer / Viewer v3
  • Basic geo map
  • Facets
  • Filtering
  • Pivot table support (?)
  • Export data (?)
  • Save data query and share
  • More and better graphing
  • Embedding! - /resource/../embed

User stories and analysis: http://ckan.okfnpad.org/feature-2283-data-viewer-v3

1334138341000000 1340623843000000
Note: See TracReports for help on using and creating reports.