{22} Trac tickets (2647 matches)

Write CKAN FAQ (Basis can be: ​http://wiki.ckan.net/FAQ)

Post preliminary questions on:

​http://notebook.okfn.org/

Write a worker that scans all packages in a ckan instance and uploads the data to storage.ckan.net or another suitable storage system.

• Naming scheme?
  • Bucket: {ckan-instance-id}-{package-name}? {ckan-instance-id}-{package-id}?
    • What happens if names change
  • File: filename? hash?
• Store hash back on ckan instance?

The caching worker will consumer update notifications and fetch packages.

Extra points for:

• Properly checking for source file modification (Last-modified, Etag)
• Using PIP VCS Backends for retrieval
• OFS/S3 Storage

Write an importer that supports most well known variants of DCat in importing the data as CKAN packages. In particular, the following sources should be supported:

• CKANrdf generated exports
• opengov.se RDF (not really DCat)
• Sunlight Nationdatacatalog as harvested by dcat-tools.

You can register a user with password and (optional) email address. But if you forget the password you can't then log in again. We need a password reset facility that sends and email with a new password.

You can't delete things from the SOLR search index if the ckan.site_id and ckan.site_url are blank.

Should assert that one of these are set up.

You configure the extension with some RSS and/or Atom feeds, it automatically reads items from these feeds and makes them available in the template context, you write a custom template to e.g. display 'news' items from a Wordpress blog on your front page.

This extension might be simpler and less fragile than ckanext-wordpresser, and also more generally useful.

Details

• Mark Pilgrim's Universal Feed Parser might be useful for reading the feeds

• Feed items should probably be cached somewhere

Suggested

• The news item 'widget' should be wrapped in a known class so that it can be styled easily regardless of the format of any HTML entry.

• For non-HTML formatted items (Atom should tell you the content type of the entry) maybe we should have a template for rendering each item along with any enclosures that it might reference

• Caching is pretty crucial and should probably obey the ttl of the feed.

You get an exception if you use the API to search packages and specify a non-ascii character in a field other than q.

For example:

http://catalogue.data.gov.uk/api/search/package?title=%E2%80%93

This "N-dash" (Unicode character 2013) causes this exception:

Module ckan.controllers.api:460 in search
<<                      if ver in u'12':
                               # Otherwise, put all unrecognised ones into the q parameter
                               params = convert_legacy_parameters_to_solr(params)
                           query = query_for(model.Package)
                           results = query.run(params)
>>  params = convert_legacy_parameters_to_solr(params)
Module ckan.lib.search.query:38 in convert_legacy_parameters_to_solr
<<      for search_key in non_solr_params:
               value_obj = legacy_params[search_key]
               value = str(value_obj).replace('+', ' ')
               if search_key == 'all_fields':
                   if value:
>>  value = str(value_obj).replace('+', ' ')
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 30: ordinal not in range(128)

This problem affects CKAN 1.5 and 1.5.1 only.

[Child of #1526]

Can't change a dataset from containing time-series resources to one containing individual resources or visa-versa.

[Child of #1526]

Create the interface used to retrieve publisher and authz information from the publisher hierarchy.

[Refactored] :: Follows on from #1506 UX changes.

• Declutter the sidebar. No yellow box.
• Facets to go on the left, rather than the right. More logical flow.
• Did you know you can search by author? Probably not. Find a nice way of presenting extended search options.
• Make Datasets in the search page look more like Datasets on the groups pages (ie. like awesome sexy search results).
  • Update that look-and-feel to include the new resource icons created in #1506

[x] The publisher_form extension and the 'publisher' component of the system will be renamed to organizations to both make its functionality clear, and to differentiate it from the publish auth profile.

Ownership of datasets is important and this ticket will enforce membership for a dataset to a single individual organization. The workflow for adding a dataset to a publisher/organization will stay the same but there will be no mechanism for adding existing datasets to an organization from within the organization forms.

[x] New datasets can be created from the organization form, in which case the organization will be set in the dataset form dropdown (with privacy set to private - see below)

[x] Datasets created from direct access to /dataset/new will have a drop down showing all of the organizations available to the current user.

[x] When editing a dataset the organization can only be changed if the current user is the admin of the current organization, and only then to one of the organizations available to the current user.

Only Organization admin can see the edit Organization form.

[x] Privacy will be enforced by changing the dataset form to have a public/private switch next to the organization drop-down which will be set as the capacity with which the dataset is added to an Organization. Private means visible only within the Organization, and should be the default.

[x] Search will be affected as the public/private status of a dataset will need to be added to the schema

[x] Only public datasets showing in site search, and private showing in organization dataset search (if user is within the organization).

[x] Make sure resources are restricted in visibility if the parent dataset is private

[x] Documentation should be updated to reflect these changes.

[x] Verify, then add, then re-add to documentation.

• A User joins or creates an Organization

• If creator of the Organization User becomes administrator of the Organization otherwise they become a Member.

• Creation of new Organization must be approved by a System Administrator

• New Members must be approved by the Organization Administrator

a simple api call that returns data like this:

{ "version": ckan_software_version,
  "contact": { "name": "Some Admin", "mbox": "[email protected]" },
  "description": "Site Description",
  "url": "http://canonical.name.ckan.net/"

add resource_status_show action function to see the state of celery tasks.

Need to get information from both out task status table and the celery_taskmeta table.

add to groups is a drop down menu where you can only choose one needs a new UI & logic allowing user to add new groups & potentially remove from other lists

allow group-level authorization on all system entities

apps dashboard available at ​http://localhost:5000/apps in main dev branch

I have got this to work in the demo however

a) it is a bit ugly

b) the template could do with some sorting eg the dropdowns should be being created in a nicer way

as per http://ckan.org/wiki/UIRedesignPackage

as title.

attempt to disengage the stats plugin from core as much as possible

be good to do this in phase 3

can we get the link going to an explanation of the 5 star system on page ​http://s031.okserver.org:2375/qa/dataset/five_stars to open in new tab? it's leaving demo all together so better to open in new tab than navigate away

cf. Tariqs mail

Updated version: ​https://spreadsheets5.google.com/ccc?key=tuOtQjD0Psoqr1pWTS8EXZQ&hl=en#gid=0

Original Version

Publishing Entity:
Publishing Entity Type: (Donor, Recipient, Community Data..)
Donor
Country
Activity period:
Verification status: enumeration of statuses (checked, not checked etc)
Resource links: to the actual IATI record
Number of activities: ...
Date record updated:
Date data updated:
License: Need this field even if it may be a standard license

So naively mashing these together, we get something like:

Identifier
Title
Abstract
Donor
Country
Publisher
Publisher Type
Verification Status
Department
Contact
Contact e-mail
Licence
Resource format
Resource URL
Resource ID
Activity period
Number of activities
Date record updated
Date data updated

changing a tag from "country-uk" to "country:uk" and updating gives server error

otherwise updating metadata / adding tags seem fine

either we should allow colons or form should return a message indicating what characters are allowed

ckan.js can be rearchitected to avoid instantiating showdown.js in the root namespace. That way, layout_base.html doesn't need to import the script (because we don't use the showdown-dependant parts of ckan.js).

ckan.net appears to have either gone down or be running ultra slowly.

this means that ckan copies running locally on my machine run very slowly indeed.

is this behaviour desirable?

This command finds lots of ​http://~~~ckan.net references in python, html and javascript files:

find ~/pyenv/src \( -name "*.py" -or -name "*.html" -or -name "*.js" \) -print0 | xargs -0 -e grep --color -nH -e "​http://.*ckan.net"

output for reference:

/home/okfn/pyenv/src/ckan/ckan/init__.py:5:Network (CKAN) site: ​http://www.ckan.net. /home/okfn/pyenv/src/ckan/ckan/lib/create_test_data.py:346:<​http://ckan.net/> /home/okfn/pyenv/src/ckan/ckan/lib/rdf.py:3:DOMAIN = '​http://ckan.net' /home/okfn/pyenv/src/ckan/ckan/lib/rdf.py:4:CKAN_NAMESPACE = '​http://ckan.net/#' /home/okfn/pyenv/src/ckan/ckan/lib/talis.py:60: 'ckan':'​http://ckan.net/ns#', /home/okfn/pyenv/src/ckan/ckan/public/scripts/bookmarklet.js:2: f='​http://ckan.net/package/new?url='+encodeURIComponent(window.location.href)+'&title='+encodeURIComponent(document.title); /home/okfn/pyenv/src/ckan/ckan/public/scripts/test_bookmarklet.html:16: addtockan.src='​http://ckan.net/scripts/bookmarklet.js'; /home/okfn/pyenv/src/ckan/ckan/public/scripts/test_bookmarklet.html:27: <p><strong>Proper bookmarklet (compressed -- need to escape &amp;):</strong> <a href="javascript:(function(){f='​http://ckan.net/package/new?url='+encodeURIComponent(window.location.href)+'&amp;title='+encodeURIComponent(document.title);if((n=document.getElementsByName('description')[0])&amp;&amp;(d=n.content)){f+='&amp;notes='+encodeURIComponent(d);}a=function(){if(!window.open(f)){location.href=f;}};if(/Firefox/.test(navigator.userAgent)){setTimeout(a,0)}else{a()}})()">Add to CKAN</a> /home/okfn/pyenv/src/ckan/ckan/templates/home/license.html:31: For convenience, all material - including all package, tag and revision information - is available in bulk, in the form of a full dump of the CKAN database. This (gzipped) dump file is updated daily and can be downloaded from <a href="​http://www.ckan.net/dump/">​http://www.ckan.net/dump/</a>. /home/okfn/pyenv/src/ckan/ckan/tests/dictization.py:71: 'notes': u'Some test notes\n\n### A 3rd level heading\n\nSome bolded text.\n\n*Some italicized text.*\n\nForeign characters:\nu with umlaut \xfc\n66-style quote \u201c\nforeign word: th\xfcmb\n \nNeeds escaping:\nleft arrow <\n\n<​http://ckan.net/>\n\n', /home/okfn/pyenv/src/ckan/ckan/tests/dictization.py:137: 'notes': u'Some test notes\n\n### A 3rd level heading\n\nSome bolded text.\n\n*Some italicized text.*\n\nForeign characters:\nu with umlaut \xfc\n66-style quote \u201c\nforeign word: th\xfcmb\n \nNeeds escaping:\nleft arrow <\n\n<​http://ckan.net/>\n\n', /home/okfn/pyenv/src/ckan/ckan/tests/dictization.py:447: 'notes': u'Some test notes\n\n### A 3rd level heading\n\nSome bolded text.\n\n*Some italicized text.*\n\nForeign characters:\nu with umlaut \xfc\n66-style quote \u201c\nforeign word: th\xfcmb\n \nNeeds escaping:\nleft arrow <\n\n<​http://ckan.net/>\n\n', /home/okfn/pyenv/src/ckan/ckan/tests/dictization.py:458: 'notes': u'Some test notes\n\n### A 3rd level heading\n\nSome bolded text.\n\n*Some italicized text.*\n\nForeign characters:\nu with umlaut \xfc\n66-style quote \u201c\nforeign word: th\xfcmb\n \nNeeds escaping:\nleft arrow <\n\n<​http://ckan.net/>\n\n', /home/okfn/pyenv/src/ckan/ckan/tests/functional/api/base.py:178: assert '"ckan_url": "​http://test.ckan.net/package/annakarenina"' in msg, msg /home/okfn/pyenv/src/ckanclient/ckanclient/init__.py:116: api e.g. ​http://ckan.net/api rather than ​http://ckan.net/api/rest) /home/okfn/pyenv/src/ckanclient/ckanclient/init__.py:261: :param base_location: default *​http://www.ckan.net/api* /home/okfn/pyenv/src/ckanclient/ckanclient/init__.py:267: base_location = '​http://www.ckan.net/api'

ckan/logic/action/create.py:resource_create():

try:

pkg_dict = _get_action('package_update')(context, pkg_dict)

except ValidationError?, e:

errors = e.error_dictresources?[-1] raise ValidationError?(errors)

This assumes that package_update always returns an error dict with 'resources' in it. This assumption doesn't hold.

ckan/templates/home/index.html contains:

<h1 class="page_heading">Welcome to ${g.site_title}!</h1>

which means that just "Welcome to" ends up in the ckan.po files for translation. This makes the phrase untranslatable in languages that have a different word ordering than English where the site title needs to be somewhere other than at the end of the sentence.

ckanext-archiver currently has a settings file (and a default) and it should be passed in all relevant information from the context.

Remove all settings (ARCHIVE_DIR and MAX_CONTENT_LENGTH and others) and pass them in from CKAN.

ckanext-moderatededits requires an old and possibly development version of CKAN. It would be good to update it for later CKAN versions.

According to the README, you need CKAN from branch feature-1141-moderated-edits-ajax but the changelog suggests this branch went into version 1.4.2. So it possibly works with 1.4.2 and 1.4.3(.1). But CKAN 1.5 has revamped templates, so the genshi stream filters definitely don't work.

BTW history_ajax/read_ajax calls have been deprecated in CKAN since 1.5.2a and will need fixing up to use the Action API too as part of this.

create a new test file ckan/model/test_user.py

add tests for the following three functions in ckan/model/user.py

number_of_edits, number_administered_packages, search

merged in in changeset 0046f83aedcf

cset:4b59ab34137d ckan/logic/action/get.py:

-            isopen = model.Package.get_license_register()[license_id].isopen()
-            result_dict['isopen'] = isopen
+            try:
+                isopen = model.Package.get_license_register()[license_id].isopen()
+                result_dict['isopen'] = isopen
+            except KeyError:
+                # TODO: create a log message this error?
+                result_dict['isopen'] = False 

This change hides problems with the license server and returns potentially incorrect values for openness.

This has been noted as 'temporary fix' but seems to be forgotten about, since it has been merged to default and gone into release 1.4.3.

I suggest the licenses are cached (I thought this was already the case when CKAN first requests them after start-up?). I suggest failure would return 503.

demo.ckan.org has a small field for adding text on homepage current text: "Welcome to the CKAN demo Try out standard CKAN functionality in a sandbox environment. Search for datasets directly from the homepage or by navigating to the Datasets search page where you can facet by tags, groups and format."

Want to make this editable for demos

ditto

doesn't belong in ckan core

e.g.

curl http://localhost:5000/api/action/package_create -d '{"name":"test5", "tags":[{"name": "tag6"}, {"name": "tag6"}]}' -H 'Authorization:tester'

Also broken in Web UI (adding a package).

Affects API and Web UI in 1.5 and I think all versions from 1.4.1

e.g. 2012_08(주요국가).xls

I tested in CKAN 2.0 and it seemed to work fine there.

e.g. ONS importer

e.g. currently no way to read full description for the 'data explorer examples' group here:http://s031.okserver.org:2375/group/data-explorer

Can we add a 'read more' link that expands and contracts the sidebar box or similar?

e.g. cz.ckan.net has lang=cz but when you click on Home page link "English" it flashes message "Language has been set to: English" but the rest of the page is still Czech.

e.g. ​http://189.9.137.65/dados/ clicking on Deutsch link is http://189.9.137.65/dados/locale?locale=de&return_to=%2Fdados%2F&hash=1dc17c315c419df850da0dd3599eefa9da76fbeb and redirect goes to http://189.9.137.65/dados/dados/?__cache=97995106 so /dados/dados/ when it should be /dados.

Affects CKAN 1.6b only (not yet released).

e.g. ​http://datahub.io/dataset/ccc-gistemp/resource/80ebdbd6-d91b-4fef-9db4-d3dfbd7e868e

CKAN attempts to render the PDF as a datastore tabular date even though the mimetype of the PDF is correctly specified.

e.g. ​http://thedatahub.org/tag/lod

Tags with less datasets work fine (e.g. ​http://thedatahub.org/tag/railways)

e.g. if I just changed a member's capacity I am not adding anything 'save' is better maybe

e.g. license, owner etc

e.g. on ​http://s031.okserver.org:2375/en/dataset/afghanistan-election-data/resource_edit/f6331f99-51f6-44d9-95b9-b20f3b74f360

& ensure consistant with sidebar of edit dataset page

e.g. searching for 'buddhist' or 'sanskrit', you don't get this newly created package: ​http://ckan.net/package/digitalsanskritbuddhistcanon

e.g. see here: ​http://s031.okserver.org:2375/apps same with related page on a dataset

e.g. tried changing the lang here: ​http://s031.okserver.org:2375/ to no effect. Doesn't work on any other pages either. Flags have also dissappeared

edit button shows up for daniel lewis for ​http://s031.okserver.org:2375/dataset/ff

eg on confirm delete

​http://localhost:5000/zh_TW/dataset/edit/1-kmwaterfractionfromnationaltopographicdatabasemapscanada

_(....) gives [object Object] not translation