{22} Trac tickets (2647 matches)

Add ckan_url attribute to REST JSON representation of a Package pointing to the (read) url of package on CKAN.

Cost: 30m

Seems some package have ended up with same tag twice: ​http://www.ckan.net/package/fred2dot0

Check whether this is still possible and if it is fix it ...

Cost: 2h

Write up search api documentation and put it in a template that shows up at api/search/.

Cost: 2h

User object should have a "created" attribute initialized to current datetime.

Require a db migration but o/w very simple.

Cost: 1.5h

Search of form: tags: ... behaves differently depending on whether there is a leading space:

• tags: postcode - tags found correctly but no packages found
• tags:postcode - tags incorrectly found but correct packages found

Let's fix this.

Cost: 0.5h

E.g. ​http://ckan.net/tag/read/postcodes this has no-postcode listed but when you look at that package 'postcodes' isn't a tag ...

I suspect this may be to do with search using deleted tags as I think no-postcode had postcodes as a tag at some point earlier ...

Create consolidated set of config by merging config from .datapkgrc and command line options and use that instead of mixture of command line options and other stuff ... (likely to be less buggy!)

Cost: 0.5h (?)

Conjecture this went missing in cset:a35db862a841

At moment order packages by title.

(Follows on from ticket:107). We want to provide facility for users to automatically upload material.

• Create an upload page
• Store the material on archive.org using s3 - ​http://www.archive.org/help/abouts3.txt
• Main issue is how we cache large datasets ...

Propose change to tabular-like format showing these attributes (perhaps should be configurable?)

• Openness status
• Title (not sure name is needed)
• Tags

Cost: 4h

Current write method just creates a bare bones package and does not write any of the available metadata other than name.

p:wui 2009-09-09 validate email address when creating an enquiry

This needs to be thought out ...

Strapline (?): Sharing, discovery and reuse of data and content

Need to consult on this.

Cost: 1h

Broken out from ticket:216

Dumping ground for various minor and miscellaneous items (mainly refactorings):rgrp

• controllers/tag.py: why does this not use lib/search.py (but controllers/packages.py search method does)?

done:

• lib/search.py: inheritance would be nicer than switching on entity in search (i.e. have SearchPackage?, SearchTag? etc)

At the moment setting of continuity_id depends on base table pkcol being id. Should not be hard to change this -- and may get for free as part of ticket:245 (composite primary keys)

Need to do i18n of forms (not in templates but in code). Should consult with tonfa about this.

Presence of license_id field in JSON input show below breaks REST API (with weird error about int not being subscriptable as if license_id field is being subscripted).

Using normal license field does not cause breakage but has no effect.

(Also think we should remove license_id from the API since there is no guarantee this cross system compatible since locally generated on install).

    ourjson = '''{
   "maintainer" : "Applications Division",
   "version" : "1999-01-01",
   "license_id" : 9,
   "name" : "geogratiswaterfraction1km_v100",
   "author_email" : "...",
   "author" : "Dissemination Branch",
   "tags" : [
      "GeoGratis",
      "Canada",
      "GCMD",
      "Hydrosphere",
      "Ground Water",
      "Surface Water",
      "Hydropattern",
      "Earth Science",
      "Land Surface",
      "Land Cover",
      "Landscape"
   ],
   "extras" : {
      "License" : "http://geogratis.ca/geogratis/en/licence.jsp",
      "Ministry" : "Natural Resources Canada",
      "Level of Government" : "Federal"
   },
   "maintainer_email" : "[email protected]",
   "notes" : "This product is a raster coverage",
   "url" : "http://geogratis.cgdi.gc.ca/geogratis/en/collection/metadata.do?id=67",
   "title" : "Geogratis - 1-km Water Fraction From National Topographic Data Base Maps, Canada"
}
'''
    def test_15_random(self):
        offset = '/api/rest/package'
        import urllib
        tmpjson = simplejson.dumps(simplejson.loads(self.ourjson))
        postparams = urllib.urlencode({tmpjson: 1})
        res = self.app.post(offset, params=postparams, status=[200],
                extra_environ=self.extra_environ)
        model.Session.remove()

At the moment the user has to figure out to login with providers -- this is not always obvious (e.g. google).

We could improve this with a small bit of javascript. As an example see: ​http://standalone.demo.civicrm.org/standalone/

Not sure exactly what library that is using (or if bespoke) so alternatives include (NB: we already use jquery):

• ​http://code.google.com/p/openid-realselector/
• ​http://code.google.com/p/openid-selector/

In addition to implementing this we also need to test logging in with main providers: google, wordpress -- as we have had reports of it not working with e.g. wordpress (not sure if this testing can be automated really -- best hope would be selenium I think -- so OK if done by hand).

Cost: 3h (2h UI), (1h testing)

Currently this raises an exception because there is no package/package_revision in existence at from_revision.

Work already here: ​http://knowledgeforge.net/ckan/trac/wiki/SearchEngine

Add knowledge there or to:

​http://wiki.okfn.org/SoftwareTools/Search

This will solve e.g. issues in formalchemy admin interface (which is due to lack of renderer for Json type)

Cost: 0.5h

from ckan.model.types import JsonType

forms.FieldSet.default_renderers[JsonType] = formalchemy.fields.TextFieldRenderer

Also put in redirect from old locations (see examples with plurals in routing.py)

Estimated cost: 20m

Actual cost: 40m

A package discussion page is like a wikipedia discussion page: an editable free text page for people to have discussion/post comments about a given package.

It provides a way for people to make suggestions about a package without needing access to main package.

Support uploading a distribution somewhere.

• New cli command: upload
• Pluggable uploader framework. Suggest initial support

Depends on ticket:306

Suggest move this below the label and make full width of screen and only 3/4 rows high (more like a wiki site).

• Also change label to: Edit summary (Briefly describe the changes you have made)
• Remove: you can markdown formatting here.
• Move author: if you have not signed in smaller and closer (like markdown instructions are nwo).
• Change commit -> save
• Remove "please save" just have the bullet points

1. Issues with quote in fields: ​http://lists.okfn.org/pipermail/ckan-discuss/2010-May/000240.html

2. Issues with package resource serialization into csv table.

<quote> In the latest dump there were 116(!) sets of the three columns (“resource-[n]-url”, “resource-[n]-format”, and “resource-[n]-description”). However, these are an extract of the packed “resource” column and I’m not sure whether they’re needed. Also, they irritatingly don’t appear in order in the CSV serialisation. If the resource columns could be ordered in the file that would be great; if a second version without the unpacked resource data would be excellent. </quote>

As a sysadmin I want to configure basic site title information for use in the site templates.

Implementation:

• ckan.site_title config variable
• set this on g in app_globals.py e.g.
  • from pylons import config; g.site_title = config.get('ckan.site_title, 'CKAN - Comprehensive Knowledge Archive Network')
• use in head title and in main site title/logo section (use it as alt on logo image)
• Also all other pages (e.g. index, about) which talk about CKAN
  • Is this needed? Would it not be better for people who want to customize the site to simply overwrite those templates?

Questions:

• Do we want a site_logo variable whic his use for site title/logo section instead of site_title if site_logo defined?
• Probably yes, but not part of this ticket.

Revisions are timestamped using the server's clock, which may not relate to the expected timezone for the site. e.g. the Norway site has a server on GMT. No timezone info is displayed either.

Would like to set timezone for a CKAN instance to use in rendering revision timestamps. For example, use CET or EST timezone.

target = _blank

Cost: 30m

A small but useful ui improvement would be to have a search box at top right on every page.

As an example see the one here on trac or on github.com or bitbucket.org.

• It would be particularly good to include a small advanced search link that took you to the full search page. Need to keep it small because screen real-estate here is limited (see how github.com does this for inspiration).

In 0.5 and 0.6 have been more oriented to getting file indexes working to support the convenience of using packages on disk directly. However we now need to go back to having a better central/default index and the best model for this is a simple db index using sqlite (which is now supported in the standard library).

NB: this is a retrospective ticket (been working on this for a couple of months)

Download Command (was install command) should support the following modes:

• Download only the first listed resource (current behaviour, slightly arbitrary)
• Download resources by interactive selection
• Download resources by MIME type (cf #235)
• Download all resources.

• (Load cli commands from setuptools entry point name datapkg.cli)
• Document how other can write new commands
• Refactor base Command class to support this pluggability (if necessary)

1. Find some relationships.
2. Add them via API
3. Notify DGU of completion and point to exemplars

The simplest way to improve performance on specific pages is to implement full page caching.

This, however, has challenges on pages with user-specific content. For example, on CKAN when a user is logged at the top of the page we have "You are logged in as: {username}". Similarly we change the visibility of some material depending on the status of users:

• Visibility of Edit or Authz link on packages and groups

Edge-side includes or partial page caching are a standard way to deal with caching of pages in which some (usually small) part of the content cannot be cached or should be cached in a different manner (e.g. much more briefly) than the rest of the page.

Edge-side includes have the advantage that they integrate with general 3rd-party caching systems such as varnish.

Introducing either partial page or ESI will require some overhaul work to change the page render processing somewhat.

Write a worker that scans all packages in a ckan instance and uploads the data to storage.ckan.net.

• Naming scheme?
  • Bucket: {ckan-instance-id}-{package-name}? {ckan-instance-id}-{package-id}?
    • What happens if names change
  • File: filename? hash?
• Store hash back on ckan instance?

WP twentyten has an excellent core html structure. Furthermore, using that structure makes us compatible with all the WP twentyten compatible themes.

• Convert to wp twentyten html
• Switch css to be based off twentyten css (and perhaps rework somewhat)

We get a lot of robot traffic. Notice that robots are visiting package edit pages. Suggest:

• Add rel=nofollow to various links
• Throttle some robots (not all seem to respect robots.txt)

Currently cache usage in home controller is inconsistent with elsewhere and, in particular, does not respect cache_disabled and is not configurable using path notation (e.g. ckan.controllers.home. ...).

This is especially frustrating in a development environment where is currently running into fact front page is being cached.

Fix this.

Move ckan trac to new ckan.org location. Will also need to re-enable user accounts etc.

Create an admin dashboard as /ckan-admin/ allowing for admin operations and overview.

Possible features:

• Purge revisions (or sets of revisions) and purge objects #1076
• Set roles for users #1075
• Put system into particular modes e.g. wiki mode (anyone can add, edit packages by default), data portal (only sysadmins or members of a special Editor group can create and edit packages)
  • WONTFIX
• Overview of activity
  • WONTFIX - already have revision log

Currently have an admin section using the formalchemy admin controller to provide basic editing of model objects. This can still be used but located at /admin/model/.

​https://bitbucket.org/okfn/ckanext-admin

Tickets

• ticket:1031 - user autocomplete api in ckan

Notes

Here's putting into restricted mode (plus creating a dedicated authz group so that others can admin sysadmin simply through that group):

# first remove permissions from roles
# this is hacky but have to do it because we hardcode assignment of 
# role permissions on package on package create (see model/authz.py)
paster roles deny editor edit
paster roles deny editor create-authorization-group
paster roles deny editor create-group
paster roles deny editor create-package
paster roles deny reader create-package
# make superuser group
# create authz group administrators / Administrators (if not exists)
paster rights make agroup:administrators admin system

Thread: ​http://lists.okfn.org/pipermail/ckan-discuss/2010-November/000718.html

Scenario 1

1. User installs Refine and CKAN extension for refine
2. On booting refine and asked to load data they can choose from any data package on CKAN.net (or any other CKAN instance)
3. They edit the dataset on Refine
4. On save (or perhaps as a separate option) they are prompted as to whether they wish to sync the dataset back to CKAN (either as a new package or as a new resource on the existing package)

NB: for the dataset sync back some form of "CKAN" storage would be required (we already have storage.ckan.net running but a closer integration would be required)

Scenario 2

1. User visits a package on CKAN.net (or another CKAN instance)
2. There is a button on the page "View and edit this dataset in Google Refine"
3. Click button -- ask them if they have Google refine installed
   • Yes: instructions for loading dataset into refine
   • No: load dataset in hosted version of google refine (we could run this)
4. User edits dataset and hits save. As in previous scenario they are prompted to sync the dataset.