{22} Trac tickets (2647 matches)

Results (801 - 900 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#2794 enhancement aron.carroll toby demo phase 3 closed fixed top buttons missing on edit resource

http://s031.okserver.org:2375/dataset/resource-markdown/resource_edit/ba1b094d-697c-47c4-a72a-6c2557da5409

should have back to resource as a minimum

1343897926000000 1343903064000000
#2797 enhancement aron.carroll toby demo phase 3 closed fixed plain_textarea styling

http://localhost:5000/ckan-admin/config uses a plain (non markup) textarea for custom css

I've added form macro plain_textarea to provide one, however it renders without a bottom border due to markup textarea styling can you fix this - thanks

1344000422000000 1344250973000000
#2800 enhancement aron.carroll shevski demo phase 5 closed worksforme tooltip on popular datasets with number of views

be good to do this in phase 3

1344243925000000 1344544461000000
#2801 defect aron.carroll shevski demo phase 3 closed fixed no way to see full description of a related item

e.g. see here: http://s031.okserver.org:2375/apps same with related page on a dataset

1344245809000000 1344531016000000
#2802 enhancement aron.carroll aron.carroll demo phase 3 closed fixed Need to tweak the JSON formatting of the localisations

This is what we need the JSON to look like. Taken from the Jed documentation (http://slexaxton.github.com/Jed/).

{

      // The empty string key is used as the configuration
      // block for each domain
      "" : {
        // Domain name
        "domain" : "the_domain",

        // Language code
        "lang" : "en",

        // Plural form function for language
        "plural_forms" : "nplurals=2; plural=(n != 1);"
      },

      // Other keys in a domain contain arrays as values
      // that map to the translations for that key.
      // Gettext suggests that you use english as your keys
      // in case the key isn't found, and it can just pass
      // the value directly through.
      // Note: by convention, the 0-index location of the translations
      // is never accessed. It's just a thing, I guess.
      "a key" : [ null, "the translation", "the plural translations", ... ],

      // The plural form string is converted into a function
      // and the value that's passed into the gettext call
      // is passed into the plural form function. It's result
      // (plus 1) is the index that the translation comes from.

      // We're using sprintf interpolation on our keys so we can
      // then sub in the _actual_ values into the result.
      "%d key" : [ null, "%d key", "%d keys" ],

      // Contexts are keys that are just prefixed with a context string
      // with a unicode \u0004 as the delimiter.
      // You can use it for anything. Usually it's just for being content aware
      // in some way (e.g. male vs. female, product vs. category)
      "context\u0004%d key": [ null, "context %d key", "context %d keys" ]
    }

This looks like an appropriate python library to do the formatting. http://pypi.python.org/pypi/pojson/ but I'll leave to your discretion.

Finally an API endpoint that returns the appropriate request would save us making requests to the i18n directory.

/api/i18n/{locale}

For example if the current url is http://localhost:5000/pl/dataset then the client would request:

http://localhost:5000/api/i18n/pl

This should return the Content-Type: application/json and a 200 with an appropriate JSON body (this can be the language requested or an appropriate fallback). I think that if the fallback is the default language (in most cases English) then we just return a 204 No Content to save data.

1344248847000000 1344269446000000
#2803 enhancement aron.carroll aron.carroll demo phase 3 closed fixed Convert all JS translations to use keywords 1344251817000000 1344256274000000
#2804 enhancement aron.carroll shevski demo phase 3 closed fixed please make download button blue

on resource pages (e.g. http://s031.okserver.org:2375/dataset/afghanistan-election-data/resource/f6331f99-51f6-44d9-95b9-b20f3b74f360), same as old ckan for continuity and to highlight!

1344252255000000 1344269865000000
#2807 enhancement aron.carroll toby demo phase 3 closed fixed autocomplete.js error

@aron,

edit dataset has a js error

http://localhost:5000/dataset/edit/1-kmwaterfractionfromnationaltopographicdatabasemapscanada

TypeError?: options is undefined [Break On This Error]

this.lookup(options.term, options.callback);

autocomplete.js (line 231)

1344266228000000 1344270190000000
#2808 enhancement aron.carroll toby demo phase 3 closed fixed Confirm delete dialog badly placed on small screen

If you make your browser window short - say half screen height on a page with delete eg http://localhost:5000/dataset/edit/1-kmwaterfractionfromnationaltopographicdatabasemapscanada

the dialog box is partially off the screen - it should try to stay on screen

1344333474000000 1345023693000000
#2809 enhancement aron.carroll toby demo phase 3 closed fixed _ not defined for modules

eg on confirm delete

http://localhost:5000/zh_TW/dataset/edit/1-kmwaterfractionfromnationaltopographicdatabasemapscanada

_(....) gives [object Object] not translation

1344335863000000 1344347814000000
#2837 enhancement aron.carroll toby demo phase 3 closed fixed checkbox styling

Is this done I know checkboxes are a pain but it looks a bit out of place compared to the other items.

1344605993000000 1344851271000000
#2840 enhancement aron.carroll toby demo phase 3 closed fixed tag line styling

@aron

Is it possible to get the tagline text to float the other way

at the moment we have

...........Title Tag line to here

I'd like

...........Title ...........Tag line to here

is this possible or too much pain

based partly on this comment Markw wrote

A minor problem visible on the home page <http://s031.okserver.org:2375/> and other pages, when viewing in both Chrome and Firefox:

if the browser width is not very wide, the tag line 'Open source data portal' vanishes off the *left* hand side of the screen. For some reason the browser doesn't even recognise this with a horizontal scroll bar (as it does when stuff is off the *right* hand side) - it just chops off the text.

1344851307000000 1344852538000000
#2841 enhancement aron.carroll amercader demo phase 3 closed fixed Slug generation fails on demo dev

On the add dataset form, slug generation fails with JS error:

Uncaught TypeError: Object [object Object] has no method 'incompleteFormWarning'    basic-form.js:5
1344855653000000 1344857415000000
#2955 defect dominik dominik ckan 2.0 closed fixed Recline should be updated 1349270534000000 1350577952000000
#2961 enhancement dominik dominik ckan 2.0 assigned Preview plugin endpoint

Users could write ckan extensions that offer previews for a certain datatype.

Advantages:

  • We can say that you can add your own previews
  • Every preview would have to define it's own dependencies
  • External development (community creates new extensions?)

Questions to answer:

  • Specification of the interface
  • Precedence of previews for the same data types
  • Plugins only define the datatype that they are responsible for or do we call a function that returns whether the extension is responsible.

As far as I can see, this should be fairly easy to implement by following these instructions: http://docs.ckan.org/en/latest/writing-extensions.html#writing-a-plugin-interface

1349696367000000 1350577990000000
#3013 defect dominik dominik new common-error-messages is unreadable

Since the update of the doc theme, the page became unreadable.

http://docs.ckan.org/en/latest/common-error-messages.html

1352553505000000 1352553505000000
#62 enhancement dread rgrp v0.10 closed fixed Change tags to contain any character (other than space)

Requires us to url encode the tag names when displaying them ...

1240585095000000 1250181376000000
#74 enhancement dread rgrp v0.10 closed fixed Add Is It Open links to package pages

On each package page in the openness field if the package is not open provide a link to is it open create enquiry page.

Details:

  • may want to add to the url a query parameter indicating which package this relates to e.g. ?ckan-package=...
  • http://isitopen.ckan.net/enquiry/create/?ckan-package=...
  • do we provide link even if package is open (e.g. so that people can make enquiries about e.g. adding open data/content buttons ...)

Cost: 0.5h

1247828218000000 1250182938000000
#75 enhancement dread rgrp closed duplicate Record and display package "usage" information
  • Number of package page visits on ckan (can we get this straight from google analytics)
  • Number of times url or download url is used - now ticket:937 (Record download stats for resoures)

How do we do this?

  • Google analytics will miss a lot of this usage (and how do we get that data out anyway)
  • Could use javascript but again misses usage.
  • One option is to redirect link but that is kind of nasty (but may be only option ...)
1247828785000000 1296341223000000
#76 enhancement dread rgrp v0.10 closed fixed Convert to use formalchemy for all forms

Current form stuff is not very good (uses formencode). Switch to formalchemy would improve this, especially on validation.

As an extra we could utilize the formalchemy pylons admin interface (pretty much for free).

Cost: 12h

Details

  1. Replace htmlfill and formencode extract in controllers/templates with formalchemy
    1. Crude and simple
    2. Suppress unwanted fields (revision, state, all revisions) and sort out ordering to be similar to before.
    3. Sort out tag field with a special renderer
  2. Validation - testing definitely required.
  3. Fix up description and pretty css etc
1247829041000000 1251301765000000
#77 enhancement dread rgrp v0.11 closed fixed Rate packages

As a User or Visitor

I wish to rate a package in some way. On the package view, below Openness, there is a 'Rating' heading with the star rating which is selectable.

Rating out of 5. One user or IP, one vote.

package_rating table:

package | user | rating annakarenina | joe.bloggs.openid.com | 4 annakarenina | 154.2.5.47 | 1

1247829193000000 1255176732000000
#79 enhancement dread rgrp v0.10 closed fixed Improvements to package search

Several things:

  • Default package search (used e.g. on front page) should deal well with multiple items: "abc xyz" means search for both abc and xyz (as separate searches)
  • Should automatically display a list of matching tags in a line at top with number of associated packages
  • have an option to only search for fully open material (or openly licensed material ...)

Since we should be moving to a more sophisticated search solution anyway (to allow prioritisation etc) maybe we should cautious as to what we do now.

1247829864000000 1265890806000000
#80 enhancement dread rgrp v0.10 closed fixed Refactor or remove modes code

Get rid of modes in the RESTful API. Do json stuff directly in controllers.

  • package will have to_dict and from_dict methods (called by rest controller) which provide and consume JSON friendly dictionaries representing the object. from_dict - class_method. stuff in forms for tags as_string, maybe factor out. reuse validation stuff.
1247844263000000 1265890912000000
#91 enhancement dread rgrp v0.10 closed fixed Add author and maintainer attributes to package

Add the following attributes to package:

  • author, author_email
  • maintainer, maintainer_email

Gives us full compatibility to: http://docs.python.org/distutils/setupscript.html#additional-meta-data

Column ordering - should come after name, title, url, download_url.

1249049780000000 1250864156000000
#94 enhancement dread rgrp v0.10 closed fixed Use sqlalchemy-migrate to handle db/model upgrades

Details of how to go about this here <http://www.rufuspollock.org/2009/07/27/sqlalchemy-migrate-with-pylons/>

Cost: 2h

1249134185000000 1250604808000000
#105 enhancement dread rgrp v0.10 closed fixed Package groups (view)

As a user I want to be able to create groups of packages.

This functionality is different from tags:

  • The ability to add a package to a given group is restricted whereas anyone can add a given tag to a package
  • Specifically groups have owners and only the owners can add a package to that group

Group properties

  • id (primarykey, uuid style)
  • name (restricted content - same rules as package)
  • title (no restrictions)
  • owners -- many:many with user object
  • description (markdown)
  • packages -- many:many with package object

Don't version groups for now.

Groups address in the WUI will be:

  • /group - browse list of groups (reuse list action)
  • /group/list - same thing for now
  • /group/<groupname> - display: group properties with links to packages. No links to user pages (yet).

Group editing and searching will be another ticket.

Cost: 3d

1251392282000000 1252488496000000
#106 enhancement dread rgrp closed duplicate Regularly convert CKAN data to RDF and put on Talis CC

Sister to ticket:90 (Link to RDF version of CKAN data on Talis Connected Commons).

Talis have already kindly done an initial conversion. We should repeat this process regularly and re-upload the data to Talis CC.

In the long run may wish to only re-convert packages changed since the last upload. However given relatively smaller size of full dataset this optimization is probably not yet required.

Attached is the ruby script used by Talis for conversion

Cost: ? (1d+ depending on e.g. how easy integration with Talis CC is)

1251454474000000 1256140649000000
#108 enhancement dread rgrp v0.10 closed fixed Package search in the REST API

Add package search facility in the rest api at /api/search

Queries can be provided as for the normal package search either by posting to that url or by performing a get with a query string.

E.g. .../api/search/package?q=xyz

Query parameters:

  • q is keyword string (searches name, title, tags by default)
    • split by words and ANDed
  • additional parameters for specific fields
  • qjson is alternative to q for Searching by specific fields (in addition to keyword string). Payload to json is a JSON-encoded dict which is a dictionary with a q field for free text (keywords) and additional key/value pairs for specific fields
    • if q and qjson specified ignore q and just process qjson value

Additional parameters in addition to query ("q" or "qjson") are:

  • limit
  • offset
  • fullinfo=0/1 - return full record for each result (default=0)
  • order_by=field_name
  • search_notes=0/1 (default 0) do we search notes field in a keyword search

Return value is json encoded dictionary with keys:

  • 'results': list of results
  • 'count': total number of results

Extras (for the future)

  • Prioritisation is not part of this ticket (requires fulltext support in DB or in external app such as Xapian)
1251915845000000 1252340511000000
#110 enhancement dread dread v0.10 closed fixed Integrate groups into packages
  1. Package needs a groups property (backref Group.packages).
  2. Put groups in the search api.
  3. Display them on the package, but not editable. (Only editable from the group page.)

Follow up to ticket:105 (Groups).

Cost: 1d

1252315956000000 1252488660000000
#113 enhancement dread dread v0.11 closed fixed Simple script to test data migration on a production db

Must not alter the db.

Related to ticket:112.

1252316523000000 1265294054000000
#114 enhancement dread dread v0.10 closed fixed Access Control - model

Create in the model basic operation of Access Control.

roles table

name | context | action
-----------------------
admin| package | edit
admin| package | edit-permissions
admin| package | read
editor| package | update
editor| package | read
reader| package | read
This data is set-up on db init and will have no interface.

user-roles table:

username | context_type | objectid | role
rgrp     | system  | n/a | admin
visitor  | package | * | reader
bob      | package | geonames | admin
visitor  | package | geonames | editor
visitor  | package | geonames | reader
john     | group | ukgov | admin
dread    | group | ukgov | editor
visitor  | group | ukgov | reader
This data will be added when someone is given permissions for the system, a package or a group.

Pseudo code:

class Package

def is_allowed(name, action):

is_allowed(name, action, context=self)

class Group

def is_allowed(name, action):

is_allowed(name, action, context=self)

def is_allowed(name, action, context=None): name: string - a username or IP for 'visitor'

action: string - 'read', 'edit', 'delete', 'edit-permissions' context: object - a Group or a Package or None (which means system)

# look up user from name. # look up in user-roles table what roles this user has for this context. # for each roles, look up in roles table what actions are allowed. # return True if action is allowed, else False.

1252494527000000 1253034529000000
#115 enhancement dread dread v0.10 closed fixed Access Control - wui constrained by model

Based on a section of AccessControl design: Reading and writing to packages in the WUI and REST API are now dependent on the authz tables.

1253034394000000 1253091426000000
#116 enhancement dread dread v0.10 closed fixed Access Control - edited in wui

Based on a section of AccessControl design: WUI gives controls to user and administrator to change permissions on a package.

1253034802000000 1253709460000000
#117 enhancement dread dread v0.10 closed fixed Access Control - group core functionality

Based on a section of AccessControl design: Group reads and edits are controlled by access control. WUI and REST interfaces covered.

1253271333000000 1254735855000000
#118 enhancement dread dread v0.10 closed fixed Use paginate in webhelpers

Take out import of paginate in setup.py. Use paginate in webhelpers instead. Make changes to take account of any i/f changes.

1253273657000000 1253784902000000
#119 enhancement dread dread v0.10 closed fixed Ensure non-active packages don't show up

Ensure pending packages don't show up in search or browse

cost: 4h

1253529414000000 1253791147000000
#120 enhancement dread dread v0.10 closed fixed Security audit

Look for all places where model is accessed and check authorization is checked.

Document holes (and, as necessary, suggestions for fixes) as new tickets. Likely areas that need looking at:

  • search i/f
  • package WUI commit

Write holes are obviously much more significant to us than read holes.

1253529427000000 1254406544000000
#121 enhancement dread dread v0.10 closed fixed Add 'Group' to main menu

And associated page to browse group.

1253694827000000 1253716782000000
#122 enhancement dread dread v0.10 closed fixed Add Group authz page 1253694842000000 1253716757000000
#123 enhancement dread dread v0.10 closed fixed Ability to edit Group in WUI

Add Group editing page.

If no permissions to change group can't edit group. Also cannot view edit page.

Editable attributes: name, title, description

No preview needed

1253708041000000 1254321447000000
#125 enhancement dread rgrp v1.0 closed duplicate Edit Generic Package Attributes in WUI

Split out from ticket:43

1253709712000000 1258377621000000
#126 enhancement dread dread v0.10 closed fixed Change package state in the WUI (delete and undelete)

As a Package Admin I want to change the state of the package. In particular I wish to delete and undelete it.

(NB: this is quite separate from "purging" objects which is the term we shall use for irrevocable removal of an object from the domain model).

  • Only Package Admins (and sysadmins) should be able to change state

Implementation Suggestions

  • 'delete' action should be renamed to 'change-state' (NB: this requires a db migration ...)
  • Have new package formalchemy form (created via inheritance?) to incorporate state attribute. Suggest this is rendered as a dropdown (and may be simple object rendering of state, i.e. do NOT need to change it to a single name such 'active').
  • This form should then be used when the user satisfies is_authorized(..., model.Action.CHANGE_STATE) instead of the usual fieldset
1253789571000000 1254740244000000
#127 enhancement dread dread v0.10 closed fixed Minor form and UI improvements
  • group form: description field for needs to be a text area
  • group form: make title and name field wider
  • package view: need to move tags and license above notes
1253798659000000 1253868048000000
#128 enhancement dread rgrp v0.11 closed fixed Add ckan_url attribute to REST JSON representation of a Package

Add ckan_url attribute to REST JSON representation of a Package pointing to the (read) url of package on CKAN.

Cost: 30m

1253866713000000 1275694573000000
#130 enhancement dread rgrp v0.10 closed fixed Create a New Group via the WUI

As a

User

I want to

Create a new group via the WUI

Details

  • locate at /group/new/
  • You must be logged in to create a group
  • Group creator is automatically given role of group admin
  • Edit screen is same as edit screen ...
1254130212000000 1254735501000000
#131 enhancement dread dread v0.10 closed fixed Groups REST interface

Controlling Groups through a REST interface.

1254307959000000 1254308115000000
#141 task dread rgrp v0.11 closed fixed Search api docs

Write up search api documentation and put it in a template that shows up at api/search/.

Cost: 2h

1254903008000000 1255007583000000
#147 enhancement dread dread v0.11 closed duplicate Parser and loader for esw.org data 1255440695000000 1255515162000000
#148 enhancement dread dread v0.11 closed fixed Parser and loader for esw.org data
  • parse like data4nr code

Follow up tickets: ticket:149 and ticket:150

1255440696000000 1255515222000000
#149 enhancement dread dread v0.11 closed fixed Loader for esw data recognises existing packages

Follows on from ticket:148. Sister to ticket:150.

1255440751000000 1257414916000000
#150 enhancement dread dread v0.11 closed fixed Loader for esw data uses REST API

(instead of sqlalchemy interface, to facilitate general usage of the loader)

  • use ckanclient

Follows on from ticket:148 and ticket:149

1255440828000000 1256751974000000
#151 enhancement dread rgrp v0.11 closed duplicate User object should have a created attribute

User object should have a "created" attribute initialized to current datetime.

Require a db migration but o/w very simple.

Cost: 1.5h

1255589694000000 1257414545000000
#152 enhancement dread dread v0.11 closed fixed Package has editable 'Extra' fields

Package edit page has fields for extra key-value pairs. Don't use formalchemy. The form displays the existing ones and new ones as follows:

Key           Value            delete
[ country   ] [ uk, spain   ]  [x]

When presenting the form, existing pairs and 3 blank pairs are provided.

When saving the data, keys that are not changed use the existing PackageExtra object (perhaps with new value). Deleted keys put the PackageExtra state to deleted. For new keys, only create a new PackageExtra if there isn't a state=deleted one to resurrect.

Validators ensure no repeated keys.

1255615316000000 1256056193000000
#153 enhancement dread dread v0.11 closed duplicate Group's packages listed alphabetically

This is so you can easily look up whether a given package is already listed - otherwise as lists get bigger becomes difficult to see what is already there.

Suggested by Jonathan Gray

1255621515000000 1258971895000000
#155 enhancement dread dread v1.0 closed duplicate Adding multiple packages to a group

Ability to add multiple packages to a group in one go (e.g. with 'add' link which makes drop down menu appear - so can add one after another - then submit simultaneously)

Use a bit of javascript to add more dropdowns.

Suggested by Jonathan Gray

1255621779000000 1271760041000000
#157 enhancement dread rgrp v0.11 closed fixed Log message not persisted through preview

Log message on package edit page is no longer persisted through a preview (no doubt due to recent form upgrades). Let's fix this.

cost: 0.5h

1255694396000000 1256062680000000
#158 enhancement dread dread v0.11 closed fixed Add 'extras' to REST API
  • Ensure you can add/change/delete extra fields from packages using the REST API.
  • Update API docs to cover this.

On Package get, ensure separation of extra attributes from other attributes.

What is the meaning of posting an incomplete dictionary (updating a package)? Option 1 - if field not present, leave field alone

  • Delete field if field value is None

Option 2 - user must supply all fields

  • Delete field if field absent

Prefer option 1. Document this.

Follows on from ticket:152

1255703259000000 1255949818000000
#159 defect dread rgrp v0.11 closed fixed Searching for tags:... resulting in lots of tags being found

Search of form: tags: ... behaves differently depending on whether there is a leading space:

  • tags: postcode - tags found correctly but no packages found
  • tags:postcode - tags incorrectly found but correct packages found

Let's fix this.

Cost: 0.5h

1256030097000000 1256060264000000
#160 enhancement dread rgrp v0.11 closed fixed Implement internal links in CKAN notes markup

Suggest form is: key:value (with optional space after ':'.

Keys would be for an object, i.e. 'package', 'tag', 'group'.

Becomes a link.

This would be implemented by implementing a 'Parser' or 'Processor' object which is run before markdown is done. Regex replace.

Cost: 4h

1256030751000000 1265312986000000
#161 defect dread rgrp v0.11 closed fixed Tag listing is incorrect

E.g. http://ckan.net/tag/read/postcodes this has no-postcode listed but when you look at that package 'postcodes' isn't a tag ...

I suspect this may be to do with search using deleted tags as I think no-postcode had postcodes as a tag at some point earlier ...

1256030925000000 1258573607000000
#162 enhancement dread rgrp v0.11 closed fixed Tags should be automatially lowere-cased if not already

Rather than raising a validation error automatically lower-case tags on submission.

Cost: 0.5h

1256048269000000 1256142462000000
#166 enhancement dread dread v0.11 closed fixed Add extras editing to ckanclient

Ensure logging is working on server.

1256203581000000 1265891199000000
#167 defect dread dread v0.11 closed fixed REST interface you can't update multiple tags

Only first tag is updated.

1256209993000000 1265891178000000
#169 enhancement dread dread closed duplicate Package derivations

A 'Derived' relationship can be applied from one package to another.

e.g. sussex-demography is derived from census-2001

'Derived' relationship is:

  • directional
  • many:many
  • stateful

'derived' table columns:

  • id (primary key)
  • source_package (foreign key)
  • result_package (foreign key)
  • description (markdown text)

Further tickets:

  • WUI - package view - shows 'derives from package x' and 'derived package y' with UML-like diagram of x -> this package -> y
  • WUI - package edit form - new option to say it 'derives from' or 'has derivation' and you select the appropriate
  • REST if - expose reading and writing this property
1256304927000000 1266928708000000
#176 enhancement dread dread closed duplicate Package dependencies

(Related to ticket:169 - Package derivations)

A 'dependency' relationship can be applied from one package to another. It implies that a package requires the download or existence of another package which it 'depends on'. (Analogous to software package dependencies.)

e.g. london-traffic-visualisation depends on road-map

'Dependency' relationship is:

  • directional
  • many:many
  • stateful

'dependency' table columns:

  • id (primary key)
  • dependent (foreign key)
  • dependency (foreign key)

Further tickets:

  • WUI - package view - have list of dependencies (do not need to list packages which depend on this one)
  • WUI - package edit form - new option to say 'depends on' (no need for 'has dependent package')
  • REST api - expose reading and writing 'depends on' property.

Issues

  • How do we deal with dependency at a particular version?
1257162812000000 1266928721000000
#179 defect dread rgrp v1.0 closed fixed Restore 404 and 500 messages in WUI

Cost: 0.5h (?)

Conjecture this went missing in cset:a35db862a841

1257412668000000 1265305549000000
#184 enhancement dread rgrp closed fixed Stats page

Create /stats/ page displaying main statistics, e.g:

  • Most highly rated packages
  • Most edited packages
  • Largest groups
  • Top tags (by packages)
  • Package addition rate
  • Users with most packages

Related to ticket:181 - Stats side-bar

Details

1257534756000000 1266837414000000
#189 enhancement dread dread v0.11 closed fixed Multiple download urls

See also: PackageResources which defines a 'distribution' as a package's payload (perhaps with metadata). In our case, at least within CKAN, the representation of the distribution will just be the reference to get it (plus some minor metadata). We are expanding this functionality to be any kind of 'resource' associated with a package since it is likely in some cases people will want to link to a file containing a part, but not the whole, of a package's data.

As a

User (package editor and package user)

I want to

(package editor) Associate multiple distributions with a package.

(package user) Easily have access to multiple distributions of a package should they exist.

Implementation

There is an object called 'Resource' with the following attributes:

  • url (text) (required in UI, but not in db table)
  • format (text)
  • description (text)
  • package (association) - a resource is always associated with 1 package

A package's list of resources will have an order which is preserved. The first in the list will be known as the 'default resource' (the presumption will always be that this is a 'distribution').

How to replace download url

download_url removed and replaced as a proxy attribute. Read returns the url of the default distribution. Write changes the url of the default resource. Ensures backward compatibility.

REST API has new attribute in package dictionary 'resouces' - an ordered list.

In WUI, replace download url in package read with a table of the distributions.

In the WUI package edit, be able to add/remove/edit/reorder distributions.

Example

Example part of package:

Distributions:

http://some.com/pollution.pdf PDF Original report
http://some.com/pollution.csv csv Original data
http://some.com/pollution sparql Linked version of data

Existing data to be migrated to new structure.

Cost: 4 days

1258386120000000 1265284961000000
#196 defect dread dread v1.0 closed fixed RDF URI to resolve on CKAN

Provide RDF version of a package at the RDF URI: http://ckan.net/package/rdf/32000-naples-florida-businesses-kml

See email on OKFN help:

For the information to be Linked Data, the following URI (from the api.talis.com SPARQL call) should be resolvable to RDF, or an RDFa enhanced HTML page.

1258727988000000 1265389771000000
#197 enhancement dread dread v0.11 closed fixed Simple dumper

As a

admin / user

I want to

Admin: export all HEAD data to JSON / CSV / RDF format dump.

User: wants to get hold of this dump.

Details

  • Just do JSON & CSV for now.
  • Put this in CLI.
  • Just do list of packages - as_dict info. CSV needs a bit of flattening.

Cost: 2h

1258972495000000 1258989700000000
#203 enhancement dread dread v0.11 closed fixed show package counts for groups in WUI

In two places in the WUI

  • list of groups
  • group page in title e.g. ukgov (5)

Cost: 1h

1260187690000000 1265891080000000
#204 enhancement dread dread v0.11 closed fixed Custom package input form

As a

admin

I want to

input government data in a constrained way

so that

you get consistency in the db which is easily searched, discovered and automatically linked.

implementation

Form has fields suitable for government data. (related to custom excel importer ticket)

In pylons config specify the form schema to use - whether the existing one or the government one.

On submission, during validation, warn if a field isn't completed (unless it's optional - agency, maintainer, maintainer_email)

Simple fields

  • all basic ckan fields
  • default license to OKD::Crown Copyright
  • co_id, precision, taxonomy url, agency

Date fields

Date format: machine readable. Can be specific or vague: year or month/year or day/month/year.

  • date_released
  • date_updated
  • temporal_coverage_from, temporal_coverage_to - two date fields combined

Strict selection

Drop down combo

  • department

Selection or free text

Editable combo

  • update_frequency - selections: never, weekly, monthly, annually, discontinued
  • geographic_granularity - selections: national, regional, local authority, ward, point
  • temporal_granularity - selections: yes, months, weeks, days, hours, point

Specialised fields

  • geographic_coverage - UI is tick boxes. store a comma separated list of all that apply from: england, scotland, wales, northern ireland, overseas, global

(Idea for future version: Select the most relevant one from a choice of OS hierarchical geographic IDs in the UK. Stored in the database as the RDF URI. Question - what to do beyond the UK?)

  • national_statisic - UI is check box, store as 'Yes' or 'No'
  • resources - use standard table, but suggestions for file type are: RDF, CSV, SPARQL, REST, XBRL, SDMX, Other XML, HTML+RDFa, XLS, DOC. Have warning label saying it is not ideal to have PDF, zip or html targets.
  • categories - multi values from strict selection of ONS categories. Stored as comma-separated.

Field data processed

  • tags - in addition to words entered here, search for known keywords in other fields: title, notes, department, agency, geographic coverage & categories.

Future - optional

Each form schemas is made available (or not) in the pylons config, along with the default schema for new packages. If there is more than just the 'basic' schema available then the 'new package' form has at the top a drop-down to select the schema with a 'switch' button. This submits the form's current data and parameter schema=xyz and the form arrives back populated with the existing options. No data should be lost switching views.

When you edit a package it detects the schema based on the extra fields that are present.

1260188965000000 1285082169000000
#207 enhancement dread nickstenning v1.0 closed fixed Reimplement tag selection in jQuery

Remove the current dependency on Yahoo YUI toolkit for doing tag autocompletion on package edit forms.

The (KForge-derived) behaviour files for CKAN can easily be switched to the new jQuery-based ones: see ticket:212. Once this is done, it would make sense to implement the tag-selection code with jQuery.

Relevant pages include:

1260200340000000 1271250740000000
#208 enhancement dread rgrp v0.11 closed fixed Obfuscate author email and maintainer email field

Obfuscate author email and maintainer email field.

Use: webhelpers.html.tools.mail_to e.g. webhelpers.html.tools.mail_to('[email protected]…', encode='javascript')

Do we want to? Surely only public addresses should be on CKAN in which case why obfuscate?

1260203938000000 1265891070000000
#213 enhancement dread dread v0.11 closed fixed Automate CKAN deployment

fabric script to automate ckan deployment

1260811707000000 1265891061000000
#221 enhancement dread dread v1.0 closed fixed UI Review - Package edit

Package edit page package/edit/mypackagename

  • Ability to add the package to groups on this page.
  • Fields could be grouped.
  • Inconsistent capitalisation (Url -> URL).
  • Tag editor YUI script appears to break occasionally.
  • Perhaps move to two column forms to save vertical space?
  • Notes field to monospace?
  • Extras fields need some jQuery love -- shouldn't have a fixed number available.
  • Typography is a bit of a mess.

1260880175000000 1271756757000000
#229 enhancement dread dread v0.11 closed fixed Resource hashes

New field for resources - hash of the resource file.

  • CKAN itself will not calculate the hash value - user just pastes it in.
  • Display text field in resource table.
  • Requires migration script.

Questions for the field's value:

  1. Which hash to use? Restrict to python hashlib and other major hash libraries.
  1. Should we use merkle trees?

Thanks to Julien D'Assanges for the suggestion.

1262686287000000 1265891612000000
#230 task dread dread v1.0 closed fixed Refactor 'package preview' to use 'package read'
  • Requires Package Read to not use the side-bar
  • Need to sync params to a package object that is taken out of the session so that it doesn't go into the database.
1262788084000000 1265293953000000
#239 task dread dread v1.0 closed fixed Test migrate scripts

Integrate new test upgrade into testmigrate stuff.

New "migrate.ini" config file programmed with the db to be used for migrate tests and a param saying where the db dump is.

1264439926000000 1265286521000000
#247 enhancement dread rgrp v1.0 closed fixed Alternative formats links on package page

"Alternative formats". Has logo for RDF (link to RDF) and JSON (link via CKAN REST i/f).

  • How do we know if RDF available? Ans: see details in ticket:90.

Cost: 1h

1265890326000000 1271248813000000
#248 enhancement dread rgrp v1.0 closed fixed i18n of forms

Need to do i18n of forms (not in templates but in code). Should consult with tonfa about this.

1265892313000000 1270745304000000
#252 enhancement dread johnbywater closed invalid Change revision object so that it has parent(s) attribute 1266519767000000 1296477560000000
#254 enhancement dread dread v1.0 closed fixed Package relationships - 1. model

New domain object: PackageRelationship? (revisioned)

Attributes:

  • subject (Package reference)
  • object (Package reference)
  • type (string)
  • comment

Relationship type values: depends_on (dependency_of) derives_from (has_derivation) child_of (parent_of)

Relationship type is stored as a string, as given in the first column. The reverse relationship (bracketed) is given just for display purposes only.

Subject / Object - as in a sentence: "Dracos is derived from OSM" is <subject> <predicate> <object>. We'll use 'type' instead of 'predicate' as that is a more familiar word.

1266928449000000 1273596180000000
#255 enhancement dread dread v1.0 closed fixed Package relationships - 2. Read in WUI

WUI:

  • View: show both sides of the relationship (but think carefully -- e.g. a given package may have *many* dependents ...)
1266928542000000 1273596174000000
#257 enhancement dread dread v1.0 closed fixed Package relationships - 4. Read in API

API:

  • Appear in package listing Example: 'relationships': [{'is_dependency_of':'osm', comments:'Since version 0.2'}, {'is_parent_of':'bobs_maps'}]
1266928630000000 1273596170000000
#260 enhancement dread rgrp v1.0 closed fixed Display group package list in standard (richer) way

Currently use old "bullet-point" style. Should change to now standard style used on e.g. package search results page.

Cost: 0.5h

1267100660000000 1267122137000000
#261 defect dread rgrp v1.0 closed fixed License does not work through REST API

Presence of license_id field in JSON input show below breaks REST API (with weird error about int not being subscriptable as if license_id field is being subscripted).

Using normal license field does not cause breakage but has no effect.

(Also think we should remove license_id from the API since there is no guarantee this cross system compatible since locally generated on install).

    ourjson = '''{
   "maintainer" : "Applications Division",
   "version" : "1999-01-01",
   "license_id" : 9,
   "name" : "geogratiswaterfraction1km_v100",
   "author_email" : "...",
   "author" : "Dissemination Branch",
   "tags" : [
      "GeoGratis",
      "Canada",
      "GCMD",
      "Hydrosphere",
      "Ground Water",
      "Surface Water",
      "Hydropattern",
      "Earth Science",
      "Land Surface",
      "Land Cover",
      "Landscape"
   ],
   "extras" : {
      "License" : "http://geogratis.ca/geogratis/en/licence.jsp",
      "Ministry" : "Natural Resources Canada",
      "Level of Government" : "Federal"
   },
   "maintainer_email" : "[email protected]",
   "notes" : "This product is a raster coverage",
   "url" : "http://geogratis.cgdi.gc.ca/geogratis/en/collection/metadata.do?id=67",
   "title" : "Geogratis - 1-km Water Fraction From National Topographic Data Base Maps, Canada"
}
'''
    def test_15_random(self):
        offset = '/api/rest/package'
        import urllib
        tmpjson = simplejson.dumps(simplejson.loads(self.ourjson))
        postparams = urllib.urlencode({tmpjson: 1})
        res = self.app.post(offset, params=postparams, status=[200],
                extra_environ=self.extra_environ)
        model.Session.remove()
1267134778000000 1267207776000000
#264 defect dread rgrp v1.0 closed fixed Investigate why there are revisions with no associated packages

Revision.packages should not include all changes so why are we getting revisions with no associated changes? See e.g. http://ckan.net/revision/read/814e39e1-2540-4449-90d1-ac9a92ab40fb?

Investigate and fix.

NB: if this is bugfix this should go against stable or metastable branch if at all possible and then be merged into default.

Cost: 1.5h (?)

1267605428000000 1272390013000000
#275 defect dread rgrp closed fixed Add forms renderer for JsonType fixing bugs in admin interface

This will solve e.g. issues in formalchemy admin interface (which is due to lack of renderer for Json type)

Cost: 0.5h

from ckan.model.types import JsonType

forms.FieldSet.default_renderers[JsonType] = formalchemy.fields.TextFieldRenderer
1269122740000000 1281002082000000
#276 defect dread rgrp v1.0 closed fixed Resources in Package form seen multiple times upon preview

Create a new package with a name 'blah' and resource format 'blah'. Hit preview. There is an error because of the lack of resource url, but in the resource input boxes, there are now four resources with format 'blah'.

1269255399000000 1272996237000000
#278 defect dread dread v1.0 closed fixed Spaces in extra field keys

If the key to an extra field has a space in it then various form code doesn't work. Either need to disallow spaces or cope with them.

1269520845000000 1271173752000000
#280 defect dread dread v1.0 closed fixed Disappearing extra fields

When I edit a package on no.ckan.net with lots of extra fields, lots of them completely disappear.

Metastable branch.

1270209684000000 1271173769000000
#281 enhancement dread dread v1.0 closed fixed Refactor forms to be plugin-able 1270542248000000 1270723675000000
#286 enhancement dread dread v1.0 closed fixed Refactor forms templates

Use Nick's examples as a basis to revamp the formalchemy usage, principally in the package edit form.

1270723513000000 1270723629000000
#287 enhancement dread dread v1.0 closed fixed Customise Canadian package edit form

Spec - see attached, which is a copy of the Pirate Pad from Lauren: http://piratepad.net/2C2iwiLDhd

1270801086000000 1270801210000000
#288 defect dread dread v1.0 closed fixed Foreign and escaped characters in extras fields

These don't work.

1271171899000000 1271173777000000
#292 defect dread dread v1.0 closed fixed PackageResources are created new on every package edit

When you edit a package, all of its resources are deleted and a fresh set created. This causes all history of them to be lost.

1271755964000000 1272286005000000
#295 enhancement dread dread v1.0 closed fixed Add multiple packages when editing a group

In Group edit page, you can only add one package at a time. It would be better to use some scripting to allow adding more than one at once, to make it easier and quicker when you have a long list to add.

Ticket is split off from ticket:221

See also ticket:155 (duplicate).

1271756713000000 1272384758000000
#303 defect dread dread v1.0 closed fixed Package history & diff & feed to show changes to tags, extras and resources

Currently a package's history page (/package/history/xyz) and related feed only shows changes to the core package i.e. ones which create a PackageResource?. This doesn't include changes to tags, extras and resources. These need to be added.

There is the complication that the 'diff' function doesn't currently work beyond PackageResource?. Unless this is quick and simple to fix, I suggest we let the diff remain slightly broken for now.

Rufus suggests alongside the existing methods:

pkg.all_revisions()
pkg.diff()

we have ones which also include the related objects (tags, extras, resources):

pkg.all_revisions_full()
pkg.diff_full()

You could have a function which returned these related objects. Ideas:

revobjlist = [ self, resources, extras ]
list of tuples: (revobj, changed_revobjs)
list of tuples: (revobj, {objtype: [changed_revobjs]})
tuple: (rev1, {Package: [ object_id, object_id], PackageTag: [object_id, object_id2] ...}
1272370659000000 1272989728000000
#310 defect dread rgrp v1.1 closed fixed Commit message box looks wrong in edit page since edit style overhaul

Suggest move this below the label and make full width of screen and only 3/4 rows high (more like a wiki site).

  • Also change label to: Edit summary (Briefly describe the changes you have made)
  • Remove: you can markdown formatting here.
  • Move author: if you have not signed in smaller and closer (like markdown instructions are nwo).
  • Change commit -> save
  • Remove "please save" just have the bullet points
1273348714000000 1279300525000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Note: See TracReports for help on using and creating reports.