{22} Trac tickets (2647 matches)

Results (101 - 200 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#101 enhancement rgrp rgrp closed fixed (Improved) Download support
  1. Given a url download from that url to disk
  2. Given a package use download url to download to disk

Once on disk should uncompress (if necessary).

Details

  • Should download either to tmp directory (defaulted in config) or a specified path on disk
  • In download would like to support as many types of target urls as possible:
    1. Single files (usually compressed filesets but could be individual files)
    2. Index pages listing files to download
    3. Revision control repositories
    4. ...

This will obviously be a significant amount of work. Suggest:

  1. Start off just supporting the first option.
  2. Have something like a plugin system to handle new target types
  3. Reuse existing work (e.g. easy_install and pip handle standard vcs such as svn, hg, git etc)

Cost: 1d

1249988866000000 1267648607000000
#102 enhancement rgrp rgrp closed fixed Switch to using UUIDs for revision ids

This is better for long term sustainability, especially if we want to "push and pull" from unrelated repositories in the future.

Details:

  • May wish to retain a number field updated in the usual incremental way (what's the cost/benefit here?)
1250068175000000 1260285104000000
#103 enhancement kindly rgrp ckan-v1.5 closed fixed View a package at a given revision

As a user I want to view a package at a given revision:

  • When I visit /package/read/xyz?rev=yyy I should be shown package at revision yyy
  • package history page should provide links to these pages

Cost: 2h

1251289897000000 1311180850000000
#104 enhancement rgrp rgrp closed fixed View a package at a given version

As a user i want to see a package at "version" X (NB: not revision X).

  • When I visit e.g. /package/read/xyz?version=0.7 I should be shown package at version 0.7 (or a message saying no such version)
  • Implementation:
    • Find revision for this version (search revision history for when version field was last 0.7)
    • Show pacakge at that revision (as in ticket:103)
  • On history page also shows versions in list of revisions associated with the package
1251292672000000 1311181303000000
#105 enhancement dread rgrp v0.10 closed fixed Package groups (view)

As a user I want to be able to create groups of packages.

This functionality is different from tags:

  • The ability to add a package to a given group is restricted whereas anyone can add a given tag to a package
  • Specifically groups have owners and only the owners can add a package to that group

Group properties

  • id (primarykey, uuid style)
  • name (restricted content - same rules as package)
  • title (no restrictions)
  • owners -- many:many with user object
  • description (markdown)
  • packages -- many:many with package object

Don't version groups for now.

Groups address in the WUI will be:

  • /group - browse list of groups (reuse list action)
  • /group/list - same thing for now
  • /group/<groupname> - display: group properties with links to packages. No links to user pages (yet).

Group editing and searching will be another ticket.

Cost: 3d

1251392282000000 1252488496000000
#106 enhancement dread rgrp closed duplicate Regularly convert CKAN data to RDF and put on Talis CC

Sister to ticket:90 (Link to RDF version of CKAN data on Talis Connected Commons).

Talis have already kindly done an initial conversion. We should repeat this process regularly and re-upload the data to Talis CC.

In the long run may wish to only re-convert packages changed since the last upload. However given relatively smaller size of full dataset this optimization is probably not yet required.

Attached is the ruby script used by Talis for conversion

Cost: ? (1d+ depending on e.g. how easy integration with Talis CC is)

1251454474000000 1256140649000000
#107 enhancement rgrp rgrp closed invalid Provide information about uploading material

Often people who are registering resources want to make the associated material available. We should:

  • Provide a dedicated upload page giving instructions as to upload process
    1. Explain we don't store the data itself on CKAN. Suggest uploading to a store somewhere and then linking using download_url (that is its purpose)
    2. Upload to
      • Talis CC for RDF
      • archive.org
      • grid.okfn.org
  • link to this page from next to download_url item on new package page

In long run (now ticket:186) we may automate this by providing a ckan upload facility which caches the data and then reuploads it to relevant service (disadvantage is "we" are the owner of the data on that service ...)

1251466274000000 1296341644000000
#108 enhancement dread rgrp v0.10 closed fixed Package search in the REST API

Add package search facility in the rest api at /api/search

Queries can be provided as for the normal package search either by posting to that url or by performing a get with a query string.

E.g. .../api/search/package?q=xyz

Query parameters:

  • q is keyword string (searches name, title, tags by default)
    • split by words and ANDed
  • additional parameters for specific fields
  • qjson is alternative to q for Searching by specific fields (in addition to keyword string). Payload to json is a JSON-encoded dict which is a dictionary with a q field for free text (keywords) and additional key/value pairs for specific fields
    • if q and qjson specified ignore q and just process qjson value

Additional parameters in addition to query ("q" or "qjson") are:

  • limit
  • offset
  • fullinfo=0/1 - return full record for each result (default=0)
  • order_by=field_name
  • search_notes=0/1 (default 0) do we search notes field in a keyword search

Return value is json encoded dictionary with keys:

  • 'results': list of results
  • 'count': total number of results

Extras (for the future)

  • Prioritisation is not part of this ticket (requires fulltext support in DB or in external app such as Xapian)
1251915845000000 1252340511000000
#109 enhancement rgrp dread closed fixed Tag search in the REST API

Similar to ticket:108 but for tags:

/api/search/tag?q

For tags search is extremely simple since you can only search by name.

Possible Extras

  • Allow "exotic" ordering of results e.g. ordering by number of packages with that tag.
  • Return this number with tag list.
1251976297000000 1291829457000000
#110 enhancement dread dread v0.10 closed fixed Integrate groups into packages
  1. Package needs a groups property (backref Group.packages).
  2. Put groups in the search api.
  3. Display them on the package, but not editable. (Only editable from the group page.)

Follow up to ticket:105 (Groups).

Cost: 1d

1252315956000000 1252488660000000
#111 enhancement rgrp dread v0.10 closed fixed Create user object

Object properties:

  • id (uuid)
  • apikey (uuid) -- migrated from the apikey table
  • name (username = openid)

/account is being renamed to /user in the wui.

1252315994000000 1252331626000000
#112 enhancement rgrp dread v0.11 closed wontfix Test db schema created via migration scripts

'paster db upgrade' should result in the same database as 'paster db create' on a clean system. In particular, all tests should pass.

Note: equivalent of paster db upgrade isn't being run from tests at the moment (we haven't been able to get this to work).

1252316495000000 1265294069000000
#113 enhancement dread dread v0.11 closed fixed Simple script to test data migration on a production db

Must not alter the db.

Related to ticket:112.

1252316523000000 1265294054000000
#114 enhancement dread dread v0.10 closed fixed Access Control - model

Create in the model basic operation of Access Control.

roles table

name | context | action
-----------------------
admin| package | edit
admin| package | edit-permissions
admin| package | read
editor| package | update
editor| package | read
reader| package | read
This data is set-up on db init and will have no interface.

user-roles table:

username | context_type | objectid | role
rgrp     | system  | n/a | admin
visitor  | package | * | reader
bob      | package | geonames | admin
visitor  | package | geonames | editor
visitor  | package | geonames | reader
john     | group | ukgov | admin
dread    | group | ukgov | editor
visitor  | group | ukgov | reader
This data will be added when someone is given permissions for the system, a package or a group.

Pseudo code:

class Package

def is_allowed(name, action):

is_allowed(name, action, context=self)

class Group

def is_allowed(name, action):

is_allowed(name, action, context=self)

def is_allowed(name, action, context=None): name: string - a username or IP for 'visitor'

action: string - 'read', 'edit', 'delete', 'edit-permissions' context: object - a Group or a Package or None (which means system)

# look up user from name. # look up in user-roles table what roles this user has for this context. # for each roles, look up in roles table what actions are allowed. # return True if action is allowed, else False.

1252494527000000 1253034529000000
#115 enhancement dread dread v0.10 closed fixed Access Control - wui constrained by model

Based on a section of AccessControl design: Reading and writing to packages in the WUI and REST API are now dependent on the authz tables.

1253034394000000 1253091426000000
#116 enhancement dread dread v0.10 closed fixed Access Control - edited in wui

Based on a section of AccessControl design: WUI gives controls to user and administrator to change permissions on a package.

1253034802000000 1253709460000000
#117 enhancement dread dread v0.10 closed fixed Access Control - group core functionality

Based on a section of AccessControl design: Group reads and edits are controlled by access control. WUI and REST interfaces covered.

1253271333000000 1254735855000000
#118 enhancement dread dread v0.10 closed fixed Use paginate in webhelpers

Take out import of paginate in setup.py. Use paginate in webhelpers instead. Make changes to take account of any i/f changes.

1253273657000000 1253784902000000
#119 enhancement dread dread v0.10 closed fixed Ensure non-active packages don't show up

Ensure pending packages don't show up in search or browse

cost: 4h

1253529414000000 1253791147000000
#120 enhancement dread dread v0.10 closed fixed Security audit

Look for all places where model is accessed and check authorization is checked.

Document holes (and, as necessary, suggestions for fixes) as new tickets. Likely areas that need looking at:

  • search i/f
  • package WUI commit

Write holes are obviously much more significant to us than read holes.

1253529427000000 1254406544000000
#121 enhancement dread dread v0.10 closed fixed Add 'Group' to main menu

And associated page to browse group.

1253694827000000 1253716782000000
#122 enhancement dread dread v0.10 closed fixed Add Group authz page 1253694842000000 1253716757000000
#123 enhancement dread dread v0.10 closed fixed Ability to edit Group in WUI

Add Group editing page.

If no permissions to change group can't edit group. Also cannot view edit page.

Editable attributes: name, title, description

No preview needed

1253708041000000 1254321447000000
#124 enhancement rgrp rgrp v1.0 closed fixed Display Generic Package Attributes in WUI

Split out from ticket:43

1253709702000000 1254735558000000
#125 enhancement dread rgrp v1.0 closed duplicate Edit Generic Package Attributes in WUI

Split out from ticket:43

1253709712000000 1258377621000000
#126 enhancement dread dread v0.10 closed fixed Change package state in the WUI (delete and undelete)

As a Package Admin I want to change the state of the package. In particular I wish to delete and undelete it.

(NB: this is quite separate from "purging" objects which is the term we shall use for irrevocable removal of an object from the domain model).

  • Only Package Admins (and sysadmins) should be able to change state

Implementation Suggestions

  • 'delete' action should be renamed to 'change-state' (NB: this requires a db migration ...)
  • Have new package formalchemy form (created via inheritance?) to incorporate state attribute. Suggest this is rendered as a dropdown (and may be simple object rendering of state, i.e. do NOT need to change it to a single name such 'active').
  • This form should then be used when the user satisfies is_authorized(..., model.Action.CHANGE_STATE) instead of the usual fieldset
1253789571000000 1254740244000000
#127 enhancement dread dread v0.10 closed fixed Minor form and UI improvements
  • group form: description field for needs to be a text area
  • group form: make title and name field wider
  • package view: need to move tags and license above notes
1253798659000000 1253868048000000
#128 enhancement dread rgrp v0.11 closed fixed Add ckan_url attribute to REST JSON representation of a Package

Add ckan_url attribute to REST JSON representation of a Package pointing to the (read) url of package on CKAN.

Cost: 30m

1253866713000000 1275694573000000
#129 enhancement rgrp dread ckan-backlog closed invalid Secure db access by channelling query generation through authz module

Controllers and templates should not access db objects directly - they should do all access via authz module giving username. They are handed by a query that has already been filtered by the packages they are authorized to read.

(Additional idea to be discussed: When they request a package object, they are handed an copy of the db object - disconnected from the database - so it the db object can't be changed.)

A couple of tests can be reenabled when this is done: ckan.tests.functional.test_authz.TestUsage?.test_admin_list_deleted ckan.tests.functional.test_authz.TestUsage?.test_search_deleted

1253886136000000 1267719162000000
#130 enhancement dread rgrp v0.10 closed fixed Create a New Group via the WUI

As a

User

I want to

Create a new group via the WUI

Details

  • locate at /group/new/
  • You must be logged in to create a group
  • Group creator is automatically given role of group admin
  • Edit screen is same as edit screen ...
1254130212000000 1254735501000000
#131 enhancement dread dread v0.10 closed fixed Groups REST interface

Controlling Groups through a REST interface.

1254307959000000 1254308115000000
#132 defect rgrp dread closed fixed Security hole - read package/group list (REST)

Using REST interface you can list packages and groups without authorization being checked.

Can be fixed using more advanced query to check authz.

1254389493000000 1273254514000000
#133 defect rgrp dread closed fixed Security hole - search package/group (WUI & REST)

Using WUI or REST interface you can search packages and groups without authorization being checked.

On the REST interface you can also read all the attributes of the packages using the 'all-fields' option.

Can be fixed using more advanced query to check authz.

1254390168000000 1273253977000000
#134 defect rgrp dread v0.11 closed fixed admin interface is only available to sysadmins 1254735314000000 1255430998000000
#135 enhancement rgrp dread v0.11 closed fixed sysadmins defined in the db 1254735368000000 1265293940000000
#136 enhancement rgrp dread v0.11 closed fixed User has publicly viewable page

A user's 'home page' is at: user/<user.id> e.g. user/28394723982-03849472

Step 1:

  • Current user home page with recent edits

Step 2: Readonly

  • Number of edits
  • Number of Packages they are admin of

How do we do a nicer URL for the home page - can we use their openid login?

Follow on tickets: ticket:138 and ticket:142

1254741650000000 1255169466000000
#137 enhancement rgrp dread closed duplicate User has editable home page
  • Generic text box for markdown about the user 'About'

Model's user table reflects these:

  • 'about' attribute
1254741703000000 1254741830000000
#138 enhancement rgrp dread v1.0 closed fixed User has editable home page
  • Generic text box for markdown about the user 'About'

Model's user table reflects these:

  • 'about' attribute
  • 'created' attribute is a default=datetime.now() - so when user was created

Need edit page for this.

Only user should be able to edit it.

Follows on from ticket:136 and ticket:142

Cost: 2h

1254741789000000 1258466074000000
#139 defect rgrp rgrp v0.11 closed worksforme Ensure you cannot add same tag twice to a package

Seems some package have ended up with same tag twice: http://www.ckan.net/package/fred2dot0

Check whether this is still possible and if it is fix it ...

Cost: 2h

1254768552000000 1255188974000000
#140 enhancement rgrp ckan-backlog new News section on front page

Have a news section (suggest as a sidebar item).

News section will link to latest 3/4 blog posts on CKAN from blog.okfn.org.

Details:

  • Suggest pulling via rss or similar.
  • Will want to cache this ...

Cost: 4h?

1254902541000000 1265625159000000
#141 task dread rgrp v0.11 closed fixed Search api docs

Write up search api documentation and put it in a template that shows up at api/search/.

Cost: 2h

1254903008000000 1255007583000000
#142 defect pudo dread closed fixed Update User object with additional fields such as 'nickname'

Suggested fields:

  • id, name (username/profile name), openid, email, password, extras (JsonType?)
    • + current items: apikey, about, created)
    • current contents of name is usually openid and should migrate to openid field (could match based on starting with http://)
  • May also want a fullname field (or perhaps that can

just go in extras - may be better to have its own field as useful to search on this )

Related changes (probably separate tickets):

  • Improve user home page to include this additional information (and have it be editable)
  • Use user 'name' (username) for home page links (rather than current user/<id>)
  • Where we show lists of revisions, each revision author is now a link to the author's user page.

Related to ticket:136 and ticket:138.

1255010314000000 1289219098000000
#143 enhancement thejimmyg dread ckan-backlog assigned Most active users listed on homepage

Display league of users' recent activity on homepage.

1255010373000000 1312372381000000
#144 enhancement rgrp dread v0.11 closed duplicate Most popular packages listed on homepage

Based on number of views.

Related to ticket:143.

1255010391000000 1265284457000000
#145 enhancement rgrp dread v0.11 closed fixed Group shows its administrators

When viewing a group you see a list of the users who have admin rights to it. (Akin to owners / moderators)

1255010403000000 1265294041000000
#146 defect pudo rgrp closed worksforme Problem when an admin user logs in between commencing editing and saving
  1. Visitor starts editing a package (or creates a new package)
  2. Visitor logs in before previewing or committing
  3. Visitor is an admin (or sysadmin)
  4. Preview or commit fails

Reason (conjecture): the form we use for package editing is now dependent on whether you have admin privileges (we allow editing of state by admins). When you login the old form is now incorrect and this results in the formalchemy fieldset breaking.

1255013773000000 1291829862000000
#147 enhancement dread dread v0.11 closed duplicate Parser and loader for esw.org data 1255440695000000 1255515162000000
#148 enhancement dread dread v0.11 closed fixed Parser and loader for esw.org data
  • parse like data4nr code

Follow up tickets: ticket:149 and ticket:150

1255440696000000 1255515222000000
#149 enhancement dread dread v0.11 closed fixed Loader for esw data recognises existing packages

Follows on from ticket:148. Sister to ticket:150.

1255440751000000 1257414916000000
#150 enhancement dread dread v0.11 closed fixed Loader for esw data uses REST API

(instead of sqlalchemy interface, to facilitate general usage of the loader)

  • use ckanclient

Follows on from ticket:148 and ticket:149

1255440828000000 1256751974000000
#151 enhancement dread rgrp v0.11 closed duplicate User object should have a created attribute

User object should have a "created" attribute initialized to current datetime.

Require a db migration but o/w very simple.

Cost: 1.5h

1255589694000000 1257414545000000
#152 enhancement dread dread v0.11 closed fixed Package has editable 'Extra' fields

Package edit page has fields for extra key-value pairs. Don't use formalchemy. The form displays the existing ones and new ones as follows:

Key           Value            delete
[ country   ] [ uk, spain   ]  [x]

When presenting the form, existing pairs and 3 blank pairs are provided.

When saving the data, keys that are not changed use the existing PackageExtra object (perhaps with new value). Deleted keys put the PackageExtra state to deleted. For new keys, only create a new PackageExtra if there isn't a state=deleted one to resurrect.

Validators ensure no repeated keys.

1255615316000000 1256056193000000
#153 enhancement dread dread v0.11 closed duplicate Group's packages listed alphabetically

This is so you can easily look up whether a given package is already listed - otherwise as lists get bigger becomes difficult to see what is already there.

Suggested by Jonathan Gray

1255621515000000 1258971895000000
#154 enhancement rgrp dread v0.11 closed wontfix Package form guidance

Give guidance on what to put in 'author' and 'maintainer' fields (e.g. name? URL?)

Suggestion from Jonathan Gray

1255621572000000 1257535066000000
#155 enhancement dread dread v1.0 closed duplicate Adding multiple packages to a group

Ability to add multiple packages to a group in one go (e.g. with 'add' link which makes drop down menu appear - so can add one after another - then submit simultaneously)

Use a bit of javascript to add more dropdowns.

Suggested by Jonathan Gray

1255621779000000 1271760041000000
#156 enhancement johnbywater dread v1.0 closed fixed Whitelist for html in mark-down field

Not just links that need to be displayed. Other html should be displayed too, such as tables which can't be represented in markdown. Do a search of the existing fields to see what is needed.

1255623973000000 1271962871000000
#157 enhancement dread rgrp v0.11 closed fixed Log message not persisted through preview

Log message on package edit page is no longer persisted through a preview (no doubt due to recent form upgrades). Let's fix this.

cost: 0.5h

1255694396000000 1256062680000000
#158 enhancement dread dread v0.11 closed fixed Add 'extras' to REST API
  • Ensure you can add/change/delete extra fields from packages using the REST API.
  • Update API docs to cover this.

On Package get, ensure separation of extra attributes from other attributes.

What is the meaning of posting an incomplete dictionary (updating a package)? Option 1 - if field not present, leave field alone

  • Delete field if field value is None

Option 2 - user must supply all fields

  • Delete field if field absent

Prefer option 1. Document this.

Follows on from ticket:152

1255703259000000 1255949818000000
#159 defect dread rgrp v0.11 closed fixed Searching for tags:... resulting in lots of tags being found

Search of form: tags: ... behaves differently depending on whether there is a leading space:

  • tags: postcode - tags found correctly but no packages found
  • tags:postcode - tags incorrectly found but correct packages found

Let's fix this.

Cost: 0.5h

1256030097000000 1256060264000000
#160 enhancement dread rgrp v0.11 closed fixed Implement internal links in CKAN notes markup

Suggest form is: key:value (with optional space after ':'.

Keys would be for an object, i.e. 'package', 'tag', 'group'.

Becomes a link.

This would be implemented by implementing a 'Parser' or 'Processor' object which is run before markdown is done. Regex replace.

Cost: 4h

1256030751000000 1265312986000000
#161 defect dread rgrp v0.11 closed fixed Tag listing is incorrect

E.g. http://ckan.net/tag/read/postcodes this has no-postcode listed but when you look at that package 'postcodes' isn't a tag ...

I suspect this may be to do with search using deleted tags as I think no-postcode had postcodes as a tag at some point earlier ...

1256030925000000 1258573607000000
#162 enhancement dread rgrp v0.11 closed fixed Tags should be automatially lowere-cased if not already

Rather than raising a validation error automatically lower-case tags on submission.

Cost: 0.5h

1256048269000000 1256142462000000
#163 enhancement rgrp dread v0.11 closed fixed Regularly put CKAN RDF on Talis CC

Related to: ticket:90 Link to RDF version of CKAN data on Talis Connected Commons ticket:164 Creating RDF data from CKAN

Regularly re-upload the CKAN RDF data to Talis CC.

In the long run may wish to only re-convert packages changed since the last upload. However given relatively smaller size of full dataset this optimization is probably not yet required.

Cost: ? (1d+ depending on e.g. how easy integration with Talis CC is)

1256052587000000 1265891374000000
#164 enhancement wwaites dread v1.0 closed fixed Creating RDF data from CKAN

Related to: ticket:90 Link to RDF version of CKAN data and ticket:163 (regularly convert CKAN data to RDF).

Convert ckan data to RDF with suitable ontology and put on semantic.ckan.net.

1256052643000000 1271251422000000
#165 enhancement rgrp dread closed invalid Use 'tag:' instead of 'tags:' for wui package search

Because people might try and do "tags:gov sweden" when "tag:gov sweden" makes more sense (sweden is searched for in all fields in these cases).

But in REST i/f search allow "tag"="gov" OR "tags"="gov sweden" in parameters.

1256200526000000 1311181391000000
#166 enhancement dread dread v0.11 closed fixed Add extras editing to ckanclient

Ensure logging is working on server.

1256203581000000 1265891199000000
#167 defect dread dread v0.11 closed fixed REST interface you can't update multiple tags

Only first tag is updated.

1256209993000000 1265891178000000
#168 enhancement rgrp dread closed duplicate Show admins for a group in group view 1256291481000000 1257414795000000
#169 enhancement dread dread closed duplicate Package derivations

A 'Derived' relationship can be applied from one package to another.

e.g. sussex-demography is derived from census-2001

'Derived' relationship is:

  • directional
  • many:many
  • stateful

'derived' table columns:

  • id (primary key)
  • source_package (foreign key)
  • result_package (foreign key)
  • description (markdown text)

Further tickets:

  • WUI - package view - shows 'derives from package x' and 'derived package y' with UML-like diagram of x -> this package -> y
  • WUI - package edit form - new option to say it 'derives from' or 'has derivation' and you select the appropriate
  • REST if - expose reading and writing this property
1256304927000000 1266928708000000
#170 enhancement rgrp dread v0.11 closed fixed Paginate search results

e.g.

http://www.ckan.net/package/search?q=science&search=Search+Packages+%C2%BB

Results are capped at 20, but can't see past first page.

1256313584000000 1265891157000000
#171 enhancement rgrp rgrp datapkg-0.7 closed fixed Create consolidated config for CLI

Create consolidated set of config by merging config from .datapkgrc and command line options and use that instead of mixture of command line options and other stuff ... (likely to be less buggy!)

1256488011000000 1297210925000000
#172 enhancement rgrp rgrp v0.11 closed fixed Build ckan documentation using sphinx and upload

Use python sphinx to build documenation in ./doc and then upload it somewhere publicly accessible.

NB: improving the documentation is another matter (as is integrating e.g. existing api docs).

Upload location (these are docs for CKAN codebase/concept not the ckan service at ckan.net so good not to associate it too closely with ckan.net):

1256489019000000 1257532331000000
#173 enhancement rgrp dread v0.11 closed fixed Diffing WUI

Use vdm diff functionality (see ticket:82) to implement diffing of packages between versions in WUI.

In package history you should be able to select to revisions and show the diff.

Estimated cost: 12h

1256565412000000 1265293968000000
#174 defect rgrp dread v0.11 closed fixed Packages search broken for 'only openly licensed'

the "only openly licensed packages" tick box seems to return blank page every time - as do queries for only openly licensed and only downloadable packages. The "only downloadable packages" tick box by itself seems to work fine. E.g. try searching for 'science'.

Raised by Jonathan Gray

1256650867000000 1265891093000000
#175 enhancement rgrp dread v0.11 closed fixed Creation date

'created' attribute on User, Group and Rating

datetime object: default=datetime.datetime.now()

See example on revision

Requires migrate script

1256754070000000 1265291037000000
#176 enhancement dread dread closed duplicate Package dependencies

(Related to ticket:169 - Package derivations)

A 'dependency' relationship can be applied from one package to another. It implies that a package requires the download or existence of another package which it 'depends on'. (Analogous to software package dependencies.)

e.g. london-traffic-visualisation depends on road-map

'Dependency' relationship is:

  • directional
  • many:many
  • stateful

'dependency' table columns:

  • id (primary key)
  • dependent (foreign key)
  • dependency (foreign key)

Further tickets:

  • WUI - package view - have list of dependencies (do not need to list packages which depend on this one)
  • WUI - package edit form - new option to say 'depends on' (no need for 'has dependent package')
  • REST api - expose reading and writing 'depends on' property.

Issues

  • How do we deal with dependency at a particular version?
1257162812000000 1266928721000000
#177 enhancement rgrp dread closed invalid Service documentation

This is docs for users of a CKAN service (e.g. ckan.net), as opposed to someone installing / administering the software. (The latter has docs on knowledgeforge.

Editability - wiki?

1257243285000000 1273050236000000
#178 enhancement rgrp dread v0.11 closed fixed Excel package importer

/tool New 'tools' section of the WUI with 'Excel package import' page.

/tool/import Import page allows you to upload a file. The format of the required Excel file is described here. List of licenses provided for copy&paste.

/tool/import/preview When you upload it, it gives you a simple preview of the file, offering import. Text as per Package Preview. 'import button' for completing process.

File format:

First row to have a cell value of 'name' or 'title' in it is the Title Row. It contains the field names, as per package. It includes:

  • license - name of license
  • tags - space separated
  • groups - space separated
  • anything - for extra key-values

When importing, values are bound into the package fieldset, validated and displayed for the preview and synced to complete.

User has to log in to use importer. Auth is checked for putting into groups.

1257244034000000 1265305558000000
#179 defect dread rgrp v1.0 closed fixed Restore 404 and 500 messages in WUI

Cost: 0.5h (?)

Conjecture this went missing in cset:a35db862a841

1257412668000000 1265305549000000
#180 enhancement rgrp jwyg v0.11 closed duplicate Tag cloud as way to view CKAN tags

Create big tag cloud with all CKAN tags - perhaps weighting with size and colour...

1257534254000000 1265284374000000
#181 enhancement rgrp jwyg closed invalid New 'stats' box on right hand sidebar

With up to date statistics such as:

  • total packages
  • total tags
  • total users
  • most active users
  • last update
  • etc.

Could also be nice to have a more dedicated stats page - with information such as graphs showing rate of packages being added and such like. See ticket:184

1257534385000000 1296339510000000
#182 enhancement rgrp v1.0 closed fixed Improve openness icons

Tick and cross aren't clear enough. Propose:

  • Icon for downloadability (nothing if absent ...(?))
  • Icon for open license? (Need something if closed ...)

Cost: 2h (?)

1257534590000000 1270567116000000
#183 enhancement rgrp rgrp closed worksforme Browse packages by rating

At moment order packages by title.

1257534606000000 1290604779000000
#184 enhancement dread rgrp closed fixed Stats page

Create /stats/ page displaying main statistics, e.g:

  • Most highly rated packages
  • Most edited packages
  • Largest groups
  • Top tags (by packages)
  • Package addition rate
  • Users with most packages

Related to ticket:181 - Stats side-bar

Details

1257534756000000 1266837414000000
#185 enhancement rgrp dread closed invalid Package form field hiding

Make parts of the package form hidden by default, to make it look simpler. Users are invited to click a button to reveal the less important fields.

Fields hidden by default: version, author, author_email, maintainer, maintainer_email

1257763437000000 1291830039000000
#186 enhancement rgrp rgrp closed duplicate Automated upload to archive.org s3

(Follows on from ticket:107). We want to provide facility for users to automatically upload material.

1257803430000000 1296341182000000
#187 enhancement rgrp rgrp v0.11 closed fixed Full-text search

Standard search should search notes field in addition to name, title and tags (discussed in ticket:108 but not done). For this to work we need proper text search since o/w we get poor ordering and lots of bad results.

If we do this we need:

  1. To weight across fields in a sensible way
  2. We can also use proper text search on title or ...

Easiest way to do this is to use existing facilities in dbs e.g. postgres has full text support since 8.3: http://www.postgresql.org/docs/8.3/static/textsearch.html

Using this with sqlalchemy: http://lowmanio.co.uk/blog/entries/postgresql-full-text-search-and-sqlalchemy/

Issues with fulltext search:

  • tags not indexed, so would need to 'or' search of tags. This would cause problems with the order_by of the query, since the tags wouldn't have a ranking.
  • if tags are indexed then perhaps we don't want them converted into lexemes? Exact match could well be better.
  • can we split the name on dash or underscore before being indexed?
  • natural language search doesn't do partial words, so search for 'gov' doesn't bring up 'government'.
  • do we keep the existing search system usable with a config file switch for if we install on a db aside from postgres?
  • we want to weight name and title higher than other fields - achievable with custom trigger.
1257869950000000 1265892403000000
#188 enhancement rgrp rgrp v0.11 closed duplicate Improve package listing views

Propose change to tabular-like format showing these attributes (perhaps should be configurable?)

  • Openness status
  • Title (not sure name is needed)
  • Tags

Cost: 4h

1257870031000000 1265294090000000
#189 enhancement dread dread v0.11 closed fixed Multiple download urls

See also: PackageResources which defines a 'distribution' as a package's payload (perhaps with metadata). In our case, at least within CKAN, the representation of the distribution will just be the reference to get it (plus some minor metadata). We are expanding this functionality to be any kind of 'resource' associated with a package since it is likely in some cases people will want to link to a file containing a part, but not the whole, of a package's data.

As a

User (package editor and package user)

I want to

(package editor) Associate multiple distributions with a package.

(package user) Easily have access to multiple distributions of a package should they exist.

Implementation

There is an object called 'Resource' with the following attributes:

  • url (text) (required in UI, but not in db table)
  • format (text)
  • description (text)
  • package (association) - a resource is always associated with 1 package

A package's list of resources will have an order which is preserved. The first in the list will be known as the 'default resource' (the presumption will always be that this is a 'distribution').

How to replace download url

download_url removed and replaced as a proxy attribute. Read returns the url of the default distribution. Write changes the url of the default resource. Ensures backward compatibility.

REST API has new attribute in package dictionary 'resouces' - an ordered list.

In WUI, replace download url in package read with a table of the distributions.

In the WUI package edit, be able to add/remove/edit/reorder distributions.

Example

Example part of package:

Distributions:

http://some.com/pollution.pdf PDF Original report
http://some.com/pollution.csv csv Original data
http://some.com/pollution sparql Linked version of data

Existing data to be migrated to new structure.

Cost: 4 days

1258386120000000 1265284961000000
#190 defect pudo dread closed fixed Package comments

Cost 7 days

When viewing a package, users can read user comments and leave their own. Users need to be logged in to leave a message. Comments appear immediately. A mechanism for deleting unwanted comments is provided to an authorized user. Comments are sorted with the most recent first. Comments are available for read, creation and deletion in both the Web UI and over the REST API.

The admin for the package and a superuser can delete unwanted comments, both on the package page and a collation of all comments on their user page. Users can delete their own comments(?) Need to consider whether over the REST API we encourage the use of a 'frontend user' APIKEY which can be used to leave comments for another, actual user.

Example at bottom of package page:

Leave a comment:

Subject _
Comment _


Submit button

Comments:

Explanation doc
Posted on 25h May 2009 by http://bertdavies.myopenid.com
It says on the pollution data web page that not all the stations have a CO2 sensor, so you have to extrapolate from the ones that do. See my visualisation of CO2 across London for an idea of what you can do: bertdavies.com/pollution-2008.jpg
More info
Posted on 24th May 2009 by http://ronsmith.myopenid.com
Excellent data, but why is there no value in the CO2 column for some of the testing stations?

Implementation details:

Comments table is with columns:

id package_id date (date) comment (multi-line text)
1258387105000000 1280820852000000
#191 enhancement johnglover dread ckan-sprint-2011-12-19 closed fixed Searching by modification date

Cost - 2 days

Search interface has new options to filter and sort the results by the date the package has been last modified in ckan. Search options are included in both Web UI and Search API.

The filter specifies a range of dates. The results can be sorted by ascending or descending dates. The last modification date is surfaced in the package.

Example search parameters:

modification-range=5/4/09- Exclude packages last modified earlier than 5/4/09
modification-range=5/4/09-5/12/09 Exclude packages last modified outside of 5/4/09-5/12/09
order_by=mod Sort by metadata modification. Defaults to newest first.
order_by=mod-newest Sort by metadata modification, newest first.
order_by=mod-oldest Sort by metadata modification, oldest first.
1258387778000000 1330020983000000
#192 enhancement rgrp dread closed fixed Time-related package field

Cost - 2 days

The time period to which a package's data applies is stored in a new field called 'Relevant Date Range'. The value of this field is up to two calendar dates: i.e. either a point in time (e.g. date of a river map is 5/10/09) or a time period (e.g. pollution measurements 1/1/09-1/4/09).

Example part of package:

Date: 5/10/09
or
Date: 1/1/09 – 1/4/09

1258388163000000 1291733895000000
#193 enhancement rgrp dread closed wontfix Searching by time-related field

Cost - 2 days

Search interface has new options to filter and sort the results by the time-related field of the package. Search options are included in both Web UI and Search API.

The filter specifies a range of dates. The results can be sorted by ascending or descending dates. The last modification date is surfaced in the package. Need to decide for a time-related field value that is date range, what date is used for the search.

Example search parameters:

reldate-range=5/4/09- Exclude packages related to earlier than 5/4/09
reldate-range=5/4/09-5/12/09 Exclude packages related to date outside of 5/4/09-5/12/09
order_by=reldate Sort by date package is related to. Defaults to newest first.
order_by=reldate-newest Sort by date package is related to, newest first.
order_by=reldate-oldest Sort by date package is related to, oldest first.

Related to ticket:192

1258388169000000 1340626463000000
#194 defect rgrp dread v0.11 closed fixed Check star ratings aren't influenced by search engine crawlers

rel=nofollow or robots.txt ?

1258471512000000 1265284389000000
#195 enhancement rgrp rgrp v0.11 closed fixed Association listings should be alphabetically sorted (e.g. a group's packages)

Where an item has an association listing (e.g. package's tags or group's packages) these should be sorted in a sensible way. For present choose to sort alphabetically by name.

Cost: 2h

1258475448000000 1258585500000000
#196 defect dread dread v1.0 closed fixed RDF URI to resolve on CKAN

Provide RDF version of a package at the RDF URI: http://ckan.net/package/rdf/32000-naples-florida-businesses-kml

See email on OKFN help:

For the information to be Linked Data, the following URI (from the api.talis.com SPARQL call) should be resolvable to RDF, or an RDFa enhanced HTML page.

1258727988000000 1265389771000000
#197 enhancement dread dread v0.11 closed fixed Simple dumper

As a

admin / user

I want to

Admin: export all HEAD data to JSON / CSV / RDF format dump.

User: wants to get hold of this dump.

Details

  • Just do JSON & CSV for now.
  • Put this in CLI.
  • Just do list of packages - as_dict info. CSV needs a bit of flattening.

Cost: 2h

1258972495000000 1258989700000000
#198 enhancement rgrp dread closed fixed Change package and tag ids to uuids

See how we did it already for other things.

Note: on ckan.net older PackageRevision?.id might not be identical to Package.id but this may need sorting at this point.

1258980613000000 1266837606000000
#199 defect rgrp casbon closed fixed Fix failing tests and provide documentation on how to run tests in datapkg

As of 80:099bdc5b07a6 datapkg has 4 test fails and 1 test error.

There is no documentation on how to run them, so I am using 'python setup.py test'.

datapkg.tests.test_cli.TestCLI.test_ckan ... FAIL
datapkg.tests.test_cli.TestCLI.test_walkthrough ... FAIL
datapkg.tests.test_index.TestDbIndex.test_get ... /Users/james/Documents/virtualenvs/okfn/src/datapkg/datapkg/index.py:97: SADeprecationWarning: Use session.add()
  self.session.update(pkg)
FAIL
datapkg.tests.test_index.TestDbIndex.test_get_when_loaded_as_new_and_init_not_called ... /Users/james/Documents/virtualenvs/okfn/src/datapkg/datapkg/tests/test_index.py:57: SADeprecationWarning: Use session.expunge_all()
  self.index.session.clear()
FAIL
1259087361000000 1267649255000000
#200 defect rgrp rgrp closed invalid PythonDistribution write method should write all available metadata

Current write method just creates a bare bones package and does not write any of the available metadata other than name.

1259229583000000 1311176118000000
1 2 3 4 5 6 7 8 9 10 11 12
Note: See TracReports for help on using and creating reports.