{22} Trac tickets (2647 matches)

Schedule worker calls to be ran several times, with certain intervals.

Runs on the commandline and prints out notifications.

Run CKAN with the publisher profile and organizations plugins enabled. Make an organization. Make a dataset and add it to the organization. Edit the organization and change e.g. its description. After saving the organization, all its datasets have been removed from the organization and now belong to no organization.

Roles on System object are important because admin role on system equates to being a 'sysadmin' (i.e. able to do anything).

• Make users sysadmin (either as separate action or as part of editing roles on system object)
• /authz subpage for editing roles on system object
  • Add and update user roles
  • Add and update authz group roles
  • List actions associated to roles at top of page (extra points for checkbox table with editability)
• Document on ​http://wiki.ckan.net/Authorization what roles on System object 'mean' esp sysadmin role on System

Related Tickets

• super ticket: #833
• authz lib improvement and authztool refactor #1050

Right now, the two logic functions that return a list of packages (package_search [1] and group_packages_list [2])use custom functions to generate the output dict. That's suboptimal because:

• The template function that renders the package listing (package_list_from_dict) could end up receiving different dicts.
• Not all package properties are available at the template level (e.g extras, tags...). These can be really useful to customize the listing from an extension.

In general only the functions present in lib/dictization/model_dictize.py should be used to build the output of a logic function, in that case package_dictize. If necessary, they can be modified to include missing properties, like on this particular case the "isopen" property, needed by the template renderer.

[1] ​https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L685 [2] ​https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L442

Right now you need to explicitly load the synchronous_search plugin in your ini file, when this is probably the behaviour that all users expect by default. We could keep a config flag to deactivate it, but synchronous search should be the default behaviour.

Right now is quite difficult to get the most current version of a package via the harvest objects, and the queries involved are prone to errors. A new "current" field will be added which will identify unequivocally the most recent harvested object. Unfortunately this will involve some major changes:

• Model update (See #1726)
• Update queries for getting harvest sources' datasets
• Update ckanext-csw queries for getting the objects
• Update inspire and ckan harvesters
• Document new field

Right now harvest sources can be inactivated (only via the cli), but we need to make sure:

• Expose this on the web interface (only for sysadmins, see #1731)
• Inactive sources are not shown in the web interface

Also for deleted packages:

a) Deleted packages get state="deleted" (not actually purged) so they are hidden on front end ACTION: need a button on the front end for "Delete" for sysamdins

Done in #1425. Not sure if the button is only shown to sysadmins.

b) No undelete: If you reharvest a deleted package, it remains deleted

• Publisher should create a new record with new identifier and same/different content as applicable.
• If publisher reharvests, a deleted document they get a error and are told to create new identifier for their updated.
• You won't get the state where a package is updated in its deleted state
• If they change the ID and reharvest, a new record appears on DGU.

c) Update CSW interface so that export doesn't export deleted packages d) Check geospatial search doesn't search deleted pacakges e) Check deleted packages don't appear in harvest source details, or in the counts or anywhere in the harvest interface or API.

Revisions aren't assigned to packages. Where are they coming from? This seems to have been occurring since the upgrade to 1.4.3b a couple of weeks ago?

Revision.packages should not include all changes so why are we getting revisions with no associated changes? See e.g. ​http://ckan.net/revision/read/814e39e1-2540-4449-90d1-ac9a92ab40fb?

Investigate and fix.

NB: if this is bugfix this should go against stable or metastable branch if at all possible and then be merged into default.

Cost: 1.5h (?)

Revision history information is not accessible, dgu want this.

Review what chunks of code we have and how to move forward with requirements.

Requirements:

• Bulk parsing is done away from the CKAN server

Possible future requirements:

• /importer for importing spreadsheets in WUI
• validation-light - 'scoring' a package

Current importers:

• /importer controller uses: PackageImporter?, form, PackageSaver?
• loader uses: package_importer -> API
• API uses: form, PackageSaver?

Modules

• package_importer:
  • xls/csv --SpreadsheetData?--> table
  • table --DataRecords?--> records
  • records --PackageImporter?--> pkg_dicts

• pkg_dict --form--> fs & validation
• fs --PackageSaver?--> package

Reverse engineer boto and work out how to get headers to support upload to google storage without holding api keys.

This would lead to an extension to OFS.

This analysis should inform (and go hand-in-hand) with the implementation of ticket:879 (Storage Auth API in CKAN).

• master ticket #852

Resource and Tag search is currently still using the old ckan query parser which is unnecessary. Refactor this out and move to the logic layer.

Requires us to url encode the tag names when displaying them ...

Requires 0.4. 0.5 is a maintenance branch, 0.6 is current.

VDM appears to work correctly (all the tests pass) with 0.5. CKAN does not -- requires more investigation to determine exactly why.

It would actually be nice to be able to run with 0.6 or 0.5, though 0.6 will require some changes to VDM as well

Requirements:

• cater naturally for multiple packages with multiple resources
• international encoding
• large number of packages

Requested by DGU.

Requested by DGU.

Repository History revision

• What is this page called? "Recent Changes", "Repository History", "/revision": standardise. Between the link in the nav and the page <title> particularly, but the route is also important. Perhaps /changes or something similar?
  • Will change page title on /revision/ to Revision History. Will not change route for the time being.
• needs copyediting.
• Pagination has similar issues to elsewhere. Also, most obvious here, is the fact that we don't need to display a link to every possible page. Please can we limit it to, say, a dozen nearby pages and an ellipsis.
• Without looking at dates, its not clear whether I'm seeing most recent or oldest changes. Change pagination to say "Older"/"More recent" rather than "Previous"/"Next". (wontfix: now have text saying we are showing most recent changes)
• Table layout is pretty ugly (yes, I'm aware this is my fault).
  • wontfix - nothing better at the moment
• Timestamps are horribly unreadable. At the absolute minimum get rid of the micros. Hover for more detail?
• Can we link to an author page? Yes!
• Atom feed should have a feed icon~~
• Why are we adding another <link rel="alternate"> to this page? We already have one for recent changes on every page in the site and the one we're adding has a less descriptive title. Which is correct? Use that one as a feed for every page.

Reported in IRC by @floapps

In previous ckan versions i could use paster commands to remove reader rights from visitors on all packages and this would then throw the user to login screen when trying to see a dataset but, when i did this in ckan 1.7, i could still see all datasets as a visitor

Using ​http://docs.ckan.org/en/latest/authorization.html#permissions-publisher-mode

paster rights remove visitor reader package:all

Also has an empty list of roles in config for default visitor roles

I tested on my local install and could replicate with that command doing nothing. Neither affects existing or newly created datasets.

Report on errors by any worker daemons, send them out via E-Mail

Replace the /revision page with a /activity page showing a site-wide activity stream of all activities in the site. Or perhaps the site-wide activity stream can go somewhere on the front page of the site instead?

Remove the current dependency on Yahoo YUI toolkit for doing tag autocompletion on package edit forms.

The (KForge-derived) behaviour files for CKAN can easily be switched to the new jQuery-based ones: see ticket:212. Once this is done, it would make sense to implement the tag-selection code with jQuery.

Relevant pages include:

• ​http://bassistance.de/jquery-plugins/jquery-plugin-autocomplete/
• e.g. ​http://ckan.net/package/edit/acawiki

Related to: ticket:90 Link to RDF version of CKAN data and ticket:163 (regularly convert CKAN data to RDF).

Convert ckan data to RDF with suitable ontology and put on semantic.ckan.net.

Regression found by updating datacatalogs.

Refactor harvesting code to simplify.

Recline views should have a default table order, being the same as the csv that was imported.

Recline allows negative ranges. The backend returns an error 500 for that which breaks the recline preview.

Reasons: more people familiar with Git, bigger Open Source community on Github.

Keeping track of the move here: ​https://docs.google.com/spreadsheet/ccc?key=0AvCDqUH8jVN8dGNVNEpXLWk3UTlJZXhFMjBPbXVtRmc&hl=en_GB#gid=0

Read/write APIs for users and publishing entities to access and register data

• Users have read access
• Publishing entities have both (though can only write to 'their' records)
• Priority: 4 for write (read access is 3)

Ratings were disabled approximately a year ago because:

• Unclear purpose and UX. What did ratings tell you? How useful were they?
• Spamming (esp by bots: you could submit an anonymous rating via a GET request which caused problems)

Both problems are solvable and it would be nice to have this feature reinstated.

• Purpose: can make this more purposable by limiting to logged in users (or at least distinguishing logged in from non-logged in users)
  • Even better we could allow ratings to be made public (I'm interested in what someone else I respect finds important)
• Spamming: limit to logged in users and / or use AJAX over an API to submit ...

Rather than raising a validation error automatically lower-case tags on submission.

Cost: 0.5h

Rather than have a separate 'extras' key all the extras fields should be consolidated into the main package dict when presenting the package internally or e.g. via the API.

Why? Extras are really just an artefact of our internal storage model. Clients of the system (both internal and external) should just see a set of key/values with no distinction between extras and non-extras.

Issues

• Possible breaking change to the API (could enforce backwards compatibility by keepings extras for the time being)

Possible subticket of forms refactoring: #961

RDFa can be used as a simple way to expose linked data or at least sameAs the API version of the data but we need to make sure we do not expose anything different from the "official" representation.

Pylons 1.0 has been out for a while and appears to be running stable on a few production sites. It also introduces a few new features, among them the ability to specifiy controllers using the entrypoint mechanism.

​http://bitbucket.org/pudo/ckanextdeliverance/src/tip/ckanext/deliverance/pylonspatch.py

This might be a bit ugly since the default behaviour of some pylons objects, such as tmpl_context, app_globals and redirect has changed.

Purging datasets (deleting them fully, not just changing the state to 'deleted') is important for users testing dataset creation over the API on a test CKAN instance.

Without this, they need to resort to more difficult methods such as:

• cleaning and reloading the database
• setting the test datasets to state 'deleted' and also appending a suffix '_00' and incrementing the number until there is no clash of names.

Requested for NHSIC.

Implementation

• This could slot into the Action API.
• Of course we would need to ensure the user's had been given the specific right to purge.
• I suggest we log the full details of the dataset being purged.

Provide way to mark a dataset as featured. Featured database show up on the front page.

TODO: detail this more.

Provide help to users in the WUI, with this linking to a detailed user guide.

• User guide (suggest at help.thedatahub.org (or possibly help.ckan.org)
• Snippets in the WUI with links to full user guide.
  • Provide helper methods for creating these links with config options for setting path to user guide so that the link location can vary on install by install basis
• General info about publishing/working with/etc data (where we don't duplicate elsewhere)

Est: (?) 5-10d

Topics

See ​http://ckan.okfnpad.org/datahub-help

• Step-by-step for adding a dataset including tips
  • Reuse: ​http://datapatterns.org/datahub.html
  • Quick overview of relevant parts of domain model (dataset and resource)
• API (perhaps link to main docs)
• Command line interface and dpm
• Using groups

Why do this

We already have wiki.ckan.org and docs.ckan.org. Why do this?

Answer for docs:

• This is administrative guide and focused on general CKAN software.
• Hard to provide completely generic help.
• Integratable with the DataHub? site

Answer for wiki:

• No spam
• General feeling that wikis are a little bit loose and untidy. We want this to be smart, clean and useful.
• Integratable with the DataHub?

Provide configuration for reverse proxying that will correctly handle the mapping of a URL to a sub-folder (using X-SCRIPT-NAME)

• Analysis of the best solution [1d]

• Implement reverse proxying in tandem with #1653 [2d]

• Document and store the configuration files. [1d]

Provide a new test install of DataGM based on the ODS demo site being completed through June.

Propose change to tabular-like format showing these attributes (perhaps should be configurable?)

• Openness status
• Title (not sure name is needed)
• Tags

Cost: 4h

Produce a video explaining some of the things which can be done on the datahub.org

Problem: When creating a related item it mentions that you can use markdown in the description, however markdown doesn't render in apps page nor in related items page. Expected: Make shortened descriptions (on apps page & related items page) render markdown

Problem: Server Error (HTTP Code 500) when trying to see information related to an RSS feed... example: ​http://s031.okserver.org:2375/en/dataset/testerisation/resource/974df29e-77bb-4b5e-982c-cb7a295c1923 Expected: A page that looks a bit more like this: ​http://s031.okserver.org:2375/dataset/epims-lod2/resource/97531bd8-e75c-4efa-bfa2-bb51d0ba90d0

Problem: On /apps page if you start in one language, filter the data, and then change languages it loses the correct format for the HTTP Get Parameters (and therefore loses the filters). Example, start in English language, change filter settings to to Application and Newest, change language to German. Expected: Retain correct format for HTTP Get Parameters so that the filter settings are retained. Solution will involve not turning "=" into "%3D" in the URL, i.e. turn off URL encoding.

Problem: Example group ( ​http://s031.okserver.org:2375/en/group/example-group ) has a broken image, image not found. Link either incorrect, or no image set, if no image set then shouldn't it show a default image? Expected: There should be an image.

Probably use ansible to do this. To create an instance, create a dir at /etc/ckan/MYSITE, and put MYSITE.wsgi, MYSITE.ini and who.ini files in it. Also put a MYSITE file in /etc/apache2/sites-available. See the example files already present in ckanbuild. Booting a new site should be a single command.

May not handle the postgres/solr/elastic-search side of things yet, could just require the user to set these up herself first and then pass them as args to the create-instance command.

Previously, a package type attribute was intended to distinguish between package data that came from government depts. and publishers location information (GEMINI). Now there is an intention use a common set of attributes to support data from either source.

Hence, there currently a v.3 of the government attributes, a document about extracting the attributes from Gemini document using XPath, and a comparison documents mapping between governemtn attributes and Gemini attributes.

That means there are potentially four different list of attributes. I'm not sure how to go about reviewing the lists and converging any differences.

But we do want to end up with a list of names, a description of their purpose, and then perhaps the XPath which would reveal the value from a Gemini doc?

Pressing the 'make an enquiry' link on the package read page gives 404. (Also links should also be changed from isitopen.ckan.net to isitopendata.org)

Example link is:

​http://isitopen.ckan.net/enquiry/create/?ckan_package=zeno

which gets forwarded to:

​http://www.isitopendata.org/enquiry/create/?ckan_package=zeno

which gives 404