{22} Trac tickets (2647 matches)

Super #1669

As groups need to be authorised in some cases, particularly publisher profiles, we need a field to denote how this is managed.

We should add an 'approval_status' field that will be used to denote its status. By default this will be "approved" but this will allow us to override it with a "pending" or "denied" status. The latter should also change the main status of the Group.

Super #1669

Authz for the publisher admin (who has permission to administer the publisher group themselves).

Right now you need to explicitly load the synchronous_search plugin in your ini file, when this is probably the behaviour that all users expect by default. We could keep a config flag to deactivate it, but synchronous search should be the default behaviour.

The 'editor', 'anon_editor' and 'reader' roles are intended to have immutable actions. This was designed to prevent their names being subverted - e.g. an editor should always be able to edit! It also meant that when we add Actions (e.g. DELETE-PACKAGE) then it can be added sensibly to these roles in an upgrade just by changing the defaults table (ckan/model/authz.py).

The problem is that this immutability is only enforced on 'db upgrade'. So you can happily change the editor role using the paster command and it works, right up until you do an upgrade and realise permissions are different.

We should stop the paster commands being able to edit these roles. Or get rid of the immutability completely. Views?

Document all of the changes and usage for #1574 so that we can mark it 'done done'.

Make sure we document any setup issues/processes and the settings.py setup required for CKAN so that Mark can announce next week and it should be easily deployable/testable.

Internal (perhaps in README somewhere or wiki): how does process work, where does data end up and in what order etc

External: here's how to configure it and here's what you get ... (goes in main sphinx docs)

ckanext-archiver currently has a settings file (and a default) and it should be passed in all relevant information from the context.

Remove all settings (ARCHIVE_DIR and MAX_CONTENT_LENGTH and others) and pass them in from CKAN.

We need an API for changing the authorization (permissions) on a model object.

This was neglected in #1253 because of various changes to authz were looming. But six months later it still hasn't been done.

Suggest this is just moving _add_user_object_role from ckan/lib/base.py to the logic layer, but no doubt it will be more complicated.

Estimate: 3 days

Missed the ckan prefix on the storage settings names so this needs to be fixed.

Add image attribute for Dataset and Group which would be shown in prominent place.

For dataset this could be a relevant image and for Group this would be more of a logo.

I think this has very high value from a UX point of view.

Issues

• Do we need to support uploading and storing the image (or do users store elsewhere)
  • Could use something like transloadit service
  • Use our own storage facility with a reserved directory
• Do we need to do resizing etc?
  • Maybe. Perhaps can prefer this in reasonably small size and then just size down in css for thumbnails etc.

Suggested by Richard Cyganiak in ​http://lists.okfn.org/pipermail/ckan-discuss/2012-January/001977.html

• new relationship type links_to / linked_from
  • Maybe with extra 'count' property?
• creating a new relationship should be post to the relationship register, not the source entity.
• add missing doc about creating a new relationship
• 'Model Formats' section seems to be missed out of the docs since release 1.4.2. See: ​http://readthedocs.org/docs/ckan/en/ckan-1.4.2/api/version2.html#model-formats

Add drupal-like "taxonomies" to CKAN.

Etherpad with user stories, feature list, design and implementation discussion:

​http://ckan.okfnpad.org/22

Branch where this is being developed is feature-1698-tag-taxonomies:

​https://github.com/okfn/ckan/compare/master...feature-1698-tag-taxonomies

Tickets related to this have ​keyword taxonomies.

Make sure accented characters are normalized when indexed and when searched for.

The prototype portal gave us a list of classifiers that are expected for the initial launch. Add then to ckanext-ecportal.

Add Vocabulary domain class, add logic functions for creating, updating, listing, getting, deleting vocabularies, add tests.

We want to make email announcements to users of thedatahub.org.

The most simple way to do this is to get a dump of the email addresses and subscribe/invite them to a mailman list.

Different ways to do it:

• 'paster dump-users-csv' creates a CSV file. Apache serves up the directory using password protection. Is this secure enough?
• CKAN admin pages provides a button to produce the CSV and download it. A config option may be necessary to enable this (disabled by default).

Something more advanced in the future could be:

• CKAN tells mailman about new subscribers using the mailman web interface. Could be brittle.
• Mailman doesn't have an Internet API, but it does have a python interface. So CKAN could tell mailman about new subscribers using a python proxy. There may be one out there, or maybe easy to do. But annoying to have to maintain another process running on machine.
• CKAN sends out the emails itself. A sysadmin is given a form. Users have an opt in/out option in their account. But receiving emails is a sensitive thing so it would need to be right first time, and there are several risks to this approach.

CKAN provides the /data page page of data.gov.uk

Breakdown of tasks:

• [x] Analysis / refinement of spec.
• [x] Log-in / register as publisher

(Waiting on publisher form)

• [x] Population of "browse by publisher"

list groups ordered by most datasets (Waiting on publisher integration)

• [ ] Browse by nation
• [ ] Featured datasets

Now a possible integration point with drupal

• [X] Tag cloud
• [X] Developers section

Dataset read page for DGU

Breakdown:

• [X] Fix missing title
• [X] Links to the social media stuff
  • "Share your app"
  • "Share your idea"
  • "Request new data"
• [ ] Presentation of additional information should be broken into sections

Content is there. Don't do anything until theming/design work is undertaken.

• [ ] "More like this": links to similar datasets.

Dropping

• [N] Stars (I think these are being removed)
• [ ] Comments

Possible integration point with Drupal. Needs it own ticket.

• [X] Give feedback to department
• [X] Tidy up the QA stars

To allow navigation between sections to still work, also potentially rephrase to avoid confusion that you have to save every time to add a tag or resource and can't say add description, tag and resource in one go

Part of #1698

See the etherpad for notes: ​http://ckan.okfnpad.org/22

ckan/logic/action/get.py:tag_list() Currently (I think) just returns all tags in the CKAN instance that the user has access to. Needs an optional argument to get only tags from a particular vocabulary? When called with no argument, should only return free tags? Also add an optional dataset argument, so you can get tags for a given dataset and/or vocabulary?

ckan/logic/action/get.py:tag_autocomplete() Add optional argument to get only tags for a given vocab? When called with no argument, should get only free tags?

ckan/logic/action/get.py:tag_search() Again, optional arguments for restricting by vocabulary and/or dataset.

Right now is quite difficult to get the most current version of a package via the harvest objects, and the queries involved are prone to errors. A new "current" field will be added which will identify unequivocally the most recent harvested object. Unfortunately this will involve some major changes:

• Model update (See #1726)
• Update queries for getting harvest sources' datasets
• Update ckanext-csw queries for getting the objects
• Update inspire and ckan harvesters
• Document new field

Add a helper function to make it easy for extensions to add new vocabularies to default_package_schema(). The helper function should take the name or ID of a vocabulary (which should already exist in the db) and add the necessary key: [schema] entry to default_package_schema(), with the necessary validation, authorisation, transformation.

Estimate: 4-5d

Only sysadmins can manage harvest sources right now, we need to allow publishers to handle their own sources.

We need to decide if there will be two different profiles on ckanext-harvest, the current one with only sysadmins allowed to manage the harvesting and a new one supporting publishers (ckan core can be configured to not use the publisher model)

Tasks include:

• Auth checks on the different controller actions (and the command line interface?)
• Modify the index page to just show sources the logged user has permissions on.

(Talk with Ross about the specific functions to use)

• Create/edit form to show a dropdown with the user available publishers.

Right now harvest sources can be inactivated (only via the cli), but we need to make sure:

• Expose this on the web interface (only for sysadmins, see #1731)
• Inactive sources are not shown in the web interface

Also for deleted packages:

a) Deleted packages get state="deleted" (not actually purged) so they are hidden on front end ACTION: need a button on the front end for "Delete" for sysamdins

Done in #1425. Not sure if the button is only shown to sysadmins.

b) No undelete: If you reharvest a deleted package, it remains deleted

• Publisher should create a new record with new identifier and same/different content as applicable.
• If publisher reharvests, a deleted document they get a error and are told to create new identifier for their updated.
• You won't get the state where a package is updated in its deleted state
• If they change the ID and reharvest, a new record appears on DGU.

c) Update CSW interface so that export doesn't export deleted packages d) Check geospatial search doesn't search deleted pacakges e) Check deleted packages don't appear in harvest source details, or in the counts or anywhere in the harvest interface or API.

Super ticket #1744

If you edit a dataset, simply add a tag and click on "Save" then you get the message "You have unsaved changed. Make sure to click 'Save Changes' below before leaving this page." inserted whilst the submission takes place. (This is because the click moves focus away from the tag field). This (reasonably common) edge-case could usefully be squashed.

Also note typo: "unsaved changed"

Super ticket: #1745

• Convert /api/rest/dataset to be search query (i.e. take arguments in ?....)
• Directly expose solr though may want to override limit. See ​https://github.com/okfn/openspending/blob/master/openspending/ui/controllers/api.py#L48

Required for some improvements to UX (such as autocomplete and better search).

Test failures in master due to tom and rufus not running tests. 0.3d.

Need to add extension point to change the data_dict that passed to the dataset view template. This extensions modify what data gets shown.

Tasks include:

• Fix up data_dict and templates so that the pkg objects do not get used in the templates. This is to assure that all data passed is modifiable. (2d) (complete)
• Add extension point to Ipackage controller (1d)
• Test (1d)

Super ticket: #1506

• Show edit summary form again, at least on edit pages (and prefill it for create to something sensible like: Creating dataset {title} {name}) [Done #1506]
• Fix drop down for resource types in resource add/edit to limit to fixed set of options (as per ​http://wiki.ckan.org/Domain_Model/Resource)
  • See also ​http://lists.okfn.org/pipermail/ckan-discuss/2011-December/001860.html
• Allow reordering of resources (or: and this is somewhat more complex but possibly simpler: allow editing of a resource_order field and we use that to sort resources (this is more like wordpress page_order -- these two options could become the same if we just hijack the existing position field on the join table and get rid of ordereredlist extension)) [DONE #1506]
• Radically simplify set of licenses [DONE #1359]
• "Unsaved changes" warning briefly when you click "Save" #1735
• "Unsaved changed" typo on dataset edit page

Currently user, package and group activity streams only return the most recent 15 activities, even though all activities are kept in the db. Do we want to add pagination - to both the API and the HTML pages - to support retrieving older activities?

At the very least we should make our JSON output conform to their spec:

​http://activitystrea.ms/

Move ckan/lib/activity.py moved to into the model - say ckan/model/activity_extension.py, because it's so tightly knit with the model code, whereas most of the lib code is used in the controllers.

• [x] hide black stars on the data-view page
• [x] Edit on view form (if you have the right permission)
• [x] RES_FORMAT ~> Resource Format
• [x] Map Based Search -> UK Location Map Based Search
• [x] Note at the top right expalining that these are seach filters (if they are) -> this requires "add a dataset" and "other access" to be moved to a different place (just some whitespace is fine)
• [x] Display publisher and provider (UKLP records only).
• [x] analysis of filter by OGL / non-OGL
• [x] provide /dump on the build server

When an extra is added to or removed from a dataset or is updated, an activity should go in the dataset's and user's activity streams.

The publisher index page should show a treeview of the publishers within the system. Probably behind a tab.

Ensure publisher edit page only allows admins to manage users. #1675 is related.

Get access to live Drupal MySQL database to get latest publisher list for migration.

Write a one-off script for import of the data.

A new form for create publisher -> Add publisher access to your account' New wizard Autocomplete publisher name if linked from publisher view. Email address used to publish should be activated/validated on first use. Original email still used to log in.

Which is where the other API tests have now been moved.

Add API calls for adding one or more tags to and removing one or more tags from a vocabulary, without affecting the other tags in that vocabulary and without having to pass the full list of the vocabulary's tags.

In many places we have @classmethods where simple module-level functions would do (and would be more idiomatic in Python), e.g. the get() classmethods in the model/ classes.

Doing it with module functions lets us type module.function() instead of module.Class.method().

In many cases we're importing classes directly with from module import Class and then doing Class.method(), but from module import foo is bad style: ​http://docs.python.org/dev/howto/doanddont.html

Add api to translation to the term_translation table.

• add to logic 2d
• test 2d

• Relationships in sidebar [WONTFIX yet without some link to documentation or datastructure, something for me to render?]
• Icons for *types* of resources in resource list
• List of 'Owners' (administrators) at top right of sidebar (a la group page)
  • Much prefer owner to administrator ...