{22} Trac tickets (2647 matches)

The Wordpresser extension seems to be working well for DataGM, but needs caching and tests.

At the bottom, it says something like

Author: Bob Bumgardner Since you have not signed in this will just be your IP address. Click here to sign in before saving (opens in new window).

I've been asked if we can do something about the overflow of the Group name in the right hand column on this page:

​http://register.data.overheid.nl/package/europese-aanbestedingen

The reason is that the list display for groups is in the form "group_tltie (group_name)", and of course group_name can't have spaces and so can't wrap nicely.

I was wondering if there's a good reason why we don't only display group_title (if it exists) and group_name only when there's not a title?

Requirements

The ODS demo site will need data adding, initially as fixtures but it would also be useful if we started evaluating datasets that we can ship with ODS installations (at least in the UK) from places such as DGU and ONS.

May wish to create a ticket for making sure the datasets within the system are reset every X hours. Perhaps.

Interface

None

User Stories

• As a new system administrator for an ODS instance, I don't want to have a site devoid of any data. Geographically relevant datasets would be welcomed.

• As a bizdev person I would like to be able to demonstrate how ODS works with real datasets.

Tasks

[ ] Identify relevant sources for datasets

[ ] Pick datasets

[ ] Set them up for import

Estimates

Need proper user stories but ...

Publisher admins/editors may need a more useful group read page showing things like:

• The current search

• Recent activity

• People within the group

• Followers

• Others?

I thought I'd made this ticket before but can't find it. When we have a fully created dataset we need to be able to edit the resources.

template: package/resource_edit.html

when you've made a form reassign to me and I'll get it working

needs review & speccing out e.g. datasets created by a user who belongs to a certain publisher (group) get auto added to this group

when language is french

a) the menu overlaps the logo

b) much of the logo becomes unclickable

@shevski

a) seems like you b) is aron

e.g. tried changing the lang here: ​http://s031.okserver.org:2375/ to no effect. Doesn't work on any other pages either. Flags have also dissappeared

Title

Visitor visits service

As a

Visitor (general web user)

I want to

Visit the website of the service (frontpage)

So that

• See the front page and discover information about the service
• Perform various activities related to the service such as registering and finding packages, registering as a user and logging in

As a

Visitor

I want to

Login to the service

So that

I am now an authenticated user and may perform the actions that only authenticated users may perform

Notes

• Login by providing username and password
• Login screen should also include a link or information about registration
• If username do not match (or the username does not exist) a login failure is noted and the visitor is re-presented with the login screen as well as information on how to register

As a

Visitor

I want to

View the details of an individual package

So that

I know more about the package. Attributes available are detailed in updating a package (../UseCase12)

When a package's name is changed, references to the old name (in the WUI and REST) are redirected to the new name (assuming the old name is not being reused by another package).

When CKAN is included in a Drupal front-end, CKAN edit pages are used in a slave-mode, such that authentication is delegated to the Drupal front-end user model.

The Drupal front-end shall have:

1. Login page - fixed location, can authenticate users, on successful authentication sets auth cookie and redirects to HTTP_REFERER.

2. Access control resource - fixed location, can authorise users, on receipt of valid auth cookie return message listing account details and permitted actions.

3. Access denied page - fixed location, static resource, gently

indicates what has happened, and how to ask for permission.

The CKAN slave edit page shall:

1. Try to detect a Drupal session key (passed as cookie or as request param).

2. Redirect to Drupal login page if no session key.

3. Check authorisation if session key is found.

4. Redirect to access denied page if session key not authorised.

5. Present the Package edit page.

6. Reject unauthenticated or unauthorised edit submissions.

7. Snag invalid edit submissions from authenticated and authorised users.

8. Respond to valid edit submissions from authenticated and authorised users, by saving the new package state, and redirecting to Package read page in Drupal front-end.

Pylons 1.0 has been out for a while and appears to be running stable on a few production sites. It also introduces a few new features, among them the ability to specifiy controllers using the entrypoint mechanism.

​http://bitbucket.org/pudo/ckanextdeliverance/src/tip/ckanext/deliverance/pylonspatch.py

This might be a bit ugly since the default behaviour of some pylons objects, such as tmpl_context, app_globals and redirect has changed.

Common requirements for running CKAN behind a (e.g Wordpress or Drupal) front-end:

1. Unrestricted total read-only access to catalogue API for general public (e.g. voluntary organisation).
   • monitored by API key
   • not monitored by API key
2. Restricted total read-write access to catalogue API for authorized clients (e.g. front-end system, bulk upload clients).
   • restricted by CKAN access controller
   • restricted by HTTP Auth
   • restricted by IP address
3. Restricted total read-write access to catalogue Web UI for authorized users (e.g. site admins).
   • restricted by CKAN access controller
   • restricted by HTTP Auth
4. Restricted partial read-write access to catalogue Web UI for authorized users (e.g. group admins).
   • restricted by CKAN access controller
   • restricted by HTTP Auth

See message and script: ​http://lists.okfn.org/pipermail/ckan-discuss/2010-September/000569.html

1500 requests shouldn't take hours.

...obviously, not all packages will have a harvested metadata document.

Whilse using CKAN web interface, you are not tempted to edit stuff:

• You know at all times this CKAN is read-only
• All editing facilities are still seen but greyed-out with an indication why it is.

Issue 1. Help command without arguments fails:

$paster help --plugin ckan --config development.ini

gives:

EITHER

Command help not recognized

OR

paster help: error: no such option: --plugin

Issue 2: Harvester command with arguments fails.

$ paster harvester Traceback (most recent call last): ...

cmd = self.args[0]

IndexError?: list index out of range

(Just like resource table, but extra fields don't need up/down buttons)

Nice to have: a blank field is added when you tab from the last filled-in field in the table.

Auth Proposal

Use case: We'd like an authenticated and authorized Drupal user to be able to edit/delete packages using the API

To do this a user would need a CKAN API key so we need some way of Drupal telling CKAN who a user is so that they can get their API key.

Proposed Implementation

A user visits the CKAN API key page to get their key. Because CKAN is at catalogue.data.gov.uk (a subdomain of the Drupal site) it can read Drupal cookies.

If there is no DRXtrArgs or DRXtrArgs2 cookie, we know the user isn't signed in so we redirect them to Drupal to sign in.

WISHLIST: It would be really nice if we could pass the URL to redirect back to Drupal so tha Druapl can send them back to the CKAN API key page

Either way, they get back to CKAN and now the cookies exist. When the first HTTP request header is sent, CKAN will read the Drupal session ID and then call a Drupal API, server to server.

TODO: Implement an API on the drupal server which accepts a Drupal session ID as an argument and returns the username and credentials, but only if the request if from the CKAN server (perhaps we specify an API key in the request)

If the session is valid CKAN will set its own auth cookie and show them the page with the API key. CKAN only considers a user signed in if both the CKAN cookie *and* Drupal session ID are present. If at any time they sign out of Drupal the Drupal session disappears so they will be signed out of CKAN too.

Now the user has an API key they can use the standard CKAN command line API tools.

The API key is the only thing the user will need to use the CKAN API. This poses a problem. What if a user is removed from Drupal but still has a CKAN API key?

Two solutions:

1. API keys will be set to only be valid for the length of a Drupal session so that a user will only be able to use the write API for a little time after credentials are revoked

2. CKAN provides an API to allow Drupal to revoke keys

Option 1. seems easier to me. If we choose this we will write a get_api_key command line tool so that Drupal user can write things like this:

::

datapkg list_packages --api_key get_api_key

The get_api_key function will prompt for username and password and then perform the steps necessary to get an API key.

As a system developer, I want to run 'create next migration version' script, so that I don't have to create migration version tests and files manually.

implement paster subcommand to delete harvest jobs

In Slovakia users are expecting from sites to be able to search without typing accents, for example if they do not have SK keyboard, just US. For example searching for: 'Obyvateľstvo' and 'obyvatelstvo' (population) or 'štatistika' and 'statistika' (statistics) should yield same results.

This should work the other way around as well, as some people might enter entries without accents into CKAN instance and others might search with accents.

For SK language simple ASCII transliteration for searched term and indexed words is sufficient (iconv ASCIITRANSLIT).

The basic purpose of this extension is provide a way to record 'apps' (applications) that relate to datasets in CKAN. Both existing apps and ideas for apps should be permitted.

The link for About > 'Project Home Page' on the CKAN.net footer should be changed to point to:

http://ckan.org/

The current link is broken.

Thanks, Jason

Public file path code adds a trailing , and thus adds / as a static file app in the Pylons middleware cascade.

Refactor ckan/lib/authztool.py so that the relevant methods are independent of the command line interface.

The extracted methods should live in a new file ckan/authz.py. authztool.py should probably move into cli.py and will just do command line parsing and printing and use ckan/authz.py. The updated web gui for authz will also use this code.

Tests should be made. There's already a file ckan/tests/test_authz.py, which looks like the appropriate place for new tests.

all to go on a branch feature-1050-refactor-authtoolz

Optional extras

• Rename ckan/authz.py to ckan/lib/authz.py or even ckan/logic/authz.py

Here are some proposed changes related to CKAN's authorization system - they aren't very big, but should provide for some forthcoming use cases including #787.

Two man reasons for the changes are:

• We have a completely refactored architecture now which introduces a logic layer. These Auth changes are designed to better support the way we work with that layer.

• Different CKAN extension apps may need radically different authentication/authorisation so we need to allow whatever we have to be override-able.

The first two changes revolve around the is_authorized method, which is called by the logic layer to ask whether a particular user (e.g. Bob) is allowed to do a certain action (e.g. edit) on a certain object (e.g. Package).

1. The first thing the is_authorized method is a hook to a plugin

which *overrides* the current call with its own implementation (note: in previous discussions we have considered allowing a chain of plugins, no longer!)

Reason: authorization can be completely delegated to another system (or partially)

2. is_authorized method currently takes (username, action, object)

but for action=create_package, the object supplied is System, and for action=edit the object supplied is the package. Instead action should always be the string name of a function in the logic layer and object should always be the object passed to that function. This means our auth system is based around the actual actions we are performing (rather than a model them) and with the actual data that forms the action (rather than a related object). You never need a System object in this model.

3. Rename these two classes to better reflect what they are
   • AuthorizationGroup? -> UserGroup?
   • Group -> PackageGroup?

4. Rename the Editor role to PriveledgeUser? since Editors sometimes can't edit.

Although this sounds a bit radical we already have auth extensions.

Read-only CKAN Web UI

(Additional requirement from #764)

Whilse using CKAN web interface, you are not tempted to edit stuff:

• You know at all times this CKAN is read-only
• All editing facilities are still seen but greyed-out with an indication why it is.

python-ckanext-solr is already available in ​http://apt-alpha.ckan.org/datanl-dev, but not yet in ​http://apt-alpha.ckan.org/debian (that is why we had to [pip-install it for DataGM). Please push into main repo.

We seem to occasionally get this exception on ckan.net.

User agents causing this include:

• Alexa ia_archiver (3/6/11)
• Maxthon browser (17/5/11)

Module ckan.controllers.error:32 in document
<<              return original_response.body
               # Otherwise, decorate original response with error template.
               c.content = literal(original_response.unicode_body) or cgi.escape(request.GET.get('message', ''))
               c.prefix=request.environ.get('SCRIPT_NAME', ''),
               c.code=cgi.escape(request.GET.get('code', str(original_response.status_int))),
>>  c.content = literal(original_response.unicode_body) or cgi.escape(request.GET.get('message', ''))
AttributeError: 'NoneType' object has no attribute 'unicode_body'

Perhaps the importer deletes them before re-importing. We shouldn't have deleted resources, so let's investigate.

Pawel knows about this so David Read, Pawel and I need to find time to discuss it.

Datasets created via the harvesting (UKLP ones) and the ones created via the form use a different schema. That could lead to problems. We need to discuss the best approach.