{22} Trac tickets (2647 matches)

We seem to occasionally get this exception on ckan.net.

User agents causing this include:

• Alexa ia_archiver (3/6/11)
• Maxthon browser (17/5/11)

Module ckan.controllers.error:32 in document
<<              return original_response.body
               # Otherwise, decorate original response with error template.
               c.content = literal(original_response.unicode_body) or cgi.escape(request.GET.get('message', ''))
               c.prefix=request.environ.get('SCRIPT_NAME', ''),
               c.code=cgi.escape(request.GET.get('code', str(original_response.status_int))),
>>  c.content = literal(original_response.unicode_body) or cgi.escape(request.GET.get('message', ''))
AttributeError: 'NoneType' object has no attribute 'unicode_body'

Perhaps the importer deletes them before re-importing. We shouldn't have deleted resources, so let's investigate.

Pawel knows about this so David Read, Pawel and I need to find time to discuss it.

Datasets created via the harvesting (UKLP ones) and the ones created via the form use a different schema. That could lead to problems. We need to discuss the best approach.

As a

Visitor

I want to

Be able to register as a user on the service

So that

One can login and be an authenticated user and perform the actions that only authenticated users can perform

Notes

• Registration involves the provisional of a minimal set of user details, namely:
  1. username (unique in lifetime of the service)
  2. fullname
  3. email
  4. password

As a

Visitor

I want to

Locate packages that I am, or might be, interested in

So that

I know what packages exist and can read the information associated with those packages

Notes

Find a package by searching by (or some combination thereof):

1. tag
2. string contained in title or description
3. license

As a

Authenticated User

I want to

Create/register a package

So that

The package will be listed in the system and visible to users

Notes

• Full package registration process may be broken down into:
  1. Creating a package by provision of minimal attributes:
     1. name: unique and web-usable
     2. auto-generation of default values such as:
        1. date-registered (today's date)
        2. first owner (one owner should be created corresponding to the authenticated user performing this action)
     3. (?) auto-generate a default release
  2. updating the created package -> ../UseCase12

As a

Registered user

I want to

Add a comment to a package

Notes

• The comment should be tagged with the username of the person making the comment
• Should we allow comments per release as well as per package? No for the time being.
• Comments will be 'flat' and not threaded (at least for the present)
• Comments should be plain text and so html should be escaped (in the future we might look to use some kind of formatter such as markdown)

On the CKAN home page there is no suggestion that you can add or register a dataset. The 'Register a dataset' option has been removed at some point! (only visible to users that have registered - why would they do that when the benefits aren't described?)

is this the expected behaviour seems crap to me, should we just see the author but just no link

For example, hide the 'edit' button on user profiles ​http://s031.okserver.org:2375/user/ross2#activity unless I'm logged in as Ross

we now have a user dashboard that needs theming not sure if we need sam to look at it

​http://localhost:5000/user/dashboard

same as yesterday: ​http://demo.ckan.org/dataset/edit/newcastle-city-council-payments-over-500 clicking update/save returns error

Super admins & owners/creators of groups need a way to delete them

Is the auth model in place for all this stuff? May be worth sitting down and going through how this should work

Currently can change state from active to deleted - this is not ery user friendly (want a button with a 2 step process maybe?) & I'd like to know who can view this option? Everyone who can edit a group? Who can edit?

tooltip on popular datasets with number of views facets to update automatically creating a dataset without reloading page between steps hover on licences information autocomplete on search terms group filtering social share buttons in lightboxes dataset counts on homepage

3. Uploading a file behaves counter-intuitively (I would suggest wrongly).

When adding a new resource by uploading a file, I select a file called say create-group.png. I expect the following to happen:

• the pathname of that file is filled in the box;
• nothing is actually uploaded till I hit 'add' (confirming that I've got the right file etc).

Instead of this,

• the file is immediately and, seemingly, irrevocably uploaded;
• the box is filled with a mysterious URL (​https://commondatastorage.googleapis.com/ckan-demo ...)
• the mysterious URL is so long I can't even see the final element, which is my file name.

On the edit dataset page... there needs to be a way for users to be able to see all the resources associated to the dataset and edit them individually.

It's because Package carries several out-of-band values, which are snagged on the way back out. Entity get response also can't be posted.

However, post response can be re-posted (because it isn't the same as the register-post/entity-get responses.

An issue for CKAN too.

Sub-ticket of #961 (form, validation, model sync meta-ticket) and depends on that work.

Among other things this will allow the tests to run much quicker.

The current formalchemy setup conflates view, controller and model code in a way that makes it hard to debug and customise.

Review existing (and potentially non-existing) frameworks with a view to porting forms over to something more explicit and lightweight.

Implement the current Package forms as an example of how this would work.

Document and circulate.

Sub-ticket of #961 (form, validation, model sync meta-ticket)

I need to completely munge the rendered data coming from the app. There's not currently a plugin hook for this, where I get access to the entire environment. The most generic form of plugin would be WSGI Middleware. I propose adding an IMiddleware plugin hook for inserting abritrary middleware via our plugin system.

I have a requirement to allow an external source to define the groups of which a user is a member.

I propose to create an IAuthzUserGroups plugin interface that allows an extension to arbitrarily extend the list of AuthzGroups? that a user is in.

The current oAuth extension tightly couples repoze.who, ckan, and data.no. These components should be decoupled; they also need more tests

Allow deployers to set a URL pointing to their own favicon

Migrate tables for activity streams

Decide which extensions are important.

1. Call ckan.logic.action.update:package_update()
2. Pass in a context dict with "allow_partial_update":True
3. Pass in a partial data dict containing only the package's ID and those fields that you want to update (e.g. title).

=> All of the package's resource will have their statuses changed to "deleted".

The "allow_partial_update" option should apply to the resource lists of packages, but it does not.

This could be implemented as a separate API call, or the rendered text versions of the activities could be added into the JSON returned by the existing API call.

This requires setting up templates for rendering activity streams items and detail items as nice, human-readable text.

There are some open questions, e.g.: Do we want the entire activity stream rendered as a block of plain text? As HTML? Or do we want a list of JSON objects, where each object contains its textual and/or HTML representations as fields?

Activity stream items and their related detail items are separate objects that each have their own textual representations.

For a mockup of the kind of text messages we want, see:

​http://datahub.pudo.org/pudo

but note that this ticket is just for creating the text snippets themselves, not rendering then in an HTML page or RSS feed. Also the mockup only show activity items and not their detail items.

See '### Task 2' in this etherpad: ​http://ckan.okfnpad.org/notifications

I have no idea yet about how to do this or how long it might take me.

A test case is currently failing for me on master:

ERROR: ckan.tests.functional.test_user.TestUserController.test_user_create_unicode
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/seanh/pyenv/lib/python2.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/seanh/pyenv/src/ckan/ckan/tests/functional/test_user.py", line 342, in test_user_create_unicode
    res = res.follow()
  File "/usr/lib/pymodules/python2.6/paste/fixture.py", line 603, in follow
    return self.test_app.get(location, **kw)
  File "/usr/lib/pymodules/python2.6/paste/fixture.py", line 208, in get
    return self.do_request(req, status=status)
  File "/usr/lib/pymodules/python2.6/paste/fixture.py", line 389, in do_request
    **req.environ)
  File "/usr/lib/pymodules/python2.6/paste/wsgilib.py", line 343, in raw_interactive
    app_iter = application(basic_environ, start_response)
  File "/usr/lib/pymodules/python2.6/paste/lint.py", line 170, in lint_app
    iterator = application(environ, start_response_wrapper)
  File "/usr/lib/pymodules/python2.6/paste/cascade.py", line 130, in __call__
    return self.apps[-1](environ, start_response)
  File "/usr/lib/pymodules/python2.6/paste/registry.py", line 350, in __call__
    app_iter = self.application(environ, start_response)
  File "/usr/lib/pymodules/python2.6/repoze/who/middleware.py", line 69, in __call__
    auth_ids = self.authenticate(environ, classification, ids)
  File "/usr/lib/pymodules/python2.6/repoze/who/middleware.py", line 201, in authenticate
    userid = plugin.authenticate(environ, identity)
  File "/home/seanh/pyenv/src/ckan/ckan/lib/authenticator.py", line 29, in authenticate
    if user.validate_password(identity.get('password')):
  File "/home/seanh/pyenv/src/ckan/ckan/model/user.py", line 113, in validate_password
    hashed_pass = sha1(password_8bit + self.password[:40])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 12: ordinal not in range(128)

Add logic functions for getting the activity stream for a dataset in JSON and rendered HTML formats, add HTML activity stream into dataset page.

CKAN should be able to send email notifications to users.

Maybe have a notifications table in the db, and a server-side job that runs periodically and consumes rows from this table, mailing them to the users.

One thing that we may want to send users notifications of is activity stream events. So the activity streams code would have to add rows to the notifications table for the mailer job to consume. But remember that email notifications feature is separate from activity streams - we may want to send notifications of other things as well.

Need to implement (at least some of) #1634 before this can be implemented, in order to have something to send notifications about.

Analysis here: ​http://ckan.okfnpad.org/27

(and add test cases for it)

The HTML and CSS for the activity streams templates needs to be tidied up. Currently the CSS is inlined in the templates themselves, and there is much unnecessary repetition of CSS.

Perhaps put this one in hold until there has been some design discussion about how the activity streams and the user, dataset and group pages with activity streams should look?

We already have simple activity streams for groups that have new group or group updated events (see #1631). Smarter/more useful group activity streams would also contain the events for things related to the group: users, datasets, other groups, etc.

This should just mean using a more complex SQL query in the group_activity_list() logic function.

Things might get tricky when a user or dataset joins a group, this may show up as two events (the user or dataset was modified, and the group was modified). We may want to have two separate events for this (one to go in the user or dataset activity stream and one for the group), or maybe we just want one 'user/dataset joined group' event.

...and show them on the tag pages.

This means adding tag_activity_list() and tag_activity_list_html() logic functions, adding the templating to render tags as HTML, and adding the tag HTML into the tag pages.

What exactly should go into a tag's activity stream? As with datasets we may want a more complex SQL query, where events from objects related to the tag (datasets, users, etc.) appear in the tag's stream.

Add drupal-like "taxonomies" to CKAN.

Etherpad with user stories, feature list, design and implementation discussion:

​http://ckan.okfnpad.org/22

Branch where this is being developed is feature-1698-tag-taxonomies:

​https://github.com/okfn/ckan/compare/master...feature-1698-tag-taxonomies

Tickets related to this have ​keyword taxonomies.

Enable ckanext's to add new vocabularies to default_package_schema(), use the new vocabularies in templates for dataset view, create, edit pages, helper function for extensions to create a vocabulary schema for a given vocabulary name (must match a vocab already in the db) and add it to default package schema.

In the tag-taxonomies branch tags now (optionally) belong to vocabularies and where tags previously had unique names, a CKAN instance can now have two tags with the same name as long as they belong to different vocabularies. The tests for the tags model should probably be updated to reflect this.

Other tests might also need to be updated.

For example, after deleting a dataset old activity stream events will still hyperlink to that dataset.

When an extra is added to or removed from a dataset or is updated, an activity should go in the dataset's and user's activity streams.

Which is where the other API tests have now been moved.

We already have API calls for adding one tag or removing one tag at a time to/from a vocab. Want more convenient ways to add or remove many in one call, or just pass in a list of tags to completely replace a vocab's current list.

• Add tags_create() function that calls the existing tag_create() many times?
• Also tags_delete()?
• Or just let tag_create() and tag_delete() accept a list?
• Add tags argument to vocabulary_update() to pass in a list of tags to replace the vocab's current tags? (But it should still be possible to simply rename a vocab without changing its tags and without having to pass in the tag list.)

In many places we have @classmethods where simple module-level functions would do (and would be more idiomatic in Python), e.g. the get() classmethods in the model/ classes.

Doing it with module functions lets us type module.function() instead of module.Class.method().

In many cases we're importing classes directly with from module import Class and then doing Class.method(), but from module import foo is bad style: ​http://docs.python.org/dev/howto/doanddont.html