{22} Trac tickets (2647 matches)

As a user I want to track changes to a particular package.

Provide an Atom feed, linked from the package read page.

As a user I want to be able to create groups of packages.

This functionality is different from tags:

• The ability to add a package to a given group is restricted whereas anyone can add a given tag to a package
• Specifically groups have owners and only the owners can add a package to that group

Group properties

• id (primarykey, uuid style)
• name (restricted content - same rules as package)
• title (no restrictions)
• owners -- many:many with user object
• description (markdown)
• packages -- many:many with package object

Don't version groups for now.

Groups address in the WUI will be:

• /group - browse list of groups (reuse list action)
• /group/list - same thing for now
• /group/<groupname> - display: group properties with links to packages. No links to user pages (yet).

Group editing and searching will be another ticket.

Cost: 3d

As a sysadmin I want to take CKAN out of wiki-mode into normal mode. In normal mode visitor and logged in users cannot create or add packages.

Suggest put this as an internal command in authz system and add to cli as paster command.

As a sysadmin I should be able to view deleted groups/publishers (publisher profile) but I can't.

In addition, there don't seem to be tests for permissions of deleted publishers.

As a User I want an image generated for my related item automatically so that I do not have to take picture myself and upload it somewhere

Implementation: (?) Use phantomjs or something like ​http://snapito.com/ Investigate for 1.8, will be needed for TDH Oct.

As a Package Admin I want to change the state of the package. In particular I wish to delete and undelete it.

(NB: this is quite separate from "purging" objects which is the term we shall use for irrevocable removal of an object from the domain model).

• Only Package Admins (and sysadmins) should be able to change state

Implementation Suggestions

• 'delete' action should be renamed to 'change-state' (NB: this requires a db migration ...)
• Have new package formalchemy form (created via inheritance?) to incorporate state attribute. Suggest this is rendered as a dropdown (and may be simple object rendering of state, i.e. do NOT need to change it to a single name such 'active').
• This form should then be used when the user satisfies is_authorized(..., model.Action.CHANGE_STATE) instead of the usual fieldset

As CKAN grows, more and more optional representations of packages will become used. Besides RDF (which is the motivation for this ticket), support for DSPL, home-grown XML, or CSV listings is imaginable.

To properly support this CKAN should have an extensible representation registry that can be extended when new output types become available (and without changing the API in the process). This needs to integrate in two places:

• We need to add support for file format extensions to the package handler and REST API.
• We need to add HTTP content type negotiation (Accept headers).

To support this we should have a registry with two registers:

• to map format extensions to mime types (e.g. "json" -> "application/json")
• to map mime types and entity types to converter functions (e.g. 8"application/json", ckan.model.Package) -> func(obj, mime_type) )

This should be set up on load_environment so that IConfigurer plugins can feed into it.

The registry should then first be added to _finish in the REST API (which needs to be refactored to be passed the {format} part of the URL if one is given. Based on the format part and HTTP headers, an appropriate representation can be generated by the registry and then be returned to the user.

Forwarding of requests to the regular WUI controllers with Accept headers set or a format specified can be implemented in a separate effort.

Appears current default search in CKAN ORs terms rather than ANDing them (i.e. adding more terms increasing number of items found rather than reducing it).

Not sure when this crept in or if it has been there for a long time.

Apparently it's working when developers run it locally but not on the demo site.

Anja is reporting this, severe bug, I think.

And as a result timeout on e.g. the datahub ...

An internal server error was reported when trying to create a new package (not when editing):

WebApp? Error: <class 'sqlalchemy.exceptions.IntegrityError?'>: (IntegrityError?) null value in column "name" violates not-null constraint 'INSERT INTO package (id, name, title, version, url, author, author_email, maintainer, maintainer_email, notes, license_id, state, revision_id) VALUES (%(id)s, %(name)s, %(title)s, %(version)s, %(url)s, %(author)s, %(author_email)s, %(maintainer)s, %(maintainer_email)s, %(notes)s, %(license_id)s, %(state)s, %(revision_id)s)' {'maintainer': None, 'name': None, 'author': None, 'author_email': None, 'notes': None, 'title': None, 'maintainer_email': None, 'url': None, 'state': 'active', 'version': None, 'license_id': None, 'revision_id': '0141c09c-4eb3-4dda-a6b1-e07f2c73b4c5', 'id': '7fe16a29-f183-471a-99c5-2c2e81dd86e4'}

The error could not be reproduced in a non-apache/mod_wsgi environment, even with a dump of the live database. Further investigation by manual editing of the package form left the "groups" field as the likely cause. After updating the packages installed, and temporarily editing the package controller the problem vanished after a restart of the web server.

Possible causes include:

• Stale pyc file
• Threading issue
• Dependency issue

As the problem is gone for now, it cannot be further investigated atm.

An extra field on datasets called "metadata_language". The language that the meta-data is in. Drop-down list needs to be translatable. Default value is "English".

An API call that returns a user's public activity stream in JSON format.

Also they seem to be missing the odd _() block. Such in the new package form.

​https://github.com/okfn/ckan/blob/2375-demo-theme-development/ckan/templates/package/snippets/package_form.html#L44

Also need to fix bug where 'Other' is never highlighted.

Already support stateful list-like collections and should extend this to dict-like collections.

(This is prerequisite for implementing versioned "extra" (key/value) attributes on packages in CKAN and elsewhere).

Allowing them to be displayed in many sizes, colours and resolutions.

​http://www.zurb.com/playground/foundation-icons

Allowing more flexible tag names:

• allowing spaces
• allow capital letters (search is case in-sensitive)
• allow all punctuation except for commas and double-quotes '"'
• allow unicode
• commas delimit tag names in the package create/edit form

Effects:

• package creation/edit form.
• /tag/{tagname} uri
• search action
• api controller (search/package-create/edit)
• web controller (search/package-create/edit)
• search api documentation
• autocomplete for tag names

Allow user to navigate directly from one resource page to another via resource box in sidebar

Allow deployers to set a URL pointing to their own favicon

Allow a user to save a partially-complete and probably invalid package-create form in order to come back to it later.

We decided on blob storage over a state field on the package:

• prevents a history of the drafts being stored in the revision history. This is a particular problem if we start auto-saving, say every 20s.
• the blob can store arbitrary extra data, such as how far through the wizard we are.

Allow a form to specify fields that are required. Currently name and resource url are required.

• Show required fields with asterisks by the input box.
• Resource fields to be included in this.

Allow 'ignored' fields on versioned objects (i.e. attributes which are not 'versioned'). This is not hard to do as we already have most of the necessary mechanisms set up in the Revisioner object.

Cost: 2h

All translated fields and vocabularies need to be added to search index in the correct fields.

All package search results on ckan.net are labelled as 'open' even when their license is closed or unknown: ​http://ckan.net/package

Against ckan-1.3.1, when I create an authz group called "administrators" and visit /authorizationgroup/edit/administrators, I am unable to add more than one user to it.

Each time I add additional users, the existing user on the list is replaced with the new one.

After moving to Bootstrap the tables on the frontend look broken.

Afaik the new CKAN 2.0 frontend has no functional tests

Administrator wants to upgrade CKAN or move it to another server. During this time the database is being administered and either edits are lost or can't be done.

Additional feature related to ticket:66.

cost: 2h

Add uuids into package in REST interface.

Add the organizations and organizations_dataset plugins to ini file, run ckan, login, create an organization, create a dataset and put in organization, edit the dataset and set organization to none, try to edit the dataset again (you must be sysadmin to do this) and get a 500:

Error - <class 'genshi.template.eval.UndefinedError'>: [] has no member named "get"
URL: http://127.0.0.1:5000/dataset/edit/fooset
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/weberror/errormiddleware.py', line 162 in __call__
  app_iter = self.application(environ, sr_checker)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/beaker/middleware.py', line 73 in __call__
  return self.app(environ, start_response)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/beaker/middleware.py', line 155 in __call__
  return self.wrap_app(environ, session_start_response)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/routes/middleware.py', line 130 in __call__
  response = self.app(environ, start_response)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/pylons/wsgiapp.py', line 125 in __call__
  response = self.dispatch(controller, environ, start_response)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/pylons/wsgiapp.py', line 324 in dispatch
  return controller(environ, start_response)
File '/home/seanh/Projects/ckan/ckan/ckan/lib/base.py', line 221 in __call__
  res = WSGIController.__call__(self, environ, start_response)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/pylons/controllers/core.py', line 221 in __call__
  response = self._dispatch_call()
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/pylons/controllers/core.py', line 172 in _dispatch_call
  response = self._inspect_call(func)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/pylons/controllers/core.py', line 107 in _inspect_call
  result = self._perform_call(func, args)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/pylons/controllers/core.py', line 60 in _perform_call
  return func(**args)
File '/home/seanh/Projects/ckan/ckan/ckan/controllers/package.py', line 499 in edit
  extra_vars=vars)
File '/home/seanh/Projects/ckan/ckan/ckan/lib/base.py', line 148 in render
  return cached_template(template_name, render_template, loader_class=loader_class)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/pylons/templating.py', line 249 in cached_template
  return render_func()
File '/home/seanh/Projects/ckan/ckan/ckan/lib/base.py', line 101 in render_template
  return literal(stream.render(method=method, encoding=None, strip_whitespace=True))
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/core.py', line 183 in render
  return encode(generator, method=method, encoding=encoding, out=out)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/output.py', line 57 in encode
  return _encode(''.join(list(iterator)))
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/output.py', line 339 in __call__
  for kind, data, pos in stream:
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/output.py', line 670 in __call__
  for kind, data, pos in stream:
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/output.py', line 771 in __call__
  for kind, data, pos in chain(stream, [(None, None, None)]):
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/output.py', line 586 in __call__
  for ev in stream:
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/core.py', line 288 in _ensure
  for event in stream:
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/template/base.py', line 605 in _include
  for event in stream:
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/template/markup.py', line 327 in _match
  for event in stream:
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/template/base.py', line 545 in _flatten
  for kind, data, pos in stream:
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/template/directives.py', line 169 in _generate
  attrs = _eval_expr(self.expr, ctxt, vars)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/template/base.py', line 277 in _eval_expr
  retval = expr.evaluate(ctxt)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/template/eval.py', line 178 in evaluate
  return eval(self.code, _globals, {'__data__': data})
File '/home/seanh/Projects/ckan/ckan/ckanext/organizations/templates/organization_package_form.html', line 110 in <Expression u"{'selected':'selected'} if organization.get('id','') == group['id'] else {}">
  <option value="${group['id']}" py:attrs="{'selected':'selected'} if organization.get('id','') == group['id'] else {}">${group['title']}</option>
File '/home/seanh/Projects/ckan/ckan/ckan/config/environment.py', line 248 in genshi_lookup_attr
  val = cls.undefined(key, owner=obj)
File '/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/genshi/template/eval.py', line 410 in undefined
  raise UndefinedError(key, owner=owner)
UndefinedError: [] has no member named "get"

Add the following attributes to package:

• author, author_email
• maintainer, maintainer_email

Gives us full compatibility to: ​http://docs.python.org/distutils/setupscript.html#additional-meta-data

Column ordering - should come after name, title, url, download_url.

Add resources into Web User Interface.

• Locate at: /dataset/{dataset}/resource/{id}
• CRUD
• Authorization

Done:

• #945 - Richer resources - Resource Groups, new fields, improved UI
• #1445 - Resource View page in WUI
• #1450 - Dataset view pages to match

Moved to superticket #1506:

• #978 - Edit Resource Extras in Web UI

Add minimal, working IDatasetForm and IGroupForm example extensions to core, with tests.

The IDatasetForm example should use tag vocabularies (two birds with one stone)

The IDatasetForm and IGroupForm docs are not very good (and are somewhat spread around different doc chapters), fix them up, and reference the new working examples.

Tab Vocabularies docs should reference IDatasetForm example.

When using convert_to/from_extras() you have to remove any free extras from the form or it won't work, this needs to be documented (in the docstring maybe)

There have been recent changes to the schemas that IDatasetForm and IGroupForm use, make sure the docs are up to date.

Add logic functions for getting the activity stream for a dataset in JSON and rendered HTML formats, add HTML activity stream into dataset page.

Add it to a 'Troubleshooting' section on the filestore page: ​http://docs.ckan.org/en/ckan-1.7.1/filestore.html

For the error messages and their solutions, see various threads on ckan-dev

Add generator=ckan tag in head of templates to identify site as generated by CKAN

Add ckan_url attribute to REST JSON representation of a Package pointing to the (read) url of package on CKAN.

Cost: 30m

Add api to translation to the term_translation table.

• add to logic 2d
• test 2d

Add an api for searching users. This is needed for any kind of ajax autocomplete (needed for anywhere we want to add users).

• API location: /api/util/user/lookup?q=querystr&limit=10
• Return json objects containing {id: ..., name: ..., fullname: ...}
• Put in a module called controllers/apiv2/user.py

Add a way to share datasets and see mentions of datasets on twitter (and elsewhere)

As a visitor I want to share a link to a dataset I have found. I also want to see how many others have shared this (or mentioned it).

• Sharing platform: simplest option is probably twitter but could generalize to e.g. sharethis system which supports, facebook, twitter etc etc.
  • Will need a config option for relevant API key e.g. ckan.sharethis.apikey
• Location: Put this at top of sidebar on dataset view (a better suggsetion?)
• Should show how many mentions / shares there are. (very important!)

Discussion

• (Tom and Ira and ...): do we really want this, versus e.g. proper follow extension?
• What about bringing conversation back in to CKAN. E.g. show all times this dataset (i.e. its url) was mentioned on twitter. If we do this I think this should be definite +1.
• Also, given our users, I think just doing twitter (identica) may be sufficient (how many people want to share links to datasets on facebook?)

Add a helper function to make it easy for extensions to add new vocabularies to default_package_schema(). The helper function should take the name or ID of a vocabulary (which should already exist in the db) and add the necessary key: [schema] entry to default_package_schema(), with the necessary validation, authorisation, transformation.

Add a config option to choose the metadata licence. Set default to Open Database License.

Currently the dataset edit form says "Important: By submitting content, you agree to release your contributions under the Open Database License." This is hard-coded, but not suitable for when DGU uses the CKAN form - they use the OGL.

Add a better docstring to for trans js command explaining what it does and why, and how to use it.

Also add this command to the CKAN Release Process as it's needed there

Add Group editing page.

If no permissions to change group can't edit group. Also cannot view edit page.

Editable attributes: name, title, description

No preview needed

Add 'package:all' to authz tool to allow mass changes of authz.

Activities about private datasets should not appear in public activity streams.

I don't think you want to actually purge the activities from the db, because you might still want them to appear in private activity streams.

I do think that when a dataset goes private all its past activity should go private, because I imagine that users are going to want to hide everything about the dataset and not have any past activities 'leaking out'

I don't think you want to consider whether the dataset was private when the activity happened, rather if a dataset is private now then all its past activities are private (and the simplest thing would be to say that if a dataset is public now then all its past activities become public as well, but is that a privacy concern?)

The easiest way to implement this is going to be by modifying the *_activity_list() action functions in get.py, after they pull their activity lists out of the db they should pass them through a function that filters out stuff about private datasets.

An activity about a private dataset is one whose object_type is 'dataset' and whose object_id matches the id of a private dataset. You should also check the object_type and object_id of all of the activity object's activity detail objects, if any of those match a private dataset then mark the whole activity as private.

Currently all activity streams are public so should have all private datasets filtered out from them, except for the dashboard activity stream which is private to the individual user. In this case private datasets that the user has permission to see should not be filtered.

According to sqlalchemy docs [1], when calling Session.is_modified(), the passive argument needs to be passed in as True.

[1] ​http://docs.sqlalchemy.org/en/rel_0_7/orm/session.html?highlight=is_modified#sqlalchemy.orm.session.Session.is_modified