{22} Trac tickets (2647 matches)

Results (201 - 300 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#207 enhancement dread nickstenning v1.0 closed fixed Reimplement tag selection in jQuery

Remove the current dependency on Yahoo YUI toolkit for doing tag autocompletion on package edit forms.

The (KForge-derived) behaviour files for CKAN can easily be switched to the new jQuery-based ones: see ticket:212. Once this is done, it would make sense to implement the tag-selection code with jQuery.

Relevant pages include:

1260200340000000 1271250740000000
#164 enhancement wwaites dread v1.0 closed fixed Creating RDF data from CKAN

Related to: ticket:90 Link to RDF version of CKAN data and ticket:163 (regularly convert CKAN data to RDF).

Convert ckan data to RDF with suitable ontology and put on semantic.ckan.net.

1256052643000000 1271251422000000
#251 enhancement johnbywater johnbywater v1.0 closed fixed Present in the API a list of revision IDs, and present individual revisions when referenced by ID

Want to have:

  • /api/search/revision/

Support querying by "since time" and "since revision". Each individual revision should be available in the REST API (read-only):

  • /api/rest/revision/{revision ID}
    • should include revision attributes plus revision.packages e.g. like { "id": "...", ... "packages": [ list of packages names from e.g. revision.packages ] }
1266519743000000 1271423917000000
#265 enhancement johnbywater dread v1.0 closed fixed More detail shown in Atom feed

Use case

As a user I want to stay abreast of package changes, such as a new package being created, newer data is available for a package or a new download is available for a package.

Implementation

Add into the Atom feed:

  1. a package is a new one
  2. the resources have been changed
  3. the last_updated field has been changed
1267708364000000 1271636891000000
#266 enhancement johnbywater dread v1.0 closed fixed Package feed

As a user I want to track changes to a particular package.

Provide an Atom feed, linked from the package read page.

1267716417000000 1271636901000000
#290 defect johnbywater dread v1.0 closed fixed Revision API - docs

doc/api.rst needs to cover the new Revision REST interface.

1271268759000000 1271636910000000
#221 enhancement dread dread v1.0 closed fixed UI Review - Package edit

Package edit page package/edit/mypackagename

  • Ability to add the package to groups on this page.
  • Fields could be grouped.
  • Inconsistent capitalisation (Url -> URL).
  • Tag editor YUI script appears to break occasionally.
  • Perhaps move to two column forms to save vertical space?
  • Notes field to monospace?
  • Extras fields need some jQuery love -- shouldn't have a fixed number available.
  • Typography is a bit of a mess.

1260880175000000 1271756757000000
#155 enhancement dread dread v1.0 closed duplicate Adding multiple packages to a group

Ability to add multiple packages to a group in one go (e.g. with 'add' link which makes drop down menu appear - so can add one after another - then submit simultaneously)

Use a bit of javascript to add more dropdowns.

Suggested by Jonathan Gray

1255621779000000 1271760041000000
#284 defect johnbywater rgrp v1.0 closed fixed Move tag/read/xxx to simple tag/xxx like package and group

Also put in redirect from old locations (see examples with plurals in routing.py)

Estimated cost: 20m

Actual cost: 40m

1270664552000000 1271852451000000
#293 defect johnbywater dread v1.0 closed fixed REST PackageResource update causes exception

This is caused by editing a package with resources over the REST interface.

Exception:

URL: http://de.ckan.net/api/rest/package/destatis
Module weberror.errormiddleware:162 in __call__
<<              __traceback_supplement__ = Supplement, self, environ
                   sr_checker = ResponseStartChecker(start_response)
                   app_iter = self.application(environ, sr_checker)
                   return self.make_catching_iter(app_iter, environ, sr_checker)
               except:
>>  app_iter = self.application(environ, sr_checker)
Module repoze.who.middleware:107 in __call__
<<          wrapper = StartResponseWrapper(start_response)
               app_iter = app(environ, wrapper.wrap_start_response)
       
               # The challenge decider almost(?) always needs information from the
>>  app_iter = app(environ, wrapper.wrap_start_response)
Module beaker.middleware:73 in __call__
<<                                                     self.cache_manager)
               environ[self.environ_key] = self.cache_manager
               return self.app(environ, start_response)
>>  return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<<                          headers.append(('Set-cookie', cookie))
                   return start_response(status, headers, exc_info)
               return self.wrap_app(environ, session_start_response)
           
           def _get_session(self):
>>  return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<<                  environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
               
               response = self.app(environ, start_response)
               
               # Wrapped in try as in rare cases the attribute will be gone already
>>  response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<          
               controller = self.resolve(environ, start_response)
               response = self.dispatch(controller, environ, start_response)
               
               if 'paste.testing_variables' in environ and hasattr(response,
>>  response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<<          if log_debug:
                   log.debug("Calling controller class with WSGI interface")
               return controller(environ, start_response)
           
           def load_test_env(self, environ):
>>  return controller(environ, start_response)
Module ckan.lib.base:51 in __call__
<<          # available in environ['pylons.routes_dict']
               try:
                   return WSGIController.__call__(self, environ, start_response)
               finally:
                   model.Session.remove()
>>  return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<<                  return response(environ, self.start_response)
               
               response = self._dispatch_call()
               if not start_response_called:
                   self.start_response = start_response
>>  response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<<              req.environ['pylons.action_method'] = func
                   
                   response = self._inspect_call(func)
               else:
                   if log_debug:
>>  response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<<                        func.__name__, args)
               try:
                   result = self._perform_call(func, args)
               except HTTPException, httpe:
                   if log_debug:
>>  result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
           
           def _inspect_call(self, func):
>>  return func(**args)
Module ckan.controllers.rest:145 in update
<<                  fs = ckan.forms.group_fs_combined
                   fs = fs.bind(entity, data=request_fa_dict)
                   validation = fs.validate_on_edit(entity.name, entity.id)
                   if not validation:
                       response.status_int = 409
>>  validation = fs.validate_on_edit(entity.name, entity.id)
Module ckan.forms.package:385 in validate_on_edit
<<              temp_name = orig_pkg_name
                   self.data['Package-%s-name' % record_id] = u'something_unique'
               validation = self.validate()
               if temp_name:
                   # restore it
>>  validation = self.validate()
Module formalchemy.forms:175 in validate
<<          if self.readonly:
                   raise Exception('Cannot validate a read-only FieldSet')
               return AbstractFieldSet.validate(self)
       
           def sync(self):
>>  return AbstractFieldSet.validate(self)
Module formalchemy.forms:104 in validate
<<          success = True
               for field in self.render_fields.itervalues():
                   success = field._validate() and success
               # run this _after_ the field validators, since each field validator
               # resets its error list. we want to allow the global validator to add
>>  success = field._validate() and success
Module formalchemy.fields:763 in _validate
<<          try:
                   value = self._deserialize()
               except validators.ValidationError, e:
                   self.errors.append(e)
>>  value = self._deserialize()
Module formalchemy.fields:1354 in _deserialize
<<          if self.is_composite_foreign_key:
                   return self.query(self.relation_type()).get(python_pk(self.renderer.deserialize()))
               return self.renderer.deserialize()
>>  return self.renderer.deserialize()
Module formalchemy.fields:183 in deserialize
<<          if self.field.is_collection:
                   return [self._deserialize(subdata) for subdata in self._serialized_value()]
               return self._deserialize(self._serialized_value())
       
           def _deserialize(self, data):
>>  return self._deserialize(self._serialized_value())
Module formalchemy.fields:167 in _serialized_value
<<          if self.field.is_collection:
                   return self._params.getall(self.name)
               return self._params.getone(self.name)
       
           def deserialize(self):
>>  return self._params.getone(self.name)
Module formalchemy.base:63 in getone
<<          if v is None or isinstance(v, basestring) or isinstance(v, cgi.FieldStorage):
                   return v
               return v[0]
           def getall(self, key):
               v = dict.get(self, key)
>>  return v[0]
TypeError: 'int' object is unsubscriptable

Thanks to Friedrich for spotting this.

1271756102000000 1271940083000000
#156 enhancement johnbywater dread v1.0 closed fixed Whitelist for html in mark-down field

Not just links that need to be displayed. Other html should be displayed too, such as tables which can't be represented in markdown. Do a search of the existing fields to see what is needed.

1255623973000000 1271962871000000
#292 defect dread dread v1.0 closed fixed PackageResources are created new on every package edit

When you edit a package, all of its resources are deleted and a fresh set created. This causes all history of them to be lost.

1271755964000000 1272286005000000
#240 defect johnbywater rgrp v1.0 closed fixed Do not allow creation of PackageResource object without a url

Currently it is possible (v0.11) to have a PackageResource? object without a url. This should be impossible to do (either at creation time or via modification).

1265399747000000 1272383770000000
#300 defect rgrp dread v1.0 closed fixed Resource ordering issue

Failing test: ckan.tests.models.test_resource.TestResourceLifecycle?.test_03_reorder_resources

Not clear how visible this is to the user.

Related to ticket:292

1272285994000000 1272384474000000
#295 enhancement dread dread v1.0 closed fixed Add multiple packages when editing a group

In Group edit page, you can only add one package at a time. It would be better to use some scripting to allow adding more than one at once, to make it easier and quicker when you have a long list to add.

Ticket is split off from ticket:221

See also ticket:155 (duplicate).

1271756713000000 1272384758000000
#264 defect dread rgrp v1.0 closed fixed Investigate why there are revisions with no associated packages

Revision.packages should not include all changes so why are we getting revisions with no associated changes? See e.g. http://ckan.net/revision/read/814e39e1-2540-4449-90d1-ac9a92ab40fb?

Investigate and fix.

NB: if this is bugfix this should go against stable or metastable branch if at all possible and then be merged into default.

Cost: 1.5h (?)

1267605428000000 1272390013000000
#304 defect johnbywater johnbywater v1.0 closed fixed Previewing package with resource with missing url is broken

Intended outcome:

See resource url validation error when previewing package with a resources that has no url.

Recurring Error:

URL: http://127.0.0.1:5000/package/new
File 'pyenv-ckan/lib/python2.5/site-packages/weberror/evalexception.py', line 431 in respond
  app_iter = self.application(environ, detect_start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/repoze/who/middleware.py', line 107 in __call__
  app_iter = app(environ, wrapper.wrap_start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/beaker/middleware.py', line 73 in __call__
  return self.app(environ, start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/beaker/middleware.py', line 152 in __call__
  return self.wrap_app(environ, session_start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/Routes-1.10.3-py2.5.egg/routes/middleware.py', line 130 in __call__
  response = self.app(environ, start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/wsgiapp.py', line 125 in __call__
  response = self.dispatch(controller, environ, start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/wsgiapp.py', line 324 in dispatch
  return controller(environ, start_response)
File 'pyenv-ckan/src/ckan/ckan/lib/base.py', line 50 in __call__
  return WSGIController.__call__(self, environ, start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/controllers/core.py', line 221 in __call__
  response = self._dispatch_call()
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/controllers/core.py', line 172 in _dispatch_call
  response = self._inspect_call(func)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/controllers/core.py', line 107 in _inspect_call
  result = self._perform_call(func, args)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/controllers/core.py', line 60 in _perform_call
  return func(**args)
File 'pyenv-ckan/src/ckan/ckan/controllers/package.py', line 216 in new
  clear_session=True)
File 'pyenv-ckan/src/ckan/ckan/controllers/package.py', line 357 in _render_edit_form
  edit_form_html = fs.render()
File 'pyenv-ckan/src/ckan/ckan/forms/builder.py', line 13 in render
  return render(self.form_template)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/templating.py', line 663 in render
  format=format, namespace=kargs, **cache_args)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/templating.py', line 546 in render
  **options)
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/plugin.py', line 110 in render
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/core.py', line 179 in render
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 60 in encode
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 425 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 592 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 698 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 532 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/core.py', line 283 in _ensure
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 569 in _include
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/markup.py', line 245 in _match
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 543 in _exec
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 533 in _eval
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 520 in _eval
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 286 in _eval_expr
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/eval.py', line 180 in evaluate
File 'pyenv-ckan/src/ckan/ckan/templates/package/form_fields.html', line 52 in <Expression u'h.literal(field.render())'>
  ${h.literal(field.render())}
File 'pyenv-ckan/lib/python2.5/site-packages/FormAlchemy-1.3.4-py2.5.egg/formalchemy/fields.py', line 1217 in render
  return self.renderer.render(**opts)
File 'pyenv-ckan/src/ckan/ckan/forms/common.py', line 311 in render
  return render('package/form_resources')
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/templating.py', line 663 in render
  format=format, namespace=kargs, **cache_args)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/templating.py', line 546 in render
  **options)
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/plugin.py', line 110 in render
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/core.py', line 179 in render
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 60 in encode
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 425 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 592 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 698 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 532 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/core.py', line 283 in _ensure
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 569 in _include
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/markup.py', line 245 in _match
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 543 in _exec
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 510 in _eval
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 520 in _eval
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 286 in _eval_expr
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/eval.py', line 180 in evaluate
File 'pyenv-ckan/src/ckan/ckan/templates/package/form_resources.html', line 14 in <Expression u"res['id'] if res else ''">
  <td><input name="${c.id}-${i}-id" type="hidden" value="${res['id'] if res else ''}" /></td>
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/eval.py', line 335 in lookup_item
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/eval.py', line 405 in undefined
UndefinedError: {'url': u'a', 'hash': u'a', 'description': u'a', 'format': u'a'} has no member named "id"

How to reproduce:

  1. Go to package edit form, enter a package resource but without a URL.
  2. Click preview.
1272379550000000 1272447296000000
#279 enhancement johnbywater dread v1.0 closed fixed Ensure package's license is an option when it's edited

With forms configurable to show a subset of licenses and ckan instances moving packages from server to server, there is a chance that a user will edit a package and the existing package's license is not displayed in the form. If the user submits this form without touching the license dropdown, then the value is lost.

So for this case we should add the package's license to the list displayed.

1270205931000000 1272451384000000
#302 enhancement johnbywater dread v1.0 closed fixed Advertise a package's feed visually

Visual display of a package's feed in the Package Read (/package/xyz) and Package History (/package/history/xyz) pages.

Motivation: each package's Atom feed is mentioned in the <link rel="alternate" type="application/atom+xml" ...> tag, which Firefox displays with a feed icon, but the other 3/4 of users may not discover it.

The Repo History (/revision) page has a visual link the feed.

PS It would be nice to display the feed icon in both these places too.

1272370397000000 1272453821000000
#267 defect johnbywater rgrp vdm-0.7 closed fixed Diffs for object that is created in that revision

Currently this raises an exception because there is no package/package_revision in existence at from_revision.

1268041383000000 1272960518000000
#303 defect dread dread v1.0 closed fixed Package history & diff & feed to show changes to tags, extras and resources

Currently a package's history page (/package/history/xyz) and related feed only shows changes to the core package i.e. ones which create a PackageResource?. This doesn't include changes to tags, extras and resources. These need to be added.

There is the complication that the 'diff' function doesn't currently work beyond PackageResource?. Unless this is quick and simple to fix, I suggest we let the diff remain slightly broken for now.

Rufus suggests alongside the existing methods:

pkg.all_revisions()
pkg.diff()

we have ones which also include the related objects (tags, extras, resources):

pkg.all_revisions_full()
pkg.diff_full()

You could have a function which returned these related objects. Ideas:

revobjlist = [ self, resources, extras ]
list of tuples: (revobj, changed_revobjs)
list of tuples: (revobj, {objtype: [changed_revobjs]})
tuple: (rev1, {Package: [ object_id, object_id], PackageTag: [object_id, object_id2] ...}
1272370659000000 1272989728000000
#305 defect johnbywater johnbywater v1.0 closed fixed Search result pagination is broken

Expect to page through results.

Only page 1 is shown, all other pages fail to display.

Reproduce by searching for something common and browsing to the second page.

1272468229000000 1272994804000000
#276 defect dread rgrp v1.0 closed fixed Resources in Package form seen multiple times upon preview

Create a new package with a name 'blah' and resource format 'blah'. Hit preview. There is an error because of the lack of resource url, but in the resource input boxes, there are now four resources with format 'blah'.

1269255399000000 1272996237000000
#177 enhancement rgrp dread closed invalid Service documentation

This is docs for users of a CKAN service (e.g. ckan.net), as opposed to someone installing / administering the software. (The latter has docs on knowledgeforge.

Editability - wiki?

1257243285000000 1273050236000000
#59 enhancement johnbywater rgrp v1.0 closed fixed Provide a basic guide to CKAN on the site
  1. Use contents from ckan package notes section (rgrp)
  2. Update that section to be more guide-like (jwyg)
1239018583000000 1273080019000000
#233 enhancement rgrp rgrp v1.1 closed fixed Allow simple site-specific customization/overriding of templates

Options:

  1. Allow for specification of genshi template paths to search in config. This way people can introduce their own templates and these templates can selectively override existing templates. (Already implemented this in shakespeare and it works)
  1. Include an extra site-specific genshi template which can then be used to customize site (e.g. by having specific calls to py:def that user can define but which are ignored if they don't exist).

Can do this using:

<xi:include href="base.html"><xi:fallback /></xi:include>

1263000396000000 1273080352000000
#133 defect rgrp dread closed fixed Security hole - search package/group (WUI & REST)

Using WUI or REST interface you can search packages and groups without authorization being checked.

On the REST interface you can also read all the attributes of the packages using the 'all-fields' option.

Can be fixed using more advanced query to check authz.

1254390168000000 1273253977000000
#132 defect rgrp dread closed fixed Security hole - read package/group list (REST)

Using REST interface you can list packages and groups without authorization being checked.

Can be fixed using more advanced query to check authz.

1254389493000000 1273254514000000
#291 defect rgrp dread closed worksforme Investigate search options encoding

On ckan.net there is this unchecked-in 'temporary hack'. It seems to be to do with foreign characters in search options. See what its doing and if necessary, put check it in.

diff -r 813ad8b5de0b ckan/lib/search.py
--- a/ckan/lib/search.py	Mon Mar 01 22:23:36 2010 +0100
+++ b/ckan/lib/search.py	Thu Apr 15 19:11:42 2010 +0200
@@ -63,6 +63,9 @@
     def query(self, options):
         '''For the given search options, returns a query object.'''
         self._options = options
+        # temporary hack!
+        if self._options.q:
+            self._options.q = self._options.q.encode('utf8')
         general_terms, field_specific_terms = self._parse_query_string()
 
         if not general_terms and \

1271351670000000 1273254895000000
#216 enhancement nickstenning dread v1.0 closed fixed UI Review - Search
  • allow clicking on 'only openly licensed'/'only downloadable' to select/deselect checkboxes: make them <label>s.
  • cut 'only's -- use word 'filter' instead
  • doesn't appear to match substrings!? A search for 'anna' doesn't return match the 'annakarenina' package.
  • autocomplete package names & tags
1260879802000000 1273596102000000
#223 enhancement nickstenning dread v1.0 closed fixed UI Review - Tag

Tags landing page tag

  • Both search for tags *and* browse the list of tags right here
  • Big cloud of tags (sorted by popularity/usage?) filterable with a search box.
  • Tags search page tag/search and list tag/list are both redundant and should be merged straight into tag.
1260880276000000 1273596111000000
#271 enhancement johnbywater johnbywater v1.0 closed fixed The system shall support distributing changes.

See DistributingChanges for analysis and functional requirements.

1268672137000000 1273596144000000
#259 defect rgrp dread v1.0 closed fixed isitopen enquiry broken

Pressing the 'make an enquiry' link on the package read page gives 404. (Also links should also be changed from isitopen.ckan.net to isitopendata.org)

Example link is:

http://isitopen.ckan.net/enquiry/create/?ckan_package=zeno

which gets forwarded to:

http://www.isitopendata.org/enquiry/create/?ckan_package=zeno

which gives 404

1267012666000000 1273596153000000
#258 enhancement rgrp dread v1.0 closed fixed Add uuids into package in REST

Add uuids into package in REST interface.

1266954722000000 1273596163000000
#257 enhancement dread dread v1.0 closed fixed Package relationships - 4. Read in API

API:

  • Appear in package listing Example: 'relationships': [{'is_dependency_of':'osm', comments:'Since version 0.2'}, {'is_parent_of':'bobs_maps'}]
1266928630000000 1273596170000000
#255 enhancement dread dread v1.0 closed fixed Package relationships - 2. Read in WUI

WUI:

  • View: show both sides of the relationship (but think carefully -- e.g. a given package may have *many* dependents ...)
1266928542000000 1273596174000000
#254 enhancement dread dread v1.0 closed fixed Package relationships - 1. model

New domain object: PackageRelationship? (revisioned)

Attributes:

  • subject (Package reference)
  • object (Package reference)
  • type (string)
  • comment

Relationship type values: depends_on (dependency_of) derives_from (has_derivation) child_of (parent_of)

Relationship type is stored as a string, as given in the first column. The reverse relationship (bracketed) is given just for display purposes only.

Subject / Object - as in a sentence: "Dracos is derived from OSM" is <subject> <predicate> <object>. We'll use 'type' instead of 'predicate' as that is a more familiar word.

1266928449000000 1273596180000000
#311 defect dread rgrp v1.1 closed fixed Reordering of package resources can lead to integri

Created a new package resource line and then moved it up above existing one and hit save. Result: 500 error. In logs have:

[Sat May 08 21:55:41 2010] [error] [client 86.26.8.30] Error - <class 'sqlalchemy.exceptions.IntegrityError'>: (IntegrityError) duplicate key value violates unique constraint "package_resource_revision_pkey", referer: http://ckan.net/package/edit/cofog
[Sat May 08 21:55:41 2010] [error] [client 86.26.8.30]  'INSERT INTO package_resource_revision (id, package_id, url, format, description, hash, position, state, revision_id, continuity_id) VALUES (%(id)s, %(package_id)s, %(url)s, %(format)s, %(description)s, %(hash)s, %(position)s, %(state)s, %(revision_id)s, %(continuity_id)s)' {'hash': '', 'description': 'The Treasury record of COFOG functions. ', 'format': 'XLS', 'url': 'http://www.hm-treasury.gov.uk/d/cofog_definitions_coins250609.xls', 'package_id': '8482334d-fe2e-4285-9114-5243130f80c0', 'state': 'active', 'continuity_id': '8bf302db-8a80-47d3-b5dc-bc07512a3928', 'position': 3, 'revision_id': 'e4e2cb2d-4bd5-414e-b646-e484f174d9ab', 'id': '8bf302db-8a80-47d3-b5dc-bc07512a3928'}, referer: http://ckan.net/package/edit/cofog
1273348968000000 1274282065000000
#319 defect dread dread closed fixed Local Authority license misnamed in migration script 18

Migration script 18 converts license "OKD Compliant::Local Authority Copyright with data.gov.uk rights" into "uklocalauthority-withrights" when it should be "localauth-withrights", according to the license service SoS 2. This causes clients difficulties looking it up.

1274349714000000 1274366882000000
#326 task dread dread v1.1 closed fixed Centralise importation of json library

Later versions of python use json which is better than simplejson, but it must be kept as an option for compatibility. So centralise the import of json to ckan.lib.helpers.

1274784223000000 1274789296000000
#329 defect rgrp dread v1.1 closed fixed Bad dates cause exception on Gov form

Reproduction

Using the government form, create a new package with name 'test' and date released of '23/5/0210'. The result is a 500 error and 'Server Error' message.

Affects all versions of CKAN.

Why it's happening

The dates module is raising an exception on the invalid date when saving the date, which is not being caught. The exception should have been raised only during the earlier 'validation' step and that would be caught.

1275060617000000 1275079189000000
#308 enhancement rgrp rgrp closed duplicate Autocomplete package names & tags in package search

Extracted from ticket:216.

Dubious of its merit.

1273050549000000 1275302577000000
#330 defect dread dread closed fixed getdata/ons timezone not recognised on non-British servers

The python time module is supposed to recognise timezones such as 'UTC', 'GMT' and 'BST' using the %Z parameter. This works fine on British installs, but the buildbot (for example) gives this error:

DateConvertError?: Could not read date as ISO format "%a, %d %b %Y %H:%M:%S %Z". Date provided: "Mon, 04 Jan 2010 09:30:00 BST"

It turns out that it only recognises local names of timezones.

The ONS import doesn't care much about timezone, so we should just ignore it, avoiding these problems.

1275300271000000 1275303122000000
#328 enhancement dread rgrp v1.1 closed fixed Mention code libraries in API documentation

Should prominently (at the top?) mention existing code libraries for working with ckan api. Have:

  • Python: ckanclient
  • Perl: luke closs wrote something
  • PHP: drupal library?
  • ...?
1274988106000000 1275318745000000
#313 enhancement rgrp dread v1.1 closed fixed Allow packages to be specified by IDs in REST interface

If a package name changes, a simple CKAN client may not be aware of this (not monitoring the push notifications, revisions or feed), so it is preferable to refer to the package by its (invariant) ID.

It is still useful to refer to a package by its name though, so both should be valid arguments in the REST interface.

1273606248000000 1275404524000000
#333 enhancement dread v1.1 closed wontfix CKAN front end requirements for package notifications

Use case: new package

  1. An external front-end system provides a web page with a list of packages. Each package has the option to edit it or and there is also a button to create a new package.
  1. User: clicks 'new package'.
  1. CKAN presents the package/new form to the user.
  1. (After a couple of previews) User: clicks 'commit'.
  1. Notification message goes from CKAN to the front-end detailing the new package.
  1. The user is redirected back to the front-end web page displaying the list of packages, which contains the new one.

The notification message (step 5) has to get through to the front-end that the new package is created before the redirect (step 6). This suggests that the message sending needs to be *synchronous*, i.e. acknowledged by the front-end, before CKAN redirects the user to the front-end package listing page (step 6).

In addition, this use case suggests the front-end listens for package notifications, to save another call to CKAN to get the package details, before the displaying the list of packages. If this isn't possible (see next use case) and it must listen for revision notifications instead, then perhaps it is worth including the full package details in the payload for the revision notification message. Would there be a problem with such a large message in the next use case, with 100 packages?

Use case: CKAN imports packages

  1. CKAN administrator runs a script that adds 100 new packages into CKAN.
  1. CKAN sends notification message to front-end to report the new packages/revisions.
  1. Knowing there are new revisions, the front-end queries the CKAN revision interface to get the list of new packages.
  1. The front-end queries CKAN for each new package one-by-one.
  1. A new user request to the front-end will include the info about the new packages.

The package addition could be achieved in 1 revision, 100 revisions or some compromise:

  • If it is 1 revision then potentially there are problems displaying the long list of packages in the 'recent changes'.
  • If it is 100 revisions, then the notification webhook would be called 100 times, which creates unnecessary load on the front-end. Suppose each Webhook call-back (step 2) triggers the front-end to make a call to CKAN to get the latest revisions (step3), in this case it would make 100 calls, most of them fruitless, causing unnecessary load on CKAN.

This use case suggests a bulk import of packages should go into one revision, and therefore generate one revision notification message and 100 package notification messages. The front-end client should listen to only revision messages.

1275324042000000 1275407987000000
#128 enhancement dread rgrp v0.11 closed fixed Add ckan_url attribute to REST JSON representation of a Package

Add ckan_url attribute to REST JSON representation of a Package pointing to the (read) url of package on CKAN.

Cost: 30m

1253866713000000 1275694573000000
#315 defect dread rgrp v1.1 closed fixed Improvements and fixes to csv dump
  1. Issues with quote in fields: http://lists.okfn.org/pipermail/ckan-discuss/2010-May/000240.html
  1. Issues with package resource serialization into csv table.

<quote> In the latest dump there were 116(!) sets of the three columns (“resource-[n]-url”, “resource-[n]-format”, and “resource-[n]-description”). However, these are an extract of the packed “resource” column and I’m not sure whether they’re needed. Also, they irritatingly don’t appear in order in the CSV serialisation. If the resource columns could be ordered in the file that would be great; if a second version without the unpacked resource data would be excellent. </quote>

1274100887000000 1275846764000000
#335 enhancement dread dread v1.1 closed fixed Post-package-edit redirect to configurable URL

As a

third-party interface to a CKAN instance

I want to

link to CKAN's package creation/editing pages. On 'commit', have the user redirected back to a URL in my interface that I can control. Also, when the package is created new, I need to be told what the new package's name is on return.

Design

  1. The 'return URL' is passed as a parameter to CKAN.
  2. CKAN substitutes the package name into the return URL.

Example

Front-end links to: http://ca.ckan.net/package/new?return_to=http://datadotgc.ca/dataset/<NAME> (but with the parameter URL-encoded)

When finished editing and the user commits, CKAN redirects the user to: http://datadotgc.ca/dataset/pollution_data

1275408834000000 1276179605000000
#342 enhancement dread dread closed fixed JSONP parameter in API

As a

CKAN client using JQuery

I want to

call the CKAN API and instead of receiving back JSON I get JSONP. i.e. "%s(%s)" % (callback, json_content)

Suggested implementation

All API calls allow the JSONP 'callback' parameter to be specified in the request and this wraps the JSON response. See suggested patch to rest.py by Donovan Hide:

http://knowledgeforge.net/ckan/trac/attachment/ticket/336/resource.patch

Test

import re import unittest

def test_jsonp_callback():

response = self.app.get('/api/search/resource/?url=http://www.scraperwiki.com&callback=jsoncallback') match = re.match('jsoncallback\(.*\);',response) self.assertTrue(match)

response = self.app.get('/api/search/resource/?url=http://www.scraperwiki.com') match = re.match('jsoncallback\(.*\);',response) self.assertFalse(match)

I think the point needs to be made that JSONP only works for GET requests and not POST/PUT/DELETE, so there needs to be a check for that in the _finish_ok method.

(thanks to Donovan Hide for test)

1276166426000000 1276278485000000
#314 defect johnbywater johnbywater closed fixed Bugs getting revisions from the REST API

Bug report regarding getting revisions:

Getting revisons by ID (on the latest ID) GET "http://test-hmg.ckan.net/api/search/revision?since_revision=44aac9b6-ba24-43a8-87a1-f6923dc523ff"

Returns a whole load of stuff (it's also quite slow - about 10 seconds)

I'm expecting it to return just an empty array - am I doing something wrong here - if so could you clarify correct use of the API?

GET "http://test-hmg.ckan.net/api/search/revision?since_time=2010-04-30T23:45" Returns the empty string - I'd expect an empty array ie []

GET "http://test-hmg.ckan.net/api/search/revision?since_time=2010-04-31T23:45" Returns an internal server error 500 - I think it should probably be "bad Request" 400 (the date is invalid)

1273743755000000 1276523983000000
#357 defect johnbywater johnbywater closed fixed Fix API documentation, to correct misleading HTTP_AUTHORIZATION statement.

Sean Burlington wrote:

I recently came across a documentation issue

http://knowledgeforge.net/ckan/doc/ckan/api.html


The key should be passed in the API request header: Header Example value HTTP_AUTHORIZATION fde34a3c-b716-4c39-8dc4-881ba115c6d4


But the header name is actually just 'AUTHORIZATION'

For example to set it in PHP

curl_setopt($ch, CURLOPT_HTTPHEADER, array('AUTHORIZATION: xxxxx-xxxx-xxxx-xxxx-xxxxxxx'));

1277459886000000 1277461466000000
#344 defect dread closed fixed REST Create package with incorrect format gives 500 error

PUT to /rest/api/package of {"name": "name", "resources": ["someurl.com/data"] } (i.e. resource is a string, not a dictionary) gives 500 error, when it should give a 400 error and helpful error message.

Exception:

Module ckan.controllers.rest:154 in create
<<              if register == 'package' and not subregister:
                       fs = ckan.forms.get_standard_fieldset()
                       request_fa_dict = ckan.forms.edit_package_dict(ckan.forms.get_package_dict(fs=fs), request_data)
                       fs = fs.bind(model.Package, data=request_fa_dict, session=model.Session)
                   elif register == 'package' and subregister in model.PackageRelationship.get_all_types():
>>  request_fa_dict = ckan.forms.edit_package_dict(ckan.forms.get_package_dict(fs=fs), request_data)
Module ckan.forms.package_dict:88 in edit_package_dict
<<                      for res_dict in value:
                               res_dict_str = {}
                               for key, value in res_dict.items():
                                   res_dict_str[str(key)] = value
                               resources.append(res_dict_str)
>>  for key, value in res_dict.items():
AttributeError: 'unicode' object has no attribute 'items'
1276341172000000 1277477712000000
#341 enhancement dread closed fixed Web UI accepts package IDs in URLs

(in the same way that we can refer to packages in the API by ID as well as name.)

As a

client of CKAN

I want to

link to a package page in the UI, referring to it by ID

1276162400000000 1277483030000000
#322 enhancement dread dread v1.1 closed fixed Client interface for Notification Service

Use cases

  • Register for package changes
  • Register for all revisions
  • Notified of a package change
  • Notified of a revision
  • Deregistration
  • Configuration of port in pylons config

Design

  • Default port: 5672 (standard for AMQP)
  • Exchange name: 'ckan'
  • Exchange type: topic exchange (most flexible)
  • Routing keys: (see below)

Routing detail

Routing key format: "OBJ_TYPE" (NB tags should be identified by their name, not ID)

Example routing keys

  • 'package' - Package edited/created
  • 'resource' - Resource edited/created
  • 'revision' - Any change
  • 'db.clean'
  • 'db.rebuild'

Example queue bindings that clients may use:

  • * - no filtering - client receives all notifications
  • package - only changes to packages
  • revision - all revisions
  • db - all database operations

Versioning

Since message payloads will be tied into the REST Entities, it makes sense to join up with the REST versioning. This could be achieved by providing new exchanges called 'ckan-1.1' perhaps?

Documentation

  • How to use
  • simple example of an external client?
1274720042000000 1277722821000000
#347 enhancement johnbywater johnbywater closed fixed The system shall present the package edit form in an API 1276523692000000 1277820440000000
#348 enhancement johnbywater johnbywater closed fixed The system shall accept package edit form submissions in an API 1276523793000000 1277820496000000
#343 defect johnbywater dread closed fixed Packages referred by ID in API

When you do a Package Search or query a Package Relationship in the API version 2, the responses have relationships which refer to packages by name, not by ID.

1276180179000000 1278066420000000
#339 defect johnbywater closed fixed Remove download_url from package data format in API Version 2, but maintain the old data format in the old version of the interface (API Version 1) 1275901499000000 1278074705000000
#273 task pudo rgrp v1.1 closed fixed Investigate search index options and create tickets

Work already here: http://knowledgeforge.net/ckan/trac/wiki/SearchEngine

Add knowledge there or to:

http://wiki.okfn.org/SoftwareTools/Search

1268996933000000 1278578527000000
#323 enhancement dread dread v1.1 closed fixed Notification message

Which events to notify on

Listed by domain object, these are the notification message 'change types' that will be sent:

  • Package
  • PackageResource

Also it is clear that it could be useful to know when db-wide maintenance is carried out:

  • db - 'clean', 'rebuild' (db is wiped and replaced with new data), 'upgrade' (migration)

Ignored domain objects

These parts of the domain model will not carry notifications as no use case has been identified for them:

  • Revision
  • Group
  • Tag
  • Rating
  • User - list of users is sensitive info
  • Relationships - complicated
  • Authz - complicated and sensitive info
  • License - change of a license's metadata is a question for the 'license service'

Message format

A notification message's header contains the routing key, identifying the object type. The client is probably interested in the object (all use cases so far), so it makes sense to send the object in the payload. This should be the JSON-encoded dictionary exactly as provided for the object's REST Entity.

For the 'db' notifications there shall be no payload.

1274723333000000 1278578841000000
#324 enhancement dread dread v1.1 closed fixed Search indexing using notifications

Currently search indexing is triggered directly using a Postgresql db callback. Now take advantage of the Notification system to register interest in all package changes and db changes to trigger this instead.

The indexing shall run in a separate shell/process, managed by supervisord.

1274723483000000 1278599927000000
#325 enhancement dread dread v1.1 closed fixed Event push notification

As a

CKAN client program

I want to

be notified when changes to the CKAN metadata occur.

Examples of use

  1. An external search engine needing to (re)index a package. (interest: Package)
  2. A front-end system that caches package info and wants to know when it changes, to keep in step. (interest: Package or Revision) See further details here: ticket:352 and previous iteration here: ticket:333.
  3. A system for automatically checking package URLs and resource URLs as they are put on the system. This could alert to bad URLs and automatically email feedback to (meta)data owners. (interest: PackageResource)
  4. Do some processing on resource (e.g. extract sample data for display) (interest: PackageResource)

Context

The current state of CKAN can be queried through the REST API, you can keep track of changes by reviewing the feeds, but there is no way to find out the instant something is changed, without costly polling.

Design

Split-off into two tickets:

  • Notification message - ticket:323
    • Which events to notify on
    • Message format
  • Interface for Notifier Service - ticket:322

Testing

To test notifications, Carrot / AMQP will be configured to use a native-Python Queue, instead of requiring RabbitMQ to be running on the machine.

1274723512000000 1278599979000000
#356 enhancement rgrp v1.1 closed fixed Search box in at top of page (UI)

A small but useful ui improvement would be to have a search box at top right on every page.

As an example see the one here on trac or on github.com or bitbucket.org.

  • It would be particularly good to include a small advanced search link that took you to the full search page. Need to keep it small because screen real-estate here is limited (see how github.com does this for inspiration).
1277235411000000 1278931830000000
#226 enhancement nickstenning dread v1.1 closed fixed UI Review - History

Repository History revision

  • What is this page called? "Recent Changes", "Repository History", "/revision": standardise. Between the link in the nav and the page <title> particularly, but the route is also important. Perhaps /changes or something similar?
    • Will change page title on /revision/ to Revision History. Will not change route for the time being.
  • needs copyediting.
  • Pagination has similar issues to elsewhere. Also, most obvious here, is the fact that we don't need to display a link to every possible page. Please can we limit it to, say, a dozen nearby pages and an ellipsis.
  • Without looking at dates, its not clear whether I'm seeing most recent or oldest changes. Change pagination to say "Older"/"More recent" rather than "Previous"/"Next". (wontfix: now have text saying we are showing most recent changes)
  • Table layout is pretty ugly (yes, I'm aware this is my fault).
    • wontfix - nothing better at the moment
  • Timestamps are horribly unreadable. At the absolute minimum get rid of the micros. Hover for more detail?
  • Can we link to an author page? Yes!
  • Atom feed should have a feed icon~~
  • Why are we adding another <link rel="alternate"> to this page? We already have one for recent changes on every page in the site and the one we're adding has a less descriptive title. Which is correct? Use that one as a feed for every page.
1260880515000000 1279130390000000
#320 enhancement dread rgrp v1.1 closed fixed site_title configuration variable which is used in template

As a sysadmin I want to configure basic site title information for use in the site templates.

Implementation:

  • ckan.site_title config variable
  • set this on g in app_globals.py e.g.
    • from pylons import config; g.site_title = config.get('ckan.site_title, 'CKAN - Comprehensive Knowledge Archive Network')
  • use in head title and in main site title/logo section (use it as alt on logo image)
  • Also all other pages (e.g. index, about) which talk about CKAN
    • Is this needed? Would it not be better for people who want to customize the site to simply overwrite those templates?

Questions:

  • Do we want a site_logo variable whic his use for site title/logo section instead of site_title if site_logo defined?
  • Probably yes, but not part of this ticket.
1274378967000000 1279130535000000
#227 enhancement nickstenning dread v1.1 closed fixed UI Review - General checks
  • There is huge inconsistency in the titles of pages across the site: to give one example "Edit Package:" vs "Package: mypkg [not linked]" vs "History - mypkg [linked to package page]"
    • Normalize and use terminology: "Data Package" - also change in navbar (but nowhere else for the time being). So hvae Data Package - mypkg, Data Package - mypkg - Edit etc
  • pagination is currently lost in amongst the results list. needs to be *much* more obvious, and should appear top and bottom, or at the very least at the bottom!
1260880552000000 1279192621000000
#317 enhancement pudo rgrp v1.1 closed fixed Make search pluggable

Make lib/search.py pluggable so that we can plug in different search systems (e.g. SOLR).

Suggest we define a base Search class from which specific search implementations inherit (e.g. SQLSearch, SOLRSearch etc). The specific one being used would then be set via a config variable.

1274295418000000 1279286041000000
#263 enhancement johnbywater rgrp v1.1 closed fixed Improve and test openid login

At the moment the user has to figure out to login with providers -- this is not always obvious (e.g. google).

We could improve this with a small bit of javascript. As an example see: http://standalone.demo.civicrm.org/standalone/

Not sure exactly what library that is using (or if bespoke) so alternatives include (NB: we already use jquery):

In addition to implementing this we also need to test logging in with main providers: google, wordpress -- as we have had reports of it not working with e.g. wordpress (not sure if this testing can be automated really -- best hope would be selenium I think -- so OK if done by hand).

Cost: 3h (2h UI), (1h testing)

1267605039000000 1279292730000000
#310 defect dread rgrp v1.1 closed fixed Commit message box looks wrong in edit page since edit style overhaul

Suggest move this below the label and make full width of screen and only 3/4 rows high (more like a wiki site).

  • Also change label to: Edit summary (Briefly describe the changes you have made)
  • Remove: you can markdown formatting here.
  • Move author: if you have not signed in smaller and closer (like markdown instructions are nwo).
  • Change commit -> save
  • Remove "please save" just have the bullet points
1273348714000000 1279300525000000
#365 enhancement dread dread closed fixed ResourceNotifications

If you change a resource then you not only get a PackageNotification?, but also a ResourceNofication?.

1279037411000000 1279300621000000
#337 enhancement johnbywater rgrp v1.1 closed fixed Download links for resources should open in new window

target = _blank

Cost: 30m

1275422151000000 1279300972000000
#367 enhancement dread dread closed fixed Notfication monitor

Runs on the commandline and prints out notifications.

1279303310000000 1279303693000000
#336 defect dread donovanhide v1.1 closed fixed Resource Search API

As a

CKAN client such as ScraperWiki

I want to

search for Package Resources, either by URL or other field, or just get them all. I want to be able to get all the resource's fields, such as URL.

Proposed implementation

Add resource search API at:

/api/search/resource

AND resource added to model API at:

api/rest/resource

(see ticket:358)

Functional differences from the ScraperWiki suggested patch:

  • URL is not normalised

  • URLs are not grouped
  • All fields of the resource object are returned, not just the URL
  • Package is identified by its ID, not name or full URL. (This is for consistency in the API - you can simple prepend 'http://ckan.net/package/' to the package ID)

This is to make our API more general, simple and consistent. It means that the ScraperWiki client has to do a bit more processing to get exactly what it needs. Is this ok?

Example search

POST

{"url": "scraperwiki.com/", "all_fields": 1}

to: /api/2/search/resource

returns JSON:

 [{"id": "a3dd8f64-9078-4f04-845c-e3f047125028",
   "package_id": "b8a325c8-af2a-43f3-8245-9db7d73dfbfe",
   "URL": "http://scraperwiki.com/lincolnshire-councillors", 
   "format": "CSV", 
   "Description": "Scrape of www.lincs.gov/councillors.pdf by ScraperWiki.",
   "hash": "", 
   "position": 2
 }]

Note use of package_id instead of package_name is something we're moving towards in the API, since names can change. When we've done ticket:341 then ckan.net/package/lincs-councillors will be a synonym of ckan.net/package/b8a325c8-af2a-43f3-8245-9db7d73dfbfe

Search Parameters

Key:  q
Description: Search all resource fields for the value

Key: url / description / format / 
Description: Search particular field for the value

Key: all_fields
Value: 0 or 1 (0 is default)
Description: If 1 (true), the full record of the package resource
(and it's package reference) are returned, rather than just the
PackageResource ID.

May also choose to introduce 'offset' and 'limit' to page through a large number of results.

JSONP achieved through API-wide parameter - see ticket:342

Search is case insensitive.

Original request

Hi, have attached a patch for adding a resource list api call. Have also added a JSONP compatible callback section, along the lines of #388.

Could also add a search version. Not sure what the best url would be for that though.

Haven't written a test as the structure seems to follow a functional spec. Is that document around somewhere?

Donovan

1275411765000000 1279373842000000
#379 task johnbywater johnbywater v1.1 closed fixed My dev task #1 1280325277000000 1280326496000000
#380 task johnbywater johnbywater v1.1 closed fixed My dev task #2 1280325304000000 1280326595000000
#378 story johnbywater johnbywater v1.1 closed fixed My user story #1 1280325186000000 1280326706000000
#374 task johnbywater johnbywater v1.1 closed fixed Design servicization of CKAN API 1280223118000000 1280398501000000
#388 task johnbywater johnbywater v1.1 closed fixed Reply to "two projects" question from RS 1280395503000000 1280405288000000
#375 task johnbywater johnbywater v1.1 closed fixed Integrate Forms API into Drupal

Requested by DGU.

1280223168000000 1280405812000000
#389 task johnbywater johnbywater v1.1 closed fixed Investigate timezone issue raised by SB 1280395554000000 1280422414000000
#387 task johnbywater johnbywater v1.1 closed fixed Fix failing tests in search sub-system trigged by recent changes 1280357804000000 1280446047000000
#338 story johnbywater johnbywater v1.1 closed Reference groups by ID in addition to name, since group names can change 1275901137000000 1280446480000000
#385 task johnbywater johnbywater v1.1 closed fixed Send email if QoS measurement breaks QoS criteria 1280355527000000 1280496127000000
#392 task johnbywater johnbywater v1.1 closed fixed Create monitoring feature to generate QoS availability log 1280446769000000 1280496140000000
#384 story johnbywater johnbywater v1.1 closed Send alert when QoS measurements break expectation 1280347841000000 1280496812000000
#393 task johnbywater johnbywater v1.1 closed fixed Create aggregation feature to aggregate QoS log 1280446811000000 1280496962000000
#332 defect pudo rgrp closed fixed Autocomplete tag on click

During tag autocompletion, clicking on a suggested tag doesn't trigger completion in the text box.

1275302887000000 1280743320000000
#309 defect pudo dread closed fixed Tag list shows unused tags

Deleted tags are visible on the /tag page and are searchable.

Delete tags are ones that were in use once and then discarded, so this includes ones which are not used on any packages any more, including misspelling, mistakes, and deprecated tags. It would makes sense to remove them.

Example tag that is viewed at /tag and searchable: http://ckan.net/tag/2rand[0

1273247648000000 1280743432000000
#334 defect pudo dread closed fixed Wrong link for package feed icon

This is a problem with the package page e.g. http://ckan.net/package/open-election-data-project . The feed icon links to [1] and the text next to it "Subscribe" links to [2]. Surely these should be the same? The second link seems to right one to me.

1275407445000000 1280743667000000
#353 defect dread closed fixed SOLR search indexing

As a

SOLR instance

I want to

keep my search index of CKAN packages up-to-date

Implementation

  • Using asynchronous event notifications
  • Running in a separate process to CKAN
1277123480000000 1280756399000000
#190 defect pudo dread closed fixed Package comments

Cost 7 days

When viewing a package, users can read user comments and leave their own. Users need to be logged in to leave a message. Comments appear immediately. A mechanism for deleting unwanted comments is provided to an authorized user. Comments are sorted with the most recent first. Comments are available for read, creation and deletion in both the Web UI and over the REST API.

The admin for the package and a superuser can delete unwanted comments, both on the package page and a collation of all comments on their user page. Users can delete their own comments(?) Need to consider whether over the REST API we encourage the use of a 'frontend user' APIKEY which can be used to leave comments for another, actual user.

Example at bottom of package page:

Leave a comment:

Subject _
Comment _


Submit button

Comments:

Explanation doc
Posted on 25h May 2009 by http://bertdavies.myopenid.com
It says on the pollution data web page that not all the stations have a CO2 sensor, so you have to extrapolate from the ones that do. See my visualisation of CO2 across London for an idea of what you can do: bertdavies.com/pollution-2008.jpg
More info
Posted on 24th May 2009 by http://ronsmith.myopenid.com
Excellent data, but why is there no value in the CO2 column for some of the testing stations?

Implementation details:

Comments table is with columns:

id package_id date (date) comment (multi-line text)
1258387105000000 1280820852000000
#242 defect pudo rgrp v1.1 closed fixed Miscellaneous tidying up (v0.11)

Dumping ground for various minor and miscellaneous items (mainly refactorings):rgrp

  • controllers/tag.py: why does this not use lib/search.py (but controllers/packages.py search method does)?

done:

  • lib/search.py: inheritance would be nicer than switching on entity in search (i.e. have SearchPackage?, SearchTag? etc)
1265715103000000 1280823876000000
#383 task johnbywater johnbywater v1.1 closed fixed Create middleware feature to generate QoS throughput log 1280347595000000 1280854578000000
#391 task johnbywater johnbywater v1.1 closed fixed Create monitoring feature to generate QoS responsiveness log 1280446714000000 1280854592000000
#382 story johnbywater johnbywater v1.1 closed Measure quality of service parameters

As a service administrator, I want to measure responsiveness, throughput, and availability of a CKAN service.

1280346974000000 1280854608000000
#397 task johnbywater johnbywater v1.1 closed fixed Implement QoS monitoring for DGU 1280854935000000 1281000745000000
#376 task johnbywater johnbywater v1.1 closed fixed Implement servicization of CKAN API

Requested by DGU.

1280223226000000 1281000755000000
#275 defect dread rgrp closed fixed Add forms renderer for JsonType fixing bugs in admin interface

This will solve e.g. issues in formalchemy admin interface (which is due to lack of renderer for Json type)

Cost: 0.5h

from ckan.model.types import JsonType

forms.FieldSet.default_renderers[JsonType] = formalchemy.fields.TextFieldRenderer
1269122740000000 1281002082000000
#386 task johnbywater johnbywater v1.1 closed fixed Set limits in /etc/security/limits.conf 1280356594000000 1281088994000000
1 2 3 4 5 6 7 8 9 10 11 12 13
Note: See TracReports for help on using and creating reports.