{22} Trac tickets (2647 matches)

Because CKAN's worth it!

Native login scenario: Complete login form, require email address. Any session is resumed.

Mandated technical requirement: use Repoze (see plugin:form of repoze.who); borrow from BlastOff?.

See CKAN or Microfacts for usage of repose with openid.

Want to have:

• /api/search/revision/

Support querying by "since time" and "since revision". Each individual revision should be available in the REST API (read-only):

• /api/rest/revision/{revision ID}
  • should include revision attributes plus revision.packages e.g. like { "id": "...", ... "packages": [ list of packages names from e.g. revision.packages ] }

There is code to create packages in-process (ckan/lib/importer) and via HTTP (ckanclient/loaders/base). There is also code to read different kinds of package source (Google Spreadsheets spreadsheet, other spreadsheet).

Propose to refactor all this to have template method in base command class (subclasses may implement command line interface, graphical user interface, or Web interface), where the template method calls on aggregated strategy objects for: reading raw spreadsheet data from spreadsheet application; for inferring CKAN package entities from the raw spreadsheet data; and for writing package entities to CKAN service.

Spreadsheet readers to be factored to allow for variations of spreadsheet application.

Raw data inferrers to allow for variations in spreadsheet cell structure.

CKAN writer strategies to allow for either CKAN client or direct linking.

Actual cases then supported by specifying existing strategies (or defining new ones) in a script that invokes the command class.

Intended outcome:

See resource url validation error when previewing package with a resources that has no url.

Recurring Error:

URL: http://127.0.0.1:5000/package/new
File 'pyenv-ckan/lib/python2.5/site-packages/weberror/evalexception.py', line 431 in respond
  app_iter = self.application(environ, detect_start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/repoze/who/middleware.py', line 107 in __call__
  app_iter = app(environ, wrapper.wrap_start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/beaker/middleware.py', line 73 in __call__
  return self.app(environ, start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/beaker/middleware.py', line 152 in __call__
  return self.wrap_app(environ, session_start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/Routes-1.10.3-py2.5.egg/routes/middleware.py', line 130 in __call__
  response = self.app(environ, start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/wsgiapp.py', line 125 in __call__
  response = self.dispatch(controller, environ, start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/wsgiapp.py', line 324 in dispatch
  return controller(environ, start_response)
File 'pyenv-ckan/src/ckan/ckan/lib/base.py', line 50 in __call__
  return WSGIController.__call__(self, environ, start_response)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/controllers/core.py', line 221 in __call__
  response = self._dispatch_call()
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/controllers/core.py', line 172 in _dispatch_call
  response = self._inspect_call(func)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/controllers/core.py', line 107 in _inspect_call
  result = self._perform_call(func, args)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/controllers/core.py', line 60 in _perform_call
  return func(**args)
File 'pyenv-ckan/src/ckan/ckan/controllers/package.py', line 216 in new
  clear_session=True)
File 'pyenv-ckan/src/ckan/ckan/controllers/package.py', line 357 in _render_edit_form
  edit_form_html = fs.render()
File 'pyenv-ckan/src/ckan/ckan/forms/builder.py', line 13 in render
  return render(self.form_template)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/templating.py', line 663 in render
  format=format, namespace=kargs, **cache_args)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/templating.py', line 546 in render
  **options)
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/plugin.py', line 110 in render
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/core.py', line 179 in render
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 60 in encode
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 425 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 592 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 698 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 532 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/core.py', line 283 in _ensure
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 569 in _include
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/markup.py', line 245 in _match
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 543 in _exec
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 533 in _eval
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 520 in _eval
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 286 in _eval_expr
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/eval.py', line 180 in evaluate
File 'pyenv-ckan/src/ckan/ckan/templates/package/form_fields.html', line 52 in <Expression u'h.literal(field.render())'>
  ${h.literal(field.render())}
File 'pyenv-ckan/lib/python2.5/site-packages/FormAlchemy-1.3.4-py2.5.egg/formalchemy/fields.py', line 1217 in render
  return self.renderer.render(**opts)
File 'pyenv-ckan/src/ckan/ckan/forms/common.py', line 311 in render
  return render('package/form_resources')
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/templating.py', line 663 in render
  format=format, namespace=kargs, **cache_args)
File 'pyenv-ckan/lib/python2.5/site-packages/Pylons-0.9.7-py2.5.egg/pylons/templating.py', line 546 in render
  **options)
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/plugin.py', line 110 in render
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/core.py', line 179 in render
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 60 in encode
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 425 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 592 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 698 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/output.py', line 532 in __call__
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/core.py', line 283 in _ensure
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 569 in _include
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/markup.py', line 245 in _match
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 543 in _exec
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 510 in _eval
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 520 in _eval
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/base.py', line 286 in _eval_expr
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/eval.py', line 180 in evaluate
File 'pyenv-ckan/src/ckan/ckan/templates/package/form_resources.html', line 14 in <Expression u"res['id'] if res else ''">
  <td><input name="${c.id}-${i}-id" type="hidden" value="${res['id'] if res else ''}" /></td>
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/eval.py', line 335 in lookup_item
File 'pyenv-ckan/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-linux-i686.egg/genshi/template/eval.py', line 405 in undefined
UndefinedError: {'url': u'a', 'hash': u'a', 'description': u'a', 'format': u'a'} has no member named "id"

How to reproduce:

1. Go to package edit form, enter a package resource but without a URL.
2. Click preview.

Bug report regarding getting revisions:

Getting revisons by ID (on the latest ID) GET "​http://test-hmg.ckan.net/api/search/revision?since_revision=44aac9b6-ba24-43a8-87a1-f6923dc523ff"

Returns a whole load of stuff (it's also quite slow - about 10 seconds)

I'm expecting it to return just an empty array - am I doing something wrong here - if so could you clarify correct use of the API?

GET "​http://test-hmg.ckan.net/api/search/revision?since_time=2010-04-30T23:45" Returns the empty string - I'd expect an empty array ie []

GET "​http://test-hmg.ckan.net/api/search/revision?since_time=2010-04-31T23:45" Returns an internal server error 500 - I think it should probably be "bad Request" 400 (the date is invalid)

Sean Burlington wrote:

I recently came across a documentation issue

​http://knowledgeforge.net/ckan/doc/ckan/api.html

---

The key should be passed in the API request header: Header Example value HTTP_AUTHORIZATION fde34a3c-b716-4c39-8dc4-881ba115c6d4

---

But the header name is actually just 'AUTHORIZATION'

For example to set it in PHP

curl_setopt($ch, CURLOPT_HTTPHEADER, array('AUTHORIZATION: xxxxx-xxxx-xxxx-xxxx-xxxxxxx'));

Set limits in /etc/security/limits.conf so that we can always ssh in at least. Requested by DGU.

Requested by DGU.

please ignore

Have TSO configure data.ov.uk/api/catalogue. Have TSO register catalogue.data.gov.uk.

Thinking about putting something like this:

Dumps Interface (this is NOT going in the API, it is another interface of the service):

• resources:

Daily Dump Folder Daily Dump File

• locations:

/dump/hmg.ckan.net-YYYY-MM-DD.[json|csv].zip

• data format:

Dump Folder is a HTML document containing a table of links to Dump Files Dump File has either JSON or CSV format.

• status codes (200, 404, 500)

Either CKAN writes to SOLR and Drupal reads from SOLR, or CKAN writes to SOLR and Drupal reads SOLR via CKAN API (so search resource locations are unaffected).

• Move script out to ckandgu repo
• Change script to convert xml into package dicts
• Test (against test.ckan.net, hmg.test.ckan.net)
• Deploy

We don't want /api to be an HTML page, but rather provide information about the API in a JSON structure.

Common requirements for running CKAN behind a (e.g Wordpress or Drupal) front-end:

1. Unrestricted total read-only access to catalogue API for general public (e.g. voluntary organisation).
   • monitored by API key
   • not monitored by API key
2. Restricted total read-write access to catalogue API for authorized clients (e.g. front-end system, bulk upload clients).
   • restricted by CKAN access controller
   • restricted by HTTP Auth
   • restricted by IP address
3. Restricted total read-write access to catalogue Web UI for authorized users (e.g. site admins).
   • restricted by CKAN access controller
   • restricted by HTTP Auth
4. Restricted partial read-write access to catalogue Web UI for authorized users (e.g. group admins).
   • restricted by CKAN access controller
   • restricted by HTTP Auth