{22} Trac tickets (2647 matches)

It doesn't pass extras_as_string

We get an error which should be improved the actual problem is this but not passed to user

⇝ KeyError?: 'ofs.impl'

View as: Interactive (full) | Text (full) | XML (full) Module ckan.controllers.storage:2 in auth_form view Module ckan.lib.jsonp:26 in jsonpify view

data = func(*args, kwargs)

Module ckan.controllers.storage:407 in auth_form view

authorize(method, bucket, label, c.userobj, self.ofs)

Module ckan.controllers.storage:200 in ofs view

StorageAPIController._ofs_impl = get_ofs()

Module ckan.controllers.storage:71 in get_ofs view

storage_backend = configofs.impl?

Module paste.registry:146 in getitem view

return self._current_obj()[key]

KeyError?: 'ofs.impl'

see spacing between 'download' button and 'data api' button ​http://s031.okserver.org:2375/dataset/afterfibre/resource/66f5b980-06db-4ccd-a005-c18de4d68a4b

Definitely do:

• Local file filestore: Remove pairtree (and OFS) and do something very simple

Options:

1. Local file only. Allow for uploaders to GS / S3 in the background
   • Advantages: simpler, if local upload can do progress bars etc etc
   • Disadvantages: strain for web-app (upload a 2GB file what happens)
     • This is probably fixable ...
2. Stick with how we are

New filestore without pairtree

Each uploaded file is located on disk at:

{uuid}/{filename}

Alternative:

yyyy/mm/dd/{uuid}/{filename}

Need the uuid to avoid collisions.

Metadata: Store no metadata.

The new ckan creates datasets as part of a 3 phase process. To allow for this partially created datasets can have a state that is 'draft' or 'draft-complete'. These datasets should not be seen as active by the activity stream.

If we click 'add dataset' and then complete the first phase of adding a dataset then we end up having a activity stream created. When we add a resource in the next phase (add data) again an activity stream item xxx added resource to dataset is created

We do not want these adding. Essentially if a dataset has a state.startswith('draft') then we want the activity stream to ignore all actions involving it.

finally when the state is changed form state.startswith('draft') to state=='active' we want a xxx has created dataset ... to be added to the stream.

The best way to do this would be to branch from 2375-demo-theme-stable and get it to work there.

Let me know if you need any help with this ticket or a better explanation of the problem.

so that data proxy works

Add minimal, working IDatasetForm and IGroupForm example extensions to core, with tests.

The IDatasetForm example should use tag vocabularies (two birds with one stone)

The IDatasetForm and IGroupForm docs are not very good (and are somewhat spread around different doc chapters), fix them up, and reference the new working examples.

Tab Vocabularies docs should reference IDatasetForm example.

When using convert_to/from_extras() you have to remove any free extras from the form or it won't work, this needs to be documented (in the docstring maybe)

There have been recent changes to the schemas that IDatasetForm and IGroupForm use, make sure the docs are up to date.

The about page for the DataHub? (thedatahub.org/about) could be improved. More importantly the default about page for a generic CKAN instance should be completely different - focus more on Open Data rather than the community hub idea (as this is more relevant for most installations) and remove specific references to tdh.

Ideally the ./bin/less command would be run when the server is started.

• Also it would be good to have a paster command to build the production CSS with {{{ ./bin/less --production }}
• The command could also detect missing node binaries and redirect to the documentation.

Currently, as a logged in admin, you can view deleted datasets just as if they are live datasets. Nothing distinguishes them apart from normal datasets other than they don't show up on the dataset page

This is potentially very confusing Can we add some kind of banner / label to deleted datasets to show they have been deleted?

& Do you guys need Sam to design this?

Related ticket for undeleting datasets - ​http://trac.ckan.org/ticket/2780

demo.ckan.org has a small field for adding text on homepage current text: "Welcome to the CKAN demo Try out standard CKAN functionality in a sandbox environment. Search for datasets directly from the homepage or by navigating to the Datasets search page where you can facet by tags, groups and format."

Want to make this editable for demos

Really we want this for CKAN Hosted & for creating demos.

Currently we're using ​http://leaflet.cloudmade.com/ - but by finding the area & taking screenshots. Would be better to interate with the app so that: a) this happens automatically and b) correct attribution will be visible on maps

e.g. if I click on download here: ​http://s031.okserver.org:2375/dataset/example-dataset/resource/d8797e51-b497-46ca-a274-8675533d110b can it take me to a new tab instead of navigating away from ckan?

Specially needed when importing large numbers of datasets.

Profiling the import command from the harvesting extension has shown some areas where improvements could be made.

This is a very slow call it would benefit from the sort of speed-ups that package-search received

for me locally this is taking 6 seconds for rufus using the datahub data I have. I think a lot of this is the dataset retrival/dictization

can we just grab json blobs from solr?

also is it possible to specify a sort order/paging?

I've put this as a demo-theme ticket as it is an big issue on the demo we are at 25 second page loads - which i can get down to about 8.8 secs so this is the main pain point now

​http://s031.okserver.org:2375/ language dropdown expands downwards it should open upwards

select2 can do this see ​http://ivaynberg.github.com/select2/#documentation can you see if we can get this in ckan demo

​http://s031.okserver.org:2375/dataset/resource-markdown/resource_edit/ba1b094d-697c-47c4-a72a-6c2557da5409

should have back to resource as a minimum

A one-pager explaining what the datahub is and with howto/examples for new users. This would make it much easier to explain the value in using the datahub for storing data.

this want to

a) be a link on the left of the form like deletes

b) needs a confirm like delete

c) wants to work with the autoform

be good to do this in phase 3

the .po file for zh_TW does not contain Plural-Forms which breaks the i18n JavaScript? Extractions for ckan-demo.

I've added the following which may or may not be correct - from web search "Plural-Forms: nplurals=1; plural=0\n"

I suspect it will get overwritten when translations are pulled from transiflex so passing to you as you are the expert in that realm

@aron,

edit dataset has a js error

​http://localhost:5000/dataset/edit/1-kmwaterfractionfromnationaltopographicdatabasemapscanada

TypeError?: options is undefined [Break On This Error]

this.lookup(options.term, options.callback);

autocomplete.js (line 231)

eg on confirm delete

​http://localhost:5000/zh_TW/dataset/edit/1-kmwaterfractionfromnationaltopographicdatabasemapscanada

_(....) gives [object Object] not translation

The 'What are Groups?' text at <​http://demo.ckan.org/group> is a bit naff. Here is a suggested replacement:

What are groups?

Groups allow you to group users and data together so that they are easier to manage. For example datasets in a group could be visible only to group members, or publicly visible but only editable by group members.

Groups can be used to collect together datasets from a single publisher, or related to a particular project, for example.

3. Uploading a file behaves counter-intuitively (I would suggest wrongly).

When adding a new resource by uploading a file, I select a file called say create-group.png. I expect the following to happen:

• the pathname of that file is filled in the box;
• nothing is actually uploaded till I hit 'add' (confirming that I've got the right file etc).

Instead of this,

• the file is immediately and, seemingly, irrevocably uploaded;
• the box is filled with a mysterious URL (​https://commondatastorage.googleapis.com/ckan-demo ...)
• the mysterious URL is so long I can't even see the final element, which is my file name.

Problem: On /apps page in the Filter Results box there is a "Only show featured items" checkbox, on selection it comes up with 0 solutions. Expected: Presumably an admin can create "featured items" so that they can be randomly selected on front page (is this correct)? If there are no "featured items" in the whole database can this check box be hidden? Bug is: no UI or obvious way to create featured items. Also the checkbox looks un-styled

Order the items so that none user fields are first from ticket #2707

Had this issue with the archiver on my local machine need to be logged in (I am admin) to see via web front end

$ paster archiver update -c ../ckan/development.ini 


2012-08-09 11:01:37,636 INFO  [ckanext.archiver.commands] Archival of dataset resource data added to celery queue: opencontext-chogha-mish-fauna (1 resources)
2012-08-09 11:01:37,671 INFO  [ckanext.archiver.commands] Getting dataset metadata: south-african-national-gov-budget-2012-13
2012-08-09 11:01:37,900 INFO  [ckan.lib.base]  /api/action/package_show render time 0.043 seconds
Traceback (most recent call last):
  File "/home/toby/okfn/pyenv/bin/paster", line 8, in <module>
    load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py", line 104, in run
    invoke(command, command_name, options, args[1:])
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py", line 143, in invoke
    exit_code = runner.run(args)
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py", line 238, in run
    result = self.command()
  File "/home/toby/okfn/pyenv/src/ckanext-archiver/ckanext/archiver/commands.py", line 98, in command
    response = app.post(api_url + '/package_show', data)
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py", line 262, in post
    expect_errors=expect_errors)
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py", line 243, in _gen_request
    return self.do_request(req, status=status)
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py", line 406, in do_request
    self._check_status(status, res)
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py", line 439, in _check_status
    res.body))
paste.fixture.AppError: Bad response: 403 Forbidden (not 200 OK or 3xx redirect for /api/action/package_show)
{"help": "Return the metadata of a dataset (package) and its resources.\n\n    :param id: the id or name of the dataset\n    :type id: string\n\n    :rtype: dictionary\n\n    ", "success": false, "error": {"message": "Access denied", "__type": "Authorization Error"}}

Allowing people to use the full range of markdown results in extremely messy output across the site. I'd suggest limiting support to only a subset of common use cases.

• Allow all inline elements, this allows bold, italic, code and links.
• Allow lists.

Disallow

• Horizontal Rules
• Headings
• Block quote and code (this may turn out to be useful and so could be included)

This way you get markdowns paragraph handling and a few inline styles without breaking the entire layout of the page.

Here's the full syntax if anyone is interested ​http://daringfireball.net/projects/markdown/syntax

I think this would be a nice feature for remote loading templates if the options.template value ends in ".html".

ckan.module('my-module', {
  options: {
    template: 'my-template.html'
  },
  initialize: function () {
    this.template // This is the loaded template.
  }
});

Client#getTemplate(name, params, success, error);

Where params, success and error are optional arguments. test/index.html already has an implementation called loadFixture().

Is this done I know checkboxes are a pain but it looks a bit out of place compared to the other items.

Create a group, create some datasets and add them to the group, then view the group's page with a sort-order specified in the URL, e.g. /group/foo?q=&sort=title_string+asc, no datasets are displayed even though there are datasets in the group that match the search query. Remove the sort order from the URL and the datasets will appear.

On the add dataset form, slug generation fails with JS error:

Uncaught TypeError: Object [object Object] has no method 'incompleteFormWarning'    basic-form.js:5

• Search needs to run of local DB (again)
• paster db clean attemps to connect to SOLR (still works as does db first but then excepts which is not nice UX)

e.g. ​http://datahub.io/dataset/ccc-gistemp/resource/80ebdbd6-d91b-4fef-9db4-d3dfbd7e868e

CKAN attempts to render the PDF as a datastore tabular date even though the mimetype of the PDF is correctly specified.

Tried to refresh the datagm harvest source to get error message:

"An error occurred: [There already is an unrun job for this source]"

This is unclear and not sure why it created an error. We should have something informative such as "A harvest has already been scheduled for this source"

Do we have a date/time for when this has been scheduled that we can display?

I've had a look at the new harvesting theme, it looks great, thanks. I did some minor changes and reworded the errors as per tickets #2852 and #2853, but I kept the messages red, as the only other ones available ("flash_notice") are blue and are not very prominent.

Toby, on the old form there was a list of descriptions for all available harvesters which could be nice to have back. I couldn't find an easy way with the new form macro, so I don't know how easy would be to implement

​http://i.imgur.com/pzS8z.png

can we add in the breadcrumb navigation to qa pages so that you can go back to ​http://s031.okserver.org:2375/qa from sub pages such as ​http://s031.okserver.org:2375/qa/dataset/five_stars

URL: ​http://thedatahub.org/el/tag/jutatt%C3%A1sok Module weberror.errormiddleware:162 in call << traceback_supplement = Supplement, self, environ

sr_checker = ResponseStartChecker?(start_response) app_iter = self.application(environ, sr_checker) return self.make_catching_iter(app_iter, environ, sr_checker)

except:

app_iter = self.application(environ, sr_checker)

Module beaker.middleware:73 in call << self.cache_manager)

environ[self.environ_key] = self.cache_manager return self.app(environ, start_response)

return self.app(environ, start_response)

Module beaker.middleware:155 in call << headers.append(('Set-cookie', cookie))

return start_response(status, headers, exc_info)

return self.wrap_app(environ, session_start_response)

def _get_session(self):

return self.wrap_app(environ, session_start_response)

Module routes.middleware:131 in call << r'\1', oldpath)

response = self.app(environ, start_response)

# Wrapped in try as in rare cases the attribute will be gone already

response = self.app(environ, start_response)

Module pylons.wsgiapp:125 in call <<

controller = self.resolve(environ, start_response) response = self.dispatch(controller, environ, start_response)

if 'paste.testing_variables' in environ and hasattr(response,

response = self.dispatch(controller, environ, start_response)

Module pylons.wsgiapp:324 in dispatch << if log_debug:

log.debug("Calling controller class with WSGI interface")

return controller(environ, start_response)

def load_test_env(self, environ):

return controller(environ, start_response)

Module ckan.lib.base:239 in call << try:

res = WSGIController.call(self, environ, start_response)

finally:

model.Session.remove()

res = WSGIController.call(self, environ, start_response)

Module pylons.controllers.core:221 in call << return response(environ, self.start_response)

response = self._dispatch_call() if not start_response_called:

self.start_response = start_response

response = self._dispatch_call()

Module pylons.controllers.core:172 in _dispatch_call << req.environpylons.action_method? = func

response = self._inspect_call(func)

else:

if log_debug:

response = self._inspect_call(func)

Module pylons.controllers.core:107 in _inspect_call << func.name, args)

try:

result = self._perform_call(func, args)

except HTTPException, httpe:

if log_debug:

result = self._perform_call(func, args)

Module pylons.controllers.core:60 in _perform_call << """Hide the traceback for everything above this method"""

traceback_hide = 'before_and_this' return func(args)

def _inspect_call(self, func):

return func(args)

Module ckan.controllers.template:30 in view << """

try:

return render(url)

except TemplateNotFound?:

if url.endswith('.html'):

return render(url)

Module ckan.lib.base:153 in render << try:

return cached_template(template_name, render_template,

loader_class=loader_class)

except ckan.exceptions.CkanUrlException?, e:

raise

loader_class=loader_class)

Module pylons.templating:249 in cached_template << return content

else:

return render_func()

return render_func()

Module ckan.lib.base:95 in render_template << template = globsapp_globals?.genshi_loader.load(template_name,

cls=loader_class)

stream = template.generate(globs)

cls=loader_class)

Module genshi.template.loader:223 in load << loadfunc = directory(loadfunc)

try:

filepath, filename, fileobj, uptodate = loadfunc(filename)

except IOError:

continue

filepath, filename, fileobj, uptodate = loadfunc(filename)

Module genshi.template.loader:286 in _load_from_directory << def _load_from_directory(filename):

filepath = os.path.join(path, filename) fileobj = open(filepath, 'U') mtime = os.path.getmtime(filepath) def _uptodate():

fileobj = open(filepath, 'U')

UnicodeEncodeError?: 'ascii' codec can't encode character u'\xe1' in position 73: ordinal not in range(128) CGI Variables CKAN_CURRENT_URL '/el/tag/jutatt%C3%A1sok' CKAN_LANG 'en' CKAN_LANG_IS_DEFAULT True CKAN_PAGE_CACHABLE True CONTENT_TYPE '; charset=utf-8' DOCUMENT_ROOT '/etc/apache2/htdocs' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT '*/*' HTTP_ACCEPT_ENCODING 'gzip,deflate' HTTP_CONNECTION 'close' HTTP_FROM 'googlebot(at)googlebot.com' HTTP_HOST 'thedatahub.org' HTTP_USER_AGENT 'Mozilla/5.0 (compatible; Googlebot/2.1; +​http://www.google.com/bot.html)' PATH_INFO '/el/tag/jutatt\xc3\xa1sok' PATH_TRANSLATED '/etc/ckan/datahub/apache.wsgi/el/tag/jutatt\xc3\xa1sok' REMOTE_ADDR '127.0.0.1' REMOTE_PORT '49520' REQUEST_METHOD 'GET' REQUEST_URI '/el/tag/jutatt%c3%a1sok' SCRIPT_FILENAME '/etc/ckan/datahub/apache.wsgi' SERVER_ADDR '127.0.0.1' SERVER_ADMIN '[no address given]' SERVER_NAME 'thedatahub.org' SERVER_PORT '80' SERVER_PROTOCOL 'HTTP/1.0' SERVER_SIGNATURE '<address>Apache/2.2.22 (Ubuntu) Server at thedatahub.org Port 80</address>\n' SERVER_SOFTWARE 'Apache/2.2.22 (Ubuntu)' WSGI Variables application <beaker.middleware.CacheMiddleware? object at 0x7f5312a51650> beaker.cache <beaker.cache.CacheManager? object at 0x7f5312a51790> beaker.get_session <bound method SessionMiddleware?._get_session of <beaker.middleware.SessionMiddleware? object at 0x7f5312a516d0>> beaker.session {'_accessed_time': 1345113298.778472, '_creation_time': 1345113298.778472} mod_wsgi.application_group 'ckan.net|' mod_wsgi.callable_object 'application' mod_wsgi.handler_script mod_wsgi.input_chunked '0' mod_wsgi.listener_host mod_wsgi.listener_port '8080' mod_wsgi.process_group 'datahub' mod_wsgi.request_handler 'wsgi-script' mod_wsgi.script_reloading '1' mod_wsgi.version (3, 3) paste.cookies (<SimpleCookie: >, ) paste.registry <paste.registry.Registry object at 0x7f5309add610> paste.throw_errors True pylons.action_method <bound method TemplateController?.view of <ckan.controllers.template.TemplateController? object at 0x7f52b0ea2d10>> pylons.controller <ckan.controllers.template.TemplateController? object at 0x7f52b0ea2d10> pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'} pylons.pylons <pylons.util.PylonsContext? object at 0x7f52b0ea2b90> pylons.routes_dict {'url': u'el/tag/jutatt\xe1sok', 'action': u'view', 'controller': u'template'} repoze.who.logger <logging.Logger object at 0x7f53127b2b50> repoze.who.plugins {'openid': <OpenIdIdentificationPlugin? 139994769488336>, 'friendlyform': <FriendlyFormPlugin? 139994769488528>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator? object at 0x7f5312d893d0>, 'auth_tkt': <AuthTktCookiePlugin? 139994771858704>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f5312d89350>} routes.route <routes.route.Route object at 0x7f531296ea10> routes.url <routes.util.URLGenerator object at 0x7f5309add810> webob._parsed_query_vars (GET([]), ) webob.adhoc_attrs {'language': 'en-us'} wsgi process 'Multi process AND threads (?)' wsgi.file_wrapper <built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f52d4004990> wsgi.version (1, 1) wsgiorg.routing_args (<routes.util.URLGenerator object at 0x7f5309add810>, {'url': u'el/tag/jutatt\xe1sok', 'action': u'view', 'controller': u'template'})

From: <thedatahub.org@…> Date: Thu, Aug 16, 2012 at 10:30 AM Subject: WebApp? Error: <class 'routes.util.GenerationException?'>: url_for can only return a string, got unicode instead: ​https://ckannet-storage.commondatastorage.googleapis.com/file/fd3e00d9-eb0d-479b-8303-38909e5400ca?Signature=cmuU3CzRHQ86F3aZ0gljxv%2B3IqQ%3D&Expires=1345109432&AWSAccessKeyId=GOOGC6OU3AYPNY47B66M&x-goog-meta-uploaded-by=d5c8ed25-70de-4035-b29d-ddbe363913c6 To: ckan-sysadmin@…, kindly@…

URL: ​http://ckan.net/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca Module weberror.errormiddleware:162 in call << traceback_supplement = Supplement, self, environ

sr_checker = ResponseStartChecker?(start_response) app_iter = self.application(environ, sr_checker) return self.make_catching_iter(app_iter, environ, sr_checker)

except:

app_iter = self.application(environ, sr_checker)

Module beaker.middleware:73 in call << self.cache_manager)

environ[self.environ_key] = self.cache_manager return self.app(environ, start_response)

return self.app(environ, start_response)

Module beaker.middleware:155 in call << headers.append(('Set-cookie', cookie))

return start_response(status, headers, exc_info)

return self.wrap_app(environ, session_start_response)

def _get_session(self):

return self.wrap_app(environ, session_start_response)

Module routes.middleware:131 in call << r'\1', oldpath)

response = self.app(environ, start_response)

# Wrapped in try as in rare cases the attribute will be gone already

response = self.app(environ, start_response)

Module pylons.wsgiapp:125 in call <<

controller = self.resolve(environ, start_response) response = self.dispatch(controller, environ, start_response)

if 'paste.testing_variables' in environ and hasattr(response,

response = self.dispatch(controller, environ, start_response)

Module pylons.wsgiapp:324 in dispatch << if log_debug:

log.debug("Calling controller class with WSGI interface")

return controller(environ, start_response)

def load_test_env(self, environ):

return controller(environ, start_response)

Module ckan.lib.base:239 in call << try:

res = WSGIController.call(self, environ, start_response)

finally:

model.Session.remove()

res = WSGIController.call(self, environ, start_response)

Module pylons.controllers.core:221 in call << return response(environ, self.start_response)

response = self._dispatch_call() if not start_response_called:

self.start_response = start_response

response = self._dispatch_call()

Module pylons.controllers.core:172 in _dispatch_call << req.environpylons.action_method? = func

response = self._inspect_call(func)

else:

if log_debug:

response = self._inspect_call(func)

Module pylons.controllers.core:107 in _inspect_call << func.name, args)

try:

result = self._perform_call(func, args)

except HTTPException, httpe:

if log_debug:

result = self._perform_call(func, args)

Module pylons.controllers.core:60 in _perform_call << """Hide the traceback for everything above this method"""

traceback_hide = 'before_and_this' return func(args)

def _inspect_call(self, func):

return func(args)

Module ckan.controllers.storage:190 in file << return fapp(request.environ, self.start_response)

else:

h.redirect_to(file_url)

h.redirect_to(file_url)

Module ckan.lib.helpers:57 in redirect_to << if are_there_flash_messages():

kw__no_cache__? = True

return _redirect_to(url_for(*args, kw))

def url(*args, kw):

return _redirect_to(url_for(*args, kw))

Module ckan.lib.helpers:79 in url_for << # fix ver to include the slash

kwver? = '/%s' % ver

my_url = _routes_default_url_for(*args, kw) kw__ckan_no_root? = no_root return _add_i18n_to_url(my_url, locale=locale, kw)

my_url = _routes_default_url_for(*args, kw)

Module routes.util:265 in url_for << if not isinstance(url, str) and url is not None:

raise GenerationException?("url_for can only return a string, got "

"unicode instead: %s" % url)

if url is None:

raise GenerationException?(

"unicode instead: %s" % url)

GenerationException?: url_for can only return a string, got unicode instead: ​https://ckannet-storage.commondatastorage.googleapis.com/file/fd3e00d9-eb0d-479b-8303-38909e5400ca?Signature=cmuU3CzRHQ86F3aZ0gljxv%2B3IqQ%3D&Expires=1345109432&AWSAccessKeyId=GOOGC6OU3AYPNY47B66M&x-goog-meta-uploaded-by=d5c8ed25-70de-4035-b29d-ddbe363913c6 CGI Variables CKAN_CURRENT_URL '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' CKAN_LANG 'en' CKAN_LANG_IS_DEFAULT True CONTENT_TYPE '; charset=utf-8' DOCUMENT_ROOT '/etc/apache2/htdocs' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT '*/*' HTTP_ACCEPT_CHARSET 'utf-8;q=0.7,iso-8859-1;q=0.2,*;q=0.1' HTTP_CONNECTION 'close' HTTP_HOST 'ckan.net' HTTP_USER_AGENT 'Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@…)' PATH_INFO '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' PATH_TRANSLATED '/etc/ckan/datahub/apache.wsgi/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' REMOTE_ADDR '127.0.0.1' REMOTE_PORT '37236' REQUEST_METHOD 'GET' REQUEST_URI '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' SCRIPT_FILENAME '/etc/ckan/datahub/apache.wsgi' SERVER_ADDR '127.0.0.1' SERVER_ADMIN '[no address given]' SERVER_NAME 'ckan.net' SERVER_PORT '80' SERVER_PROTOCOL 'HTTP/1.0' SERVER_SIGNATURE '<address>Apache/2.2.22 (Ubuntu) Server at ckan.net Port 80</address>\n' SERVER_SOFTWARE 'Apache/2.2.22 (Ubuntu)' WSGI Variables application <beaker.middleware.CacheMiddleware? object at 0x7f5312a51650> beaker.cache <beaker.cache.CacheManager? object at 0x7f5312a51790> beaker.get_session <bound method SessionMiddleware?._get_session of <beaker.middleware.SessionMiddleware? object at 0x7f5312a516d0>> beaker.session {'_accessed_time': 1345109433.273328, '_creation_time': 1345109433.273328} mod_wsgi.application_group 'ckan.net|' mod_wsgi.callable_object 'application' mod_wsgi.handler_script mod_wsgi.input_chunked '0' mod_wsgi.listener_host mod_wsgi.listener_port '8080' mod_wsgi.process_group 'datahub' mod_wsgi.request_handler 'wsgi-script' mod_wsgi.script_reloading '1' mod_wsgi.version (3, 3) paste.cookies (<SimpleCookie: >, ) paste.registry <paste.registry.Registry object at 0x7f5303b05610> paste.throw_errors True pylons.action_method <bound method StorageController?.file of <ckan.controllers.storage.StorageController? object at 0x7f530891fa10>> pylons.controller <ckan.controllers.storage.StorageController? object at 0x7f530891fa10> pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'} pylons.pylons <pylons.util.PylonsContext? object at 0x7f52f864f850> pylons.routes_dict {'action': u'file', 'controller': u'ckan.controllers.storage:StorageController', 'label': u'file/fd3e00d9-eb0d-479b-8303-38909e5400ca'} repoze.who.logger <logging.Logger object at 0x7f53127b2b50> repoze.who.plugins {'openid': <OpenIdIdentificationPlugin? 139994769488336>, 'friendlyform': <FriendlyFormPlugin? 139994769488528>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator? object at 0x7f5312d893d0>, 'auth_tkt': <AuthTktCookiePlugin? 139994771858704>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f5312d89350>} routes.route <routes.route.Route object at 0x7f531296e690> routes.url <routes.util.URLGenerator object at 0x7f52f82cd450> webob._parsed_query_vars (GET([]), ) webob.adhoc_attrs {'language': 'en-us'} wsgi process 'Multi process AND threads (?)' wsgi.file_wrapper <built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f52f5870cd8> wsgi.version (1, 1) wsgiorg.routing_args (<routes.util.URLGenerator object at 0x7f52f82cd450>, {'action': u'file', 'controller': u'ckan.controllers.storage:StorageController', 'label': u'file/fd3e00d9-eb0d-479b-8303-38909e5400ca'})

From: <thedatahub.org@…> Date: Thu, Aug 16, 2012 at 3:20 AM Subject: WebApp? Error: <class 'genshi.template.eval.UndefinedError?'>: "tag_list" not defined To: ckan-sysadmin@…, kindly@…

URL: ​http://ckan.net/dataset/uk-postboxes Module weberror.errormiddleware:162 in call << traceback_supplement = Supplement, self, environ

sr_checker = ResponseStartChecker?(start_response) app_iter = self.application(environ, sr_checker) return self.make_catching_iter(app_iter, environ, sr_checker)

except:

app_iter = self.application(environ, sr_checker)

Module beaker.middleware:73 in call << self.cache_manager)

environ[self.environ_key] = self.cache_manager return self.app(environ, start_response)

return self.app(environ, start_response)

Module beaker.middleware:155 in call << headers.append(('Set-cookie', cookie))

return start_response(status, headers, exc_info)

return self.wrap_app(environ, session_start_response)

def _get_session(self):

return self.wrap_app(environ, session_start_response)

Module routes.middleware:131 in call << r'\1', oldpath)

response = self.app(environ, start_response)

# Wrapped in try as in rare cases the attribute will be gone already

response = self.app(environ, start_response)

Module pylons.wsgiapp:125 in call <<

controller = self.resolve(environ, start_response) response = self.dispatch(controller, environ, start_response)

if 'paste.testing_variables' in environ and hasattr(response,

response = self.dispatch(controller, environ, start_response)

Module pylons.wsgiapp:324 in dispatch << if log_debug:

log.debug("Calling controller class with WSGI interface")

return controller(environ, start_response)

def load_test_env(self, environ):

return controller(environ, start_response)

Module ckan.lib.base:239 in call << try:

res = WSGIController.call(self, environ, start_response)

finally:

model.Session.remove()

res = WSGIController.call(self, environ, start_response)

Module pylons.controllers.core:221 in call << return response(environ, self.start_response)

response = self._dispatch_call() if not start_response_called:

self.start_response = start_response

response = self._dispatch_call()

Module pylons.controllers.core:172 in _dispatch_call << req.environpylons.action_method? = func

response = self._inspect_call(func)

else:

if log_debug:

response = self._inspect_call(func)

Module pylons.controllers.core:107 in _inspect_call << func.name, args)

try:

result = self._perform_call(func, args)

except HTTPException, httpe:

if log_debug:

result = self._perform_call(func, args)

Module pylons.controllers.core:60 in _perform_call << """Hide the traceback for everything above this method"""

traceback_hide = 'before_and_this' return func(args)

def _inspect_call(self, func):

return func(args)

Module ckan.controllers.package:322 in read << template = template[:template.index('.') + 1] + format

return render(template, loader_class=loader)

def comments(self, id):

return render(template, loader_class=loader)

Module ckan.lib.base:153 in render << try:

return cached_template(template_name, render_template,

loader_class=loader_class)

except ckan.exceptions.CkanUrlException?, e:

raise

loader_class=loader_class)

Module pylons.templating:249 in cached_template << return content

else:

return render_func()

return render_func()

Module ckan.lib.base:102 in render_template << if loader_class == NewTextTemplate?:

return literal(stream.render(method="text", encoding=None))

return literal(stream.render(method=method, encoding=None,

return literal(stream.render(method="text", encoding =None))

Module genshi.core:183 in render << method = self.serializer or 'xml'

generator = self.serialize(method=method, kwargs) return encode(generator, method=method, encoding=encoding, out=out)

def select(self, path, namespaces=None, variables=None):

return encode(generator, method=method, encoding=encoding, out =out)

Module genshi.output:57 in encode << _encode = lambda string: string

if out is None:

return _encode(.join(list(iterator)))

for chunk in iterator:

out.write(_encode(chunk))

return _encode(.join(list(iterator)))

Module genshi.output:569 in call << def call(self, stream):

strip_markup = self.strip_markup for event in stream:

if event[0] is TEXT:

data = event[1]

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module genshi.filters.transform:1145 in call << :param stream: The marked event stream to filter

""" for mark, event in stream:

yield mark, event if mark is ENTER:

for mark, event in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module genshi.filters.transform:1145 in call << :param stream: The marked event stream to filter

""" for mark, event in stream:

yield mark, event if mark is ENTER:

for mark, event in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module genshi.filters.transform:1145 in call << :param stream: The marked event stream to filter

""" for mark, event in stream:

yield mark, event if mark is ENTER:

for mark, event in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module ckanext.googleanalytics.plugin:93 in download_adder << [downloaded %s times]</span>

count = None for mark, (kind, data, pos) in stream:

if mark and kind == START:

href = data[1].get('href')

for mark, (kind, data, pos) in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module genshi.filters.transform:1175 in call << """

callable_value = hasattr(self.value, 'call') for mark, (kind, data, pos) in stream:

if mark is ENTER:

if callable_value:

for mark, (kind, data, pos) in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module genshi.filters.transform:1145 in call << :param stream: The marked event stream to filter

""" for mark, event in stream:

yield mark, event if mark is ENTER:

for mark, event in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.template.base:605 in _include << from genshi.template.loader import TemplateNotFound?

for event in stream:

if event[0] is INCLUDE:

href, cls, fallback = event[1]

for event in stream:

Module genshi.template.base:565 in _flatten << elif kind is EXPR:

result = _eval_expr(data, ctxt, vars) if result is not None:

# First check for a string, otherwise the iterable test

result = _eval_expr(data, ctxt, vars)

Module genshi.template.base:277 in _eval_expr << if vars:

ctxt.push(vars)

retval = expr.evaluate(ctxt) if vars:

ctxt.pop()

retval = expr.evaluate(ctxt)

Module genshi.template.eval:178 in evaluate << traceback_hide = 'before_and_this'

_globals = self._globals(data) return eval(self.code, _globals, {'data': data})

return eval(self.code, _globals, {'data': data})

Module ?:51 in <Expression u"tag_list(c.pkg_dict.get('tags', ))"> << <li py:if="c.pkg_dict.get('tags')" class="sidebar-section">

Fanstatic does not support

@import ...

relative imports for images etc

need to get a fix for these asap

In progress

1. [major] q does not seem to work reliably. e.g. using the setup from this gist ​https://gist.github.com/1930806 and doing a ?q=DE yields no results (does not work with "q=de" either)
   • q=second does work ...
2. [major] q does not work with 2 values (see below)
3. Query on search with limit 0 results in total of 0 (should either be null or correct total). Queries with other limits yield correct total AFAICT
   • Also weird fact that limit is returned but as as as string - should it not be an integer
4. Types on fields: could these not be canonical and as per recline (or is it important to allow exact sql types ...)

Multiple query values

Try a query such as: "second UK" and you will get 500 error:

​http://localhost:5000/api/3/action/datastore_search?resource_id=4f1299ab-a100-4e5f-ba81-e6d234a2f3bd&q=second%20UK

ProgrammingError: (ProgrammingError) syntax error in tsquery: "second UK" 'select "_id", "id", "date", "x", "y", "z", "country", "title", "lat", "lon", count(*) over() as "_full_count"\n from "4f1299ab-a100-4e5f-ba81-e6d234a2f3bd" where _full_text @@ to_tsquery(%s) limit 100 offset 0' (u'second UK',)

Suggestions

Filter support: should think in more detail about this (may want to follow recline style)

Simple filters in query parameters would be nice too ...

The Package model should have a boolean field added to it describing its visibility beyond that supplied via auth.

The related permission for viewing packages where public is set to False would be package.view (show, whatever), and *never* visible for unauthenticated users.

Here's a draft: ​https://gist.github.com/3414107

Related items (Apps & Ideas) have a collection of types, which are currently fixed. Some of these types are ambiguous, such as ideas in that they may not have a link - and if they do it is likely to be to a blog post - another existing type.

Whilst the URL is required, and we think this should stay required, we should also allow users to change the types found in Apps & Ideas to a shortened list that suits their requirements.

For example, edit the warandpeace dataset and add an extra with key "comments" and value "[{u'date': u'Tue Aug 21 13:01:48 GMT 2012', u'comment': u'testing ckan', u'userId': u'11702'}, {u'date': u'Wed Aug 22 13:01:48 GMT 2012', u'comment': u'testing ckan again', u'userId': u'11702'}]", now visit API URLs such as:

/api/search/dataset?sort=metadata_modified+desc&all_fields=1&limit=10

/api/rest/dataset/warandpeace

/api/action/package_show?id=warandpeace

The value of the extra comes out as a string in the dataset JSON, rather than having its contents unpacked into a list of JSON objects.