CKAN: Ticket Query

#1259: "Add a row" for Extras on Package form

pudo — Tue, 02 Aug 2011 16:31:33 GMT

The default package form offers 4 empty extras fields. Like the resource section, it should have an "add more" button to add another row.

#2989: "Add dataset to organization" should auto-select the organization

seanh — Mon, 15 Oct 2012 11:04:12 GMT

'Add dataset to organization' button, when you get through to the third stage of the new dataset form the organization you came from is not selected.

I don't see any option to choose the group when adding or updating a dataset, but if I add a dataset via the "Add dataset to group" button on a group's page, then the dataset seems to get added to that group. (And I can also add/remove existing datasets by editing the group.) I wonder if organizations should work the same way, instead of having an Organization drop-down when creating or updating a dataset. The add dataset page needs to somehow indicate that you're adding a dataset to a certain group or organization though, doesn't currently.

#2971: "Are ytou sure you want to delete this member?" should say which member

seanh — Mon, 15 Oct 2012 10:27:20 GMT

when deleting members from groups and orgs

#369: "Package Listing Key" should appear on Tag results

shudson@… — Thu, 22 Jul 2010 18:00:34 GMT

Currently there's a nice legend titled "Package Listing Key" that appears in right side of "Browse Packages" results. The same key should show on other search results like when searching for a tag.

#2887: "Welcome to CKAN!" on front page is untranslatable

seanh — Wed, 22 Aug 2012 09:17:59 GMT

ckan/templates/home/index.html contains:

<h1 class="page_heading">Welcome to ${g.site_title}!</h1>

which means that just "Welcome to" ends up in the ckan.po files for translation. This makes the phrase untranslatable in languages that have a different word ordering than English where the site title needs to be somewhere other than at the end of the sentence.

#2993: "logged_in" and "visitor" show in user list at /users

seanh — Wed, 17 Oct 2012 09:42:02 GMT

#2407: 'Access denied' message is unhelpful

markw — Mon, 21 May 2012 15:01:40 GMT

This chap tried to use the API to add a dataset to a group and got the uninformative message 'Access denied':

http://lists.okfn.org/pipermail/ckan-dev/2012-May/002229.html

The problem was he didn't have access to the group. A message like 'You don't have access to the group "LODcloud"' would have been clearer.

#2966: 'Add' button text is wrong when editing organization members

seanh — Mon, 15 Oct 2012 10:15:49 GMT

e.g. if I just changed a member's capacity I am not adding anything 'save' is better maybe

#2265: 'More Like This' for a dataset

dread — Tue, 27 Mar 2012 16:20:20 GMT

When viewing a dataset, it would be nice to show a couple of 'Related Datasets'. i.e. ones that are similar.

SOLR has a feature for finding documents similar to a particular document, called 'More Like This'.

We would like this for DGU.

#1827: 'Register' link should be hidden if you not allowed to register

dread — Wed, 22 Feb 2012 15:35:39 GMT

I have just deny visitors the create-user permission:
sudo -u ckanstd paster --plugin=ckan roles deny reader create-user -c /etc/ckan/std/std.ini
sudo -u ckanstd paster --plugin=ckan roles deny anon_editor create-user -c /etc/ckan/std/std.ini
and after restarting, the register link is *not* hidden, but now when you access the register page, it shows you this message "Unauthorized to create a user" (when not logged in). But anyway that is an improvement.

#2607: 'Upload a file' appears on resource form when storage not enabled

seanh — Wed, 27 Jun 2012 13:30:08 GMT

if the user tries to upload a file they will get "Failed to get credentials for storage upload. Upload cannot proceed"

Maybe add a test for it this time, this bug has appeared and reappeared before

#2411: .ini file is a mess

markw — Tue, 22 May 2012 11:41:40 GMT

The .ini file needs cleaning up in assorted ways which Toby will expand upon.

#2870: 1.8 tag_list not defined

ross — Thu, 16 Aug 2012 10:58:52 GMT

From: <thedatahub.org@…> Date: Thu, Aug 16, 2012 at 3:20 AM Subject: WebApp? Error: <class 'genshi.template.eval.UndefinedError?'>: "tag_list" not defined To: ckan-sysadmin@…, kindly@…

URL: http://ckan.net/dataset/uk-postboxes Module weberror.errormiddleware:162 in call << traceback_supplement = Supplement, self, environ

sr_checker = ResponseStartChecker?(start_response) app_iter = self.application(environ, sr_checker) return self.make_catching_iter(app_iter, environ, sr_checker)

except:

app_iter = self.application(environ, sr_checker)

Module beaker.middleware:73 in call << self.cache_manager)

environ[self.environ_key] = self.cache_manager return self.app(environ, start_response)

return self.app(environ, start_response)

Module beaker.middleware:155 in call << headers.append(('Set-cookie', cookie))

return start_response(status, headers, exc_info)

return self.wrap_app(environ, session_start_response)

def _get_session(self):

return self.wrap_app(environ, session_start_response)

Module routes.middleware:131 in call << r'\1', oldpath)

response = self.app(environ, start_response)

# Wrapped in try as in rare cases the attribute will be gone already

response = self.app(environ, start_response)

Module pylons.wsgiapp:125 in call <<

controller = self.resolve(environ, start_response) response = self.dispatch(controller, environ, start_response)

if 'paste.testing_variables' in environ and hasattr(response,

response = self.dispatch(controller, environ, start_response)

Module pylons.wsgiapp:324 in dispatch << if log_debug:

log.debug("Calling controller class with WSGI interface")

return controller(environ, start_response)

def load_test_env(self, environ):

return controller(environ, start_response)

Module ckan.lib.base:239 in call << try:

res = WSGIController.call(self, environ, start_response)

finally:

model.Session.remove()

res = WSGIController.call(self, environ, start_response)

Module pylons.controllers.core:221 in call << return response(environ, self.start_response)

response = self._dispatch_call() if not start_response_called:

self.start_response = start_response

response = self._dispatch_call()

Module pylons.controllers.core:172 in _dispatch_call << req.environpylons.action_method? = func

response = self._inspect_call(func)

else:

if log_debug:

response = self._inspect_call(func)

Module pylons.controllers.core:107 in _inspect_call << func.name, args)

try:

result = self._perform_call(func, args)

except HTTPException, httpe:

if log_debug:

result = self._perform_call(func, args)

Module pylons.controllers.core:60 in _perform_call << """Hide the traceback for everything above this method"""

traceback_hide = 'before_and_this' return func(args)

def _inspect_call(self, func):

return func(args)

Module ckan.controllers.package:322 in read << template = template[:template.index('.') + 1] + format

return render(template, loader_class=loader)

def comments(self, id):

return render(template, loader_class=loader)

Module ckan.lib.base:153 in render << try:

return cached_template(template_name, render_template,

loader_class=loader_class)

except ckan.exceptions.CkanUrlException?, e:

raise

loader_class=loader_class)

Module pylons.templating:249 in cached_template << return content

else:

return render_func()

return render_func()

Module ckan.lib.base:102 in render_template << if loader_class == NewTextTemplate?:

return literal(stream.render(method="text", encoding=None))

return literal(stream.render(method=method, encoding=None,

return literal(stream.render(method="text", encoding =None))

Module genshi.core:183 in render << method = self.serializer or 'xml'

generator = self.serialize(method=method, kwargs) return encode(generator, method=method, encoding=encoding, out=out)

def select(self, path, namespaces=None, variables=None):

return encode(generator, method=method, encoding=encoding, out =out)

Module genshi.output:57 in encode << _encode = lambda string: string

if out is None:

return _encode(.join(list(iterator)))

for chunk in iterator:

out.write(_encode(chunk))

return _encode(.join(list(iterator)))

Module genshi.output:569 in call << def call(self, stream):

strip_markup = self.strip_markup for event in stream:

if event[0] is TEXT:

data = event[1]

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module genshi.filters.transform:1145 in call << :param stream: The marked event stream to filter

""" for mark, event in stream:

yield mark, event if mark is ENTER:

for mark, event in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module genshi.filters.transform:1145 in call << :param stream: The marked event stream to filter

""" for mark, event in stream:

yield mark, event if mark is ENTER:

for mark, event in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module genshi.filters.transform:1145 in call << :param stream: The marked event stream to filter

""" for mark, event in stream:

yield mark, event if mark is ENTER:

for mark, event in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module ckanext.googleanalytics.plugin:93 in download_adder << [downloaded %s times]</span>

count = None for mark, (kind, data, pos) in stream:

if mark and kind == START:

href = data[1].get('href')

for mark, (kind, data, pos) in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module genshi.filters.transform:1175 in call << """

callable_value = hasattr(self.value, 'call') for mark, (kind, data, pos) in stream:

if mark is ENTER:

if callable_value:

for mark, (kind, data, pos) in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.core:288 in _ensure << # unchanged

yield event for event in stream:

yield event

for event in stream:

Module genshi.filters.transform:686 in _unmark << def _unmark(self, stream):

for mark, event in stream:

kind = event[0] if not (kind is None or kind is ATTR or kind is BREAK):

for mark, event in stream:

Module genshi.filters.transform:1145 in call << :param stream: The marked event stream to filter

""" for mark, event in stream:

yield mark, event if mark is ENTER:

for mark, event in stream:

Module genshi.filters.transform:714 in call << stream = iter(stream)

next = stream.next for mark, event in stream:

if mark is None:

yield mark, event

for mark, event in stream:

Module genshi.filters.transform:682 in _mark << def _mark(self, stream):

for event in stream:

yield OUTSIDE, event

for event in stream:

Module genshi.template.base:605 in _include << from genshi.template.loader import TemplateNotFound?

for event in stream:

if event[0] is INCLUDE:

href, cls, fallback = event[1]

for event in stream:

Module genshi.template.base:565 in _flatten << elif kind is EXPR:

result = _eval_expr(data, ctxt, vars) if result is not None:

# First check for a string, otherwise the iterable test

result = _eval_expr(data, ctxt, vars)

Module genshi.template.base:277 in _eval_expr << if vars:

ctxt.push(vars)

retval = expr.evaluate(ctxt) if vars:

ctxt.pop()

retval = expr.evaluate(ctxt)

Module genshi.template.eval:178 in evaluate << traceback_hide = 'before_and_this'

_globals = self._globals(data) return eval(self.code, _globals, {'data': data})

return eval(self.code, _globals, {'data': data})

Module ?:51 in <Expression u"tag_list(c.pkg_dict.get('tags', ))"> << <li py:if="c.pkg_dict.get('tags')" class="sidebar-section">

<h3>Tags</h3> ${tag_list(c.pkg_dict.get('tags', ))}

</li>

${tag_list(c.pkg_dict.get('tags', ))}

Module genshi.template.eval:309 in lookup_name << val = BUILTINS.get(name, val)

if val is UNDEFINED:

val = cls.undefined(name)

return val

val = cls.undefined(name)

Module genshi.template.eval:410 in undefined << """Raise an UndefinedError? immediately."""

traceback_hide = True raise UndefinedError?(key, owner=owner)

raise UndefinedError?(key, owner=owner)

UndefinedError?: "tag_list" not defined CGI Variables CKAN_CURRENT_URL '/dataset/uk-postboxes' CKAN_LANG 'en' CKAN_LANG_IS_DEFAULT True CKAN_PAGE_CACHABLE True CONTENT_TYPE '; charset=utf-8' DOCUMENT_ROOT '/etc/apache2/htdocs' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT 'text/html,text/plain,text/xml' HTTP_ACCEPT_CHARSET 'ISO-8859-1,utf-8;q=0.7,*;q=0.7' HTTP_ACCEPT_ENCODING 'gzip' HTTP_ACCEPT_LANGUAGE 'vi,en-us;q=0.7,en;q=0.3' HTTP_CONNECTION 'close' HTTP_HOST 'ckan.net' HTTP_USER_AGENT 'coccoc/1.0 ()' PATH_INFO '/dataset/uk-postboxes' PATH_TRANSLATED '/etc/ckan/datahub/apache.wsgi/dataset/uk-postboxes' REMOTE_ADDR '127.0.0.1' REMOTE_PORT '47065' REQUEST_METHOD 'GET' REQUEST_URI '/dataset/uk-postboxes' SCRIPT_FILENAME '/etc/ckan/datahub/apache.wsgi' SERVER_ADDR '127.0.0.1' SERVER_ADMIN '[no address given]' SERVER_NAME 'ckan.net' SERVER_PORT '80' SERVER_PROTOCOL 'HTTP/1.0' SERVER_SIGNATURE '<address>Apache/2.2.22 (Ubuntu) Server at ckan.net Port 80</address>\n' SERVER_SOFTWARE 'Apache/2.2.22 (Ubuntu)' WSGI Variables application <beaker.middleware.CacheMiddleware? object at 0x7f5312a51650> beaker.cache <beaker.cache.CacheManager? object at 0x7f5312a51790> beaker.get_session <bound method SessionMiddleware?._get_session of <beaker.middleware.SessionMiddleware? object at 0x7f5312a516d0>> beaker.session {'_accessed_time': 1345083636.079552, '_creation_time': 1345083636.079552} mod_wsgi.application_group 'ckan.net|' mod_wsgi.callable_object 'application' mod_wsgi.handler_script mod_wsgi.input_chunked '0' mod_wsgi.listener_host mod_wsgi.listener_port '8080' mod_wsgi.process_group 'datahub' mod_wsgi.request_handler 'wsgi-script' mod_wsgi.script_reloading '1' mod_wsgi.version (3, 3) paste.cookies (<SimpleCookie: >, ) paste.registry <paste.registry.Registry object at 0x7f52e47477d0> paste.throw_errors True pylons.action_method <bound method PackageController?.read of <ckan.controllers.package.PackageController? object at 0x7f52e477b810>> pylons.controller <ckan.controllers.package.PackageController? object at 0x7f52e477b810> pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'} pylons.pylons <pylons.util.PylonsContext? object at 0x7f52e477be90> pylons.routes_dict {'action': u'read', 'controller': u'package', 'id': u'uk-postboxes'} repoze.who.logger <logging.Logger object at 0x7f53127b2b50> repoze.who.plugins {'openid': <OpenIdIdentificationPlugin? 139994769488336>, 'friendlyform': <FriendlyFormPlugin? 139994769488528>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator? object at 0x7f5312d893d0>, 'auth_tkt': <AuthTktCookiePlugin? 139994771858704>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f5312d89350>} routes.route <routes.route.Route object at 0x7f5312955a50> routes.url <routes.util.URLGenerator object at 0x7f52eb8c9090> webob._parsed_query_vars (GET([]), ) webob.adhoc_attrs {'language': 'en-us'} wsgi process 'Multi process AND threads (?)' wsgi.file_wrapper <built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f5308983a80> wsgi.version (1, 1) wsgiorg.routing_args (<routes.util.URLGenerator object at 0x7f52eb8c9090>, {'action': u'read', 'controller': u'package', 'id': u'uk-postboxes'})

#2867: 1.8 url_for error

ross — Thu, 16 Aug 2012 10:57:49 GMT

From: <thedatahub.org@…> Date: Thu, Aug 16, 2012 at 10:30 AM Subject: WebApp? Error: <class 'routes.util.GenerationException?'>: url_for can only return a string, got unicode instead: https://ckannet-storage.commondatastorage.googleapis.com/file/fd3e00d9-eb0d-479b-8303-38909e5400ca?Signature=cmuU3CzRHQ86F3aZ0gljxv%2B3IqQ%3D&Expires=1345109432&AWSAccessKeyId=GOOGC6OU3AYPNY47B66M&x-goog-meta-uploaded-by=d5c8ed25-70de-4035-b29d-ddbe363913c6 To: ckan-sysadmin@…, kindly@…

URL: http://ckan.net/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca Module weberror.errormiddleware:162 in call << traceback_supplement = Supplement, self, environ

sr_checker = ResponseStartChecker?(start_response) app_iter = self.application(environ, sr_checker) return self.make_catching_iter(app_iter, environ, sr_checker)

except:

app_iter = self.application(environ, sr_checker)

Module beaker.middleware:73 in call << self.cache_manager)

environ[self.environ_key] = self.cache_manager return self.app(environ, start_response)

return self.app(environ, start_response)

Module beaker.middleware:155 in call << headers.append(('Set-cookie', cookie))

return start_response(status, headers, exc_info)

return self.wrap_app(environ, session_start_response)

def _get_session(self):

return self.wrap_app(environ, session_start_response)

Module routes.middleware:131 in call << r'\1', oldpath)

response = self.app(environ, start_response)

# Wrapped in try as in rare cases the attribute will be gone already

response = self.app(environ, start_response)

Module pylons.wsgiapp:125 in call <<

controller = self.resolve(environ, start_response) response = self.dispatch(controller, environ, start_response)

if 'paste.testing_variables' in environ and hasattr(response,

response = self.dispatch(controller, environ, start_response)

Module pylons.wsgiapp:324 in dispatch << if log_debug:

log.debug("Calling controller class with WSGI interface")

return controller(environ, start_response)

def load_test_env(self, environ):

return controller(environ, start_response)

Module ckan.lib.base:239 in call << try:

res = WSGIController.call(self, environ, start_response)

finally:

model.Session.remove()

res = WSGIController.call(self, environ, start_response)

Module pylons.controllers.core:221 in call << return response(environ, self.start_response)

response = self._dispatch_call() if not start_response_called:

self.start_response = start_response

response = self._dispatch_call()

Module pylons.controllers.core:172 in _dispatch_call << req.environpylons.action_method? = func

response = self._inspect_call(func)

else:

if log_debug:

response = self._inspect_call(func)

Module pylons.controllers.core:107 in _inspect_call << func.name, args)

try:

result = self._perform_call(func, args)

except HTTPException, httpe:

if log_debug:

result = self._perform_call(func, args)

Module pylons.controllers.core:60 in _perform_call << """Hide the traceback for everything above this method"""

traceback_hide = 'before_and_this' return func(args)

def _inspect_call(self, func):

return func(args)

Module ckan.controllers.storage:190 in file << return fapp(request.environ, self.start_response)

else:

h.redirect_to(file_url)

h.redirect_to(file_url)

Module ckan.lib.helpers:57 in redirect_to << if are_there_flash_messages():

kw__no_cache__? = True

return _redirect_to(url_for(*args, kw))

def url(*args, kw):

return _redirect_to(url_for(*args, kw))

Module ckan.lib.helpers:79 in url_for << # fix ver to include the slash

kwver? = '/%s' % ver

my_url = _routes_default_url_for(*args, kw) kw__ckan_no_root? = no_root return _add_i18n_to_url(my_url, locale=locale, kw)

my_url = _routes_default_url_for(*args, kw)

Module routes.util:265 in url_for << if not isinstance(url, str) and url is not None:

raise GenerationException?("url_for can only return a string, got "

"unicode instead: %s" % url)

if url is None:

raise GenerationException?(

"unicode instead: %s" % url)

GenerationException?: url_for can only return a string, got unicode instead: https://ckannet-storage.commondatastorage.googleapis.com/file/fd3e00d9-eb0d-479b-8303-38909e5400ca?Signature=cmuU3CzRHQ86F3aZ0gljxv%2B3IqQ%3D&Expires=1345109432&AWSAccessKeyId=GOOGC6OU3AYPNY47B66M&x-goog-meta-uploaded-by=d5c8ed25-70de-4035-b29d-ddbe363913c6 CGI Variables CKAN_CURRENT_URL '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' CKAN_LANG 'en' CKAN_LANG_IS_DEFAULT True CONTENT_TYPE '; charset=utf-8' DOCUMENT_ROOT '/etc/apache2/htdocs' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT '*/*' HTTP_ACCEPT_CHARSET 'utf-8;q=0.7,iso-8859-1;q=0.2,*;q=0.1' HTTP_CONNECTION 'close' HTTP_HOST 'ckan.net' HTTP_USER_AGENT 'Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@…)' PATH_INFO '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' PATH_TRANSLATED '/etc/ckan/datahub/apache.wsgi/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' REMOTE_ADDR '127.0.0.1' REMOTE_PORT '37236' REQUEST_METHOD 'GET' REQUEST_URI '/storage/f/file/fd3e00d9-eb0d-479b-8303-38909e5400ca' SCRIPT_FILENAME '/etc/ckan/datahub/apache.wsgi' SERVER_ADDR '127.0.0.1' SERVER_ADMIN '[no address given]' SERVER_NAME 'ckan.net' SERVER_PORT '80' SERVER_PROTOCOL 'HTTP/1.0' SERVER_SIGNATURE '<address>Apache/2.2.22 (Ubuntu) Server at ckan.net Port 80</address>\n' SERVER_SOFTWARE 'Apache/2.2.22 (Ubuntu)' WSGI Variables application <beaker.middleware.CacheMiddleware? object at 0x7f5312a51650> beaker.cache <beaker.cache.CacheManager? object at 0x7f5312a51790> beaker.get_session <bound method SessionMiddleware?._get_session of <beaker.middleware.SessionMiddleware? object at 0x7f5312a516d0>> beaker.session {'_accessed_time': 1345109433.273328, '_creation_time': 1345109433.273328} mod_wsgi.application_group 'ckan.net|' mod_wsgi.callable_object 'application' mod_wsgi.handler_script mod_wsgi.input_chunked '0' mod_wsgi.listener_host mod_wsgi.listener_port '8080' mod_wsgi.process_group 'datahub' mod_wsgi.request_handler 'wsgi-script' mod_wsgi.script_reloading '1' mod_wsgi.version (3, 3) paste.cookies (<SimpleCookie: >, ) paste.registry <paste.registry.Registry object at 0x7f5303b05610> paste.throw_errors True pylons.action_method <bound method StorageController?.file of <ckan.controllers.storage.StorageController? object at 0x7f530891fa10>> pylons.controller <ckan.controllers.storage.StorageController? object at 0x7f530891fa10> pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'} pylons.pylons <pylons.util.PylonsContext? object at 0x7f52f864f850> pylons.routes_dict {'action': u'file', 'controller': u'ckan.controllers.storage:StorageController', 'label': u'file/fd3e00d9-eb0d-479b-8303-38909e5400ca'} repoze.who.logger <logging.Logger object at 0x7f53127b2b50> repoze.who.plugins {'openid': <OpenIdIdentificationPlugin? 139994769488336>, 'friendlyform': <FriendlyFormPlugin? 139994769488528>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator? object at 0x7f5312d893d0>, 'auth_tkt': <AuthTktCookiePlugin? 139994771858704>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f5312d89350>} routes.route <routes.route.Route object at 0x7f531296e690> routes.url <routes.util.URLGenerator object at 0x7f52f82cd450> webob._parsed_query_vars (GET([]), ) webob.adhoc_attrs {'language': 'en-us'} wsgi process 'Multi process AND threads (?)' wsgi.file_wrapper <built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f52f5870cd8> wsgi.version (1, 1) wsgiorg.routing_args (<routes.util.URLGenerator object at 0x7f52f82cd450>, {'action': u'file', 'controller': u'ckan.controllers.storage:StorageController', 'label': u'file/fd3e00d9-eb0d-479b-8303-38909e5400ca'})

#1697: A Configurable list of states for a Dataset

rgrp — Tue, 24 Jan 2012 10:23:50 GMT

Currently have 'active' and 'deleted' suggest also:

'draft'
'hidden'
(Do we need both). Also write out workflows related to these.

#2433: API uses name not id for some version 3 calls

toby — Fri, 25 May 2012 14:54:08 GMT

see https://github.com/okfn/ckan/pull/5

#3002: API v1/2 'legacy' search parameters must be escaped before they are put into a Solr query string

amercader — Fri, 19 Oct 2012 09:32:22 GMT

Just to track @tauberer patch on Github. Would be nice to write a test for it. Probably going to 1.8.1

#2773: About page needs improving

markw — Mon, 30 Jul 2012 11:13:15 GMT

The about page for the DataHub? (thedatahub.org/about) could be improved. More importantly the default about page for a generic CKAN instance should be completely different - focus more on Open Data rather than the community hub idea (as this is more relevant for most installations) and remove specific references to tdh.

#1438: Action API - parameter discovery/checking

dread — Tue, 01 Nov 2011 15:38:10 GMT

Many actions in the Action API require parameters. What params are needed should be listed and checked. Because currently, if you get them wrong you simply get a useless 500 error.

Currently they are listed in the docs, extracted from the code manually.

So you could GET /action/api/package_list to receive not only the help text, but a list of arguments.

And if you send an extra or missing argument then an intelligent error message can be returned.

implementation

How about some sort of decorator on the action function:

@logic_params(id, offset, limit) def get_package_list(context, data_dict): ...
This would do the param checking, and is there a way to extract these params from the function? Or do a registration of the logic function?

I'd certainly like to keep the list of the list of params for the function with the function, for ease of reading the code.

Another good thing would be to pass in the params named as themselves, rather than having them contained in the data_dict.

#1439: Action API discoverablility

dread — Tue, 01 Nov 2011 15:39:30 GMT

A good service API needs to be discoverable, so you are not always having to refer to the documentation html.

Maybe /api/action should return a list of actions available? (Currently this returns a 404.)

It would be nice to sort these into get/create/update/delete.
#1438 Parameters for each of the actions must be discoverable too
/api/action/{action_name} should also return the help text / parameters allowable. (Currently this returns 400 error)

#1322: Action API improvements

dread — Thu, 08 Sep 2011 09:39:09 GMT

Focusing on improving Action API as the v3 API:

have an optional parameter of the data_dict called "options". Options would contain items that would get passed into the context. e.g. {"options" : {"ref_package_by": "id"}}.
instead of using API version to change the way packages are referenced, use the ref_package_by.
All package_show, group_show etc. to accept an object 'name' as an alternative to object 'id'.
Action API is v3 of api, replacement for v1 & v2. Default for most urls is still v1, but if url is /api/action then default to v3.
Next steps:

Add search API (package, resource,
Add Util API
Clarify JSONP still applies
Add doc strings, clarifying parameters

#2641: Adapt spatial widgets to new theme

amercader — Mon, 09 Jul 2012 15:02:27 GMT

Dataset extent map and spatial filter need to adapted to the new theme, as they are not showing up now

#2850: Add JSON support for IE7

aron.carroll — Tue, 14 Aug 2012 10:38:12 GMT

The JavaScript? just isn't going to work without it. Look into either the newer JSON3:

http://bestiejs.github.com/json3/

or the old faithful:

https://github.com/douglascrockford/JSON-js/

#2997: Add activity streams to dataset pages

seanh — Wed, 17 Oct 2012 14:31:46 GMT

Add an activity stream tab to dataset pages, like we have on user profile pages. Dataset activity streams are already implemented in the backend.

#2998: Add activity streams to group pages

seanh — Wed, 17 Oct 2012 14:33:14 GMT

Before we do this we should implement smarter group activity streams (i.e. what activities should appear in a group's activity stream?) and following of groups.

#2999: Add activity streams to organization pages

seanh — Wed, 17 Oct 2012 14:34:17 GMT

Before we do this we need to merge the organizations branch into master, we need to implement smarter activity streams for organizations, and we need to implement following of organizations.

#1667: Add an extension point for rendering activity streams

seanh — Tue, 17 Jan 2012 10:29:11 GMT

Currently rendering of activity stream events to HTML works by looking up a rendering function in a dictionary that maps activity types ('new package', 'changed group', etc.) to rendering functions that take an activity stream event and return the rendered HTML.

There needs to be an extension point where extensions can register their own rendering functions for particular activity types.

#2329: Add back in RSS/Atom links on relevant pages

rgrp — Thu, 26 Apr 2012 12:15:40 GMT

E.g. on dataset page, on revision page, on user page and on search results.

#2775: Add bin/less to paster serve command

aron.carroll — Mon, 30 Jul 2012 22:01:26 GMT

Ideally the ./bin/less command would be run when the server is started.

Also it would be good to have a paster command to build the production CSS with {{{ ./bin/less --production }}
The command could also detect missing node binaries and redirect to the documentation.

#2991: Add blocks to header.html template

seanh — Tue, 16 Oct 2012 11:10:20 GMT

Currently if you want to customise the header.html template you have to copy the whole thing, it isn't broken up into blocks like other templates are.

#2905: Add dataset URL key behaviour

johnmartin — Wed, 05 Sep 2012 14:28:03 GMT

On step 1 for adding a dataset can we add a ajax ping that checks if the URL that is generated by the JS is taken? Behaviour should be as follows:

User types title in box (as current)
URL key is regexed by JS (as current)
The JS does a ajax call to check if the generated URL key is valid. The ajax callback returns a JSON dump which contains what the URL key should be (e.g. if 'test' was taken it returns 'test-1')
If the URL key returned in the ajax is different than the one generated by the JS then:
4a. If the user hasn't clicked the edit button then: it automatically updates the URL key field with the URL key supplied in the ajax

4b. If the user has clicked the edit button then: a form error next to the input appears saying that the URL key is taken and within that error is suggests the URL key returned by the ajax as alternative (the user can click said URL key within the error to automatically choose that one)

The URL key isn't reserved by the ajax and upon form submission the usual validation should still apply.

#2994: Add dataset and user popovers throughout the site

seanh — Wed, 17 Oct 2012 14:26:04 GMT

If you hover the mouse over a dataset or user in an activity stream, you now get a popover with some info about that dataset or user and a follow/unfollow button.

These popovers should appear wherever datasets or users are listed, throughout the site.

#2932: Add docstring to system_info.py

seanh — Mon, 17 Sep 2012 14:50:29 GMT

What is the system_info table for? It looks like a way to override config file settings in the database, perhaps so that admins can use a web interface to change site title etc. Could do with a docstring explaining the intended purpose of the table.

#2921: Add docstring to top of lib/extract.py file

seanh — Thu, 13 Sep 2012 10:00:07 GMT

I think it couldn't hurt for this module to have a docstring at the top of the file explaining what the module is for. I know from setup.py that it's there to provide the extract_ckan() string extractor function for Babel, but I think looking at the file on its own it's not so obvious.

Also a couple of other small fixes:

jinja2_cleaner looks like a helper function only meant to be used by extract_ckan(), might make things clearer if it was called _jinja2_cleaner(), just so it doesn't look like a public function the module wants to export.

jinja_extensions seems to be a module-level variable that is only used in one function, could be moved into the function, unless you think there might be more functions that want it in future.

#2370: Add examples of all config settings in default config file

seanh — Thu, 03 May 2012 10:50:37 GMT

The default config file should contains examples (maybe commented out) for all config variables. Makes it easier to edit the config. For example openid_enabled is not in there.

#2477: Add extensions section to readthedocs

shevski — Wed, 30 May 2012 15:54:04 GMT

Need a central up to date place for currently supported extensions (closest we have is http://wiki.ckan.org/List_of_Extensions) & their documentation. I suggest this gets added to a dedicated section of docs.ckan.org

#1643: Add fixed tags to thedatahub for better browsing

shevski — Thu, 12 Jan 2012 18:34:53 GMT

Similar to publicdata.eu, want to have themed areas such as finance, environment, census, etc and country tags

#3005: Add following/unfollowing of groups: model, API, frontend, tests

seanh — Tue, 23 Oct 2012 17:35:44 GMT

#2914: Add form_to_db_schema_options() to IDatasetForm and IGroupForm

seanh — Wed, 12 Sep 2012 10:29:55 GMT

This optional method is supported when creating or updating groups and packages and is used in DefaultGroupForm? and DefaultPackageForm?, but seems to be missing from the interfaces. Should be added with docstring.

#2883: Add high level dev overview to 'For CKAN Developers' section of docs

seanh — Tue, 21 Aug 2012 10:52:35 GMT

Here's a draft: https://gist.github.com/3414107

#2625: Add i18n strings from non-core but supported extensions to ckan.pot file

seanh — Mon, 02 Jul 2012 13:48:23 GMT

Have to decide which non-core extensions are going to be supported first.

#1165: Add multi-site support to ckan

nils.toedtmann — Thu, 26 May 2011 12:41:07 GMT

Currently, each ckan site needs its own ckan wsgi process. That eats a lot of resources where many ckan sites are served from one machine (e.g. eu3).

That would dramatically change if a ckan process could behave like multiple ckans (e.g. like Apache's "<VirtualHost?>", or tracd). Depending on the "Host:" header in the HTTP1.1 request, it would choose which local ckan ini file to obey.

I see two ways to constitute the map hostname-to-ini-file map:

ckan reads a set of ini files, and each ini file declares which servers names it is responsible for
In a global ini file, there are directives mapping servernames to ini files.
In either case there should be a global ckan ini having the default settings for all local ckan sites. Each site ini could be very short then, just having e.g. title, name, database credentials, active plugins etc.

#2538: Add multiple-instance support to ckanbuild

seanh — Fri, 15 Jun 2012 15:51:39 GMT

Probably use ansible to do this. To create an instance, create a dir at /etc/ckan/MYSITE, and put MYSITE.wsgi, MYSITE.ini and who.ini files in it. Also put a MYSITE file in /etc/apache2/sites-available. See the example files already present in ckanbuild. Booting a new site should be a single command.

May not handle the postgres/solr/elastic-search side of things yet, could just require the user to set these up herself first and then pass them as args to the create-instance command.

#2683: Add no-cache header to _tracking API call's response to make sure it doesn't get cached

seanh — Mon, 16 Jul 2012 13:49:37 GMT

#2541: Add non-core extensions to ckanbuild

seanh — Fri, 15 Jun 2012 15:57:06 GMT

We want some extensions from outside of CKAN core to be included in ckanbuild. These would be pip installed into the virtualenv before packaging the debian package. Decide which extensions to include.

#3000: Add number of followers to dataset pages

seanh — Wed, 17 Oct 2012 14:36:00 GMT

User profile pages now show some interesting stats about the user: num. datasets, num. edits and num. followers. Would be nice to add some similar info to dataset pages. At least num. followers can be done (already implemented in backend).

#2861: Add qa breadcrumb

shevski — Thu, 16 Aug 2012 10:47:26 GMT

can we add in the breadcrumb navigation to qa pages so that you can go back to http://s031.okserver.org:2375/qa from sub pages such as http://s031.okserver.org:2375/qa/dataset/five_stars

#3025: Add requests to core requirements

amercader — Tue, 27 Nov 2012 21:35:24 GMT

Because yes please

#2298: Add sort-by controls to the search results page

icmurray — Mon, 16 Apr 2012 09:39:22 GMT

Sort-by functionality was exposed through the package controller in [1]. But no controls were added to the search-page.

What should the sortBy controls/widget look like?
Which fields should be exposed?
The above commit is in the release-1.6.1 branch, so this work is dependant upon release-1.6.1 from being merged into master.

[1] https://github.com/okfn/ckan/commit/8685c6000d1cb211928b4dbc63990fb72d884f8c

#895: Add version number (or simular) to css/js includes query string

memespring — Thu, 06 Jan 2011 19:49:42 GMT

Updates to css after a new deploy don't come through without a hard refresh. Adding the version number to the include urls will solve this e.g.

mycssfile.css?v=12345678

#1824: Add vocabulary pages

seanh — Tue, 21 Feb 2012 17:24:49 GMT

For a free tag foo you can visit the page at /tag/foo and see a list of all the datasets that have the tag foo, and when the tag appears on dataset view pages etc. it's linked to this tag page.

We should do the same thing for vocabulary tags. A tag bar in vocabulary baz should be hyperlinked to a page /tag/baz/bar, or perhaps /vocab/baz/bar.

#2200: Add vocabulary_id option to tag_show() logic action function

seanh — Wed, 29 Feb 2012 16:41:18 GMT

Currently tag_show() only works with free tags, it's not possible to get a tag_show for a vocabulary tag.

#3007: Adding a dataset creates multiple activities

seanh — Mon, 29 Oct 2012 13:15:07 GMT

Adding a new dataset creates multiple activity stream activities, e.g. seanh created the dataset foo, seanh add the resource bar to the dataset foo, and seanh updated the dataset foo are all created when I add a new dataset. I wonder if these can be collapsed into a single activity.

#1565: Admin dashboard finished?

jilly mathews — Mon, 19 Dec 2011 11:11:32 GMT

Is testing complete and ready for release?

#1185: Administrators can't delete packages from web UI

timmcnamara — Wed, 15 Jun 2011 04:23:38 GMT

Administrators have "View", "Edit" and "History" tabs. However, I can't see a way to delete a package from the web UI.

Version: CKAN.net as of today

#1749: Allow creating activity details through API

seanh — Sun, 05 Feb 2012 18:10:06 GMT

Currently the activity_create() logic action function only lets you create top-level activity stream items, and not their related activity details. It should handle activity details via nested dicts.

#1188: Allow diffing against initial (blank) package version

nickstenning — Wed, 15 Jun 2011 15:52:40 GMT

Currently the history page only allows diffing between different versions of a package, but there doesn't appear to be any easy way to see the changes introduced by the first version of a package.

I'm requesting the ability to diff against a "blank slate" initial state of a project, so I can see the content of the first project commit.

Not sure if this is a vdm feature, so I'm putting this ticket in against ckan.

#2956: Allow for resource editing in CKAN 2.0

johnmartin — Wed, 03 Oct 2012 15:11:40 GMT

On the edit dataset page... there needs to be a way for users to be able to see all the resources associated to the dataset and edit them individually.

#2688: Allow ordering of groups in WUI

ross — Tue, 17 Jul 2012 10:27:55 GMT

Currently the group_index page just shows the entire list of groups, forcing the ordering to be by name. It would be better if it could be sortable by name (or reversed) or by package_count (or reversed)

#2785: Allow resources to be re-ordered

aron.carroll — Wed, 01 Aug 2012 10:22:03 GMT

Not sure where this functionality should be added, possibly in one of the sidebar widgets when editing a resource?

Ira, what are your thoughts?

#1550: Allow simple auth via the API

ross — Fri, 16 Dec 2011 15:29:29 GMT

It should be possible to pass userid/username and api key and obtain a response from CKAN for external services that use CKAN auth. Those services shouldn't be talking to the DB directly.

#1257: Anti-Spam tools

dread — Tue, 02 Aug 2011 11:12:45 GMT

We are getting more and more spam on ckan.net and we need to improve our strategy of combating it. It is bad because google ranks who we link to (which we do want for legitimate links), and our front page contains the latest package edits, so spam is immediately very visible.

Spam:

Package creation
Package edit
User creation
Group creation
Systems to consider:

Automatic
captcha
bayesian scoring
Sysadmin
a tool to quickly analyse and remove packages, edits and users
Helpful users
button to click indicating spam, leading to any/all of:
add TODO item (which emails package admin and sysadmins)
quarantine the package/user so it doesn't show up on the front page and a warning is shown when you view it
General thoughts:

We should learn from the wikipedia tools, as they are the experts
This is a serious problem that is only going to get worse and is a threat to the whole site.

#2968: Anyone can access organization members page

seanh — Mon, 15 Oct 2012 10:19:15 GMT

The button will not show if you are not authorized but browse to /organization/members/foo and you can edit the members, it does stop you when you try to save your changes, but you shouldn't be able to get to the page at all

#1573: Apps and Ideas

jilly mathews — Mon, 19 Dec 2011 11:36:33 GMT

Estimate 2 weeks for someone to finish and test.

#2709: Atom feeds are undocumented

markw — Wed, 18 Jul 2012 15:11:50 GMT

There doesn't seem to be any documentation yet for Atom feeds.

#1120: Atom feeds of each tag

tsm — Mon, 02 May 2011 04:08:05 GMT

Tags could/should have an Atom feed. This would mean that every edit to relevant packages could be easily monitored. See [1].

[1] http://lists.okfn.org/pipermail/ckan-discuss/2011-May/001162.html

#2984: Auto-populate context with user, model, session, etc.

seanh — Mon, 15 Oct 2012 10:48:21 GMT

From Toby: make the context auto-populated with things like user/model/session if not user supplied - especially for extensions (pre 2.0) maybe via a helper or else a decorator on the action - helper seems nicer maybe done via get_action()

#2404: Auto-screenshot for related items

rgrp — Mon, 21 May 2012 06:55:47 GMT

As a User I want an image generated for my related item automatically so that I do not have to take picture myself and upload it somewhere

Implementation: (?) Use phantomjs or something like http://snapito.com/ Investigate for 1.8, will be needed for TDH Oct.

#2924: Better docs for trans js command, and add to release process

seanh — Thu, 13 Sep 2012 10:04:31 GMT

Add a better docstring to for trans js command explaining what it does and why, and how to use it.

Also add this command to the CKAN Release Process as it's needed there

#2922: Better docstring for CKANInternationalizationExtension

seanh — Thu, 13 Sep 2012 10:01:47 GMT

I'm unsure about what's going on here. As I understand it, when we run python setup.py extract_messages it's going to use the extract_ckan() function from ckan/lib/extract.py to process the HTML files. For the HTML files that are jinja2 templates, extract_ckan() will call jinja2_cleaner() which will call regularise_html() on the strings. So the strings are regularised when they are extracted from the source files.

But then we have the parse() method of CkanInternationalizationExtension? also calling regularise_html(). I don't get what's happening here. Why do the strings need to be regularised twice?

My guess is that CkanInternationalizationExtension? is used when the strings are extracted from the templates at runtime, and they need to be regularised at this time in order to match them against the regularised strings in the mo files to find the translations to output?

Maybe CkanInternationalizationExtension? needs a better docstring saying what it does?

#2931: Better docstring for app_globals.py

seanh — Mon, 17 Sep 2012 14:13:14 GMT

The application's Globals object is not very informative.

#2480: Better message when dataset has no resources

markw — Thu, 31 May 2012 08:31:33 GMT

If a dataset has no resources the resources list currently says '(none)'.

Here is a suggested improvement, provided that a maintainer is named: 'There are no data resources here yet. For information about this data, contact the dataset maintainer.'

#1581: Blog post about Google Analytics extension for CKAN

johnglover — Tue, 20 Dec 2011 17:40:00 GMT

The CKAN Google Analytics extension has been updated to work with the latest version of CKAN, could make for a nice blog post.

Can ping John Glover in January for any details required.

Key link is: http://thedatahub.org/analytics/dataset/top though this should probably move to be under stats (e.g. http://thedatahub.org/stats/usage)

#1457: Bug with DataNL instance

jilly.mathews — Thu, 10 Nov 2011 11:37:21 GMT

"when logging into http://register.data.overheid.nl/ with OpenID, the /user/me page gives a 404. CKAN version 1.3.4."

n the manual it says an API key kan be created via http://test.ckan.net/user/me /. However, when I try the corresponding http://register.data.overheid.nl/user/me, I get a 404 error (not found).

#2877: Bugs with datastore v2

rgrp — Sat, 18 Aug 2012 00:33:22 GMT

In progress

~~[major] q does not seem to work reliably. e.g. using the setup from this gist https://gist.github.com/1930806 and doing a ?q=DE yields no results (does not work with "q=de" either)~~
~~q=second does work ...~~
[major] q does not work with 2 values (see below)
Query on search with limit 0 results in total of 0 (should either be null or correct total). Queries with other limits yield correct total AFAICT
Also weird fact that limit is returned but as as as string - should it not be an integer
Types on fields: could these not be canonical and as per recline (or is it important to allow exact sql types ...)
Multiple query values

Try a query such as: "second UK" and you will get 500 error:

http://localhost:5000/api/3/action/datastore_search?resource_id=4f1299ab-a100-4e5f-ba81-e6d234a2f3bd&q=second%20UK

ProgrammingError: (ProgrammingError) syntax error in tsquery: "second UK" 'select "_id", "id", "date", "x", "y", "z", "country", "title", "lat", "lon", count(*) over() as "_full_count"\n from "4f1299ab-a100-4e5f-ba81-e6d234a2f3bd" where _full_text @@ to_tsquery(%s) limit 100 offset 0' (u'second UK',)
Suggestions

Filter support: should think in more detail about this (may want to follow recline style)

Simple filters in query parameters would be nice too ...

#1542: Buttons to purge spam datasets and groups

dread — Thu, 08 Dec 2011 17:22:10 GMT

A sysadmin should be able to easily examine a suspect group or package, determine if it was created by a spammer (as opposed to being a legitimate object that has been graffitied by a spammer) and purge it.

The existing two-stage revision delete is currently unreliable and perhaps too laborious.

Olav and Richard have needs along this line.

#3016: CKAN 2.0 template tweaks

johnmartin — Tue, 13 Nov 2012 13:30:17 GMT

Just a ticket to keep track of a few suggested template changes.

#837: CKAN integration with freebase gridworks / google refine

rgrp — Tue, 30 Nov 2010 18:10:09 GMT

Thread: http://lists.okfn.org/pipermail/ckan-discuss/2010-November/000718.html

Scenario 1

User installs Refine and CKAN extension for refine
On booting refine and asked to load data they can choose from any data package on CKAN.net (or any other CKAN instance)
They edit the dataset on Refine
On save (or perhaps as a separate option) they are prompted as to whether they wish to sync the dataset back to CKAN (either as a new package or as a new resource on the existing package)
NB: for the dataset sync back some form of "CKAN" storage would be required (we already have storage.ckan.net running but a closer integration would be required)

Scenario 2

User visits a package on CKAN.net (or another CKAN instance)
There is a button on the page "View and edit this dataset in Google Refine"
Click button -- ask them if they have Google refine installed
Yes: instructions for loading dataset into refine
No: load dataset in hosted version of google refine (we could run this)
User edits dataset and hits save. As in previous scenario they are prompted to sync the dataset.

#1384: CKAN wiki needs updating to refer to thedatahub.org instead of ckan.net and datasets instead of packages

shevski — Wed, 12 Oct 2011 10:07:57 GMT

Most articles still refer and link to ckan.net, wiki.ckan.net and to packages

#728: CSW Harvesting shall be optimised in respect of reharvesting only records that have changed

johnbywater — Thu, 21 Oct 2010 15:35:40 GMT

Hi Will, this is important again because some CSW servers we use have over 300 documents in. Could you take a look at modifying the filter please?

#1577: Can't upload file with foreign chars in filename

dread — Mon, 19 Dec 2011 18:00:59 GMT

Looks like uploading a file with foreign characters fails due to encoding reasons.

URL: http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf Module weberror.errormiddleware:162 in __call__ << __traceback_supplement__ = Supplement, self, environ sr_checker = ResponseStartChecker(start_response) app_iter = self.application(environ, sr_checker) return self.make_catching_iter(app_iter, environ, sr_checker) except: >> app_iter = self.application(environ, sr_checker) Module beaker.middleware:73 in __call__ << self.cache_manager) environ[self.environ_key] = self.cache_manager return self.app(environ, start_response) >> return self.app(environ, start_response) Module beaker.middleware:152 in __call__ << headers.append(('Set-cookie', cookie)) return start_response(status, headers, exc_info) return self.wrap_app(environ, session_start_response) def _get_session(self): >> return self.wrap_app(environ, session_start_response) Module routes.middleware:130 in __call__ << environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1] response = self.app(environ, start_response) # Wrapped in try as in rare cases the attribute will be gone already >> response = self.app(environ, start_response) Module pylons.wsgiapp:125 in __call__ << controller = self.resolve(environ, start_response) response = self.dispatch(controller, environ, start_response) if 'paste.testing_variables' in environ and hasattr(response, >> response = self.dispatch(controller, environ, start_response) Module pylons.wsgiapp:324 in dispatch << if log_debug: log.debug("Calling controller class with WSGI interface") return controller(environ, start_response) def load_test_env(self, environ): >> return controller(environ, start_response) Module ckan.lib.base:123 in __call__ << # available in environ['pylons.routes_dict'] try: return WSGIController.__call__(self, environ, start_response) finally: model.Session.remove() >> return WSGIController.__call__(self, environ, start_response) Module pylons.controllers.core:221 in __call__ << return response(environ, self.start_response) response = self._dispatch_call() if not start_response_called: self.start_response = start_response >> response = self._dispatch_call() Module pylons.controllers.core:172 in _dispatch_call << req.environ['pylons.action_method'] = func response = self._inspect_call(func) else: if log_debug: >> response = self._inspect_call(func) Module pylons.controllers.core:107 in _inspect_call << func.__name__, args) try: result = self._perform_call(func, args) except HTTPException, httpe: if log_debug: >> result = self._perform_call(func, args) Module pylons.controllers.core:60 in _perform_call << """Hide the traceback for everything above this method""" __traceback_hide__ = 'before_and_this' return func(**args) def _inspect_call(self, func): >> return func(**args) Module ckanext.storage.controller:2 in auth_form Module ckan.lib.jsonp:26 in jsonpify << Very much modelled after pylons.decorators.jsonify . """ data = func(*args, **kwargs) return to_jsonp(data) >> data = func(*args, **kwargs) Module ckanext.storage.controller:301 in auth_form << method = 'POST' authorize(method, bucket, label, c.userobj, self.ofs) data = self._get_form_data(label) return data >> authorize(method, bucket, label, c.userobj, self.ofs) Module ckanext.storage.controller:79 in authorize << if method != 'GET': # do not allow overwriting if ofs.exists(bucket, key): abort(409) # now check user stuff >> if ofs.exists(bucket, key): Module ofs.remote.botostore:53 in exists << if bucket is None: return False return (label is None) or (label in bucket) def claim_bucket(self, bucket): >> return (label is None) or (label in bucket) Module boto.s3.bucket:87 in __contains__ << def __contains__(self, key_name): return not (self.get_key(key_name) is None) def startElement(self, name, attrs, connection): >> return not (self.get_key(key_name) is None) Module boto.s3.bucket:144 in get_key << response = self.connection.make_request('HEAD', self.name, key_name, headers=headers, query_args=query_args) # Allow any success status (2xx) - for example this lets us # support Range gets, which return status 206: >> query_args=query_args) Module boto.s3.connection:388 in make_request << if isinstance(key, Key): key = key.name path = self.calling_format.build_path_base(bucket, key) boto.log.debug('path=%s' % path) auth_path = self.calling_format.build_auth_path(bucket, key) >> path = self.calling_format.build_path_base(bucket, key) Module boto.s3.connection:88 in build_path_base << def build_path_base(self, bucket, key=''): return '/%s' % urllib.quote(key) class SubdomainCallingFormat(_CallingFormat): >> return '/%s' % urllib.quote(key) Module urllib:1222 in quote << safe_map[c] = (c in safe) and c or ('%%%02X' % i) _safemaps[cachekey] = safe_map res = map(safe_map.__getitem__, s) return ''.join(res) >> res = map(safe_map.__getitem__, s) KeyError: u'\xed' CGI Variables AUTH_TYPE 'cookie' CONTENT_TYPE '; charset=utf-8' DOCUMENT_ROOT '/htdocs' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT '*/*' HTTP_ACCEPT_CHARSET 'ISO-8859-1,utf-8;q=0.7,*;q=0.3' HTTP_ACCEPT_ENCODING 'gzip,deflate,sdch' HTTP_ACCEPT_LANGUAGE 'en-US,en;q=0.8' HTTP_CACHE_CONTROL 'max-age=259200' HTTP_CONNECTION 'keep-alive' HTTP_COOKIE 'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; ckan_user=elenaibp; ckan_display_name="Elena Mondo"; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; __utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10.1324298316; __utmc=119670349; __utmz=119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' HTTP_HOST 'thedatahub.org' HTTP_REFERER 'http://thedatahub.org/dataset/edit/budget-library-czeck-republic' HTTP_USER_AGENT 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7' HTTP_VIA '1.1 localhost (squid/3.0.STABLE19)' HTTP_X_FORWARDED_FOR '87.114.74.190' HTTP_X_REQUESTED_WITH 'XMLHttpRequest' PATH '/usr/local/bin:/usr/bin:/bin' PATH_INFO '/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf' PATH_TRANSLATED '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf' REMOTE_ADDR '193.34.146.142' REMOTE_PORT '55419' REMOTE_USER u'elenaibp' REMOTE_USER_DATA 'userid_type:unicode' REMOTE_USER_TOKENS [''] REQUEST_METHOD 'GET' REQUEST_URI '/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ%C3%AD-%C4%8Cesk%C3%A9-republiky-_-P%C5%99%C3%ADprava-rozpo%C4%8Dtu.pdf' SCRIPT_FILENAME '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py' SCRIPT_URI 'http://thedatahub.org/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf' SCRIPT_URL '/api/storage/auth/form/2011-12-19T124447/Ministerstvo-financ\xc3\xad-\xc4\x8cesk\xc3\xa9-republiky-_-P\xc5\x99\xc3\xadprava-rozpo\xc4\x8dtu.pdf' SERVER_ADDR '193.34.146.146' SERVER_ADMIN '[no address given]' SERVER_NAME 'thedatahub.org' SERVER_PORT '80' SERVER_PROTOCOL 'HTTP/1.0' SERVER_SIGNATURE '<address>Apache/2.2.14 (Ubuntu) Server at thedatahub.org Port 80</address>\n' SERVER_SOFTWARE 'Apache/2.2.14 (Ubuntu)' WSGI Variables application <beaker.middleware.CacheMiddleware object at 0x7f22601c7dd0> beaker.cache <beaker.cache.CacheManager object at 0x7f22601c7b50> beaker.get_session <bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x7f22601c7a90>> beaker.session {'_accessed_time': 1324298703.071357, '_creation_time': 1324293077.4139669} mod_wsgi.application_group 'ckan.net|' mod_wsgi.callable_object 'application' mod_wsgi.listener_host '' mod_wsgi.listener_port '80' mod_wsgi.process_group 'ckan.net' mod_wsgi.reload_mechanism '1' mod_wsgi.script_reloading '1' mod_wsgi.version (2, 8) paste.cookies (<SimpleCookie: __utma='119670349.1809834699.1323782464.1324293066.1324298316.4' __utmb='119670349.3.10.1324298316' __utmc='119670349' __utmz='119670349.1323782464.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' auth_tkt='f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode' ckan_apikey='decd48b1-49ee-4250-bff4-98ccca9c02a5' ckan_display_name='Elena Mondo' ckan_user='elenaibp' hide_welcome_message='1' thedatahub_net='27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e'>, 'thedatahub_net=27a7f095fcca1ea6b36df996d595e3278b16f4538862bf7f88d49e2000b9246547c8fd0e; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; auth_tkt="f9c6ab2b0d9fcd71c4c2408bc12fab544eef1c45elenaibp!userid_type:unicode"; ckan_user=elenaibp; ckan_display_name="Elena Mondo"; ckan_apikey=decd48b1-49ee-4250-bff4-98ccca9c02a5; hide_welcome_message=1; _ _utma=119670349.1809834699.1323782464.1324293066.1324298316.4; __utmb=119670349.3.10...)|utmcmd=(none)') paste.registry <paste.registry.Registry object at 0x7f226194df50> paste.throw_errors True pylons.action_method <bound method StorageAPIController.auth_form of <ckanext.storage.controller.StorageAPIController object at 0x7f2261dad990>> pylons.controller <ckanext.storage.controller.StorageAPIController object at 0x7f2261dad990> pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'} pylons.pylons <pylons.util.PylonsContext object at 0x7f2261daddd0> pylons.routes_dict {'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'} repoze.who.identity <repoze.who identity (hidden, dict-like) at 139785645747120> repoze.who.logger <logging.Logger instance at 0x7f225e23c098> repoze.who.plugins {'openid': <OpenIdIdentificationPlugin 139785625065680>, 'friendlyform': <FriendlyFormPlugin 139785618095248>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator object at 0x7f2260874c10>, 'auth_tkt': <AuthTktCookiePlugin 139785625065808>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f2260874c90>} routes.route <routes.route.Route object at 0x7f22601a1090> routes.url <routes.util.URLGenerator object at 0x7f2261dadf50> webob._parsed_query_vars (GET([]), '') webob.adhoc_attrs {'language': 'en-us'} wsgi process 'Multiprocess' wsgi.file_wrapper <built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f2261da9af8> wsgiorg.routing_args (<routes.util.URLGenerator object at 0x7f2261dadf50>, {'action': u'auth_form', 'controller': u'ckanext.storage.controller:StorageAPIController', 'label': u'2011-12-19T124447/Ministerstvo-financ\xed-\u010cesk\xe9-republiky-_-P\u0159\xedprava-rozpo\u010dtu.pdf'})

#3019: Cannot delete dataset extras

seanh — Wed, 14 Nov 2012 18:44:38 GMT

Deleting extras in the web interface is broken

#2725: Case sensitivity on tags

shevski — Sun, 22 Jul 2012 09:34:27 GMT

My feeling is that 'country-US' and 'country-us' should be the same tag. However currently tags with caps are treated differently

see http://s031.okserver.org:2375/en/dataset/test-dataset

with TEST and test - there also get indexed twice in the search page

#2679: Change default behaviour of TemplateController.view to 404.

icmurray — Mon, 16 Jul 2012 10:55:33 GMT

The current behaviour of TemplateController?.view() (which is the fallback controller should all others fail) is to attempt to render (as a genshi template) the requested file.

Although this may be a feature that some instances want. In general, it leads to:

500s when attempting to access a normal template (eg - http://datahub.io/importer/preview)
A way of inadvertantly serving things you may not want to serve. (Small risk, as it needs to be renderable as a genshi template).
Solution:

Change the controller to 404
Ensure there's a way for existing ckan instances to override that behaviour should they need it.

#2923: Change regularise -> regularize

seanh — Thu, 13 Sep 2012 10:03:02 GMT

The function is called regularise_html(), can't remember what file it's in.

#1534: Change revisions to record userid rather than username

rgrp — Wed, 07 Dec 2011 17:26:30 GMT

The use of username is problematic because username's can change.

Change all revision creation code to use user id (simplest is to change c.author field in lib/base.py (?))
(?) Add a field ipaddr for ip address of anonymous users? (or just keep putting this in author field on Revision and then acception that those won't match when we do a look up against user table)
Change user view page to look up against user id rather than name
Perform migration on existing Revision objects
Match should probably be against both openid and username when searching Revisions' author field (especially true on CKAN where some people have already changed their username from being their openid)

#2959: Changing a Group's name through the action api disassociates it from its datasets in the index

icmurray — Thu, 04 Oct 2012 15:18:55 GMT

Repro:

Create a new Group, named "test-group".
Add a dataset to it.
Verify the dataset belongs to the group by visiting the Group's read-page
Update the Group through the action api (group_update), using the uid in the "id" field, and a new name in the "name" field.
Visit the group's read-page. The list of datasets will be empty.
This was an issue when editing a Group through the web interface, which was fixed in [1]. However it only fixes the issue in the group controller.

[1] https://github.com/okfn/ckan/commit/dbe25d8b8d7fabfc40c5d794a920b91cec349335

#2795: Check validation of HTML, CSS, JS

toby — Thu, 02 Aug 2012 10:25:28 GMT

Ensure that we are being standards compliant

#2943: Chrome does not resize preview

dominik — Mon, 01 Oct 2012 11:08:06 GMT

Chrome does not resize iframe after a full refresh/ on first load

#1171: Citation instructions on dataset and resource view pages

dread — Wed, 01 Jun 2011 09:33:19 GMT

Some sort of citation helper. Something small on the dataset and resource page that would show how to cite.

wwaites: Some related thoughts on this from opb: http://homepages.inf.ed.ac.uk/opb/papers/ssdbm2006.pdf

timclicks: I'm looking at Dataverse for the first time[0]. It seems very popular in the social sciences. I noticed that there is a recommended citation for each dataset. For example, [1] is has this one: "Targeted Input Programme (TIP) 2000-01", http://hdl.handle.net/1902.1/SSC-MWI-TIP2000-01-M1 V1 [Version]"

Implementation

Add a small box at bottom of dataset / resource page (or in sidebar on dataset page) with title "Cite this" with contents like:

%title. %author. Retrieved %date. %site_title.

For resource: %title = %dataset_title. %resource_name.

Could also add export to ref managers (e.g. to bibtex) but that is for later.

#2874: Clean up bin directory

rgrp — Fri, 17 Aug 2012 08:01:48 GMT

Full of obsolete material

#1790: Click to delete tags, rather than have all existing tags in the tag text box

dread — Fri, 10 Feb 2012 15:44:34 GMT

From Pablo:

Editing the tags field is clumsy when there are too many tags. Could show existing effectively as tags (like delicious), then allow clicks to delete. New tags added via text box.

#2835: Client module needs a template loading method

aron.carroll — Thu, 09 Aug 2012 17:10:33 GMT

Client#getTemplate(name, params, success, error);
Where params, success and error are optional arguments. test/index.html already has an implementation called loadFixture().

#2890: Collect data previews and data store docs in one chapter

seanh — Tue, 28 Aug 2012 10:20:36 GMT

Currently there is this page:

http://docs.ckan.org/en/latest/data-viewer.html

which covers Recline Data Explorer and other kinds of data preview in CKAN. It is under the Publishing Datasets section in the documentation. I had to to a search for 'recline' to find it.

Separately there is this page: http://docs.ckan.org/en/ckan-1.7.1/datastore.html which covers datastore, datastorer, and the data api.

I suggest collecting this together in one chapter called 'Data Previews'. If I understand it right the general gist would be:

CKAN has builtin previews of data resources on resource pages, enabled by default.

Images, Google Documents, and web page resources will be loaded into embedded iframes for preview.

Text-like files will be displayed raw.

CSV or Excel files uploaded to CKAN will be previewed using Recline Data Explorer.

Additionally, you can enable CKAN's DataStore?, requires you to install ElasticSearch? and nginx and put datastore.enabled=1 in your ini file. Lets you use the Data API to query data.

Does having DataStore? enabled mean you get preview of more types of resources? Any resource that's available via the Data API will be previewed using Recline,

You can install ckanext-datastorer, and then CSV and Excel files _linked to_ as CKAN resources will be previewed using Recline also. Requires celeryd.

#1182: Comments from deleted packages appear in "Recent Comments" feed

timmcnamara — Thu, 09 Jun 2011 22:24:11 GMT

When a package has been deleted, say for spam moderation, comments still appear in the recent comments section.

This is a problem because non-admin users will be shown a warning that they're not authorised to view the package if they click on the link.

At CKAN.net currently, this affects the most recent comment.

#1557: Complete Webstore Preview Extension

jilly mathews — Mon, 19 Dec 2011 10:40:53 GMT

Finish any work out standing on web store preview extension to be able to package and release.

Ref James and I going through existing features and trying to mention any polishing that needed doing to get exiting features ready for release with projects such as CKAN hosted.

#2886: Configurable related items

ross — Wed, 22 Aug 2012 08:55:18 GMT

Related items (Apps & Ideas) have a collection of types, which are currently fixed. Some of these types are ambiguous, such as ideas in that they may not have a link - and if they do it is likely to be to a blog post - another existing type.

Whilst the URL is required, and we think this should stay required, we should also allow users to change the types found in Apps & Ideas to a shortened list that suits their requirements.

#2415: Confusing message on attempting to register new account with upper case

markw — Wed, 23 May 2012 13:19:49 GMT

On registering a new account, if you try to use a capital letter in your 'Login', you get the unhelpful error message:

"The form contains invalid entries: Name: Url must be purely lowercase alphanumeric (ascii) characters and these symbols: -_"

What does 'Name' or 'Url' have to do with anything? It means 'Login must be ...'

But even better would be to make the login case-insensitive and allow capitals to be entered.

#2813: Confusing sidebar on demo dataset page

markw — Wed, 08 Aug 2012 10:03:26 GMT

On a dataset page on demo.ckan.org, the left sidebar is confusing.

It starts with some random links. Actually they are links to groups which the dataset is in, but this isn't clear.
The sidebar elements that are actually part of the dataset are 'Datset extent' and 'License', so these should be right at the top (if they belong in the sidebar at all), instead of which they are right at the bottom in the junk part of the page (i.e. probably lower than the bottom of the main page, and hence lower than anyone will scroll).

#2287: Consistent datetime and time period strings localization

seanh — Thu, 12 Apr 2012 18:08:43 GMT

Find all places where date, times and time periods are rendered as strings in CKAN. Make them all go through the same code path (e.g. the helper function in helpers.py) with localization support.

For example, on the user index page right now there are time period strings like "less than 1 month" that currently do not get translated (and the way they're implemented doesn't look i18n-friendly).

ISO date format everywhere might be a good idea.

Need to consider local timezone issues.

Python standard library and Babel may have useful helpers for this.

Should the local timezone displayed be a global setting per CKAN instance? Or should it change depending on the location of the user who is viewing the page?