{22} Trac tickets (2647 matches)

• Create page on ​http://wiki.ckan.org/ about setting up a queue (this will merge into core docs when this is stable) - having a queue will now become a generic requirement for a bunch of extensions
• README with install instructions in ckanext-archiver
• Create an entry in ​http://wiki.ckan.org/Extensions
• Write a blog post

• Customize text at bottom of forms
• Also need to make clear that this does not apply to the data itself (that is covered by the license you choose on your dataset ...)

• Deleting a group changes state to 'deleted' rather than purging it
• Adding authz tests for deleted groups

• Deserialize forms to new dict format.
• Replace controllers/forms to use dictization.

• Ensure use case is in design - "an admin writes to the CKAN API" (and see JB notes on paper)
• Communicate how this is planned to be achieved in design to JF

• Ensure you can add/change/delete extra fields from packages using the REST API.
• Update API docs to cover this.

On Package get, ensure separation of extra attributes from other attributes.

What is the meaning of posting an incomplete dictionary (updating a package)? Option 1 - if field not present, leave field alone

• Delete field if field value is None

Option 2 - user must supply all fields

• Delete field if field absent

Prefer option 1. Document this.

Follows on from ticket:152

• Generic text box for markdown about the user 'About'

Model's user table reflects these:

• 'about' attribute

• Get rid of jquery UI
  • Requires switching to bootstrap lookahead plus a small bit of coding for ajax part (see ​https://gist.github.com/2007241)
• Remove modernizr (what do we use it for?)
• Convert jquery.tmpl to jquery mustache (?)
• Remove ckanjs moving core parts of it directly into our code.

Compress everything for releases and add minified version to build (with switch to enable use of this?)

• Hide remove button when adding new field
• Finish takes you to the newly created dataset

• Major issue with spam signup on thedatahub.org
• Suggest using recaptcha

Est cost: 1d

• Move into core (keeping as an extension) and default enable
• Do we need tests for controller? (No: already there)
• Documentation (?) - e.g. of basic config options
• Remove lib/stats.py which is still in core
  • dread moved it to extension in ​https://github.com/okfn/ckanext-stats/commit/293b2544737d8ab7610884bd5e999ee37f1ab813 but did not remove from core
• Deprecation notice on ckanext-stats repo and update extension registry

Est: 0.5d.

Questions:

• Why do this?
  • tiny extension with few dependencies - and really nice to have out of the box
  • trial for doing this on larger scale
• Do we keep as extension (even if in core)?
  • Ans: Yes, keep as extension because:
    • Already set up that way
    • Cleaner
    • Easier to disable / enable
• How do we integrate with main them (e.g. have stats link)
  • Ans: not sure (this is part of more general issue of how we update theme for varying changes elsewhere). Best answer is to have some widgetization in theme.
• Hide ratings section (at least until we reinstate ratings #1598)
  • Ans: no, let's not bother (and having ratings there encourages us to do #1598 and/or find out whether people are interested in ratings)

• No tests.
• 500 error given when user is not found.

• Propose three-column package listing: two lists of highest-scorers on some metric and last column is an invitation to register a new package, with a little copy explaining what's involved and why you'd want to.
• Add in search widget

• Relationships in sidebar [WONTFIX yet without some link to documentation or datastructure, something for me to render?]
• Icons for *types* of resources in resource list
• List of 'Owners' (administrators) at top right of sidebar (a la group page)
  • Much prefer owner to administrator ...

• Review process of releases and updating strings (pot) and translations (po) files
  • c.f. ​http://lists.okfn.org/pipermail/ckan-dev/2011-May/000718.html
• Get language changing thing to get home page to mostly work - push to key strings translated (or don't advertise the translation?)
• Review long list of strings for improvement
  • Consolidate any?
  • Get rid of i18n for strings in api
  • Try and remove javascript
  • Add in extensions
• #1374 Fix switching to en from other default language
• #1417 Fix browser language detection

• Set hover names on navigation links.
• Do we really need to display "About CKAN" everywhere? Just on home page.
• Redirect in routing /groups and /tags rather than /group and /tag. At the very least make the plural versions forward to the singular versions.

• Show resources in the sidebar
• Allow people to go backward and forward
• Have a back button on each form
• Lose resource type
• Add extras as optional metadata
• Remove save
• Add back/Previous
• Add Another

• There is huge inconsistency in the titles of pages across the site: to give one example "Edit Package:" vs "Package: mypkg [not linked]" vs "History - mypkg [linked to package page]"
  • Normalize and use terminology: "Data Package" - also change in navbar (but nowhere else for the time being). So hvae Data Package - mypkg, Data Package - mypkg - Edit etc
• pagination is currently lost in amongst the results list. needs to be *much* more obvious, and should appear top and bottom, or at the very least at the bottom!

• Upgrade to new DataStore? system
• Better support for unicode etc
• ? Rename to DataStorer?
• Deploy

• Use what is on iatiregistry.org at the moment.

• allow clicking on 'only openly licensed'/'only downloadable' to select/deselect checkboxes: make them <label>s.
• cut 'only's -- use word 'filter' instead
• doesn't appear to match substrings!? A search for 'anna' doesn't return match the 'annakarenina' package.
• autocomplete package names & tags

• code review with D.R. for ckanext-ecportal
• merge and test all multilingual code
• fix server issues: routing, Git access to CKAN repository

Bugs to fix:

• rename groups to publishers
• update home page layout (recent updates along side publishers)
• don't add 'en' to main menu links if english
• resource edit page, fix missing images
• change dataset 'settings' to 'edit'
• translate 'published by' drop-down list
• fix JS on 'add a dataset page'
• prepopulate 'published_by' when clicking 'add a dataset' from group
• chosen broken (dataset edit) if in different language

• created repo for final list of instructions: ​http://github.com/okfn/ecportal-server-setup

• data.london.gov.uk (1d, ping DERI)
• catalogues from Paris and Vienna (1d)

Others?

• digitaliser.dk (1d)
• data-publica.com (2d)
• data.suomi.fi (2d)
• dati.piedmonte.it (1d)
• portalu.de (2d)

• deploy celery
• deploy QA and archiver tasks
• write up a blog post announcing QA on thedatahub

• examples of posting a new package
• example license_id - explain

• group form: description field for needs to be a text area
• group form: make title and name field wider
• package view: need to move tags and license above notes

• parse like data4nr code

Follow up tickets: ticket:149 and ticket:150

• see how this currently works with DGU first

• write importer for publisher info (currently in JSON format), add to ckanext-ecportal
• add group translations to translations table
• run on test server

• Search needs to run of local DB (again)
• paster db clean attemps to connect to SOLR (still works as does db first but then excepts which is not nice UX)

• [x] hide black stars on the data-view page
• [x] Edit on view form (if you have the right permission)
• [x] RES_FORMAT ~> Resource Format
• [x] Map Based Search -> UK Location Map Based Search
• [x] Note at the top right expalining that these are seach filters (if they are) -> this requires "add a dataset" and "other access" to be moved to a different place (just some whitespace is fine)
• [x] Display publisher and provider (UKLP records only).
• [x] analysis of filter by OGL / non-OGL
• [x] provide /dump on the build server

1. Go to new dataset
2. Add a slug
3. Click next

Get error.

URL: http://localhost:5000/dataset/new
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/weberror/evalexception.py', line 431 in respond
  app_iter = self.application(environ, detect_start_response)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
  resp = self.call_func(req, *args, **self.kwargs)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
  return self.func(req, *args, **kwargs)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/fanstatic/publisher.py', line 234 in __call__
  return request.get_response(self.app)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
  application, catch_exc_info=False)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
  app_iter = application(self.environ, start_response)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
  resp = self.call_func(req, *args, **self.kwargs)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
  return self.func(req, *args, **kwargs)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/fanstatic/injector.py', line 52 in __call__
  response = request.get_response(self.app)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
  application, catch_exc_info=False)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
  app_iter = application(self.environ, start_response)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/beaker/middleware.py', line 73 in __call__
  return self.app(environ, start_response)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/beaker/middleware.py', line 152 in __call__
  return self.wrap_app(environ, session_start_response)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/routes/middleware.py', line 130 in __call__
  response = self.app(environ, start_response)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/pylons/wsgiapp.py', line 125 in __call__
  response = self.dispatch(controller, environ, start_response)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/pylons/wsgiapp.py', line 324 in dispatch
  return controller(environ, start_response)
File '/Users/Aron/.virtualenvs/ckan/src/ckan/ckan/lib/base.py', line 258 in __call__
  res = WSGIController.__call__(self, environ, start_response)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/pylons/controllers/core.py', line 221 in __call__
  response = self._dispatch_call()
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/pylons/controllers/core.py', line 172 in _dispatch_call
  response = self._inspect_call(func)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/pylons/controllers/core.py', line 107 in _inspect_call
  result = self._perform_call(func, args)
File '/Users/Aron/.virtualenvs/ckan/lib/python2.7/site-packages/pylons/controllers/core.py', line 60 in _perform_call
  return func(**args)
File '/Users/Aron/.virtualenvs/ckan/src/ckan/ckan/controllers/package.py', line 422 in new
  return self._save_new(context)
File '/Users/Aron/.virtualenvs/ckan/src/ckan/ckan/controllers/package.py', line 703 in _save_new
  return self.new(data_dict, errors, error_summary)
File '/Users/Aron/.virtualenvs/ckan/src/ckan/ckan/controllers/package.py', line 430 in new
  data['tag_string'] = ', '.join(h.dict_list_reduce(c.pkg_dict['tags'], 'name'))
TypeError: string indices must be integers, not str

1. Package needs a groups property (backref Group.packages).
2. Put groups in the search api.
3. Display them on the package, but not editable. (Only editable from the group page.)

Follow up to ticket:105 (Groups).

Cost: 1d

1. Remove contributors list from about page (out-of-date and does not belong there)
   • A "Powered by CKAN" link in footer is enough
2. Add contributors / team / credits page to ckan.org
   • May want to separate credits (e.g. software use), from contributors (also is team different from contributors)
   • (perhaps with photos!) so that people can see who is part of it and from which organisations. (this is medium-term, not really part of this ticket)
3. ? CREDITS.txt file to the source repository and add a link to it on bitbucket for more detailed credits info.

Cost: 1h

1. file uploads on dataset creation
2. create dataset validation
3. create dataset slugs
4. Add dataset to group process: pop-up allowing you to quickly search for a pick a dataset to add to group. Must check you have necc auth over dataset to be added validation of user name
5. Language box
6. toggling activity stream
7. Search sorting
8. autocomplete on tags
9. add and remove custom fields on forms
10. the rest: tooltip on popular datasets with number of views

facets to update automatically

creating a dataset without reloading page between steps

hover on licences information

autocomplete on search terms

group filtering

add comment to datasets

social share buttons in lightboxes

dataset counts on homepage?

Hello,

I have done a recent update to mu ckan install, and I am now getting this error for all my packages:

URL: http://ckan.emap.fgv.br/package/dengue-net
Module weberror.errormiddleware:162 in __call__
<<              __traceback_supplement__ = Supplement, self, environ
                   sr_checker = ResponseStartChecker(start_response)
                   app_iter = self.application(environ, sr_checker)
                   return self.make_catching_iter(app_iter, environ, sr_checker)
               except:
>>  app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<<                                                     self.cache_manager)
               environ[self.environ_key] = self.cache_manager
               return self.app(environ, start_response)
>>  return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<<                          headers.append(('Set-cookie', cookie))
                   return start_response(status, headers, exc_info)
               return self.wrap_app(environ, session_start_response)
          
           def _get_session(self):
>>  return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<<                  environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
              
               response = self.app(environ, start_response)
              
               # Wrapped in try as in rare cases the attribute will be gone already
>>  response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<         
               controller = self.resolve(environ, start_response)
               response = self.dispatch(controller, environ, start_response)
              
               if 'paste.testing_variables' in environ and hasattr(response,
>>  response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<<          if log_debug:
                   log.debug("Calling controller class with WSGI interface")
               return controller(environ, start_response)
          
           def load_test_env(self, environ):
>>  return controller(environ, start_response)
Module ckan.lib.base:118 in __call__
<<          # available in environ['pylons.routes_dict']   
               try:
                   return WSGIController.__call__(self, environ, start_response)
               finally:
                   model.Session.remove()
>>  return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<<                  return response(environ, self.start_response)
              
               response = self._dispatch_call()
               if not start_response_called:
                   self.start_response = start_response
>>  response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<<              req.environ['pylons.action_method'] = func
                  
                   response = self._inspect_call(func)
               else:
                   if log_debug:
>>  response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<<                        func.__name__, args)
               try:
                   result = self._perform_call(func, args)
               except HTTPException, httpe:
                   if log_debug:
>>  result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
          
           def _inspect_call(self, func):
>>  return func(**args)
Module ?:2 in read
Module ckan.lib.cache:167 in wrapper
<<      log = __import__("logging").getLogger("proxy_cache")
           def wrapper(func, *args, **kwargs):
               result = func(*args, **kwargs)
      
               pylons = get_pylons(args)
>>  result = func(*args, **kwargs)
Module ckan.controllers.package:208 in read
<<          #render the package
               PackageSaver().render_package(c.pkg)
               return render('package/read.html')
      
           def comments(self, id):
>>  return render('package/read.html')
Module ckan.lib.base:74 in render
<<     
           return cached_template(template_name, render_template, cache_key=cache_key,
                                  cache_type=cache_type, cache_expire=cache_expire)
                                  #, ns_options=('method'), method=method)
>>  cache_type=cache_type, cache_expire=cache_expire)
Module pylons.templating:249 in cached_template
<<          return content
           else:
               return render_func()
>>  return render_func()
Module ckan.lib.base:63 in render_template
<<              stream = item.filter(stream)
              
               return literal(stream.render(method=method, encoding=None))
          
           if 'Pragma' in response.headers:
>>  return literal(stream.render(method=method, encoding=None))
Module genshi.core:183 in render
<<              method = self.serializer or 'xml'
               generator = self.serialize(method=method, **kwargs)
               return encode(generator, method=method, encoding=encoding, out=out)
      
           def select(self, path, namespaces=None, variables=None):
>>  return encode(generator, method=method, encoding=encoding, out=out)
Module genshi.output:57 in encode
<<          _encode = lambda string: string
           if out is None:
               return _encode(''.join(list(iterator)))
           for chunk in iterator:
               out.write(_encode(chunk))
>>  return _encode(''.join(list(iterator)))
Module genshi.output:339 in __call__
<<          for filter_ in self.filters:
                   stream = filter_(stream)
               for kind, data, pos in stream:
                   cached = cache_get((kind, data))
                   if cached is not None:
>>  for kind, data, pos in stream:
Module genshi.output:670 in __call__
<<          _gen_prefix = _gen_prefix().next
      
               for kind, data, pos in stream:
                   output = cache_get((kind, data))
                   if output is not None:
>>  for kind, data, pos in stream:
Module genshi.output:771 in __call__
<<          push_text = textbuf.append
               pop_text = textbuf.pop
               for kind, data, pos in chain(stream, [(None, None, None)]):
      
                   if kind is TEXT:
>>  for kind, data, pos in chain(stream, [(None, None, None)]):
Module genshi.output:586 in __call__
<<      def __call__(self, stream):
               prev = (None, None, None)
               for ev in stream:
                   if prev[0] is START:
                       if ev[0] is END:
>>  for ev in stream:
Module genshi.core:288 in _ensure
<<      # unchanged
           yield event
           for event in stream:
               yield event
>>  for event in stream:
Module genshi.template.base:618 in _include
<<                      tmpl = self.loader.load(href, relative_to=event[2][0],
                                                   cls=cls or self.__class__)
                           for event in tmpl.generate(ctxt, **vars):
                               yield event
                       except TemplateNotFound:
>>  for event in tmpl.generate(ctxt, **vars):
Module genshi.template.base:618 in _include
<<                      tmpl = self.loader.load(href, relative_to=event[2][0],
                                                   cls=cls or self.__class__)
                           for event in tmpl.generate(ctxt, **vars):
                               yield event
                       except TemplateNotFound:
>>  for event in tmpl.generate(ctxt, **vars):
Module genshi.template.base:618 in _include
<<                      tmpl = self.loader.load(href, relative_to=event[2][0],
                                                   cls=cls or self.__class__)
                           for event in tmpl.generate(ctxt, **vars):
                               yield event
                       except TemplateNotFound:
>>  for event in tmpl.generate(ctxt, **vars):
Module genshi.template.base:605 in _include
<<          from genshi.template.loader import TemplateNotFound
      
               for event in stream:
                   if event[0] is INCLUDE:
                       href, cls, fallback = event[1]
>>  for event in stream:
Module genshi.template.markup:378 in _match
<<                      for event in self._match(self._flatten(template, ctxt,
                                                                  **vars),
                                                    ctxt, start=idx + 1, **vars):
                               yield event
>>  ctxt, start=idx + 1, **vars):
Module genshi.template.markup:327 in _match
<<                      break
      
               for event in stream:
      
                   # We (currently) only care about start and end events for matching
>>  for event in stream:
Module genshi.template.base:565 in _flatten
<<                  elif kind is EXPR:
                           result = _eval_expr(data, ctxt, vars)
                           if result is not None:
                               # First check for a string, otherwise the iterable test
>>  result = _eval_expr(data, ctxt, vars)
Module genshi.template.base:277 in _eval_expr
<<      if vars:
               ctxt.push(vars)
           retval = expr.evaluate(ctxt)
           if vars:
               ctxt.pop()
>>  retval = expr.evaluate(ctxt)
Module genshi.template.eval:178 in evaluate
<<          __traceback_hide__ = 'before_and_this'
               _globals = self._globals(data)
               return eval(self.code, _globals, {'__data__': data})
>>  return eval(self.code, _globals, {'__data__': data})
Module ?:16 in <Expression u"h.subnav_link(c, h.icon('comments') + _('Comments & Questions'), controller='package', action='comments', id=c.pkg.name)">
<<        </li>
             <li py:if="g.has_commenting">
                 ${h.subnav_link(c, h.icon('comments') + _('Comments &amp; Questions'), controller='package', action='comments', id=c.pkg.name)}
             </li>
             <li>${h.subnav_link(c, h.icon('page_white_stack') + _('History'), controller='package', action='history', id=c.pkg.name)}</li>
>>  ${h.subnav_link(c, h.icon('comments') + _('Comments &amp; Questions'), controller='package', action='comments', id=c.pkg.name)}
Module ckan.lib.helpers:126 in subnav_link
<<      return link_to(
               text,
               url_for(action=action, **kwargs),
               class_=('active' if c.action == action else '')
           )
>>  url_for(action=action, **kwargs),
Module routes.util:280 in url_for
<<          raise GenerationException(
                   "url_for could not generate URL. Called with args: %s %s" % \
                   (args, kargs))
           return url
>>  (args, kargs))
GenerationException: url_for could not generate URL. Called with args: () {'action': 'comments', 'controller': 'package', 'id': 'dengue-net'}
CGI Variables
AUTH_TYPE 	'cookie'
DOCUMENT_ROOT 	'/etc/apache2/htdocs'
GATEWAY_INTERFACE 	'CGI/1.1'
HTTP_ACCEPT 	'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
HTTP_ACCEPT_CHARSET 	'ISO-8859-1,utf-8;q=0.7,*;q=0.7'
HTTP_ACCEPT_ENCODING 	'gzip, deflate'
HTTP_ACCEPT_LANGUAGE 	'pt-br,pt;q=0.8,en-us;q=0.5,en;q=0.3'
HTTP_CONNECTION 	'keep-alive'
HTTP_COOKIE 	'__utma=4669863.1260802706.1295364409.1306935693.1306960483.86; __utmz=4669863.1295364409.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); SESS61faf7f7d2406929b2f9eb4cbfff17d2=a3306f5d1194cf0d8a51dbf7b42c1259; ckan=d4951177ef995d92f0f987d481bd9180aa0f89e4518921db9cd9b6ac75fdd3fab4a25d06; auth_tkt="c6c3c3e0b3fe7bb765356659992760b24da43221flavio!userid_type:unicode"; auth_tkt="c6c3c3e0b3fe7bb765356659992760b24da43221flavio!userid_type:unicode"; ckan_user="flavio"; ckan_display_name="Fl\xc3\xa1vio Code\xc3\xa7o Coelho"; ckan_apikey="c139718d-918f-4a2b-b219-e33cb05cbe23"; __utmc=4669863'
HTTP_HOST 	'ckan.emap.fgv.br'
HTTP_KEEP_ALIVE 	'115'
HTTP_REFERER 	'http://ckan.emap.fgv.br/'
HTTP_USER_AGENT 	'Mozilla/5.0 (X11; Linux x86_64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1'
PATH_INFO 	'/package/dengue-net'
PATH_TRANSLATED 	'/home/flavio/var/srvc/ckan.emap.fgv.br/pyenv/bin/ckan.emap.fgv.br.py/package/dengue-net'
REMOTE_ADDR 	'10.250.48.110'
REMOTE_PORT 	'47624'
REMOTE_USER 	u'flavio'
REMOTE_USER_DATA 	'userid_type:unicode'
REMOTE_USER_TOKENS 	['']
REQUEST_METHOD 	'GET'
REQUEST_URI 	'/package/dengue-net'
SCRIPT_FILENAME 	'/home/flavio/var/srvc/ckan.emap.fgv.br/pyenv/bin/ckan.emap.fgv.br.py'
SERVER_ADDR 	'10.252.2.60'
SERVER_ADMIN 	'[no address given]'
SERVER_NAME 	'ckan.emap.fgv.br'
SERVER_PORT 	'80'
SERVER_PROTOCOL 	'HTTP/1.1'
SERVER_SIGNATURE 	'<address>Apache/2.2.16 (Debian) Server at ckan.emap.fgv.br Port 80</address>\n'
SERVER_SOFTWARE 	'Apache/2.2.16 (Debian)'
WSGI Variables
application 	<beaker.middleware.CacheMiddleware object at 0x7f051c802690>
beaker.cache 	<beaker.cache.CacheManager object at 0x7f051c802750>
beaker.get_session 	<bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x7f051c8026d0>>
beaker.session 	{'locale': u'pt_BR', '_accessed_time': 1307358102.6266389, '_creation_time': 1302605470.0245121}
mod_wsgi.application_group 	'dck093|'
mod_wsgi.callable_object 	'application'
mod_wsgi.handler_script 	''
mod_wsgi.input_chunked 	'0'
mod_wsgi.listener_host 	''
mod_wsgi.listener_port 	'80'
mod_wsgi.process_group 	'ckan'
mod_wsgi.request_handler 	'wsgi-script'
mod_wsgi.script_reloading 	'1'
mod_wsgi.version 	(3, 3)
paste.cookies 	(<SimpleCookie: SESS61faf7f7d2406929b2f9eb4cbfff17d2='a3306f5d1194cf0d8a51dbf7b42c1259' __utma='4669863.1260802706.1295364409.1306935693.1306960483.86' __utmc='4669863' __utmz='4669863.1295364409.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' auth_tkt='c6c3c3e0b3fe7bb765356659992760b24da43221flavio!userid_type:unicode' ckan='d4951177ef995d92f0f987d481bd9180aa0f89e4518921db9cd9b6ac75fdd3fab4a25d06' ckan_apikey='c139718d-918f-4a2b-b219-e33cb05cbe23' ckan_display_name='Fl\xc3\xa1vio Code\xc3\xa7o Coelho' ckan_user='flavio'>, '__utma=4669863.1260802706.1295364409.1306935693.1306960483.86; __utmz=4669863.1295364409.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); SESS61faf7f7d2406929b2f9eb4cbfff17d2=a3306f5d1194cf0d8a51dbf7b42c1259; ckan=d4951177ef995d92f0f987d481bd9180aa0f89e4518921db9cd9b6ac75fdd3fab4a25d06; auth_tkt="c6c3c3e0b3fe7bb765356659992760b24da43221flavio!userid_type:unicode"; auth_tkt="c6c3c3e0b3fe7bb765356659992760b24da43221flavio!userid_type:unicode";... __utmc=4669863')
paste.registry 	<paste.registry.Registry object at 0x7f051d7e5710>
paste.throw_errors 	True
pylons.action_method 	<bound method PackageController.read of <ckan.controllers.package.PackageController object at 0x7f051d7e5b10>>
pylons.controller 	<ckan.controllers.package.PackageController object at 0x7f051d7e5b10>
pylons.environ_config 	{'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons 	<pylons.util.PylonsContext object at 0x7f051d7e5a50>
pylons.routes_dict 	{'action': u'read', 'controller': u'package', 'id': u'dengue-net'}
repoze.who.identity 	<repoze.who identity (hidden, dict-like) at 139659946193344>
repoze.who.logger 	<logging.Logger instance at 0x7f051c801128>
repoze.who.plugins 	{'openid': <OpenIdIdentificationPlugin 139659931643152>, 'friendlyform': <FriendlyFormPlugin 139659929725584>, 'ckan.lib.authenticator:UsernamePasswordAuthenticator': <ckan.lib.authenticator.UsernamePasswordAuthenticator object at 0x7f051c9e7610>, 'auth_tkt': <AuthTktCookiePlugin 139659931643280>, 'ckan.lib.authenticator:OpenIDAuthenticator': <ckan.lib.authenticator.OpenIDAuthenticator object at 0x7f051c9dcfd0>}
routes.route 	<routes.route.Route object at 0x7f051c69fd50>
routes.url 	<routes.util.URLGenerator object at 0x7f051d7e57d0>
webob._parsed_cookies 	({'ckan': 'd4951177ef995d92f0f987d481bd9180aa0f89e4518921db9cd9b6ac75fdd3fab4a25d06', '__utmz': '4669863.1295364409.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)', 'auth_tkt': 'c6c3c3e0b3fe7bb765356659992760b24da43221flavio!userid_type:unicode', 'ckan_display_name': 'Fl\xc3\xa1vio Code\xc3\xa7o Coelho', 'ckan_apikey': 'c139718d-918f-4a2b-b219-e33cb05cbe23', 'ckan_user': 'flavio', '__utma': '4669863.1260802706.1295364409.1306935693.1306960483.86', '__utmc': '4669863', 'SESS61faf7f7d2406929b2f9eb4cbfff17d2': 'a3306f5d1194cf0d8a51dbf7b42c1259'}, '__utma=4669863.1260802706.1295364409.1306935693.1306960483.86; __utmz=4669863.1295364409.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); SESS61faf7f7d2406929b2f9eb4cbfff17d2=a3306f5d1194cf0d8a51dbf7b42c1259; ckan=d4951177ef995d92f0f987d481bd9180aa0f89e4518921db9cd9b6ac75fdd3fab4a25d06; auth_tkt="c6c3c3e0b3fe7bb765356659992760b24da43221flavio!userid_type:unicode"< wbr>; auth_tkt="c6c3c3e0b3fe7bb765356659992760b24da43221flavio!... __utmc=4669863')
webob._parsed_query_vars 	(GET([]), '')
webob.adhoc_attrs 	{'language': 'en-us'}
wsgi process 	'Multi process AND threads (?)'
wsgi.file_wrapper 	<built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f051cc1ad50>
wsgi.version 	(1, 1)
wsgiorg.routing_args 	(<routes.util.URLGenerator object at 0x7f051d7e57d0>, {'action': u'read', 'controller': u'package', 'id': u'dengue-net'})

"Alternative formats". Has logo for RDF (link to RDF) and JSON (link via CKAN REST i/f).

• How do we know if RDF available? Ans: see details in ticket:90.

Cost: 1h

The application's Globals object is not very informative.

Proposer: John Glover
Seconder: James Gardner

Abstract

We are trying to achieve these goals:

• To get people involved with making edits to CKAN metadata.
• To have an ownership model as to who can moderate and validate these changes
• To not put too huge a burden on these owners.

This feature allows anyone to edit a package and create a new revision, but requires an owner/moderator to approve a revision before it is are made "official".

There have been a lot of discussions around the revisioning system side of this ticket (CREP 0002) and I think these are now largely resolved. We now want to discuss the user interface.

The Problem

We require the following functionality:

• Allow a group of changes to be stored as a new revision.
• Allow a linear stack of "community" revisions.
• Provide a way for the editor and moderator to compare previous revisions to the current one.
• When a moderator approves a change it creates a new revision flagged "moderated" (this is analogous to a merge commit)
• Provide a way for the editor and moderator comment on revisions if necessary.

Extra features:

• Need a way to summarise the changes (as part of the preview perhaps)
• Sysadmin needs to purge a revision completely

Specification

UI/UX

UI Mockup:

Revisions:

• Revisions are per package rather than per field.
• Internally CKAN has separate revisions for resources, extras and package metadata. From a user's point of view this could be confusing to expose, so everything that they see on a package form when they hit save is a single revision.

On the Edit page:

• We have a panel on the right, listing all the revisions with the current moderated one selected. Moderated revisions are highligted in some way (red and bold?).
• The values displayed in the form are by default populated from the latest revision (whether community or moderated)
• Under each field is a "shadow", showing the value of the field in the revision selected in the panel, if it is different from the value in the field. By default the shadow values are populated from the latest moderated revision which is the one selected in the revision panel by default too.
• When you change the value of a field, a shadow may appear or disappear accordingly. If they disappear a box saying that they are the same replaces it
• If you want to edit values from a previous revision, you first select that revision to get the shadows populated. There is a button named "Replace fields with values from this revision" under the revision list. You click this, a warning pops up and then you say "Yes". You then select the moderated revision again.
• We also allow package comments the same way as the todo extension works at the moment. Additionally, we need to be able to differentiate between what the moderator wrote and what a community member wrote, and so we may need to make a small change to the todo extension to facilitate this.
• In addition to package comments, each revision will have a revision log (analogous to a commit message).

Technical Details

• This CREP will result in a new CKAN extension.
• It depends heavily on the new revisioning system (CREP0002), some of the details of which are yet to be finalised.
• This CREP therefore requires working closely with David Raznick to come up with an API that the UI AJAX calls can use.
• We will then use suitable test data to mimic these API calls until CREP0002 is ready.

Why do it this way

This hopefully provides a clear and consistent mechanism allowing both a community member to make new revisions and a moderator to view and approve revisions, with largely the same UI/UX.

Implementation plan

Deliverables

A new CKAN extension, consisting of:

• Code: Python, HTML, CSS, Javascript
• Unit tests
• Localization
• Documentation

Participants

John Glover to do it.

Progress

John has implemented the bulk of this UI. Just some things to tidy up before it is complete:

• Genshi stream filters to be updated with CKAN 1.5 / 1.5.1 templates
• history_ajax / read_ajax to be replaced with calls to Action API (or Util REST API)

I've split these two off into a new ticket #1604.

Related Progress

The Todo extension is written and available at: ​https://bitbucket.org/johnglover/ckanext-todo.

In the section 'The Problem', under extra features, we mention a need for the sysadmin to be able to purge a revision already. This is already done.

See also

#1129 Backend work

Proposer: Seb Bacon
Seconder: Rufus Pollock

Abstract

When adding major new features to CKAN, a longer, more formal discussion will improve software design quality and documentation, better engage the wider community, and ensure the core team are up to date with latest developments.

I propose a formal process (CREP -- CKAN Revision and Enhancement Proposal) for making this happen.

The Problem

The current workflow for introducing major new features into CKAN is very informal, typically based around one person's great idea, which they've discussed with one or two other people in the team. The originator of the idea is typically the only person with access to all the input they've had through such discussions. Often, the only location of this information is in that person's head.

However, there is a lot of experience embodied in the CKAN community which should be drawn on before making large design decisions. This will lead to better software. Additionally, building consensus in the community around a proposal before implementation ensures positive community engagement and buy-in to new features, making them more likely to be a success.

We aren't great at documenting new features. Documentation after coding is complete is an unrewarding experience for most programmers. Requiring skeleton documentation before code is written is a good discipline that can form the basis of better documentation in the future (e.g. by a writer rather than a programmer).

Specification

Minor features don't require a CREP, and can just be entered in the issue tracking system as a bug or feature. As a rule of thumb, a feature is major if it will take more than a day to implement, or is likely to involve matters of opinion in its design.

A developer may decide that a CREP is too formal and long-winded. The decision to write a CREP is at at their discretion; however, new features MUST always be proposed via email, even if this is just a couple of sentences.

If a feature requires a CREP, the proposer should find a seconder for their idea. This sanity check step happens before a CREP is written to ensure at least the possibility of consensus on the CREP.

Next the proposer should write a CREP, starting by copying and pasting the ​template on the wiki into a new Trac ticket. This will be with a status of "new" and Type of "CREP". The proposer should notify the ​ckan-dev mailing list, and possibly the ​ckan-discuss list for less technical CREPs.

The draft can be discussed via email, verbally, or via the trac ticket. In any case, it is the proposer's responsibility to keep the CREP updated to reflect the current consensus.

Once consensus has been reached, the ticket should be marked with the "accepted" status and assigned to a CKAN release milestone.

When an accepted CREP has been implemented, it should be resolved as "fixed".

If no consensus can be reached on a draft CREP, or for some reason an accepted CREP doesn't get completed, it should be marked as or "wontfix".

If a completed CREP becomes obsolete, it should be marked as "invalid", with a note pointing to the obsoleting ticket(s)

Why do it this way

Given the distributed nature of the core team plus other volunteers, some kind of written procedure is necessary to ensure a fully documented and discussed proposal.

The idea of "Enhancement Proposals" which can be semi-formally proposed and discussed prior to implementation is common in the Open Source world (​PEPs, ​DEPs, ​PLIPs, to name three).

Existing historic proposals exist, called CEPs. The proposed system is called CREP (CKAN revision or enhancement proposal) to disambiguate it from the legacy proposals, and from the delicious fungus Boletus Edulis.

Giving a formal structure to the proposal is useful as it gives the community a means to identify a CREP that's not had sufficient thought or discussion. An informal email thread can easily be lost and important questions (such as backwards compatibility) overlooked. The use of the proposed template empowers any community member to ask the proposer to expand on rationale, deliverables, etc.

The structure chosen is somewhere between Debian's and Plone's. It aims to give a structure to the debate, a clear start at documentation, and also prompt some thinking about implementation and timescales.

All this policy about structure should not be construed as mandatory. In particular, the later fields in the CREP template regarding Implementation Plan may be omitted if the author doesn't find them helpful.

Some projects (e.g. Debian) keep their enhancement proposals in a versioning repository; others (e.g. Plone) keep them in an issue tracking system. Trac is proposed for CKAN because we already use it for small feature proposals and for team planning. It seems unlikely that change tracking on an individual CREP will be useful; a CREP that changes sufficiently from its original form should probably be marked "obselete" and a new CREP started. Using an issue tracking system also means we can easily track CREPs by state.

Backwards Compatibility

Some [​https://bitbucket.org/okfn/ceps/src/76b274888bcf/cep/ legacy enhancement proposals], called CEPs, have previously been started.

They are currently all marked as "active". Any which require discussion should be altered by the proposer to match the new CREP specification and submitted to trac. The original CEP should be updated with a banner at the top pointing a reader to the new CREP.

Any that are now obselete should be clearly marked as such in a banner at the top, pointing a reader to the trac for new CREPs.

Implementation plan

Deliverables

• This CREP, agreed
• Support for proposed statuses in Trac
• Canned reports for listing CREPs in Trac

Risks and mitigations

• That this CREP is agreed, but rarely acted on. This risk can be mitigated by nominating a CREP champion in the community or core team, whose job it is to say "where's the CREP for that?" and generally own the quality of CREPS

Participants

Seb Bacon: as current Documentation Czar (May 2011), responsible for ensuring CREPs are up to date.

Progress

This document is the entire proposal.

'Publisher' entities in the model. They are hierarchical.

'User-Publisher' connections with one or more roles (e.g. drafter, moderator).

Authorization settings can control who can set what values in a 'published by' type field.

Publishers and User-Publishers available to read in the API.

Future tickets will provide:

• API to write Publishers and User-Publishers
• UI to edit Publishers and User-Publishers

(This feature deprecates authorization groups)

'paster db upgrade' should result in the same database as 'paster db create' on a clean system. In particular, all tests should pass.

Note: equivalent of paster db upgrade isn't being run from tests at the moment (we haven't been able to get this to work).

(2008-09-10) at present when setup.py is edited info command may not give up to date information if pkg-info (in egg.info) is not rebuilt.

Suggest: info command needs to rebuilds pkg-info in python-type distributions.

(2009-03-09) Do not create a distribution at path X if path X already exists and contains material (unless forced via a force option).

Cost: 1h

(Follows on from ticket:107). We want to provide facility for users to automatically upload material.

• Create an upload page
• Store the material on archive.org using s3 - ​http://www.archive.org/help/abouts3.txt
• Main issue is how we cache large datasets ...

(Just creating this ticket as somewhere to keep notes)

• Decide on REST api versus action API
  • Do we want to support both?
• Tidying
  • Unify on /api/v{version num}/... structure (do we want a default option that points to current default? e.g. /api/default/ ...)
  • extras merged into normal field list in package
  • Get rid of /rest/ so just have api/v1/package
  • Get rid of separation of search api from 'rest' api
    • Propose that GET on REST index is search e.g. /package/?q=...
      • This is also resolves issue whereby GET at root returns whole package set (a *bad* idea) as this would now become the matchall search query (with a default limit on items returned)
• Resource read/write in API (separate from package)
  • Does this need authorization work?
• user/account API - read/write
• Remove autocomplete -- can just use search
  • Do not worry about backwards compat as should only be used in our js (if others using it too bad!)

(Related to ticket:169 - Package derivations)

A 'dependency' relationship can be applied from one package to another. It implies that a package requires the download or existence of another package which it 'depends on'. (Analogous to software package dependencies.)

e.g. london-traffic-visualisation depends on road-map

'Dependency' relationship is:

• directional
• many:many
• stateful

'dependency' table columns:

• id (primary key)
• dependent (foreign key)
• dependency (foreign key)

Further tickets:

• WUI - package view - have list of dependencies (do not need to list packages which depend on this one)
• WUI - package edit form - new option to say 'depends on' (no need for 'has dependent package')
• REST api - expose reading and writing 'depends on' property.

Issues

• How do we deal with dependency at a particular version?

(because if you forget, you get difficult to understand errors, and more than one person has tripped up on this)

(in the same way that we can refer to packages in the API by ID as well as name.)

As a

client of CKAN

I want to

link to a package page in the UI, referring to it by ID