{22} Trac tickets (2647 matches)

John did this a while ago.

This package:

​http://ckan.net/package/dbtune-audioscrobbler

was:

1. created by Richard (logged-in)
2. edited by Richard (logged-in)

(According to the logs, at this point the state was changed from 'active' to 'deleted') -- RP was it set to 'deleted' or just ?

Several questions to answer (and see file in email). To respond to the Finnish team and copy all Q&A to the ckan-discuss list.

1 ) I have played with two different datasets to get the metadata from those imported to CKAN. The other dataset is in PCAXIS format and other one is in Excel format. Please find the scripts + sample files attached in a zip file that contains instructions too to run the scripts. I looked at import scripts written in object oriented manner and getdata samples in CKAN but it was easier for me too write these from scratch (copied the unicrap function from CKAN samples). If I run the PCAXIS import script I get Unicode decode error (see errors file). When I use the unicrap function then the error does not appear in logs. I assume this is some Python environment setting issue. Of course I would like to run the script without unicrap function to get scandic letter imported correctly to ckan(and displayed in WUI). In Excel case there is no scandic letter issue which is kind of strange but this probably relates how the Excel library handles encoding.

To be honest the whole unicode/encoding/decoding issue has been quite difficult to deal with and debug. I hope I would get proper environment configuration to handle scandic letters.

2 ) It seem in CKAN by default the localisation file contains 462 strings. The transifex link Rufus sent contains 400 strings. Is transifex the one we should use as localisation file ? Where are the rest 62 strings then fetched from or are they needed at all. I have found that as I have been using the .po file coming in CKAN source (which has API translations too) this also causes similar unicode problems when an error in API occurs (the API error messages containing scandic letters provide the similar UnicodeDecoding? errors as mentioned above). So, it seems the unicode issues appear in various cases. Maybe a proper environment settings will be able to prevent also these problems... If transifex is the one to use I can prepare it and maybe then you could upload it to fi.ckan.net as there will be also other people checking the translations. To run the test and see the API translation errors especially in pcaxis case I would recommend to install attached FI translations to your environment and include lang = fi configuration. It looks like the API errors does not come when running default lang = en.

3 ) Is it possible to change the license selection/options available in license field drop-down ? Where are these located and can these be modified locally ?

4) It looks like the values stored in package extra fields are not searchable. Is this intentional ? It looks more or less that all other fields can be searched using free-text search in CKAN.

5) How package dependencies are defined and managed ? For instance we want to import some packages in a manner that the other package is a child package of another. Does API support this and/or is this possible to do in WUI ?

Best Regards, Henri

We don't want /api to be an HTML page, but rather provide information about the API in a JSON structure.

Common requirements for running CKAN behind a (e.g Wordpress or Drupal) front-end:

1. Unrestricted total read-only access to catalogue API for general public (e.g. voluntary organisation).
   • monitored by API key
   • not monitored by API key
2. Restricted total read-write access to catalogue API for authorized clients (e.g. front-end system, bulk upload clients).
   • restricted by CKAN access controller
   • restricted by HTTP Auth
   • restricted by IP address
3. Restricted total read-write access to catalogue Web UI for authorized users (e.g. site admins).
   • restricted by CKAN access controller
   • restricted by HTTP Auth
4. Restricted partial read-write access to catalogue Web UI for authorized users (e.g. group admins).
   • restricted by CKAN access controller
   • restricted by HTTP Auth

Investigate exception occured occasionally in last couple of days on ckan.net:

WebApp Error: <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'key' 					

URL: http://ckan.net/revision/diff/dbtune-audioscrobbler?diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c
Module weberror.errormiddleware:162 in __call__
<<              __traceback_supplement__ = Supplement, self, environ
                   sr_checker = ResponseStartChecker(start_response)
                   app_iter = self.application(environ, sr_checker)
                   return self.make_catching_iter(app_iter, environ, sr_checker)
               except:
>>  app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<<                                                     self.cache_manager)
               environ[self.environ_key] = self.cache_manager
               return self.app(environ, start_response)
>>  return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<<                          headers.append(('Set-cookie', cookie))
                   return start_response(status, headers, exc_info)
               return self.wrap_app(environ, session_start_response)
           
           def _get_session(self):
>>  return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<<                  environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
               
               response = self.app(environ, start_response)
               
               # Wrapped in try as in rare cases the attribute will be gone already
>>  response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<          
               controller = self.resolve(environ, start_response)
               response = self.dispatch(controller, environ, start_response)
               
               if 'paste.testing_variables' in environ and hasattr(response,
>>  response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<<          if log_debug:
                   log.debug("Calling controller class with WSGI interface")
               return controller(environ, start_response)
           
           def load_test_env(self, environ):
>>  return controller(environ, start_response)
Module ckan.lib.base:73 in __call__
<<          # available in environ['pylons.routes_dict']    
               try:
                   return WSGIController.__call__(self, environ, start_response)
               finally:
                   model.Session.remove()
>>  return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<<                  return response(environ, self.start_response)
               
               response = self._dispatch_call()
               if not start_response_called:
                   self.start_response = start_response
>>  response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<<              req.environ['pylons.action_method'] = func
                   
                   response = self._inspect_call(func)
               else:
                   if log_debug:
>>  response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<<                        func.__name__, args)
               try:
                   result = self._perform_call(func, args)
               except HTTPException, httpe:
                   if log_debug:
>>  result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
           
           def _inspect_call(self, func):
>>  return func(**args)
Module ckan.controllers.revision:119 in diff
<<          c.revision_to = model.Session.query(model.Revision).get(
                   request.params.getone('diff'))
               diff = pkg.diff(c.revision_to, c.revision_from)
               c.diff = diff.items()
               c.diff.sort()
>>  diff = pkg.diff(c.revision_to, c.revision_from)
Module ckan.model.package:340 in diff
<<                              display_id = to_obj_rev.tag.name
                               elif obj_class.__name__ == 'PackageExtra':
                                   display_id = to_obj_rev.key
                               else:
                                   display_id = related_obj_id[:4]
>>  display_id = to_obj_rev.key
AttributeError: 'NoneType' object has no attribute 'key'
CGI Variables
DOCUMENT_ROOT	'/htdocs'
GATEWAY_INTERFACE	'CGI/1.1'
HTTP_ACCEPT	'*/*'
HTTP_ACCEPT_ENCODING	'gzip'
HTTP_ACCEPT_LANGUAGE	'zh-cn,zh-tw'
HTTP_CONNECTION	'close'
HTTP_HOST	'ckan.net'
HTTP_USER_AGENT	'Baiduspider+(+http://www.baidu.com/search/spider.htm)'
PATH	'/usr/local/bin:/usr/bin:/bin'
PATH_INFO	'/revision/diff/dbtune-audioscrobbler'
PATH_TRANSLATED	'/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/revision/diff/dbtune-audioscrobbler'
QUERY_STRING	'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c'
REMOTE_ADDR	'123.125.66.32'
REMOTE_PORT	'63767'
REQUEST_METHOD	'GET'
REQUEST_URI	'/revision/diff/dbtune-audioscrobbler?diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c'
SCRIPT_FILENAME	'/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py'
SCRIPT_URI	'http://ckan.net/revision/diff/dbtune-audioscrobbler'
SCRIPT_URL	'/revision/diff/dbtune-audioscrobbler'
SERVER_ADDR	'10.226.226.118'
SERVER_ADMIN	'[no address given]'
SERVER_NAME	'ckan.net'
SERVER_PORT	'80'
SERVER_PROTOCOL	'HTTP/1.1'
SERVER_SIGNATURE	'<address>Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2 Server at ckan.net Port 80</address>\n'
SERVER_SOFTWARE	'Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2'
WSGI Variables
application	<beaker.middleware.CacheMiddleware object at 0x9f603ec>
beaker.cache	<beaker.cache.CacheManager object at 0x9f6042c>
beaker.get_session	<bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x9f602ac>>
beaker.session	{'_accessed_time': 1282385101.4243281, '_creation_time': 1282385101.4243281}
mod_wsgi.application_group	'ckan.net|'
mod_wsgi.callable_object	'application'
mod_wsgi.listener_host	''
mod_wsgi.listener_port	'80'
mod_wsgi.process_group	''
mod_wsgi.reload_mechanism	'0'
mod_wsgi.script_reloading	'1'
mod_wsgi.version	(2, 5)
paste.cookies	(<SimpleCookie: >, '')
paste.parsed_querystring	([('diff', '66a47b9e-232a-4838-8674-66fa1a5c76e1'), ('oldid', 'a99c98be-767a-4e49-9025-2472b2d18b9c')], 'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c')
paste.registry	<paste.registry.Registry object at 0x104552ec>
paste.throw_errors	True
pylons.action_method	<bound method RevisionController.diff of <ckan.controllers.revision.RevisionController object at 0xfb17aec>>
pylons.controller	<ckan.controllers.revision.RevisionController object at 0xfb17aec>
pylons.environ_config	{'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons	<pylons.util.PylonsContext object at 0x10286d4c>
pylons.routes_dict	{'action': u'diff', 'controller': u'revision', 'id': u'dbtune-audioscrobbler'}
repoze.who.logger	<logging.Logger instance at 0xa16e0cc>
repoze.who.plugins	{'openid': <OpenIdIdentificationPlugin 167584972>, 'auth_tkt': <AuthTktCookiePlugin 169253516>}
routes.route	<routes.route.Route object at 0x9f3690c>
routes.url	<routes.util.URLGenerator object at 0xfd8d7cc>
webob._parsed_query_vars	(GET([('diff', '66a47b9e-232a-4838-8674-66fa1a5c76e1'), ('oldid', 'a99c98be-767a-4e49-9025-2472b2d18b9c')]), 'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c')
webob.adhoc_attrs	{'language': 'en-us'}
wsgi process	'Multi process AND threads (?)'
wsgi.file_wrapper	<built-in method file_wrapper of mod_wsgi.Adapter object at 0x103a5bf0>
wsgiorg.routing_args	(<routes.util.URLGenerator object at 0xfd8d7cc>, {'action': u'diff', 'controller': u'revision', 'id': u'dbtune-audioscrobbler'})

Occasionally data4nr removes a data package from its registry. When the latest full data4nr data is imported, these removed packages should be found and removed.

No need for requirement - closing in favour of #505

• Create a branch named ultrastable that represents latest version of 1.0 as found on dgu
  • Going forward ultrastable will act as stable - 1 (and either ultrastable or stable will be on the rock-solid production systems such as dgu)
• Do not deprecate the stable or metastable branch for the present
• Ask for recommendations on mailing list about best practice for branching (e.g. versioned based versus a simple HEAD versus stable) and how this affects standard operations such as applying bugfixes.
  • For useful discussion of how to do backporting and applying bugfix patches see: ​http://www.python.org/dev/peps/pep-0374/#backport
  • NB: in general do not always have to apply in stable/ultrastable and merge upwards for bugfixes -- can use transplant extension as well.

When you create a package then the 'location' header gets set. This doesn't happen for any other domain objects. I think this should be consistent - either none or all.

I've removed the info about the header in the docs in the meantime.

At: ​http://ckan.net/revision you can see: "Next »" on pager This doesn't seem to be a problem with earlier (1.0.1) version.

• Create custom form

• e.g. find/browse by country and by publishing entity and by donor

• Publishing entity emails system editors
• System editor creates publishing entity account and emails publishing entity with details
• Priority: 4/5

Limit registering data to authenticated users either via API or manually.

• Edits may only be made by authenticated entities
• Priority: 5

Assumes 8 is done:

8. Tool for validation an IATI XML record (referenced by a URL) for compliance with the IATI specification

• Outside of scope of registry
• Multiple validators with different levels of rigour

From lopad:

Publishing Entity (often/always? equal to the logged in user)
    * Domain logic: only can add entries for "yourself"
Donor
Country
Activity period: period to which this record relates
Verification status: enumeration of statuses (checked, not checked etc)
Resource links: to the actual IATI record
Number of activities: ...
Date record updated:
Date data updated: 
License: Need this field even if it may be a standard license

Configure buildbot to use postgres db which has been moved to eu5.okfn.org

pudo said: Could you perhaps also comment on ​http://knowledgeforge.net/okfn/tasks/ticket/466 with a hint on how to update this properly? I only edited /home/buildslave/okfn/full/build/buildandsmoke/buildandsmoke.ini which was the only occurence of the DB credentials I could find there.

Multiple ONS records can go into one package e.g. Stats for Jan, Stats for Feb, Stats for Mar etc.

We already identify an existing package by 'external_id'. Now need to find package by title and an extra field (department in ONS case) and allow it to merge in resource and tags. Alert if rest of meta data is different.

Update report queries with more recent versions (below are report queries shipped with trac 0.11.7):

NB: Report 1 has already been done.

id | query

1 |

: SELECT p.value AS color, : id AS ticket, summary, component, version, milestone, t.type AS type, : owner, status, : time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status <> 'closed' : ORDER BY CAST(p.value AS integer), milestone, t.type, time :

2 |

: SELECT p.value AS color, : version AS group, : id AS ticket, summary, component, version, t.type AS type, : owner, status, : time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status <> 'closed' : ORDER BY (version IS NULL),version, CAST(p.value AS integer), t.type, time :

3 |

: SELECT p.value AS color,

: 'Milestone '

milestone AS group,

: id AS ticket, summary, component, version, t.type AS type, : owner, status, : time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status <> 'closed' : ORDER BY (milestone IS NULL),milestone, CAST(p.value AS integer), t.type, time :

4 |

: : SELECT p.value AS color, : owner AS group, : id AS ticket, summary, component, milestone, t.type AS type, time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status = 'accepted' : ORDER BY owner, CAST(p.value AS integer), t.type, time :

5 |

: SELECT p.value AS color, : owner AS group, : id AS ticket, summary, component, milestone, t.type AS type, time AS created, : description AS _description_, : changetime AS _changetime, reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status = 'accepted' : ORDER BY owner, CAST(p.value AS integer), t.type, time :

6 |

: SELECT p.value AS color, : t.milestone AS group, : (CASE status : WHEN 'closed' THEN 'color: #777; background: #ddd; border-color: #ccc;' : ELSE : (CASE owner WHEN $USER THEN 'font-weight: bold' END) : END) AS style, : id AS ticket, summary, component, status, : resolution,version, t.type AS type, priority, owner, : changetime AS modified, : time AS _time,reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : ORDER BY (milestone IS NULL), milestone DESC, (status = 'closed'), : (CASE status WHEN 'closed' THEN changetime ELSE (-1) * CAST(p.value AS integer) END) DESC :

7 |

: SELECT p.value AS color, : (CASE status WHEN 'accepted' THEN 'Accepted' ELSE 'Owned' END) AS group, : id AS ticket, summary, component, version, milestone, : t.type AS type, priority, time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE t.status <> 'closed' AND owner = $USER : ORDER BY (status = 'accepted') DESC, CAST(p.value AS integer), milestone, t.type, time :

8 |

: SELECT p.value AS color, : (CASE owner : WHEN $USER THEN 'My Tickets' : ELSE 'Active Tickets' : END) AS group, : id AS ticket, summary, component, version, milestone, t.type AS type, : owner, status, : time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status <> 'closed' : ORDER BY (COALESCE(owner, ) = $USER) DESC, CAST(p.value AS integer), milestone, t.type, time

To allow for full page caching even on customized pages for logged in users we can use cookie+javascript techniques as described here: ​http://www.ibm.com/developerworks/web/library/wa-rails2/

In the first place we just need to apply this to:

• You are logged in as
• Package/group edit/authz tabs

This is in IATI, would be nice to have in generic CKAN.

• ​http://bitbucket.org/okfn/iati/changeset/378431974c76