{22} Trac tickets (2647 matches)

Results (1001 - 1100 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#866 enhancement memespring memespring ckan-v1.3-sprint-1 closed fixed Include download formats in search results

as above

1291812213000000 1292240191000000
#867 enhancement dread dread ckan-v1.3-sprint-1 closed fixed ckanclient raises exceptions

To be more pythonic, raise exceptions when ckanclient gets status which isn't 200.

1291915162000000 1299866685000000
#869 enhancement rgrp rgrp ckan-v1.3-sprint-1 closed fixed Add plugin interface to allow editing and extending of config by extensions

Use case: extensions need to add to template and public search paths if they wish to have templates or extra static files (such as js etc).

Allowing plugins to edit the config dictionary on load would allow this (and more!).

Are there any security considerations here? (Is is bad to allow extensions to make arbitrary changes to the config information?)

1291989395000000 1292235262000000
#870 defect memespring ckan-v1.3-sprint-1 closed fixed Menu items not selecting correctly

Since the main menu items have changed (part of the UI redesign) the selected items are not functioning correctly.

Specifically if Home / About or Add Package / Search are selected both menu items are highlighted.

This is because the code assumes only one top-level menu item per controller.

1292243583000000 1294914243000000
#231 defect pudo rgrp ckan-v1.3 closed fixed Make group versioned

Make group object revisioned/versioned.

  • Involves upgrade script both for group object and group_revision object
  • No changes required to WUI for present
1262853923000000 1288516929000000
#346 defect dread johnbywater ckan-v1.3 closed wontfix Revision search API (response data format and documentation issue)

Whilst going through the API docs for the revision search API, it was noticed that the "Gdu" SoS doc doesn't match up. It returns revision IDs (perhaps this is useful to note in the spec?) so the format is probably not 'limitedstring'. Also, they appear to be ordered youngest first, not oldest as stated.

And in the revision model, it refers to 'simplestring' which it doesn't define - I guess the names should be 'limitedstring'?

Could this be checked out?

1276523517000000 1296477510000000
#364 defect dread ckan-v1.3 closed fixed Search for 'statistic' returns nothing

On ckan.net there are plenty of packages (and indeed their tags) with the word 'statistic' in them, but no packages turn up when you search for it:

http://ckan.net/package/search?q=statistic&search=Search+Packages+%C2%BB

(Using Postgres full text search)

1278949620000000 1291637291000000
#395 task pudo ckan-v1.3 closed duplicate Set up profiling to analyze performance issues

At the moment, some pages within CKAN tend to load slowly. We should create a profiling setup in which we can measure response times for complete requests and individual methods calls.

This could be used to identify bottlenecks and find an appropriate caching or tuning strategy to improve CKAN performance.

NB: We should also agree on a maximum request latency.

TODO: Read up on all those QoS tickets to avoid overlapping efforts.

1280824739000000 1294417538000000
#398 task johnbywater johnbywater ckan-v1.3 closed fixed Create Public API documentation for DGU (draft) 1281000828000000 1294407372000000
#402 task pudo pudo ckan-v1.3 closed duplicate Archiving worker to back up package resources from a CKAN instance

Write a worker that scans all packages in a ckan instance and uploads the data to storage.ckan.net or another suitable storage system.

  • Naming scheme?
    • Bucket: {ckan-instance-id}-{package-name}? {ckan-instance-id}-{package-id}?
      • What happens if names change
    • File: filename? hash?
  • Store hash back on ckan instance?

The caching worker will consumer update notifications and fetch packages.

Extra points for:

  • Properly checking for source file modification (Last-modified, Etag)
  • Using PIP VCS Backends for retrieval
  • OFS/S3 Storage
1281018912000000 1296467635000000
#413 task johnbywater johnbywater ckan-v1.3 closed invalid Write functional tests for DGU caching 1281431633000000 1294407458000000
#422 story dread ckan-v1.3 closed fixed Administrator imports meta-data into CKAN

Requirements:

  • cater naturally for multiple packages with multiple resources
  • international encoding
  • large number of packages
1281433490000000 1292586586000000
#438 requirement dread dread ckan-v1.3 closed invalid Data importers use minimal processing

e.g. ONS importer

1282224383000000 1294661079000000
#441 requirement dread dread ckan-v1.3 closed duplicate CKAN read-only state

When performing maintenance on CKAN it may be necessary to make CKAN obviously read-only, telling the users and restricting access to 'edit' pages.

Examples of use:

  • Administrator wants to upgrade CKAN or move it to another server. During this time the database is being administered and either edits are lost or can't be done.
  • A CKAN is used just for distributing metadata and so is always read-only. Updates may still arrive through direct db manipulation, e.g.:
    • another (but writable) CKAN instance is connected to the same db
    • restoring database dumps from another CKAN db
  • Should a security be breached, all editing could be stopped
1282227314000000 1292586309000000
#442 requirement dread dread ckan-v1.3 closed worksforme Importing data

Review what chunks of code we have and how to move forward with requirements.

Requirements:

  • Bulk parsing is done away from the CKAN server

Possible future requirements:

  • /importer for importing spreadsheets in WUI
  • validation-light - 'scoring' a package

Current importers:

Modules

  • pkg_dict --form--> fs & validation
  • fs --PackageSaver?--> package
1282228977000000 1294661028000000
#451 task dread dread ckan-v1.3 closed invalid Show/Allow only departments associated with user

Use drupal api

1282299641000000 1292587233000000
#463 defect dread ckan-v1.3 closed worksforme package diff exception

Investigate exception:

URL: http://ckan.net/revision/diff/dbtune-audioscrobbler?diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c
Module weberror.errormiddleware:162 in __call__
<<              __traceback_supplement__ = Supplement, self, environ
                   sr_checker = ResponseStartChecker(start_response)
                   app_iter = self.application(environ, sr_checker)
                   return self.make_catching_iter(app_iter, environ, sr_checker)
               except:
>>  app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<<                                                     self.cache_manager)
               environ[self.environ_key] = self.cache_manager
               return self.app(environ, start_response)
>>  return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<<                          headers.append(('Set-cookie', cookie))
                   return start_response(status, headers, exc_info)
               return self.wrap_app(environ, session_start_response)
           
           def _get_session(self):
>>  return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<<                  environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
               
               response = self.app(environ, start_response)
               
               # Wrapped in try as in rare cases the attribute will be gone already
>>  response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<          
               controller = self.resolve(environ, start_response)
               response = self.dispatch(controller, environ, start_response)
               
               if 'paste.testing_variables' in environ and hasattr(response,
>>  response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<<          if log_debug:
                   log.debug("Calling controller class with WSGI interface")
               return controller(environ, start_response)
           
           def load_test_env(self, environ):
>>  return controller(environ, start_response)
Module ckan.lib.base:73 in __call__
<<          # available in environ['pylons.routes_dict']    
               try:
                   return WSGIController.__call__(self, environ, start_response)
               finally:
                   model.Session.remove()
>>  return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<<                  return response(environ, self.start_response)
               
               response = self._dispatch_call()
               if not start_response_called:
                   self.start_response = start_response
>>  response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<<              req.environ['pylons.action_method'] = func
                   
                   response = self._inspect_call(func)
               else:
                   if log_debug:
>>  response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<<                        func.__name__, args)
               try:
                   result = self._perform_call(func, args)
               except HTTPException, httpe:
                   if log_debug:
>>  result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
           
           def _inspect_call(self, func):
>>  return func(**args)
Module ckan.controllers.revision:119 in diff
<<          c.revision_to = model.Session.query(model.Revision).get(
                   request.params.getone('diff'))
               diff = pkg.diff(c.revision_to, c.revision_from)
               c.diff = diff.items()
               c.diff.sort()
>>  diff = pkg.diff(c.revision_to, c.revision_from)
Module ckan.model.package:340 in diff
<<                              display_id = to_obj_rev.tag.name
                               elif obj_class.__name__ == 'PackageExtra':
                                   display_id = to_obj_rev.key
                               else:
                                   display_id = related_obj_id[:4]
>>  display_id = to_obj_rev.key
AttributeError: 'NoneType' object has no attribute 'key'
CGI Variables
AUTH_TYPE	'cookie'
DOCUMENT_ROOT	'/htdocs'
GATEWAY_INTERFACE	'CGI/1.1'
HTTP_ACCEPT	'application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'
HTTP_ACCEPT_CHARSET	'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
HTTP_ACCEPT_ENCODING	'gzip'
HTTP_ACCEPT_LANGUAGE	'en-US,en;q=0.8'
HTTP_CACHE_CONTROL	'max-age=0'
HTTP_COOKIE	'__utmz=27730403.1281000628.1.6.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=ckan; auth_tkt="e919c7bb40f890a0184ed7be95465e1a4c6d130bhttp://pudo.org/!"; auth_tkt="e919c7bb40f890a0184ed7be95465e1a4c6d130bhttp://pudo.org/!"; __utma=27730403.1381387877.1281962773.1282248244.1282250173.11; __utmc=27730403; __utmb=27730403.6.10.1282250173'
HTTP_HOST	'ckan.net'
HTTP_REFERER	'http://ckan.net/package/history/dbtune-audioscrobbler'
HTTP_USER_AGENT	'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.6 (KHTML, like Gecko) Chrome/6.0.490.1 Safari/534.6'
PATH	'/usr/local/bin:/usr/bin:/bin'
PATH_INFO	'/revision/diff/dbtune-audioscrobbler'
PATH_TRANSLATED	'/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/revision/diff/dbtune-audioscrobbler'
QUERY_STRING	'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c'
REMOTE_ADDR	'92.226.62.168'
REMOTE_PORT	'55102'
REMOTE_USER	'http://pudo.org/'
REMOTE_USER_TOKENS	['']
REQUEST_METHOD	'GET'
REQUEST_URI	'/revision/diff/dbtune-audioscrobbler?diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c'
SCRIPT_FILENAME	'/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py'
SCRIPT_URI	'http://ckan.net/revision/diff/dbtune-audioscrobbler'
SCRIPT_URL	'/revision/diff/dbtune-audioscrobbler'
SERVER_ADDR	'10.226.226.118'
SERVER_ADMIN	'[no address given]'
SERVER_NAME	'ckan.net'
SERVER_PORT	'80'
SERVER_PROTOCOL	'HTTP/1.1'
SERVER_SIGNATURE	'<address>Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2 Server at ckan.net Port 80</address>\n'
SERVER_SOFTWARE	'Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2'
WSGI Variables
application	<beaker.middleware.CacheMiddleware object at 0xbb528cc>
beaker.cache	<beaker.cache.CacheManager object at 0xbb528ec>
beaker.get_session	<bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0xbb5286c>>
beaker.session	{'_accessed_time': 1282252047.603267, '_creation_time': 1282252047.603267}
mod_wsgi.application_group	'ckan.net|'
mod_wsgi.callable_object	'application'
mod_wsgi.listener_host	''
mod_wsgi.listener_port	'80'
mod_wsgi.process_group	''
mod_wsgi.reload_mechanism	'0'
mod_wsgi.script_reloading	'1'
mod_wsgi.version	(2, 5)
paste.cookies	(<SimpleCookie: __utma='27730403.1381387877.1281962773.1282248244.1282250173.11' __utmb='27730403.6.10.1282250173' __utmc='27730403' __utmz='27730403.1281000628.1.6.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=ckan' auth_tkt='e919c7bb40f890a0184ed7be95465e1a4c6d130bhttp://pudo.org/!'>, '__utmz=27730403.1281000628.1.6.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=ckan; auth_tkt="e919c7bb40f890a0184ed7be95465e1a4c6d130bhttp://pudo.org/!"; auth_tkt="e919c7bb40f890a0184ed7be95465e1a4c6d130bhttp://pudo.org/!"; __utma=27730403.1381387877.1281962773.1282248244.1282250173.11; __utmc=27730403; __utmb=27730403.6.10.1282250173')
paste.parsed_querystring	([('diff', '66a47b9e-232a-4838-8674-66fa1a5c76e1'), ('oldid', 'a99c98be-767a-4e49-9025-2472b2d18b9c')], 'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c')
paste.registry	<paste.registry.Registry object at 0xf702aec>
paste.throw_errors	True
pylons.action_method	<bound method RevisionController.diff of <ckan.controllers.revision.RevisionController object at 0xf6eaf0c>>
pylons.controller	<ckan.controllers.revision.RevisionController object at 0xf6eaf0c>
pylons.environ_config	{'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons	<pylons.util.PylonsContext object at 0xf6eaa8c>
pylons.routes_dict	{'action': u'diff', 'controller': u'revision', 'id': u'dbtune-audioscrobbler'}
repoze.who.identity	<repoze.who identity (hidden, dict-like) at 280815340>
repoze.who.logger	<logging.Logger instance at 0xbcdfccc>
repoze.who.plugins	{'openid': <OpenIdIdentificationPlugin 196612236>, 'auth_tkt': <AuthTktCookiePlugin 198048140>}
routes.route	<routes.route.Route object at 0xbbc44cc>
routes.url	<routes.util.URLGenerator object at 0xf5e55ec>
webob._parsed_query_vars	(GET([('diff', '66a47b9e-232a-4838-8674-66fa1a5c76e1'), ('oldid', 'a99c98be-767a-4e49-9025-2472b2d18b9c')]), 'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c')
webob.adhoc_attrs	{'language': 'en-us'}
wsgi process	'Multi process AND threads (?)'
wsgi.file_wrapper	<built-in method file_wrapper of mod_wsgi.Adapter object at 0xfe0b0b0>
wsgiorg.routing_args	(<routes.util.URLGenerator object at 0xf5e55ec>, {'action': u'diff', 'controller': u'revision', 'id': u'dbtune-audioscrobbler'})
1282306037000000 1294916148000000
#465 requirement johnbywater ckan-v1.3 closed wontfix Require API key for all API requests. 1282308904000000 1294411534000000
#466 requirement dread johnbywater ckan-v1.3 closed wontfix Name of HTTP header for API key shall be configurable 1282308995000000 1294835610000000
#467 story johnbywater ckan-v1.3 closed duplicate Admin configures CKAN to expect API key in named HTTP header 1282310562000000 1294411681000000
#470 task dread johnbywater ckan-v1.3 closed fixed Adjust API key's HTTP header name on DGU "live" and "test" 1282310854000000 1292587187000000
#476 requirement johnbywater ckan-v1.3 closed wontfix Daily database dumps shall be discoverable and downloadable 1282313433000000 1294411741000000
#477 story johnbywater ckan-v1.3 closed duplicate Discover location of the daily database dumps 1282313788000000 1294411761000000
#481 story johnbywater ckan-v1.3 closed fixed Unrestricted total read-only access to catalogue API 1282422697000000 1294248359000000
#486 requirement johnbywater ckan-v1.3 closed duplicate Catalogue service shall notify and query SOLR service 1282425790000000 1291639321000000
#502 requirement dread rgrp ckan-v1.3 closed wontfix Data4nr uploading script shall remove deleted packages

Occasionally data4nr removes a data package from its registry. When the latest full data4nr data is imported, these removed packages should be found and removed.

1282659304000000 1292586466000000
#504 requirement dread rgrp ckan-v1.3 closed invalid The DGU fieldset shall conform to v3 of DGU metadata specification

No need for requirement - closing in favour of #505

1282660246000000 1292586686000000
#511 requirement dread ckan-v1.3 closed worksforme Better warnings and errors when using API 1282754677000000 1297075354000000
#533 requirement johnbywater ckan-v1.3 closed invalid The performance of CKAN shall scale

CKAN uses a database. We can put the database on another machine and then clone the CKAN machine. Then there are two machines calling the database and converting SQL results into HTTP responses.

How many machines can you have before the database becomes the bottleneck? At that point, what QoS can be obtained with normal hardware?

Afterwards, can CKAN be made to scale further than this with a little development? Can its Postgres database be mirrored in realtime? Or can we change SQL writes (create/updates) to write to many databases?

Are there any other options for scaling the persistence mechanism?

1282910499000000 1292957374000000
#544 requirement pudo ckan-v1.3 closed duplicate Backport facet browsing to CKAN 1.2

This is in IATI, would be nice to have in generic CKAN.

1283267292000000 1291638966000000
#557 defect pudo ckan-v1.3 closed invalid Remove siblings from link_to/from relationship display

Displaying siblings is not needed for link_to/from.

1283542595000000 1297075159000000
#564 requirement johnbywater ckan-v1.3 closed wontfix The system shall support location based search 1284033829000000 1294412714000000
#565 requirement thejimmyg johnbywater ckan-v1.3 closed invalid The system shall support UKLP code lists 1284033864000000 1294417752000000
#566 requirement johnbywater johnbywater ckan-v1.3 closed invalid The system shall support harvesting remote metadata 1284039619000000 1294232284000000
#568 story johnbywater johnbywater ckan-v1.3 closed Pull metadata documents from given harvest source entity 1284040131000000 1288038207000000
#570 story johnbywater johnbywater ckan-v1.3 closed duplicate Validate metadata document against UKLP schematron 1284040256000000 1294407974000000
#571 story johnbywater johnbywater ckan-v1.3 closed fixed Save metadata document and associate with harvest source entity 1284040495000000 1288038218000000
#572 story johnbywater johnbywater ckan-v1.3 closed Write CKAN package from metadata document

The attributes we need to read in are being defined here: http://okfnpad.org/uklii

The design advice is to prepare, test, and maintain a dictionary of XPaths for each of the attributes.

Work effort is to be directed to the maintainability of the XPath statements. We need to know which are needed for which documents, and to have a way of preventing cruft.

1284040784000000 1288038237000000
#576 defect sebbacon wwaites ckan-v1.3 closed fixed CKAN Requires Old Version of SQLAlchemy

Requires 0.4. 0.5 is a maintenance branch, 0.6 is current.

VDM appears to work correctly (all the tests pass) with 0.5. CKAN does not -- requires more investigation to determine exactly why.

It would actually be nice to be able to run with 0.6 or 0.5, though 0.6 will require some changes to VDM as well

1284141573000000 1294753848000000
#611 task johnbywater johnbywater ckan-v1.3 closed invalid Load local copy of XML schema for metadata document 1284218639000000 1294408239000000
#612 task johnbywater johnbywater ckan-v1.3 closed duplicate Check given XML schema validates given metadata document 1284218750000000 1294408188000000
#617 task johnbywater johnbywater ckan-v1.3 closed duplicate Check UKLP schematron validates given metadata document 1284219298000000 1294408164000000
#621 task johnbywater johnbywater ckan-v1.3 closed fixed Metadata document links extracted from WAF response 1284220707000000 1288038326000000
#647 requirement pudo pudo ckan-v1.3 closed fixed Authorization Groups for group-based access control

allow group-level authorization on all system entities

1284889418000000 1287391981000000
#648 requirement pudo ckan-v1.3 closed fixed Enable lock-down of package creation
  • copy exisiting tests, modify authz in setUp, adapt and extend tests
    • Problem: default_role_actions is read by init_db
    • Solution: nuke db after monkey-patching
    • role = model.Role('Reader'), del role.actions[...]

self.PRE_MAUTHZ_RULES = copy(mauthz.default_role_actions) mauthz.default_role_actions.remove((Role.READER, Action.CREATE)) #raise Exception(mauthz.default_role_actions) model.Session.remove() model.repo.rebuild_db()

  1. Start from the functional

is_authorized(user, Action.Create, model.Package)

-> Doing this will put 'Package' in the context field of the user_object_role table. This will trigger SQLAlchemy to attempt a join towards PackageRole? in all queries. Since for class-level role assignments there never is a PackageRole? join table entry, this will never return any results.

  • have a ckan install that would not allow visitors to either list packages or list groups
    • two paths: create the listing, but for each group/pkg decide that you cannot show this
    • lock down the whole page (/package/list)
      • this is class-based, not object-based

is_authorized(user, Action.Package_Create, model.System()) is_authorized(user, Action.Group_Create, model.System())

[Separating Package and Group roles may be useful going forward: PackageEditor?, GroupEditor? etc]

  1. Find a standard way to lock down classes
    • possibly add default rows in user_object_role
    • introduce lock-down into controllers: group new, package new, REST equivalents,
    • confirm tests
1284889519000000 1284889656000000
#661 task dread ckan-v1.3 closed fixed Create CKAN site tester

From Rufus:

'blackbox' testing command (read-only only would be fine) -- very
useful for upgrade on sunday

On blackbox thinking of something like:

paster test-site http://hmg.ckan.net {package-name}

Or (maybe better):

paster test-site --site-spec {myconfig.json}

my-config is a json file with info for testing (could overload
--config rather than site-spec):

site-url: http://hmg.ckan.net/
package-name:
package-title:
...etc ...

Then:

Test front-page
Test /revision/
Test /package/{page}  -- check title
Test /package/history
Test /api/rest/package/ ...
Test /api/search/package ...
etc
1285348558000000 1288271266000000
#665 requirement johnbywater johnbywater ckan-v1.3 closed duplicate The system shall support withdrawing a harvested dataset or service from publication

Discussion between John and Peter:

Given we can identify a document, does the disappearance of a document from a registered source imply the disappearance of the metadata (such that we delete packages once the documents disappear from the registered source)?

I would expect a more explicit 'delete'. The UKLP Use Case Library describes this as "withdraw a dataset or service from publication" (part of UCD03 Maintain resources).

1285588250000000 1297268097000000
#667 defect thejimmyg dread ckan-v1.3 closed wontfix API is slow

See message and script: http://lists.okfn.org/pipermail/ckan-discuss/2010-September/000569.html

1500 requests shouldn't take hours.

1285885228000000 1297069121000000
#675 task johnbywater johnbywater ckan-v1.3 closed fixed Draft specification of CSW harvesting behaviour with guide for publishing agents 1286214877000000 1288038316000000
#676 requirement thejimmyg johnbywater ckan-v1.3 closed wontfix The system shall support browsing to correct version of the CKAN documentation for a given service 1286276136000000 1295869107000000
#677 task dread dread ckan-v1.3 closed fixed License change script
  • Uses API to change licenses of all packages in a CKAN instance to a particular one.
  • Run it on DGU.
1286384756000000 1292587315000000
#679 task thejimmyg dread ckan-v1.3 closed fixed Improve standard package edit form 1286386050000000 1294166120000000
#681 task dread dread ckan-v1.3 closed fixed Department field bug

https://trac.dataco.coi.gov.uk/projects/datagov/ticket/613

1286386685000000 1290507180000000
#689 task johnbywater johnbywater ckan-v1.3 closed fixed Updated package for updated metadata document 1286789134000000 1288038308000000
#692 requirement johnbywater johnbywater ckan-v1.3 closed invalid The system shall support presenting map thumbnail for bounding box.

To do this for the UKLP, the Ordnance Survey would need to add to their map service an operation to return image for a given bounding box, and COI would need to present the image on the search result page.

Otherwise, it would be possible to use Google.

1286823049000000 1294247841000000
#711 requirement thejimmyg johnbywater ckan-v1.3 closed fixed Packages shall support both UK government and location information with a common set of attributes

Previously, a package type attribute was intended to distinguish between package data that came from government depts. and publishers location information (GEMINI). Now there is an intention use a common set of attributes to support data from either source.

Hence, there currently a v.3 of the government attributes, a document about extracting the attributes from Gemini document using XPath, and a comparison documents mapping between governemtn attributes and Gemini attributes.

That means there are potentially four different list of attributes. I'm not sure how to go about reviewing the lists and converging any differences.

But we do want to end up with a list of names, a description of their purpose, and then perhaps the XPath which would reveal the value from a Gemini doc?

1287580866000000 1296592589000000
#712 story johnbywater johnbywater ckan-v1.3 closed invalid Model government data package with "v0.4" of the attributes

Where said "v0.4" would be the result of converging four lists mentioned in requirement #711.

1287581039000000 1294408364000000
#713 task johnbywater johnbywater ckan-v1.3 closed fixed Review four definitions of DGU package attributes 1287581158000000 1288260915000000
#731 bug dread dread ckan-v1.3 closed worksforme Geo coverage field losses in Form API

Sometimes editing a package via api on dgu results in some countries being lost in geo coverage.

1287738539000000 1288038226000000
#732 task dread dread ckan-v1.3 closed fixed Reproduce and investigate (geo coverage issue) 1287738761000000 1288038335000000
#733 task dread dread ckan-v1.3 closed invalid Fix problem (geo coverage issue) 1287738764000000 1288038341000000
#734 task dread dread ckan-v1.3 closed fixed Investigate department issue (ONS import)

Creation of duplicate packages named with trailing underscore

ww 22/9/10:

Need to ... merge some packages (the "find packages by attributes" is not 100% reliable because of inconsistent data already in there).

The "find by tags" algorithm uses (title, department). Unfortunately some of the data already there has no department. This causes it to make a new package (with trailing underscore).

dread 24/9/10:

I've done a test for the missing department issue and I can't reproduce the problem. This may be due to my general tidy up and getting tests passing using the latest ckanclient. Anyway, can you check over my test to see if this is what you mean?: dgu/ckanext/dgu/tests/ons/test_ons_loader.py:TestOnsLoadMissingDept

ww 20/10/10:

you should uncover it if you run against (a copy of) the dgu database.

1287747273000000 1292587603000000
#739 requirement thejimmyg johnbywater ckan-v1.3 closed fixed The system shall support retrieving a copy of the harvested metadata for a given package

...obviously, not all packages will have a harvested metadata document.

1287779710000000 1296592842000000
#742 task dread dread ckan-v1.3 closed fixed Release ckan 1.2 to metastable

Need to make sure #700 is fixed.

1288002397000000 1289824163000000
#745 requirement johnbywater ckan-v1.3 closed wontfix Web UI shall accept requests for harvesting jobs 1288013148000000 1294412820000000
#752 task johnbywater johnbywater ckan-v1.3 closed duplicate Change package attribute names used by Gemini harvesting to DGU "v.4" 1288039205000000 1294408472000000
#753 requirement johnbywater johnbywater ckan-v1.3 closed wontfix The system shall support filtered harvest of CSW sources 1288040290000000 1294247922000000
#755 task johnbywater johnbywater ckan-v1.3 closed duplicate Add filter attribute to harvest source entity 1288040506000000 1294408632000000
#756 task johnbywater johnbywater ckan-v1.3 closed duplicate Add filter field to harvest source form 1288040545000000 1294408642000000
#757 task thejimmyg johnbywater ckan-v1.3 closed duplicate Create migration script to add harvest source filter attribute to existing tables 1288040584000000 1296593448000000
#758 task johnbywater johnbywater ckan-v1.3 closed duplicate Change API documentation to indicate harvest source entity has filter attribute 1288040643000000 1294409053000000
#759 story johnbywater johnbywater ckan-v1.3 closed duplicate Construct and send filtered CSW GetRecords request 1288040753000000 1294408652000000
#760 task johnbywater johnbywater ckan-v1.3 closed duplicate Change "CSW Get Records" request class to accept and used given CSW filter 1288040993000000 1294409111000000
#766 requirement wwaites johnbywater ckan-v1.3 closed duplicate The system shall support harvesting metadata from ArcGIS

Perhaps 80% of the users of the UKLII may be using ArcGIS.

1288177933000000 1296593519000000
#768 task johnbywater johnbywater ckan-v1.3 closed duplicate Obtain example ArcGIS CSW server 1288178134000000 1294409308000000
#769 task johnbywater johnbywater ckan-v1.3 closed duplicate Tests for harvesting metadata from ArcGIS CSW API 1288178179000000 1294409318000000
#770 defect thejimmyg johnbywater ckan-v1.3 closed wontfix Improve the paster harvester help command

Issue 1. Help command without arguments fails:

$paster help --plugin ckan --config development.ini

gives:

EITHER

Command help not recognized

OR

paster help: error: no such option: --plugin

Issue 2: Harvester command with arguments fails.

$ paster harvester Traceback (most recent call last): ...

cmd = self.args[0]

IndexError?: list index out of range

1288190220000000 1296593925000000
#771 story johnbywater ckan-v1.3 closed worksforme Run CLI help command without arguments

Should return something sensible (currently reports an error).

1288190318000000 1294413256000000
#772 story johnbywater ckan-v1.3 closed duplicate Run CLI harvester command without arguments

Should return help for the harvester (currently crashes).

1288190373000000 1294413313000000
#773 task johnbywater ckan-v1.3 closed duplicate Fix CLI help command to return CLI help message 1288190475000000 1294413400000000
#774 task johnbywater ckan-v1.3 closed duplicate Fix CLI harvester command to return harvester help message 1288190522000000 1294413412000000
#777 task dread dread ckan-v1.3 closed fixed Put 503-returning middleware into a module

Also configure hmgckan, hmgapi, test-hmg and dev-hmg machines.

1288608324000000 1292586843000000
#778 task thejimmyg dread ckan-v1.3 closed fixed Package edit form - name auto-generated from title 1288630777000000 1294416567000000
#779 task thejimmyg dread ckan-v1.3 closed fixed Package edit form - stylesheet in DGU updated 1288630800000000 1294416581000000
#780 task thejimmyg dread ckan-v1.3 closed fixed Package edit form - 'preview' button shows preview, with form lower in page

This ticket refers to the form shown in Drupal. This doesn't have a preview function, but we don't have a requirement for it right now, so leaving ticket closed.

1288630829000000 1294660720000000
#781 task thejimmyg dread ckan-v1.3 closed duplicate Package edit form - 'extra' fields have 'add' and 'delete' buttons

(Just like resource table, but extra fields don't need up/down buttons)

Nice to have: a blank field is added when you tab from the last filled-in field in the table.

1288630935000000 1294415081000000
#782 defect pudo rgrp ckan-v1.3 closed fixed Add and remove users from sysadmin role from the command line

Need to be able to create (and remove) sysadmins easily (used to be able to do this via ini file but this functionality was removed a while back in authz system reorganization it seems).

Suggest creating a new paster command as first step.

# user identifier = openid (?)
paster sysadmin add {user identifier}
paster sysadmin remove {user identifier}

Suggest this also be added to main CKAN docs.

This should go into metastable as well as default so that it goes into the v1.2 release (can do this via a transplant from one branch to the other)

1289205287000000 1289218661000000
#783 story johnbywater johnbywater ckan-v1.3 closed duplicate Delete harvested documents when remote metadata disappears from harvest source

We need a condition for 'disappears'. Asked AW about this (prob. in July) but should chase for an answer.

1289210029000000 1294409399000000
#785 task johnbywater johnbywater ckan-v1.3 closed fixed Review document from PP regarding CSW and WAF guidance

Scope of CSW guidance document has been broadened to include WAF.

1289210338000000 1289482499000000
#786 requirement wwaites johnbywater ckan-v1.3 closed duplicate The system shall support deleting a registered harvest source 1289210982000000 1294410004000000
#788 story johnbywater johnbywater ckan-v1.3 closed duplicate API user deletes harvest source

Needs to delete all harving jobs and harvested documents, and probably to purge all packages (perhaps just delete?).

1289211196000000 1294410007000000
#789 requirement wwaites johnbywater ckan-v1.3 closed duplicate The harvest source form shall validate location field by positive identification of source type 1289212106000000 1296593257000000
#790 story johnbywater johnbywater ckan-v1.3 closed duplicate Submit harvest source form with valid CSW location 1289212479000000 1294409709000000
#791 story johnbywater johnbywater ckan-v1.3 closed duplicate Submit harvest source form with invalid CSW location 1289212751000000 1294409723000000
#792 task johnbywater johnbywater ckan-v1.3 closed fixed Meeting to review pre-alpha testing report from TM 1289212851000000 1289482513000000
#793 task johnbywater johnbywater ckan-v1.3 closed invalid Fix up changes in changeset aa9aa32e00a9 1289212907000000 1294409836000000
#795 story thejimmyg johnbywater ckan-v1.3 closed duplicate "Match up" GEMINI2 "metadata point of contact" with registered entities

TM wants to know how this will be "matched up". I don't know whether this always the data provider, or sometimes the publisher.

1289228025000000 1296593361000000
#796 defect pudo ckan-v1.3 closed invalid AlphaPage for non-latin languages

The AlphaPage? implementation assumes that all packages start with a latin character. This is not true for Russian, Chinese, Arabian and Persian languages etc. We should either use a unicode solution (i.e. display a full class of characters based on the current locale) or use generic paging instead.

1289296662000000 1291640040000000
#797 enhancement rgrp rgrp ckan-v1.3 closed fixed Rework core html layout to mirror wordpress twentyten

WP twentyten has an excellent core html structure. Furthermore, using that structure makes us compatible with all the WP twentyten compatible themes.

  • Convert to wp twentyten html
  • Switch css to be based off twentyten css (and perhaps rework somewhat)
1289402873000000 1289402982000000
#798 task dread dread ckan-v1.3 closed fixed cospread importer using API

Move existing getdata/cospread.py into dgu repo and use the importer/loader mechanism.

1289475362000000 1294916973000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Note: See TracReports for help on using and creating reports.