id,type,owner,reporter,milestone,status,resolution,summary,description,PosixTime,ModifiedTime
140,enhancement,,rgrp,ckan-backlog,new,,News section on front page,"Have a news section (suggest as a sidebar item).
News section will link to latest 3/4 blog posts on CKAN from blog.okfn.org.
Details:
* Suggest pulling via rss or similar.
* Will want to cache this ...
Cost: 4h?",1254902541000000,1265625159000000
182,enhancement,,rgrp,v1.0,closed,fixed,Improve openness icons,"Tick and cross aren't clear enough. Propose:
* Icon for downloadability (nothing if absent ...(?))
* Icon for open license? (Need something if closed ...)
Cost: 2h (?)",1257534590000000,1270567116000000
253,enhancement,,dread,ckan-backlog,assigned,,Package relationships,"= Overview =
Functionality to formally associate packages. We see a need for specific parent-child, inheriting or dependency relations. Not only should this help navigation between packages in the web interface, but it also provides a mechanism to automatically pull dependencies when downloading a data package, in a similar manner as we see in software package management.
= Examples =
1. There are 27 packages in data.gov.uk to do with the Data4NR's Health Poverty Index. There is currently no common link between these, unless you search for 'HPI' (which also brings up House Price Index), or look under tag 'health' (which also has 600 other results). There should be a link on each HPI package page to navigate to the other 'sibling' HPI packages, and to a 'root' package that has info about the set. This could be partially achieved using the existing tag or group concepts, but a more explicit/official/obvious marking of their relationship could be beneficial.
2. In ckan.net is freedict, a collection of translation dictionaries. You could make each dictionary a child package and use this system. But it would probably be better to make each dictionary a different resource in the same package. (There are other ideas to denote a resource as the data making up a 'portion' of package, or a 'whole' of the package, to help people downloading datasets in the software package style.)
3. OSM has had some Naptan data imported (bus stops), with special permission - i.e. a more liberal license. It would be useful to show this link on both OSM and Naptan packages in CKAN: OSM 'derives from' Naptan with a comment about the license change. I'm not sure this is useful to an automatic download or use of these datasets, but may aid exploration on the CKAN website and understanding the provenance of the bus stop data on it.
4. IPCC collection of data linked / mirrored. Not sure if there are useful relationships here?
5. Dracos gets postbox locations from crowd sourcing and OSM. We could say Dracos 'derives from' OSM.
See more examples discussed here: http://trac.ckan.org/ticket/253
= Implementation =
This is split into four tickets:
* Model: ticket:254
* Read in WUI: ticket:255
* Edit in WUI: ticket:256
* API: ticket:257
No need for write access to be provided API for the moment.
This ticket also encompasses ticket:169 (Package derivations) and ticket:176 (Package dependencies).",1266854721000000,1339774726000000
256,requirement,,dread,ckan-backlog,assigned,,Package relationships - 3. Edit in WUI,"WUI:
* Editable as part of package or separately? (e.g. like authz)
* Do we normalize to only one type name of the pair?
* Do we allow create of relationship from both ends (e.g. only from dependency to dependent or either way?)
",1266928561000000,1339774714000000
285,enhancement,,rgrp,,assigned,,Paginate list of packages on tag read page,"Is this worth doing? On hmg.ckan.net start to have a lot of packages with a given tag ...
",1270664606000000,1340631923000000
301,enhancement,,rgrp,,assigned,,Package discussion pages,"A package discussion page is like a wikipedia discussion page: an editable free text page for people to have discussion/post comments about a given package.
It provides a way for people to make suggestions about a package without needing access to main package.",1272301033000000,1340632055000000
331,enhancement,,rgrp,ckan-backlog,new,,Timezone of CKAN timestamps should be configurable,"Revisions are timestamped using the server's clock, which may not relate to the expected timezone for the site. e.g. the Norway site has a server on GMT. No timezone info is displayed either.
Would like to set timezone for a CKAN instance to use in rendering revision timestamps. For example, use CET or EST timezone.",1275302440000000,1339774701000000
333,enhancement,,dread,v1.1,closed,wontfix,CKAN front end requirements for package notifications,"== Use case: new package ==
1. An external front-end system provides a web page with a list of packages. Each package has the option to edit it or and there is also a button to create a new package.
2. User: clicks 'new package'.
3. CKAN presents the package/new form to the user.
4. (After a couple of previews) User: clicks 'commit'.
5. Notification message goes from CKAN to the front-end detailing the new package.
6. The user is redirected back to the front-end web page displaying the list of packages, which contains the new one.
The notification message (step 5) has to get through to the front-end that the new package is created before the redirect (step 6). This suggests that the message sending needs to be *synchronous*, i.e. acknowledged by the front-end, before CKAN redirects the user to the front-end package listing page (step 6).
In addition, this use case suggests the front-end listens for package notifications, to save another call to CKAN to get the package details, before the displaying the list of packages. If this isn't possible (see next use case) and it must listen for revision notifications instead, then perhaps it is worth including the full package details in the payload for the revision notification message. Would there be a problem with such a large message in the next use case, with 100 packages?
== Use case: CKAN imports packages ==
1. CKAN administrator runs a script that adds 100 new packages into CKAN.
2. CKAN sends notification message to front-end to report the new packages/revisions.
3. Knowing there are new revisions, the front-end queries the CKAN revision interface to get the list of new packages.
4. The front-end queries CKAN for each new package one-by-one.
5. A new user request to the front-end will include the info about the new packages.
The package addition could be achieved in 1 revision, 100 revisions or some compromise:
* If it is 1 revision then potentially there are problems displaying the long list of packages in the 'recent changes'.
* If it is 100 revisions, then the notification webhook would be called 100 times, which creates unnecessary load on the front-end. Suppose each Webhook call-back (step 2) triggers the front-end to make a call to CKAN to get the latest revisions (step3), in this case it would make 100 calls, most of them fruitless, causing unnecessary load on CKAN.
This use case suggests a bulk import of packages should go into one revision, and therefore generate one revision notification message and 100 package notification messages. The front-end client should listen to only revision messages.",1275324042000000,1275407987000000
339,defect,,johnbywater,,closed,fixed,"Remove download_url from package data format in API Version 2, but maintain the old data format in the old version of the interface (API Version 1)",,1275901499000000,1278074705000000
341,enhancement,,dread,,closed,fixed,Web UI accepts package IDs in URLs,"(in the same way that we can refer to packages in the API by ID as well as name.)
== As a ==
client of CKAN
== I want to ==
link to a package page in the UI, referring to it by ID",1276162400000000,1277483030000000
344,defect,,dread,,closed,fixed,REST Create package with incorrect format gives 500 error,"PUT to /rest/api/package of {{{ {""name"": ""name"", ""resources"": [""someurl.com/data""] } }}} (i.e. resource is a string, not a dictionary) gives 500 error, when it should give a 400 error and helpful error message.
Exception:
{{{
Module ckan.controllers.rest:154 in create
<< if register == 'package' and not subregister:
fs = ckan.forms.get_standard_fieldset()
request_fa_dict = ckan.forms.edit_package_dict(ckan.forms.get_package_dict(fs=fs), request_data)
fs = fs.bind(model.Package, data=request_fa_dict, session=model.Session)
elif register == 'package' and subregister in model.PackageRelationship.get_all_types():
>> request_fa_dict = ckan.forms.edit_package_dict(ckan.forms.get_package_dict(fs=fs), request_data)
Module ckan.forms.package_dict:88 in edit_package_dict
<< for res_dict in value:
res_dict_str = {}
for key, value in res_dict.items():
res_dict_str[str(key)] = value
resources.append(res_dict_str)
>> for key, value in res_dict.items():
AttributeError: 'unicode' object has no attribute 'items'
}}}",1276341172000000,1277477712000000
350,enhancement,,dread,ckan-backlog,reopened,,Search engine optimisation,"Need to research what can easily be done to improve CKAN packages in the search rankings.
Comments from Glen Barnes:
We've been pretty successful at SEO without even really trying (see http://www.google.co.nz/search?client=safari&rls=en&q=auckland+google+transit+feed&ie=UTF-8&oe=UTF-8&redir_esc=&ei=dsYSTOzJLs2eceuZiI8I as an example). This to me is key. If we are to make data available it has to be findable which is the main reason for a catalogue. There are probably things we should be doing on CKAN like using slugged urls (http://www.ckan.net/package/ascoe -> http://www.ckan.net/package/ascoe/atmospheric-chemistry-studies-in-the-oceanic-environment), setting the H1 tag correctly (""Atmospheric Chemistry Studies in the Oceanic Environment"" on the example above). Some basic SEO 101 on page optimisations.
",1276594541000000,1339774690000000
351,enhancement,,dread,ckan-backlog,new,,"Homepage: list new, updated and 'hot' packages","Have a simpler list of exciting data, as opposed to the big revision list.
For example:
{{{
Hot data
===========
New packages: package1, package2, package3
Updated resources: package1, package2, package3
Popular packages:
}}}",1276595816000000,1339774677000000
353,defect,,dread,,closed,fixed,SOLR search indexing,"== As a ==
SOLR instance
== I want to ==
keep my search index of CKAN packages up-to-date
== Implementation ==
* Using asynchronous event notifications
* Running in a separate process to CKAN",1277123480000000,1280756399000000
356,enhancement,,rgrp,v1.1,closed,fixed,Search box in at top of page (UI),"A small but useful ui improvement would be to have a search box at top right on every page.
As an example see the one here on trac or on github.com or bitbucket.org.
* It would be particularly good to include a small advanced search link that took you to the full search page. Need to keep it small because screen real-estate here is limited (see how github.com does this for inspiration).",1277235411000000,1278931830000000
364,defect,,dread,ckan-v1.3,closed,fixed,Search for 'statistic' returns nothing,"On ckan.net there are plenty of packages (and indeed their tags) with the word 'statistic' in them, but no packages turn up when you search for it:
http://ckan.net/package/search?q=statistic&search=Search+Packages+%C2%BB
(Using Postgres full text search)",1278949620000000,1291637291000000
368,defect,,anonymous,,closed,wontfix,500 Server error when creating package,"
I've just started writing the importer from cap.open.org.nz and was running an initial import of just the package name and titles. The packages were created however I received a 500 error in response:
""publish: New Zealand Coastline (new-zealand-coastline)""
opening connection to nz.ckan.net...
opened
<- ""POST /api/rest/package HTTP/1.1\r\nContent-Type: application/x-www-form-urlencoded\r\nConnection: close\r\nAuthorization: 81179ade-fa4a-4632-9b89-3d0c98bfc8b8\r\nContent-Length: 64\r\nHost: nz.ckan.net\r\n\r\n""
<- ""{\""name\"":\""new-zealand-coastline\"",\""title\"":\""New Zealand Coastline\""}""
-> ""HTTP/1.1 500 Internal Server Error\r\n""
-> ""Date: Tue, 20 Jul 2010 10:20:35 GMT\r\n""
-> ""Server: Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2\r\n""
-> ""Vary: Accept-Encoding\r\n""
-> ""Content-Type: text/html; charset=iso-8859-1\r\n""
-> ""Cache-Control: proxy-revalidate\r\n""
-> ""Content-Length: 640\r\n""
-> ""Connection: close\r\n""
-> ""\r\n""
reading 640 bytes...
-> ""\n
\n500 Internal Server Error\n\nInternal Server Error
\nThe server encountered an internal error or\nmisconfiguration and was unable to complete\nyour request.
\nPlease contact the server administrator,\n [no address given] and inform them of the time the error occurred,\nand anything you might have done that may have\ncaused the error.
\nMore information about this error may be available\nin the server error log.
\n
\nApache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2 Server at nz.ckan.net Port 80\n\n""
read 640 bytes
Conn close
The server should not respond with a 500 error if the package was created. ",1279622462000000,1291831811000000
369,enhancement,,shudson@…,ckan-backlog,new,,"""Package Listing Key"" should appear on Tag results","Currently there's a nice legend titled ""Package Listing Key"" that appears in right side of ""Browse Packages"" results. The same key should show on other search results like when searching for a tag.",1279821634000000,1339774666000000
370,enhancement,,shudson@…,ckan-backlog,new,,Use better email encryption for author_email and maintainer_email,"The JavaScript email encryption used is not very reassuring. Google's MailHide is a much better solution that is easily implemented.
http://www.google.com/recaptcha/mailhide/
Check on the Mailhide API where there are even some Python libraries already built.",1279821819000000,1339774649000000
377,enhancement,,glen@…,ckan-future,closed,wontfix,API Should return JSON in all cases,"When the API returns any response it should always retunr JSON. For example a 404 response should return something like
{{{
{'status': 404, 'message': 'package somename not found'}
}}}
When errors are encountered when creating a package:
{{{
{'status': nnn, 'message': 'Validation Errors', 'errors':['description':'name cannot be
blank', 'description':'license code must be an integer']}
}}}
Not authorised:
{{{
{'status': 503, 'message': 'You are not authorised to create this package.'}
}}}
The reason being that screeds of html coming back in the response just makes it harder to debug and understand. Having the proper error codes/messages means that you can see why your package was not created.
",1280268158000000,1338206349000000
381,defect,,johnbywater,,closed,invalid,Test defect,please ignore,1280345937000000,1294410466000000
395,task,,pudo,ckan-v1.3,closed,duplicate,Set up profiling to analyze performance issues,"At the moment, some pages within CKAN tend to load slowly. We should create a profiling setup in which we can measure response times for complete requests and individual methods calls.
This could be used to identify bottlenecks and find an appropriate caching or tuning strategy to improve CKAN performance.
NB: We should also agree on a maximum request latency.
TODO: Read up on all those QoS tickets to avoid overlapping efforts. ",1280824739000000,1294417538000000
403,requirement,,pudo,,closed,fixed,Create web service for OFS storage,"Create a thin web service that will allow users to upload files to OFS on S3 without knowing S3 credentials. This could also provide static urls for all stored resources.
* relates to #307",1281346110000000,1287392451000000
407,requirement,,dread,,closed,wontfix,Link to Scraperwiki,"== As a ==
user browsing a CKAN package which has a resource in a bad format
== I want to ==
be alerted to the possibility of scraping it on scraperwiki
== Implementation ==
This should be done in a CKAN extension. It should appear in a side-bar on the package read page.
If the package does not have a resource with scraperwiki.com in the url, then it displays this message:
Title: Scrape it[[BR]]
Image: (Scraperwiki dumper truck)[[BR]]
Body text: Could this data be in a better format? Why not use Scraperwiki to extract data from tables, Excel or PDF for everyone's benefit? You can [http://scraperwiki.com/market/request/ request] this data is scraped or even [http://scraperwiki.com/editor/ write the scraper] yourself.
",1281348368000000,1330769956000000
412,requirement,,johnbywater,,closed,fixed,Data format for package importing shall be documented,"Requirements:
* cater naturally for multiple packages with multiple resources
* international encoding
* large number of packages",1281430809000000,1289822092000000
415,task,,dread,ckan-v1.2,closed,fixed,Chase Talis about loading RDF from CKAN.,,1281431656000000,1288003954000000
418,task,,dread,,closed,invalid,"With COI, get SOLR working",,1281431691000000,1291637919000000
422,story,,dread,ckan-v1.3,closed,fixed,Administrator imports meta-data into CKAN,"Requirements:
* cater naturally for multiple packages with multiple resources
* international encoding
* large number of packages",1281433490000000,1292586586000000
428,requirement,,dread,ckan-v1.2,closed,fixed,Daily dump of DGU CKAN available,End users want to get hold of a JSON & CSV dump of the DGU data.,1281703251000000,1285757316000000
429,requirement,,dread,ckan-v1.2,closed,fixed,ONS data is imported daily to DGU,,1281703294000000,1285757274000000
433,defect,,wwaites,datapkg-0.8,closed,fixed,Data package metadata in the Egg,"Still not sure if we shouldn't use the existing setuptools machinery to manage this -- there is already a way to get at the metadata. In any event, I've just made an addition to datapkg that makes it possible to put datapkg_sources as a dictionary in your package's setup.py. Afterwards it is possible to pull the metadata out of the egg. Of course this could easily be changed to save the information in whatever form, indeed if you pass it a string instead of a dictionary it will just write whatever you gave it into the datapkg_sources.spec. The point is, I think that the egg is a good place to stuff this information.
For non-python users, it is always possible to simply put up the datapkg_sources.spec somewhere on the web so they can directly retrieve the data files.
From the docstring::
{{{
This is the implementation for an [egg_info.writers] entrypoint.
Datapkg adds an argument to setuptools's setup() function called
datapkg_sources. The argument should be a dictionary of the form:
.. code-block:: python
setup(
...,
datapkg_sources = {
""cra2009"" : ""http://www.hm-treasury.gov.uk/d/cra_2009_db.csv""
}
)
The result of this is that there will be a file in the egg called
datapkg_sources.spec that looks like this::
[sources]
cra2009=http://www.hm-treasury.gov.uk/d/cra_2009_db.csv
How do you get at this data? Simple::
.. code-block:: python
import pkg_resources
dist = pkg_resources.get_distribution(""ukgov_treasury_cra"")
spec = dist.get_metadata(""datapkg_sources.spec"")
and 'spec' will be the contents of the file as a string.
}}}
",1282078660000000,1283183647000000
434,defect,,dread,ckan-v1.2,closed,fixed,Logged in user to si.ckan.net causes genshi exception,"Here's the trace:
{{{
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] Error - : list index out of range
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] mod_wsgi (pid=6195): Exception occurred processing WSGI script '/home/okfn/var/srvc/si.ckan.net/py
env/bin/si.ckan.net.py'.
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] Traceback (most recent call last):
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/paste/cascade.py"", line
130, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return self.apps[-1](environ, start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/paste/registry.py"", line
375, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] app_iter = self.application(environ, start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/repoze/who/middleware.py
"", line 107, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] app_iter = app(environ, wrapper.wrap_start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/pylons/middleware.py"", l
ine 214, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] self.app, new_environ, catch_exc_info=True)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/pylons/util.py"", line 94
, in call_wsgi_application
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] app_iter = application(environ, start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/weberror/errormiddleware
.py"", line 156, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return self.application(environ, start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/beaker/middleware.py"", l
ine 73, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return self.app(environ, start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/beaker/middleware.py"", l
ine 152, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return self.wrap_app(environ, session_start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/routes/middleware.py"", l
ine 130, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] response = self.app(environ, start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/pylons/wsgiapp.py"", line
125, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] response = self.dispatch(controller, environ, start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/pylons/wsgiapp.py"", line
324, in dispatch
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return controller(environ, start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/src/ckan/ckan/lib/base.py"", line 73, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return WSGIController.__call__(self, environ, start_response)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/pylons/controllers/core.
py"", line 221, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] response = self._dispatch_call()
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/pylons/controllers/core.
py"", line 172, in _dispatch_call
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] response = self._inspect_call(func)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/pylons/controllers/core.
py"", line 107, in _inspect_call
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] result = self._perform_call(func, args)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/pylons/controllers/core.
py"", line 60, in _perform_call
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return func(**args)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/src/ckan/ckan/controllers/error.py"", line 32, in doc
ument
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] ckan_template = render('error_document_template.html')
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/src/ckan/ckan/lib/base.py"", line 38, in render
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return render_template()
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/src/ckan/ckan/lib/base.py"", line 36, in render_templ
ate
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return literal(stream.render(method=method, encoding=None))
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/core.py"", line 18
3, in render
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return encode(generator, method=method, encoding=encoding, out=out)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/output.py"", line
57, in encode
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] return _encode(''.join(list(iterator)))
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/output.py"", line 339, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for kind, data, pos in stream:
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/output.py"", line 670, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for kind, data, pos in stream:
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/output.py"", line 771, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for kind, data, pos in chain(stream, [(None, None, None)]):
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/output.py"", line 586, in __call__
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for ev in stream:
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/core.py"", line 288, in _ensure
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for event in stream:
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/template/base.py"", line 618, in _include
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for event in tmpl.generate(ctxt, **vars):
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/template/base.py"", line 618, in _include
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for event in tmpl.generate(ctxt, **vars):
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/template/base.py"", line 605, in _include
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for event in stream:
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/template/markup.py"", line 327, in _match
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for event in stream:
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/template/base.py"", line 545, in _flatten
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for kind, data, pos in stream:
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/filters/i18n.py"", line 177, in _generate
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] for event in msgbuf.translate(gettext(msgbuf.format())):
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/filters/i18n.py"", line 1029, in translate
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] parts = parse_msg(string)
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] File ""/home/okfn/var/srvc/si.ckan.net/pyenv/lib/python2.5/site-packages/genshi/filters/i18n.py"", line 1143, in parse_msg
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] parts.append((stack[-1], string))
[Wed Aug 18 11:10:19 2010] [error] [client 89.212.104.197] IndexError: list index out of range
}}}
ckan: 1e403927f2bb (metastable)
genshi: 0.6
lang: slovenian",1282130515000000,1283189807000000
436,bug,,dread,ckan-v1.2,closed,wontfix,Investigate exception: resource search JSON,"Here's the dump from 22:10 last night:
URL: http://ckan.net/api/search/resource?all_fields=1&offset=0&limit=20&qjson=%3Cspan%20class=
Module weberror.errormiddleware:162 in __call__
<< __traceback_supplement__ = Supplement, self, environ
sr_checker = ResponseStartChecker(start_response)
app_iter = self.application(environ, sr_checker)
return self.make_catching_iter(app_iter, environ, sr_checker)
except:
>> app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<< self.cache_manager)
environ[self.environ_key] = self.cache_manager
return self.app(environ, start_response)
>> return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<< headers.append(('Set-cookie', cookie))
return start_response(status, headers, exc_info)
return self.wrap_app(environ, session_start_response)
def _get_session(self):
>> return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<< environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
response = self.app(environ, start_response)
# Wrapped in try as in rare cases the attribute will be gone already
>> response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<
controller = self.resolve(environ, start_response)
response = self.dispatch(controller, environ, start_response)
if 'paste.testing_variables' in environ and hasattr(response,
>> response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<< if log_debug:
log.debug(""Calling controller class with WSGI interface"")
return controller(environ, start_response)
def load_test_env(self, environ):
>> return controller(environ, start_response)
Module ckan.lib.base:73 in __call__
<< # available in environ['pylons.routes_dict']
try:
return WSGIController.__call__(self, environ, start_response)
finally:
model.Session.remove()
>> return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<< return response(environ, self.start_response)
response = self._dispatch_call()
if not start_response_called:
self.start_response = start_response
>> response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<< req.environ['pylons.action_method'] = func
response = self._inspect_call(func)
else:
if log_debug:
>> response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<< func.__name__, args)
try:
result = self._perform_call(func, args)
except HTTPException, httpe:
if log_debug:
>> result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<< """"""Hide the traceback for everything above this method""""""
__traceback_hide__ = 'before_and_this'
return func(**args)
def _inspect_call(self, func):
>> return func(**args)
Module ckan.controllers.rest:400 in search
<< response.status_int = 400
return gettext('Blank qjson parameter')
params = json.loads(request.params['qjson'])
elif request.params.values() and request.params.values() != [u''] and request.params.values() != [u'1']:
params = request.params
>> params = json.loads(request.params['qjson'])
Module simplejson:384 in loads
<< parse_constant is None and object_pairs_hook is None
and not use_decimal and not kw):
return _default_decoder.decode(s)
if cls is None:
cls = JSONDecoder
>> return _default_decoder.decode(s)
Module simplejson.decoder:402 in decode
<< """"""
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
end = _w(s, end).end()
if end != len(s):
>> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
Module simplejson.decoder:420 in raw_decode
<< obj, end = self.scan_once(s, idx)
except StopIteration:
raise JSONDecodeError(""No JSON object could be decoded"", s, idx)
return obj, end
>> raise JSONDecodeError(""No JSON object could be decoded"", s, idx)
JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)
CGI Variables
DOCUMENT_ROOT '/htdocs'
GATEWAY_INTERFACE 'CGI/1.1'
HTTP_ACCEPT 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8, application/json'
HTTP_ACCEPT_CHARSET 'ISO-8859-1,utf-8;q=0.7,*;q=0.7'
HTTP_ACCEPT_ENCODING 'gzip,deflate'
HTTP_ACCEPT_LANGUAGE 'en-us,en;q=0.5'
HTTP_CONNECTION 'keep-alive'
HTTP_COOKIE '__utma=27730403.1245320310.1281386803.1281386803.1282164955.2; __utmz=27730403.1282164955.2.2.utmcsr=jira|utmccn=(referral)|utmcmd=referral|utmcct=/browse/PLATFORM-892; __utmb=27730403.3.10.1282164955; __utmc=27730403'
HTTP_HOST 'ckan.net'
HTTP_KEEP_ALIVE '300'
HTTP_REFERER 'http://jira/browse/PLATFORM-892'
HTTP_USER_AGENT 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7'
PATH '/usr/local/bin:/usr/bin:/bin'
PATH_INFO '/api/search/resource'
PATH_TRANSLATED '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/api/search/resource'
QUERY_STRING 'all_fields=1&offset=0&limit=20&qjson=%3Cspan%20class='
REMOTE_ADDR '64.235.97.218'
REMOTE_PORT '20720'
REQUEST_METHOD 'GET'
REQUEST_URI '/api/search/resource?all_fields=1&offset=0&limit=20&qjson=%3Cspan%20class='
SCRIPT_FILENAME '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py'
SCRIPT_URI 'http://ckan.net/api/search/resource'
SCRIPT_URL '/api/search/resource'
SERVER_ADDR '10.226.226.118'
SERVER_ADMIN '[no address given]'
SERVER_NAME 'ckan.net'
SERVER_PORT '80'
SERVER_PROTOCOL 'HTTP/1.1'
SERVER_SIGNATURE 'Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2 Server at ckan.net Port 80\n'
SERVER_SOFTWARE 'Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2'
WSGI Variables
application
beaker.cache
beaker.get_session >
beaker.session {'_accessed_time': 1282165818.0880959, '_creation_time': 1282165818.0880959}
mod_wsgi.application_group 'ckan.net|'
mod_wsgi.callable_object 'application'
mod_wsgi.listener_host ''
mod_wsgi.listener_port '80'
mod_wsgi.process_group ''
mod_wsgi.reload_mechanism '0'
mod_wsgi.script_reloading '1'
mod_wsgi.version (2, 5)
paste.cookies (, '__utma=27730403.1245320310.1281386803.1281386803.1282164955.2; __utmz=27730403.1282164955.2.2.utmcsr=jira|utmccn=(referral)|utmcmd=referral|utmcct=/browse/PLATFORM-892; __utmb=27730403.3.10.1282164955; __utmc=27730403')
paste.parsed_querystring ([('all_fields', '1'), ('offset', '0'), ('limit', '20'), ('qjson', '
paste.throw_errors True
pylons.action_method >
pylons.controller
pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons
pylons.routes_dict {'action': u'search', 'controller': u'rest', 'register': u'resource'}
repoze.who.logger
repoze.who.plugins {'openid': , 'auth_tkt': }
routes.route
routes.url
webob._parsed_query_vars (GET([('all_fields', '1'), ('offset', '0'), ('limit', '20'), ('qjson', '
wsgiorg.routing_args (, {'action': u'search', 'controller': u'rest', 'register': u'resource'})
",1282206959000000,1288003983000000
437,bug,,dread,ckan-v1.2,closed,fixed,Buildbot test failures - ascii codec,"On today's buildbot: http://buildbot.okfn.org/builders/buildbot-test/builds/201
2 failures about ascii (ignore other 2)",1282223640000000,1288004009000000
439,task,,dread,,closed,fixed,Plan server requirements for DGU,"Using use cases for security etc.
Draw diagram to satisfy use cases.
Communicate to Paul and therefore TSO.",1282224452000000,1294413903000000
444,task,,dread,,closed,duplicate,Discuss package relationships ideas with JF,"* Create test data on visible ckan
* Discuss with JF",1282299238000000,1294414008000000
446,task,,dread,,closed,invalid,Server configuration query (?)," * Ensure use case is in design - ""an admin writes to the CKAN API"" (and see JB notes on paper)
* Communicate how this is planned to be achieved in design to JF",1282299385000000,1294414077000000
450,task,,dread,,closed,fixed,Assist with proxying of static media for forms,John did this a while ago.,1282299573000000,1287997487000000
452,task,,dread,,closed,fixed,Assist with using package update error form,,1282299647000000,1287997540000000
455,task,,johnbywater,,closed,invalid,SOLR - suggest 1 pager about how system would work,"Either CKAN writes to SOLR and Drupal reads from SOLR, or CKAN writes to SOLR and Drupal reads SOLR via CKAN API (so search resource locations are unaffected).",1282299913000000,1291637172000000
459,bug,,johnbywater,ckan-v1.2,closed,fixed,Versions on branches are broken,,1282299973000000,1282921783000000
463,defect,,dread,ckan-v1.3,closed,worksforme,package diff exception,"Investigate exception:
{{{
URL: http://ckan.net/revision/diff/dbtune-audioscrobbler?diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c
Module weberror.errormiddleware:162 in __call__
<< __traceback_supplement__ = Supplement, self, environ
sr_checker = ResponseStartChecker(start_response)
app_iter = self.application(environ, sr_checker)
return self.make_catching_iter(app_iter, environ, sr_checker)
except:
>> app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<< self.cache_manager)
environ[self.environ_key] = self.cache_manager
return self.app(environ, start_response)
>> return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<< headers.append(('Set-cookie', cookie))
return start_response(status, headers, exc_info)
return self.wrap_app(environ, session_start_response)
def _get_session(self):
>> return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<< environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
response = self.app(environ, start_response)
# Wrapped in try as in rare cases the attribute will be gone already
>> response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<
controller = self.resolve(environ, start_response)
response = self.dispatch(controller, environ, start_response)
if 'paste.testing_variables' in environ and hasattr(response,
>> response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<< if log_debug:
log.debug(""Calling controller class with WSGI interface"")
return controller(environ, start_response)
def load_test_env(self, environ):
>> return controller(environ, start_response)
Module ckan.lib.base:73 in __call__
<< # available in environ['pylons.routes_dict']
try:
return WSGIController.__call__(self, environ, start_response)
finally:
model.Session.remove()
>> return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<< return response(environ, self.start_response)
response = self._dispatch_call()
if not start_response_called:
self.start_response = start_response
>> response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<< req.environ['pylons.action_method'] = func
response = self._inspect_call(func)
else:
if log_debug:
>> response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<< func.__name__, args)
try:
result = self._perform_call(func, args)
except HTTPException, httpe:
if log_debug:
>> result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<< """"""Hide the traceback for everything above this method""""""
__traceback_hide__ = 'before_and_this'
return func(**args)
def _inspect_call(self, func):
>> return func(**args)
Module ckan.controllers.revision:119 in diff
<< c.revision_to = model.Session.query(model.Revision).get(
request.params.getone('diff'))
diff = pkg.diff(c.revision_to, c.revision_from)
c.diff = diff.items()
c.diff.sort()
>> diff = pkg.diff(c.revision_to, c.revision_from)
Module ckan.model.package:340 in diff
<< display_id = to_obj_rev.tag.name
elif obj_class.__name__ == 'PackageExtra':
display_id = to_obj_rev.key
else:
display_id = related_obj_id[:4]
>> display_id = to_obj_rev.key
AttributeError: 'NoneType' object has no attribute 'key'
CGI Variables
AUTH_TYPE 'cookie'
DOCUMENT_ROOT '/htdocs'
GATEWAY_INTERFACE 'CGI/1.1'
HTTP_ACCEPT 'application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'
HTTP_ACCEPT_CHARSET 'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
HTTP_ACCEPT_ENCODING 'gzip'
HTTP_ACCEPT_LANGUAGE 'en-US,en;q=0.8'
HTTP_CACHE_CONTROL 'max-age=0'
HTTP_COOKIE '__utmz=27730403.1281000628.1.6.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=ckan; auth_tkt=""e919c7bb40f890a0184ed7be95465e1a4c6d130bhttp://pudo.org/!""; auth_tkt=""e919c7bb40f890a0184ed7be95465e1a4c6d130bhttp://pudo.org/!""; __utma=27730403.1381387877.1281962773.1282248244.1282250173.11; __utmc=27730403; __utmb=27730403.6.10.1282250173'
HTTP_HOST 'ckan.net'
HTTP_REFERER 'http://ckan.net/package/history/dbtune-audioscrobbler'
HTTP_USER_AGENT 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.6 (KHTML, like Gecko) Chrome/6.0.490.1 Safari/534.6'
PATH '/usr/local/bin:/usr/bin:/bin'
PATH_INFO '/revision/diff/dbtune-audioscrobbler'
PATH_TRANSLATED '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/revision/diff/dbtune-audioscrobbler'
QUERY_STRING 'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c'
REMOTE_ADDR '92.226.62.168'
REMOTE_PORT '55102'
REMOTE_USER 'http://pudo.org/'
REMOTE_USER_TOKENS ['']
REQUEST_METHOD 'GET'
REQUEST_URI '/revision/diff/dbtune-audioscrobbler?diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c'
SCRIPT_FILENAME '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py'
SCRIPT_URI 'http://ckan.net/revision/diff/dbtune-audioscrobbler'
SCRIPT_URL '/revision/diff/dbtune-audioscrobbler'
SERVER_ADDR '10.226.226.118'
SERVER_ADMIN '[no address given]'
SERVER_NAME 'ckan.net'
SERVER_PORT '80'
SERVER_PROTOCOL 'HTTP/1.1'
SERVER_SIGNATURE 'Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2 Server at ckan.net Port 80\n'
SERVER_SOFTWARE 'Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2'
WSGI Variables
application
beaker.cache
beaker.get_session >
beaker.session {'_accessed_time': 1282252047.603267, '_creation_time': 1282252047.603267}
mod_wsgi.application_group 'ckan.net|'
mod_wsgi.callable_object 'application'
mod_wsgi.listener_host ''
mod_wsgi.listener_port '80'
mod_wsgi.process_group ''
mod_wsgi.reload_mechanism '0'
mod_wsgi.script_reloading '1'
mod_wsgi.version (2, 5)
paste.cookies (, '__utmz=27730403.1281000628.1.6.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=ckan; auth_tkt=""e919c7bb40f890a0184ed7be95465e1a4c6d130bhttp://pudo.org/!""; auth_tkt=""e919c7bb40f890a0184ed7be95465e1a4c6d130bhttp://pudo.org/!""; __utma=27730403.1381387877.1281962773.1282248244.1282250173.11; __utmc=27730403; __utmb=27730403.6.10.1282250173')
paste.parsed_querystring ([('diff', '66a47b9e-232a-4838-8674-66fa1a5c76e1'), ('oldid', 'a99c98be-767a-4e49-9025-2472b2d18b9c')], 'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c')
paste.registry
paste.throw_errors True
pylons.action_method >
pylons.controller
pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons
pylons.routes_dict {'action': u'diff', 'controller': u'revision', 'id': u'dbtune-audioscrobbler'}
repoze.who.identity
repoze.who.logger
repoze.who.plugins {'openid': , 'auth_tkt': }
routes.route
routes.url
webob._parsed_query_vars (GET([('diff', '66a47b9e-232a-4838-8674-66fa1a5c76e1'), ('oldid', 'a99c98be-767a-4e49-9025-2472b2d18b9c')]), 'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c')
webob.adhoc_attrs {'language': 'en-us'}
wsgi process 'Multi process AND threads (?)'
wsgi.file_wrapper
wsgiorg.routing_args (, {'action': u'diff', 'controller': u'revision', 'id': u'dbtune-audioscrobbler'})
}}}",1282306037000000,1294916148000000
465,requirement,,johnbywater,ckan-v1.3,closed,wontfix,Require API key for all API requests.,,1282308904000000,1294411534000000
467,story,,johnbywater,ckan-v1.3,closed,duplicate,Admin configures CKAN to expect API key in named HTTP header,,1282310562000000,1294411681000000
469,task,,johnbywater,ckan-v1.2,closed,fixed,Unit test to check API key in correctly named HTTP header is accepted by server,,1282310760000000,1282932682000000
471,story,,johnbywater,ckan-v1.2,closed,,API user sends API key in correctly named header,,1282312088000000,1283248729000000
472,story,,johnbywater,ckan-v1.2,closed,,API user sends API key in incorrectly named header,,1282312108000000,1282932740000000
473,story,,johnbywater,ckan-v1.2,closed,,API user discovers correct header for sending API key,,1282312203000000,1283248736000000
475,task,,johnbywater,ckan-v1.2,closed,fixed,Unit test to check API key in incorrectly named HTTP header is not accepted by server,,1282312802000000,1282932705000000
476,requirement,,johnbywater,ckan-v1.3,closed,wontfix,Daily database dumps shall be discoverable and downloadable,,1282313433000000,1294411741000000
477,story,,johnbywater,ckan-v1.3,closed,duplicate,Discover location of the daily database dumps,,1282313788000000,1294411761000000
478,task,,johnbywater,ckan-v1.2,closed,fixed,"Unit test for ""interface resource"" in API","We don't want /api to be an HTML page, but rather provide information about the API in a JSON structure.",1282315067000000,1283248713000000
481,story,,johnbywater,ckan-v1.3,closed,fixed,Unrestricted total read-only access to catalogue API,,1282422697000000,1294248359000000
483,story,,johnbywater,,closed,invalid,Restricted total read-write access to catalogue Web UI,,1282422801000000,1294417216000000
484,story,,johnbywater,,closed,invalid,Restricted partial read-write access to catalogue Web UI,,1282422858000000,1294417248000000
485,story,,johnbywater,,closed,duplicate,Performance beats QoS criteria,,1282425219000000,1294411946000000
486,requirement,,johnbywater,ckan-v1.3,closed,duplicate,Catalogue service shall notify and query SOLR service,,1282425790000000,1291639321000000
487,story,,johnbywater,,closed,duplicate,Notify SOLR service of model events,,1282425910000000,1291639404000000
488,requirement,,johnbywater,,closed,wontfix,Catalogue service shall notify RDF service,,1282426021000000,1320930240000000
490,requirement,,johnbywater,,closed,fixed,The system shall support registering a harvest source via the CKAN API,,1282426875000000,1286200804000000
491,story,,johnbywater,ckan-v1.2,closed,,Get form for creating harvest source entity,,1282427008000000,1284493173000000
492,story,,johnbywater,ckan-v1.2,closed,,Submit harvest source create form response to the API,,1282427042000000,1284493145000000
493,story,,johnbywater,ckan-v1.2,closed,,Get harvest source entity,,1282427083000000,1284493130000000
494,story,,johnbywater,ckan-v1.2,closed,,Get form for updating remote metadata entity,,1282427106000000,1285198894000000
495,story,,johnbywater,ckan-v1.2,closed,,Put form for updating remote metadata entity,,1282427150000000,1285198898000000
498,story,,johnbywater,,closed,invalid,"Search packages within location ""bounding box""",,1282427412000000,1294412520000000
500,defect,,dread,ckan-v1.2,closed,duplicate,Exception from diff,"Investigate exception occured occasionally in last couple of days on ckan.net:
{{{
WebApp Error: : 'NoneType' object has no attribute 'key'
URL: http://ckan.net/revision/diff/dbtune-audioscrobbler?diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c
Module weberror.errormiddleware:162 in __call__
<< __traceback_supplement__ = Supplement, self, environ
sr_checker = ResponseStartChecker(start_response)
app_iter = self.application(environ, sr_checker)
return self.make_catching_iter(app_iter, environ, sr_checker)
except:
>> app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<< self.cache_manager)
environ[self.environ_key] = self.cache_manager
return self.app(environ, start_response)
>> return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<< headers.append(('Set-cookie', cookie))
return start_response(status, headers, exc_info)
return self.wrap_app(environ, session_start_response)
def _get_session(self):
>> return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<< environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
response = self.app(environ, start_response)
# Wrapped in try as in rare cases the attribute will be gone already
>> response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<
controller = self.resolve(environ, start_response)
response = self.dispatch(controller, environ, start_response)
if 'paste.testing_variables' in environ and hasattr(response,
>> response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<< if log_debug:
log.debug(""Calling controller class with WSGI interface"")
return controller(environ, start_response)
def load_test_env(self, environ):
>> return controller(environ, start_response)
Module ckan.lib.base:73 in __call__
<< # available in environ['pylons.routes_dict']
try:
return WSGIController.__call__(self, environ, start_response)
finally:
model.Session.remove()
>> return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<< return response(environ, self.start_response)
response = self._dispatch_call()
if not start_response_called:
self.start_response = start_response
>> response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<< req.environ['pylons.action_method'] = func
response = self._inspect_call(func)
else:
if log_debug:
>> response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<< func.__name__, args)
try:
result = self._perform_call(func, args)
except HTTPException, httpe:
if log_debug:
>> result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<< """"""Hide the traceback for everything above this method""""""
__traceback_hide__ = 'before_and_this'
return func(**args)
def _inspect_call(self, func):
>> return func(**args)
Module ckan.controllers.revision:119 in diff
<< c.revision_to = model.Session.query(model.Revision).get(
request.params.getone('diff'))
diff = pkg.diff(c.revision_to, c.revision_from)
c.diff = diff.items()
c.diff.sort()
>> diff = pkg.diff(c.revision_to, c.revision_from)
Module ckan.model.package:340 in diff
<< display_id = to_obj_rev.tag.name
elif obj_class.__name__ == 'PackageExtra':
display_id = to_obj_rev.key
else:
display_id = related_obj_id[:4]
>> display_id = to_obj_rev.key
AttributeError: 'NoneType' object has no attribute 'key'
CGI Variables
DOCUMENT_ROOT '/htdocs'
GATEWAY_INTERFACE 'CGI/1.1'
HTTP_ACCEPT '*/*'
HTTP_ACCEPT_ENCODING 'gzip'
HTTP_ACCEPT_LANGUAGE 'zh-cn,zh-tw'
HTTP_CONNECTION 'close'
HTTP_HOST 'ckan.net'
HTTP_USER_AGENT 'Baiduspider+(+http://www.baidu.com/search/spider.htm)'
PATH '/usr/local/bin:/usr/bin:/bin'
PATH_INFO '/revision/diff/dbtune-audioscrobbler'
PATH_TRANSLATED '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py/revision/diff/dbtune-audioscrobbler'
QUERY_STRING 'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c'
REMOTE_ADDR '123.125.66.32'
REMOTE_PORT '63767'
REQUEST_METHOD 'GET'
REQUEST_URI '/revision/diff/dbtune-audioscrobbler?diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c'
SCRIPT_FILENAME '/home/okfn/var/srvc/ckan.net/pyenv/bin/ckan.net.py'
SCRIPT_URI 'http://ckan.net/revision/diff/dbtune-audioscrobbler'
SCRIPT_URL '/revision/diff/dbtune-audioscrobbler'
SERVER_ADDR '10.226.226.118'
SERVER_ADMIN '[no address given]'
SERVER_NAME 'ckan.net'
SERVER_PORT '80'
SERVER_PROTOCOL 'HTTP/1.1'
SERVER_SIGNATURE 'Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2 Server at ckan.net Port 80\n'
SERVER_SOFTWARE 'Apache/2.2.9 (Debian) mod_wsgi/2.5 Python/2.5.2'
WSGI Variables
application
beaker.cache
beaker.get_session >
beaker.session {'_accessed_time': 1282385101.4243281, '_creation_time': 1282385101.4243281}
mod_wsgi.application_group 'ckan.net|'
mod_wsgi.callable_object 'application'
mod_wsgi.listener_host ''
mod_wsgi.listener_port '80'
mod_wsgi.process_group ''
mod_wsgi.reload_mechanism '0'
mod_wsgi.script_reloading '1'
mod_wsgi.version (2, 5)
paste.cookies (, '')
paste.parsed_querystring ([('diff', '66a47b9e-232a-4838-8674-66fa1a5c76e1'), ('oldid', 'a99c98be-767a-4e49-9025-2472b2d18b9c')], 'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c')
paste.registry
paste.throw_errors True
pylons.action_method >
pylons.controller
pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.pylons
pylons.routes_dict {'action': u'diff', 'controller': u'revision', 'id': u'dbtune-audioscrobbler'}
repoze.who.logger
repoze.who.plugins {'openid': , 'auth_tkt': }
routes.route
routes.url
webob._parsed_query_vars (GET([('diff', '66a47b9e-232a-4838-8674-66fa1a5c76e1'), ('oldid', 'a99c98be-767a-4e49-9025-2472b2d18b9c')]), 'diff=66a47b9e-232a-4838-8674-66fa1a5c76e1&oldid=a99c98be-767a-4e49-9025-2472b2d18b9c')
webob.adhoc_attrs {'language': 'en-us'}
wsgi process 'Multi process AND threads (?)'
wsgi.file_wrapper
wsgiorg.routing_args (, {'action': u'diff', 'controller': u'revision', 'id': u'dbtune-audioscrobbler'})
}}}",1282553033000000,1287747652000000
501,requirement,,pudo,ckan-v1.2,closed,duplicate,Read-only maintenance mode,"CKAN should have a read-only maintenance mode with a nice little banner on all pages, appropriate REST messages etc. Bonus points if this is triggered via an environment variable and thus can be triggered by the surrounding apache. ",1282554617000000,1282724566000000
507,story,,johnbywater,,closed,fixed,Upgrade a CKAN service,,1282661646000000,1282909852000000
511,requirement,,dread,ckan-v1.3,closed,worksforme,Better warnings and errors when using API,,1282754677000000,1297075354000000
512,story,,dread,,closed,duplicate,User creates package via API with incorrect core fields specified,,1282754750000000,1294917121000000
513,task,,dread,,closed,wontfix,Creating a package with API should warn if there are incorrect fields,"When you create a package using the API, you might get a core field name wrong e.g. 'title' mispelt as 'titel' or 'groups' (which can't be specified in package creation). Currently that field will be ignored by the API. It should warn or give an error, rather than give 200 OK.
",1282754802000000,1302774329000000
514,defect,,dread,ckan-v1.2,closed,duplicate,Inconsistent use of 'location' header in API,"When you create a package then the 'location' header gets set. This doesn't happen for any other domain objects. I think this should be consistent - either none or all.
I've removed the info about the header in the docs in the meantime.",1282757357000000,1282757391000000
516,defect,,dread,ckan-v1.2,closed,fixed,Double-escaped text visible - revision page,"At: http://ckan.net/revision
you can see:
""Next »"" on pager
This doesn't seem to be a problem with earlier (1.0.1) version.",1282829539000000,1288002933000000
533,requirement,,johnbywater,ckan-v1.3,closed,invalid,The performance of CKAN shall scale,"CKAN uses a database. We can put the database on another machine and then clone the CKAN machine. Then there are two machines calling the database and converting SQL results into HTTP responses.
How many machines can you have before the database becomes the bottleneck? At that point, what QoS can be obtained with normal hardware?
Afterwards, can CKAN be made to scale further than this with a little development? Can its Postgres database be mirrored in realtime? Or can we change SQL writes (create/updates) to write to many databases?
Are there any other options for scaling the persistence mechanism?",1282910499000000,1292957374000000
535,defect,,dread,ckan-v1.2,closed,duplicate,genshi error when logged into sl.ckan.net,Genshi exception when rendering the page whilst logged in to sl.ckan.net.,1283165774000000,1283167040000000
544,requirement,,pudo,ckan-v1.3,closed,duplicate,Backport facet browsing to CKAN 1.2,"This is in IATI, would be nice to have in generic CKAN.
* http://bitbucket.org/okfn/iati/changeset/378431974c76",1283267292000000,1291638966000000
557,defect,,pudo,ckan-v1.3,closed,invalid,Remove siblings from link_to/from relationship display,Displaying siblings is not needed for link_to/from.,1283542595000000,1297075159000000
558,defect,,pudo,ckan-v1.2,closed,fixed,"Sort extras, relationships by key/object alphabetically",,1283542680000000,1283781883000000
561,defect,,pudo,ckan-v1.2,closed,duplicate,Deleted packages are returned in the API,"Anja is reporting this, severe bug, I think.",1283775578000000,1283775711000000
562,defect,,pudo,ckan-v1.2,closed,fixed,OpenID Login fails using Yahoo and Google,"reported by LOD folks.
subsidary bug: error messages from OpenID providers are not shown on the form return page. ",1283941119000000,1284063574000000
564,requirement,,johnbywater,ckan-v1.3,closed,wontfix,The system shall support location based search,,1284033829000000,1294412714000000
567,story,,johnbywater,ckan-v1.2,closed,,Post new harvest job for given harvest source,,1284039943000000,1284689374000000
573,story,,johnbywater,,closed,duplicate,Add metadata entity to harvesting queue,,1284045353000000,1284220987000000
574,story,,johnbywater,,closed,duplicate,Create UKLII package with attributes from remote metadata record,,1284045805000000,1284222410000000
575,requirement,,johnbywater,ckan-v1.2,closed,fixed,The system shall notify remote metadata harvesting errors,,1284047959000000,1288012749000000
577,story,,johnbywater,ckan-v1.2,closed,,Get remote metadata harvest job errors,,1284209381000000,1285348049000000
578,story,,johnbywater,ckan-v1.2,closed,,Get remote metadata harvest job,,1284209407000000,1284493202000000
579,story,,johnbywater,ckan-v1.2,closed,,Delete remote metadata harvest job,,1284209440000000,1285260356000000
580,story,,johnbywater,,closed,duplicate,Write (create or update) CKAN package for metadata document,,1284210730000000,1284223068000000
582,task,,johnbywater,ckan-v1.2,closed,fixed,Domain object class for harvest source,,1284211104000000,1284261402000000
583,task,,johnbywater,ckan-v1.2,closed,fixed,Define fieldset object for harvest source forms,,1284211169000000,1284261424000000
584,task,,johnbywater,ckan-v1.2,closed,fixed,Migration script to add harvest source table,,1284211204000000,1284483760000000
585,task,,johnbywater,ckan-v1.2,closed,fixed,Method on API controller to return harvest source create form,,1284211302000000,1284261448000000
586,task,,johnbywater,ckan-v1.2,closed,fixed,Routing configuration for harvest source create form API resource,,1284211336000000,1284261447000000
587,task,,johnbywater,ckan-v1.2,closed,fixed,Documentation for getting harvest source create form from API,,1284211364000000,1284492381000000
588,task,,johnbywater,ckan-v1.2,closed,fixed,Method on API controller to accept valid harvest source create form submission,,1284211575000000,1284318555000000
589,task,,johnbywater,ckan-v1.2,closed,fixed,Method on API controller to reject invalid harvest source create form submission,,1284211620000000,1284318561000000
590,task,,johnbywater,ckan-v1.2,closed,fixed,Documentation for submitting harvest source create form submission to API,,1284211659000000,1284492397000000
591,task,,johnbywater,ckan-v1.2,closed,fixed,Routing configuration for harvest source entity API resource,,1284212082000000,1284342925000000
592,task,,johnbywater,ckan-v1.2,closed,fixed,Methods to return harvest source entity as JSON,,1284212338000000,1284342904000000
593,task,,johnbywater,ckan-v1.2,closed,fixed,Method on API controller to return remote metadata edit form,,1284212498000000,1285198853000000
594,task,,johnbywater,ckan-v1.2,closed,fixed,Routing configuration for remote metadata edit form API resource,,1284212532000000,1285198859000000
595,task,,johnbywater,ckan-v1.2,closed,fixed,Documentation for getting remote metadata edit form from API,,1284212572000000,1285198863000000
596,task,,johnbywater,ckan-v1.2,closed,fixed,Method on API controller to accept valid remote metadata edit form submission,,1284212662000000,1285198874000000
597,task,,johnbywater,ckan-v1.2,closed,fixed,Method on API controller to reject invalid remote metadata edit form submission,,1284212689000000,1285198885000000
598,story,,johnbywater,ckan-v1.2,closed,,List remote metadata entities for given publisher,,1284213730000000,1286200795000000
599,task,,johnbywater,ckan-v1.2,closed,fixed,Routing configuration for listing remote metadata entities for a given publisher,,1284213814000000,1284347290000000
600,task,,johnbywater,ckan-v1.2,closed,fixed,Method on API controller for listing remote metadata entities for a given publisher,,1284213854000000,1284347296000000
601,task,,johnbywater,ckan-v1.2,closed,fixed,Documentation for listing remote metadata entities for a given publisher via API,,1284213928000000,1284493045000000
602,task,,johnbywater,ckan-v1.2,closed,fixed,Method on API controller to accept valid harvest job,,1284217587000000,1284336801000000
603,task,,johnbywater,ckan-v1.2,closed,fixed,Method on API controller to reject invalid harvest job,,1284217624000000,1284338918000000
604,task,,johnbywater,ckan-v1.2,closed,fixed,Domain object class for harvest job,,1284217689000000,1284326615000000
605,task,,johnbywater,ckan-v1.2,closed,fixed,Routing configuration for harvest job register,,1284217966000000,1284336788000000
606,task,,johnbywater,ckan-v1.2,closed,fixed,Methods to return harvest job as JSON,,1284218102000000,1284344127000000
607,task,,johnbywater,ckan-v1.2,closed,fixed,Routing configuration for harvest job entity,,1284218136000000,1284344113000000
608,task,,johnbywater,ckan-v1.2,closed,fixed,Routing configuration for register of harvest jobs with error status,,1284218378000000,1285347281000000
609,task,,johnbywater,ckan-v1.2,closed,fixed,Methods to return harvest jobs with error status as JSON,,1284218437000000,1285347285000000
618,task,,johnbywater,ckan-v1.2,closed,fixed,Domain object class for metadata document,,1284219669000000,1284598039000000
619,task,,johnbywater,ckan-v1.2,closed,fixed,Get HTTP response for given resource locator,,1284220327000000,1284675079000000
620,task,,johnbywater,ckan-v1.2,closed,fixed,"Identify remote metadata entity as metadata document, WAF, or CSW",,1284220527000000,1284689400000000
624,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute 'title' from local copy of metadata document,,1284221083000000,1284675746000000
625,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute 'abstract' from local copy of metadata document,,1284221101000000,1284675753000000
626,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute 'point of contact' from local copy of metadata document,,1284221117000000,1285443038000000
627,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute 'guid' from local copy of metadata document,,1284221134000000,1284598028000000
628,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute 'bounding-box' from local copy of metadata document,,1284221156000000,1285443047000000
629,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute 'keywords' from local copy of metadata document,,1284221191000000,1285443055000000
630,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute 'use constraints' from local copy of metadata document,,1284221210000000,1285443069000000
631,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute 'resource-locator' from local copy of metadata document,,1284221223000000,1284675741000000
632,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute 'topic category' from local copy of metadata document,,1284221246000000,1285443078000000
633,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute '' from local copy of metadata document,,1284221272000000,1285443112000000
634,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute '' from local copy of metadata document,,1284221284000000,1285443115000000
635,task,,johnbywater,ckan-v1.2,closed,fixed,Read attribute '' from local copy of metadata document,,1284221305000000,1285443119000000
636,task,,johnbywater,ckan-v1.2,closed,fixed,Method on API controller to delete given harvest job,,1284221790000000,1285260342000000
637,task,,johnbywater,,closed,invalid,Read UKLII package data from given metadata document,,1284222536000000,1284223104000000
638,task,,johnbywater,ckan-v1.2,closed,fixed,Documentation for getting harvest source entity from API,,1284225805000000,1284491185000000
639,task,,johnbywater,ckan-v1.2,closed,fixed,Documentation for posting harvest job to API,,1284226008000000,1284491239000000
640,task,,johnbywater,ckan-v1.2,closed,fixed,Documentation for getting harvest job entity from API,,1284226036000000,1284491233000000
641,task,,johnbywater,ckan-v1.2,closed,fixed,Documentation for getting harvest job errors from API,,1284226049000000,1285348033000000
642,task,,johnbywater,ckan-v1.2,closed,fixed,Migration script to add harvest job table,,1284226154000000,1284483747000000
643,task,,johnbywater,ckan-v1.2,closed,fixed,Migration script to add metadata document table,,1284226255000000,1284675721000000
644,task,,johnbywater,ckan-v1.2,closed,fixed,Documentation for deleting harvest job entity,,1284226402000000,1285260346000000
646,requirement,,johnbywater,,closed,wontfix,Web UI shall support registering harvest sources,,1284740303000000,1294413500000000
648,requirement,,pudo,ckan-v1.3,closed,fixed,Enable lock-down of package creation," - copy exisiting tests, modify authz in setUp, adapt and extend tests
* Problem: default_role_actions is read by init_db
* Solution: nuke db after monkey-patching
* role = model.Role('Reader'), del role.actions[...]
self.PRE_MAUTHZ_RULES = copy(mauthz.default_role_actions)
mauthz.default_role_actions.remove((Role.READER, Action.CREATE))
#raise Exception(mauthz.default_role_actions)
model.Session.remove()
model.repo.rebuild_db()
3. Start from the functional
* Both the api controllers and wui controllers to do
* tests/function/test_authz.py extended to check CREATE in lockedDown mode. fails nicely.
http://bitbucket.org/pudo/ckan-authz2
is_authorized(user, Action.Create, model.Package)
-> Doing this will put 'Package' in the context field of the user_object_role table. This will trigger SQLAlchemy to attempt a join towards PackageRole in all queries. Since for class-level role assignments there never is a PackageRole join table entry, this will never return any results.
* cf branch ""classes"": http://bitbucket.org/pudo/ckan-authz2/src/6fd0475e0c66
* No possibility of including ""instance"" column in inheritance decision.
* Even if this can be cirumvented, it must happen on a per-query level and would require a major re-modelling.
* have a ckan install that would not allow visitors to either list packages or list groups
* two paths: create the listing, but for each group/pkg decide that you
cannot show this
* lock down the whole page (/package/list)
- this is class-based, not object-based
is_authorized(user, Action.Package_Create, model.System())
is_authorized(user, Action.Group_Create, model.System())
[Separating Package and Group roles may be useful going forward: PackageEditor, GroupEditor etc]
4. Find a standard way to lock down classes
* possibly add default rows in user_object_role
* introduce lock-down into controllers: group new, package new, REST equivalents,
* confirm tests",1284889519000000,1284889656000000
649,story,,pudo,,closed,fixed,Introduce Authorization Groups in model,"Authorization Groups:
* Rename UserObjectRole to SubjectObjectRole
* Create database migration to do the rename in DB
* Make sure all the tests are passing again
(Doing this first seems more stable since we first want to ""re-gain"" exisiting functionality)
* Create authz.py and model tests for authorization_group-based auth
* tests: functional: create fixtures, do actual requests
lib-tests: test for membership addition, removal, group-right additions affecting all members, group-right removal, group rights affecting non-members.
# * Adapt model to get these passing
* Integration into existing controllers should be negligible but may want ot have tests
- do not want that. would mean design is broken
* Need to create a new controller for editing authz groups (will have same permissions as any other object -- Reader, Editor, Administrator)
",1284889884000000,1284889983000000
651,requirement,,pudo,,closed,fixed,Improve javascript helper for package/user addition on groups/authzgroups,This needs to be abstracted and we still need to be able to remove items before saving. ,1284972211000000,1311171104000000
652,requirement,,pudo,ckan-v1.2,closed,fixed,Allow for addition and removal of groups on package/edit,"Much more natural place to do this, even at the risk of overloading the package edit form. ",1284998134000000,1284998201000000
653,requirement,,dread,ckan-backlog,new,,Trackback links for packages,"When people link to a package, a track-back link is auto-created. (Similar system as for blogs).
As suggested by Tim Davies:
Allowing some form of ‘track back’ against datasets
When a non-technical user comes to look at a dataset it would be really useful for them to be able to see if anyone has created an interface interpretation of it already.
I found quite a few cases in research of end-users struggling to make sense of a dataset when good interfaces to that data had already been built and blogged about, but without there being any link from the dataset listing to those data uses.
Accepting track backs could also make it easier for technical users to find blog posts / shared code etc. relating to a given dataset.
",1285062025000000,1339774636000000
654,bug,,johnbywater,ckan-v1.2,closed,,Harvest sources and jobs should return 404 when missing (not 500),,1285170717000000,1285254097000000
655,story,,johnbywater,ckan-v1.2,closed,,Return status code 404 when harvest source is not found,,1285252399000000,1285254088000000
656,story,,johnbywater,ckan-v1.2,closed,,Return status code 404 when harvesting job is not found,,1285252429000000,1285254084000000
657,task,,johnbywater,ckan-v1.2,closed,fixed,Fix harvesting job entity get method to default to returning 'None' rather than raising an exception,,1285252536000000,1285254068000000
658,task,,johnbywater,ckan-v1.2,closed,fixed,Fix harvest source entity get method to default to returning 'None' rather than raising an exception,,1285252579000000,1285254072000000
660,requirement,,dread,,closed,invalid,Check CKAN instance works,As an admin I want to check a CKAN instance works having just upgraded it or configured it.,1285348463000000,1311183115000000
661,task,,dread,ckan-v1.3,closed,fixed,Create CKAN site tester,"From Rufus:
{{{
'blackbox' testing command (read-only only would be fine) -- very
useful for upgrade on sunday
On blackbox thinking of something like:
paster test-site http://hmg.ckan.net {package-name}
Or (maybe better):
paster test-site --site-spec {myconfig.json}
my-config is a json file with info for testing (could overload
--config rather than site-spec):
site-url: http://hmg.ckan.net/
package-name:
package-title:
...etc ...
Then:
Test front-page
Test /revision/
Test /package/{page} -- check title
Test /package/history
Test /api/rest/package/ ...
Test /api/search/package ...
etc
}}}",1285348558000000,1288271266000000
666,requirement,,johnbywater,ckan-v1.2,closed,fixed,The system shall support updating packages from new versions of previously harvested dataset or service metadata,,1285590374000000,1288013526000000
673,task,,johnbywater,,closed,,Construct and send CSW GetRecordById request,,1286214712000000,1286786758000000
674,task,,johnbywater,,closed,,Extract metadata documents from CSW GetRecordById,,1286214780000000,1286787032000000
693,requirement,,wwaites,ckan-v1.2,closed,fixed,Support ETag in cache decorator,... and make cache-control header optional,1286831051000000,1286831188000000
714,bug,,johnbywater,,closed,fixed,DGU package form shall have a read only field for ??? attribute,"So that Drupal doesn't need to make any adjustments to the form, and risk data being lost on submission.",1287581408000000,1291733788000000
729,bug,,pudo,,closed,fixed,"Assets to be loaded from assets.okfn.org, not m.okfn.org",Move from hetzner box to s3. ,1287736519000000,1288012854000000
730,task,,rgrp,,closed,duplicate,Back up package data from all CKAN packages to storage.ckan.net,"Write a worker that scans all packages in a ckan instance and uploads the data to storage.ckan.net.
* Naming scheme?
* Bucket: {ckan-instance-id}-{package-name}? {ckan-instance-id}-{package-id}?
* What happens if names change
* File: filename? hash?
* Store hash back on ckan instance?",1287737109000000,1291139609000000
737,enhancement,,dread,ckan-backlog,new,,Markdown syntax summary page,"I suggest we produce a quick Markdown cheat-sheet page, showing the key runes: e.g. create a title and quote some text. This page can link to the full Markdown docs for advanced users.
A user going to the Markdown docs that we link will have to read a couple of pages of the raison-d'etre of Markdown before he gets to the syntax. And it's not very easy to read, and being white on black it looks like proper geek stuff.
",1287766749000000,1323170239000000
741,enhancement,,ollyc,ckan-v1.2,closed,fixed,Extension / Plugin system for CKAN,"Create plugin system using
Specification in CEP-003 (browser:doc/cep/cep-003.txt) and
",1287996890000000,1291989194000000
743,story,,johnbywater,,closed,wontfix,Create harvest source registration via Web UI,,1288012860000000,1294412807000000
744,story,,johnbywater,,closed,wontfix,Update harvest source registration via Web UI,,1288012960000000,1294412813000000
745,requirement,,johnbywater,ckan-v1.3,closed,wontfix,Web UI shall accept requests for harvesting jobs,,1288013148000000,1294412820000000
746,story,,johnbywater,,closed,wontfix,Request new harvesting job via Web UI,,1288013198000000,1294412827000000
747,story,,johnbywater,,closed,wontfix,Track harvesting jobs via Web UI,,1288013217000000,1294412834000000
748,story,,johnbywater,,closed,duplicate,Link new sample package to previous sample package of continuous series,,1288013849000000,1294412976000000
749,story,,johnbywater,,closed,duplicate,Fold up continuous series in search results behind newest sample package,,1288014002000000,1294412986000000
751,story,,johnbywater,,closed,duplicate,Get harvested document for a given package,,1288014518000000,1288014616000000
761,story,,dread,,closed,invalid,Read-only CKAN for maintenance,Administrator wants to upgrade CKAN or move it to another server. During this time the database is being administered and either edits are lost or can't be done.,1288090334000000,1288091977000000
762,story,,dread,,closed,invalid,Permanently Read-Only CKAN instance,"A CKAN is used just for distributing metadata. Updates may still arrive through direct db manipulation, e.g.:
* another (but writable) CKAN instance is connected to the same db
* restoring database dumps from another CKAN db
",1288090578000000,1288091982000000
763,enhancement,,dread,ckan-future,assigned,,Read-only mode - Setup,"Admin configures entering read-only mode in one of two places:
* CKAN config file (e.g. ckan.ini)
* environment variable from Apache config
Once enabled, no writes can occur to the database (including user ratings and other usage stats).",1288091506000000,1338206204000000
765,enhancement,,dread,ckan-backlog,assigned,,Read-only mode - API usage,"All writes to the API are captured and you are returned an error explaining the reason.
Possible errors:
* 503 temporary maintenance
* 403 forbidden (if server if permanently read-only)",1288091897000000,1338206123000000
771,story,,johnbywater,ckan-v1.3,closed,worksforme,Run CLI help command without arguments,Should return something sensible (currently reports an error).,1288190318000000,1294413256000000
772,story,,johnbywater,ckan-v1.3,closed,duplicate,Run CLI harvester command without arguments,Should return help for the harvester (currently crashes).,1288190373000000,1294413313000000
773,task,,johnbywater,ckan-v1.3,closed,duplicate,Fix CLI help command to return CLI help message,,1288190475000000,1294413400000000
774,task,,johnbywater,ckan-v1.3,closed,duplicate,Fix CLI harvester command to return harvester help message,,1288190522000000,1294413412000000
796,defect,,pudo,ckan-v1.3,closed,invalid,AlphaPage for non-latin languages,"The AlphaPage implementation assumes that all packages start with a latin character. This is not true for Russian, Chinese, Arabian and Persian languages etc. We should either use a unicode solution (i.e. display a full class of characters based on the current locale) or use generic paging instead.
",1289296662000000,1291640040000000
807,enhancement,,rgrp,ckan-v1.3,closed,wontfix,Handle robot traffic better,"We get a lot of robot traffic. Notice that robots are visiting package edit pages. Suggest:
* Add rel=nofollow to various links
* Throttle some robots (not all seem to respect robots.txt)
",1289899826000000,1297075372000000
809,requirement,,pudo,ckan-v1.3,closed,wontfix,Link to the Wiki pages in package form,"This would allow for a community-driven definition of field values, conventions, etc.
Plus: create wiki.ckan.net ",1289992231000000,1297075561000000
811,defect,,cygri,ckan-backlog,new,,Extra field editing form layout breaks when there are long field names,The layout of the editing section for extra fields breaks when a field name is slightly too long. Field names jump over to the right. See http://ckan.net/package/edit/dbpedia for examples.,1289994812000000,1323170289000000
812,defect,,cygri,ckan-backlog,new,,Package edit form only allows three extra fields,"= Rationale =
The package edit form is restricted to three extra fields. To enter more than three fields, one has to save the package and hit edit again (or hit preview).
= Implementation =
A mechanism similar to the one for resources (where you can add lines as you go) would solve this. So, have a button that adds more extra field rows via JS. (Extra fields don't need up/down buttons that the Resource table has)
Nice to have: a blank field is added when you tab from the last filled-in field in the table.",1289995010000000,1311176917000000
813,requirement,,cygri,ckan-v1.3,closed,fixed,Add link to “Register new package” to the homepage,"A link to the “register new package” page would be handy somewhere on the front page. As a frequent editor, I often go to http://ckan.net in order to create a new package. I don't want to hunt around for the link to the “new package” form.
The link could go into the bar next to “About / Statistics”. Or into the “More information” sidebar. Neither of these truly make sense, but it's a frequently accessed link and that's reason enough to put it somewhere onto the frontpage.
",1289995253000000,1294411047000000
814,requirement,,cygri,ckan-v1.3,closed,fixed,Have an “About CKAN” link on every page,"A link “About CKAN” pointing to http://ckan.net/about should be visible on every page. This is important because people often land on subpages and may have trouble figuring out what CKAN is.
I would put this link into the main navigation bar, on the very right next to “Revision History”. Then I would also remove the “Home” item from the main navigation bar because it is redundant. The CKAN logo is already a link to the homepage. The only other subpage in the “Home” section is “Statistics” and that's already linked from the homepage sidebar.
But anywhere else would be fine as well.",1289995821000000,1294411109000000
815,requirement,,cygri,ckan-v1.3,closed,duplicate,Autocomplete for the search field,"The search field (on the homepage and in the top right corner of each page) should have autocomplete for package name. If a package name is selected, it should not do a search but go straight to the package page.",1289995951000000,1294410951000000
817,requirement,,cygri,ckan-v1.3,closed,worksforme,Proposed redesign of Resources table on package pages,"There are some problems with the Resources table on package pages:
1. The column labelled “URL” doesn't contain URLs, but just the word “Download”.
2. The links labelled “Download” often are not download links, but something else (example links, API endpoints, documentation links).
Attached is a screenshot for a proposed redesign. The changes are:
1. Move the Format column to the right of the Description column (description is more important)
2. Make the description a clickable link
3. Drop the “URL/Download” column because it's now redundant
4. Rename “Description” to “Resource” (not that important)",1289997384000000,1297073724000000
818,requirement,,cygri,ckan-backlog,new,,Rethinking the author and maintainer fields,"The semantics of the Author and Maintainer fields are really unclear at the moment. This leads to very inconsistent usage. Also, perhaps Name and Email are not the only fields that are needed for a contact.
Here is a table that shows the current usage of these fields in CKAN:
http://richard.cyganiak.de/2010/ckan/ckan-ppl.html
We note several problems:
* Author and Maintainer are often the same
* Author and Maintainer are often used interchangeably
* People really want to specify URLs for the contacts and stick them into random places because there is no field for it
* Multiple comma-separated names in a single field
I'm not sure what to do about this, but a redesign is necessary in my opinion.
Some ideas:
* Remove the maintainer field?
* Make really clear that Author doesn't refer to the metadata on CKAN, but to the original data
* Add an “author URL” field?",1290003524000000,1339774621000000
819,defect,,cygri,ckan-backlog,closed,fixed,Tag autocompletion widget broken,"The widget for tag autocompletion is broken in various ways.
For example, if I edit a package that is tagged ""music"", and just tab through the form fields to get to the Author field, then the widget changes to ""industrial-music"" as I tab through it.
Or if the tag is ""foo bar"" and I hit alt-right to jump to the end of the text field (with the intent of adding a third tag), then the contents change to ""foo barbecue"".",1290004121000000,1323171435000000
820,defect,,dread,,closed,fixed,Exception browsing /user/apikey when not logged in,"Should fail gracefully (not the existing 500 error) when you go to ckan.net/user/apikey (manually typing in the url) but haven't logged in.
{{{
WebApp Error: : 'NoneType' object has no attribute 'apikey'
URL: http://test-hmg.ckan.net/user/apikey
}}}",1290006079000000,1311325226000000
821,defect,,dread,ckan-v1.4-sprint-2,closed,worksforme,Some resources have disappeared in CKAN,"Simon has spotted 31 packages whose resources have disappeared in CKAN, but still appear in the Drupal front-end.
Here are the details:
* in CKAN's package view (in web interface, API and dump) no resources display
* but they WERE created in CKAN, as shown by the revision diffs.
* they are in the Drupal front-end
So these resources must have been in the CKAN API at some point and then disappeared without trace/revision to alert Drupal.
Packages affected: anti-social-behaviour-orders-1999-2007
asylum-applications-jan-mar-2009
control-of-immigration-quarterly-statistical-summary-united-kingdom-2009-october-december
coroners-statistics-england-and-wales
courts-statistics-user-survey-england-and-wales
court-statistics-company-insolvency-and-bankruptcy-england-and-wales
court-statistics-england-and-wales
court-statistics-mortages-and-landlord-possession-england-and-wales
crime-in-england-and-wales
crime-statistics-local-reoffending-england-and-wales
crime-statistics-prison-and-probation-england-and-wales
crime-statistics-reoffending-of-adults-england-and-wales
crime-statistics-reoffending-of-juvenilles-england-and-wales
data_gov_uk-datasets
digest-uk-energy-statistics-2008
directgov-central-hottest-pages-monthly
directgov-central-internal-search-terms-monthly
directgov-section-visits-monthly
electricity-consumption-2007
electricity-gas-consumption-2007
energy-consumption-uk-2008
final-energy-consumption-2007
foi-statistics-uk-central-government
fuel-poverty-statistics-2007
gas-consumption-2007
gb-reported-bicycling-accidents
gb-road-traffic-counts
gb-traffic-matrix
greenhouse-gas-emissions-2008
high-level-indicators-energy-use-2006
judicial-and-court-statistics-england-and-wales
laboratory-tests-and-prices
local-authority-carbon-dioxide-emissions-2007
magistrates-courts-statistics-survey-england-and-wales
monthly-energy-prices
monthly-energy-trends
ni_012_refused_and_deferred_houses_in_multiple_occupation_hmos_licence_applications_leading_to_immig
ni_013_migrants_english_language_skills_and_knowledge
ni_023_perceptions_that_people_in_the_area_treat_one_another_with_respect_and_consideration
ni_024_satisfaction_with_the_way_the_police_and_local_council_dealt_with_anti-social_behaviour
ni_025_satisfaction_of_different_groups_with_the_way_the_police_and_local_council_dealt_with_anti-so
ni_026_specialist_support_to_victims_of_a_serious_sexual_offence
ni_029_gun_crime_rate
ni_031_re-offending_rate_of_registered_sex_offenders
ni_032_repeat_incidents_of_domestic_violence
ni_034_domestic_violence_-_murder
ni_036_protection_against_terrorist_attack
ni_038_drug_related_class_a_offending_rate
ni_078_reduction_in_number_of_schools_where_fewer_than_30_of_pupils_achieve_5_or_more_a-_c_grades_at
ni_101_looked_after_children_achieving_5_a-c_gcses_or_equivalent_at_key_stage_4_including_english_an
ni_109_delivery_of_sure_start_childrens_centres
ni_126_early_access_for_women_to_maternity_services
ni_127_self_reported_experience_of_social_care_users
ni_128_user_reported_measure_of_respect_and_dignity_in_their_treatment
ni_181_time_taken_to_process_housing_benefit-council_tax_benefit_new_claims_and_change_events
ni_184_food_establishments_in_the_area_which_are_broadly_compliant_with_food_hygiene_law
ni_185_co2_reduction_from_local_authority_operations
ni_190_achievement_in_meeting_standards_for_the_control_system_for_animal_health
ni_194_air_quality_-_reduction_in_nox_and_primary_pm10_emissions_through_local_authorities_estate_an
other-fuels-consumption-2006
police-use-firearms-england-wales-2007-2008
prison-end-of-custody-licence-releases-and-recalls-england-and-wales
prison-population-england-and-wales
probation-offender-management-caseload-statistics-england-and-wales
probation-statistics-quarterly-brief-england-and-wales
quality-indicators-energy-data-2007
quarterly-energy-prices
quarterly-energy-trends
road-transport-energy-consumption-2007
sentencing-statistics-england-and-wales
statistics-terrorism-arrests-outcomes-2001-2008
ukba-control-of-immigration-statistics-2008
ukba-control-of-immigration-statistics-2008-supplementary-tables
uk-energy-in-brief-2008
uk-energy-sector-indicators-background-2008
uk-energy-sector-indicators-key-supporting-2008
uk-exportcontrollists
uk-exportcontrol-sanctions
uk-export-control-statistics
uk-glossary-exportcontrol
uk-ipo-offences
weekly-fuel-prices",1290015776000000,1298486642000000
822,enhancement,,Javier de la Cueva ,,closed,fixed,Docs: Configuration as site-available instead of site-enabled for apache,"The documentation for setting up Apache uses sites-enabled instead of sites-available and a2ensite demo.ckan.net
",1290107103000000,1290506354000000
830,enhancement,,rgrp,,closed,fixed,Themes for CKAN,"Already have the basic capacity using *_extras paths for templates and public directory (see http://wiki.okfn.org/ckan/doc/theme).
However, would be nice to have a proper 'theme' structure (e.g. would allow for theme switching) that also avoids directly polluting 'public'. Could be inspired by: http://packages.python.org/Flask-Themes/",1290760612000000,1311180263000000
834,task,,Alexander,ckan-v1.3,closed,worksforme,Searching in CKAN,"Hello.
I've installed stable CKAN v1.1 from PyPI.
I can't find any docs about using CKAN API in order to query packages.
Query ./api/search/package?q=str works fine, but with extra parameters, such as limit, offset, fullinfo, order_by, search_notes, don't. Should I use new version for this? How can I perform this query via Ckanclient? Is it possible?
Also I'm interested how to find open-licensed files? I tied URL ./api/search/package?q=str&open_only=1&downloadable_only=1 and Ckanclient:
result = ckan.package_search('str', {'open_only': 1, 'downloadable_only': 1})
As result nothing found.
Thanks.",1290769564000000,1291633657000000
836,enhancement,,rgrp,ckan-v1.3-sprint-1,closed,fixed,Use site_url config option in templates,"A user reported wanting to install ckan at a suburl. Apparently this does not work well because some urls are hard-coded: http://lists.okfn.org/pipermail/ckan-discuss/2010-November/000726.html
Instead we should use the site_url config option where necessary.
Cost: 1h",1291132434000000,1302882808000000
837,enhancement,,rgrp,ckan-backlog,new,,CKAN integration with freebase gridworks / google refine,"Thread: http://lists.okfn.org/pipermail/ckan-discuss/2010-November/000718.html
=== Scenario 1 ===
1. User installs Refine and CKAN extension for refine
2. On booting refine and asked to load data they can choose from any data package on CKAN.net (or any other CKAN instance)
3. They edit the dataset on Refine
4. On save (or perhaps as a separate option) they are prompted as to whether they wish to sync the dataset back to CKAN (either as a new package or as a new resource on the existing package)
NB: for the dataset sync back some form of ""CKAN"" storage would be
required (we already have storage.ckan.net running but a closer
integration would be required)
=== Scenario 2 ===
1. User visits a package on CKAN.net (or another CKAN instance)
2. There is a button on the page ""View and edit this dataset in Google Refine""
3. Click button -- ask them if they have Google refine installed
* Yes: instructions for loading dataset into refine
* No: load dataset in hosted version of google refine (we could run this)
4. User edits dataset and hits save. As in previous scenario they are prompted to sync the dataset.",1291140609000000,1339774605000000
843,enhancement,,wwaites,,closed,fixed,user list ui for group curation," * perhaps we should show the openid as well to distinguish between users with the same name.
* perhaps on account creation, the user should be redirected to their personal details page to encourage them to fill in a human readable name.
* also the list is much too long. can we make it work some other way? javascript is an option, but must be careful to fail gracefully when the browser does not support it.",1291484245000000,1319721601000000
845,enhancement,,dread,ckan-v1.3-sprint-1,closed,fixed,Required fields,"Allow a form to specify fields that are required. Currently name and resource url are required.
* Show required fields with asterisks by the input box.
* Resource fields to be included in this.",1291634995000000,1291723492000000
848,enhancement,,pudo,ckan-v1.3-sprint-1,closed,fixed,Ability to hide extras fields on package read,for helsinki regional infoshare. ,1291714316000000,1291751275000000
849,defect,,pudo,,closed,duplicate,Import extra CSS from main template,"The CSS @import directive at the bottom of the main CKAN css file is not understood in all browsers, move this to a seperate import. ",1291714923000000,1291715179000000
857,defect,,dread,,closed,invalid,Tests for CLI,Test coverage for ckan.lib.cli is 34%. Need to improve that.,1291724131000000,1311182581000000
870,defect,,memespring,ckan-v1.3-sprint-1,closed,fixed,Menu items not selecting correctly,"Since the main menu items have changed (part of the UI redesign) the selected items are not functioning correctly.
Specifically if Home / About or Add Package / Search are selected both menu items are highlighted.
This is because the code assumes only one top-level menu item per controller.",1292243583000000,1294914243000000
871,defect,,nils.toedtmann,,closed,invalid,Check whether localhost-only exim installtions need upgrading too,"The infamous [http://www.exim.org/lurker/message/20101207.215955.bb32d4f2.en.html exim bug] only needs one mail with prepared headers to travel through a exim system infect it. All local processes could do that, and some services (e.g. cron, webapps) send messages and might be convinced by malicious remote users to produce evil headers.
We should either rule out that this could happen on our systems, or upgrade all exims regardless of whether they are localhost-only or not.
BTW did we already run a rootkit checker like [http://rkhunter.sourceforge.net/ Rootkit hunter] on eu1? If not we should maybe do it now - there was already an exploit out in the wild. ByteMark has (a) already observed infections and (b) notified us because they remotely fingerprinted our mailer to be exim<4.70 (our EHLO banner contains the exim version), just as anyone could.
",1292264117000000,1296340558000000
874,enhancement,,pudo,ckan-v1.3-sprint-2,closed,fixed,Extract solr search backend into an extension,"Move the solr SearchBackend to a seperate code base, remove any mentions of the CKAN model. ",1292844422000000,1293057151000000
880,defect,,dread,ckan-v1.3-sprint-2,closed,fixed,ONS loader tripping up on deleted packages,"Deleted packages with the same title and extra fields as an active package trip up the loader. It finds them in the search, but due to ckanclient not using the apikey when you 'get' the package to examine it further, it causes an exception.",1292939379000000,1292945137000000
893,defect,,pudo,ckan-v1.4,closed,wontfix,ExtrasField should not overwrite more specific extras,"At the moment, ExtrasField cannot be used in conjunction with more specific extra fields, such as TextExtraField or SuggestTextExtraField.",1294135783000000,1298293527000000
895,defect,,memespring,ckan-backlog,new,,Add version number (or simular) to css/js includes query string,"Updates to css after a new deploy don't come through without a hard refresh. Adding the version number to the include urls will solve this e.g.
mycssfile.css?v=12345678",1294343382000000,1339774593000000
896,story,,thejimmyg,,closed,duplicate,Distributed Data and Syncing Between CKAN Instances,"Based on the ideas in:
* DistributingChanges
* SyncingInstances
Merging the tickets #296, #297, #298, #299 into this one.
Related ideas include:
* Using dcat to exchange data
* Using couchDB to have a stub package record synchronised between multiple CKAN instances, each linking to the main record in the correct instance
This should be considered a wishlist item for the timebeing.
",1294406999000000,1300217375000000
900,defect,,dread,,closed,fixed,Intermittent error with new revision not having id,"There are still some intermittent errors due to objects not having a revision_id, if you save an object at the same time as the revision. We add the uuid earlier to fix this. https://bitbucket.org/kindly/vdm/changeset/0050d023ca4e.
From David Raznick",1294659696000000,1294916587000000
907,enhancement,,rgrp,ckan-v1.3-sprint-3,closed,fixed,Data previews: preview package resource data in the WUI,"As a user I want to preview a dataset ('resource') before downloading it.
== Implementation ==
This will be implemented as an [wiki:Extensions Extension]
NB: This ticket requires a way to get data out of a remote url (the dataset). We do '''not want to put this data load/streaming into ckan core''' due to a) complexity b) load problems (do not want to take ckan out by accidentally pulling in a 1GB file)
1. Need a 'data proxy' - see ticket:698 and ticket:888
2. Preview using javascript in WUI - do this in an extension http://bitbucket.org/okfn/ckanext-dataapi
",1295261173000000,1297072303000000
909,enhancement,,pudo,ckan-backlog,assigned,,DCat importer for CKAN,"Write an importer that supports most well known variants of DCat in importing the data as CKAN packages. In particular, the following sources should be supported:
* CKANrdf generated exports
* opengov.se RDF (not really DCat)
* Sunlight Nationdatacatalog as harvested by dcat-tools. ",1295265958000000,1346669602000000
924,enhancement,,dread,ckan-backlog,new,,Search box has no search button,"The search box at the top-right of CKAN's page doesn't have a 'go' button. I feel that a larger percentage of users expect a 'go' or 'search' button on the right-hand side of the box to press to start searching. Techies tend to know the keyboard shortcut of pressing 'carriage-return' but it might be better to follow standard practise on this.
Examples with 'search' button: Internet Explorer, Firefox, Google, Amazon, trac
Examples without: ?",1295867533000000,1323170436000000
925,defect,,dread,ckan-backlog,closed,fixed,Change the search box icon to remove the down arrow,Is there a good reason why the search box has a 'down arrow' icon when there is no drop-down menu? Or can this be usefully removed?,1295867593000000,1323168588000000
933,enhancement,,kindly,ckan-v1.4,closed,fixed,get rid of self when it should be cls in classmethods,use pylint,1296174526000000,1297348975000000
934,enhancement,,kindly,,closed,invalid,Key value store for extensions to store data,"We need to allow plugins to store information.
Each plug-in should have its own name space and allow any data to be set.
There are two options.
* make a key value table in ckan that will hold random information.
* use redis (or other key value store)
If in sql suggest schema of form:
* (namespace, obj_id, key, value [, value_type])
* value is json",1296228758000000,1323171047000000
939,enhancement,,rgrp,,closed,fixed,Notification bar at top of site shown to new users pointing them to simple instructions,"On http://getthedata.org/ or stackoverflow there is a pop-up bar for new users that give them some simple instructions.
Talking with users it is clear that some people are not clear how ckan.net (or other sites) work and what they are and are not 'allowed to do. This could help make this better.
== Implementation ==
* Pop-up bar (based on a cookie or just being not logged in?) - uses ticket:938 (message flashing)
* FAQ/instructions page (use the new wiki?)
== Depends ==
* ticket:938
== Estimate ==
* Cost: 1h",1296399008000000,1323171158000000
947,enhancement,,rgrp,,closed,duplicate,Move the assignment of roles to a package on package creation to an extension,"At the moment the assignment of user roles on package creation is done in setup_user_roles method in ckan/model/authz.py and is hard-coded.
This can be a pain to override (you can use a Plugin listening for package create events) and makes some things such as putting system in restricted mode more complex than it should be (see ticket:833).
An elegant solution would be to move this into an Extension this simplifies the code and make it easier for people override (just remove the default extension and plugin your own).",1296557478000000,1314031310000000
948,enhancement,,dread,ckan-future,assigned,,Highlight (to a sysadmin) which packages are deleted,"When a customer logs in as a sysadmin then he/she see all packages, including deleted and pending ones. These are hidden to the average user, but the sysadmin has no idea of this until he clicks on the package and sees at the bottom 'state: deleted'.
It should be more obvious than that on the search view - an icon, message or crossed-out name to packages are deleted.",1296646369000000,1338206280000000
951,defect,,adrian.pohl@…,,closed,invalid,Can't add a package to group,"I can't add a package (e.g. http://ckan.net/package/ub-konstanz) to a group (e.g. http://ckan.net/group/bibliographic). It's neither possible when editing a package (the only group in drop down menu is ""history"") nor on the group page.",1296726886000000,1314031006000000
958,enhancement,,kindly,ckan-v1.5,closed,duplicate,ui for new kind field,Ui for related new field in resource (#957),1296822896000000,1320664462000000
974,defect,,dread,ckan-v1.3,closed,fixed,Document site_description,Configuration option 'ckan.site_description' isn't documented,1297342254000000,1297342599000000
976,defect,,dread,ckan-v1.3,closed,fixed,Documentation references knowledgeforge.net,,1297346649000000,1297346954000000
977,defect,,dread,ckan-v1.3,closed,fixed,db_upgrade errors," * db_upgrade (in ckan/model/__init__) makes call to validate_authorization_setup but forgets about rest of initialisation.
* it uses self.metadata.bind.url when it should be self.metadata.bind it seems (confused by sqlmigrate update?)",1297420558000000,1297420742000000
990,defect,,rene.kapusta,,closed,fixed,umlauts in title / text,"It's not possible to use umlauts (like ""CKAN Österreich"") in the title / text.",1297801036000000,1311180850000000
991,defect,,dread,ckan-v1.3,closed,fixed,Checkbox defaults to True,"Form for new package has CheckboxExtraField checked, when the value is False.
(as used in ckanext-dgu package v3 form)",1298035175000000,1298037717000000
1020,defect,,kindly,,closed,fixed,harvesting doc revision table,add revisioning to harvested document table,1299205012000000,1300196215000000
1029,defect,,kindly,ckan-v1.4-sprint-3,closed,fixed,synchronous search erroring when harvesting is run.,Errors are caused when harvesting documents. This also makes the count not show up correctly on the ckan search page.,1299768337000000,1301311643000000
1034,defect,,dread,,closed,duplicate,Flash message cached," 1. Login
(shows ckan home page with flash message ""Welcome back xyz"")
2. Click away, to ""Add package"" say
3. Click back to ckan ""Home"".
It shows ""Welcome back xyz"" again - it shouldn't, I assume?
",1299845308000000,1320174353000000
1043,defect,,kindly,,closed,fixed,stop sqlalchemy message saying at least one scoped seession already present,see summary.,1300310326000000,1300321033000000
1049,defect,,dread,,closed,invalid,Fix database errors on offener.datenkatalog.at etc.,"We're getting these sorts of exceptions from http://offener.datenkatalog.at/ like we had from ckan.net, which was due to small errors in migration scripts. Is it worth fixing this database, and in fact all the ckans on eu3 at once?
{{{
WebApp Error: : (IntegrityError) duplicate key value violates unique constraint ""group_revision_pkey"" 'INSERT INTO group_revision (id, name, title, description, created, state, revision_id, continuity_id) VALUES (%(id)s, %(name)s, %(title)s, %(description)s, %(created)s, %(state)s, %(revision_id)s, ...
}}}
Thoughts James and Rufus?",1300444912000000,1323169424000000
1057,defect,,dread,,closed,fixed,JSONP parameter isn't escaped,"{{{
$ curl ""http://127.0.0.1:5000/api/rest/package/annakarenina?callback=