{22} Trac tickets (2647 matches)

Results (1401 - 1500 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1416 defect dread dread ckan-sprint-2011-11-07 closed fixed Registering with blank email can cause exception

This might well be a bot, but not POSTing the email field should not cause an exception - it should be 400 and then the sysadmin doesn't get these nags.

http://www.publicdata.eu/user/register
...
Module ckan.controllers.user:79 in register
<<              c.login = request.params.getone('login')
                   c.fullname = request.params.getone('fullname')
                   c.email = request.params.getone('email')
                   if not model.User.check_name_available(c.login):
                       h.flash_error(_("That username is not available."))
>>  c.email = request.params.getone('email')
Module webob.multidict:335 in getone
<<          values were found.
               """
               return self._decode_value(self.multi.getone(self._encode_key(key)))
       
           def mixed(self):
>>  return self._decode_value(self.multi.getone(self._encode_key(key)))
Module webob.multidict:99 in getone
<<          v = self.getall(key)
               if not v:
                   raise KeyError('Key not found: %r' % key)
               if len(v) > 1:
                   raise KeyError('Multiple values match %r: %r' % (key, v))
>>  raise KeyError('Key not found: %r' % key)
KeyError: "Key not found: 'email'"
HTTP_USER_AGENT	'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MRA 4.3 (build 01218); .NET CLR 1.1.4322)'
webob._parsed_post_vars	(MultiDict([('login', 'ahusernames9'), ('fullname', 'ahusernames9'), ('password1', '******'), ('password2', '******'), ('s', 'Sign up')]), <FakeCGIBody at 0xb9d7c5ec viewing MultiDict([('lo...p')])>)
1319533989000000 1320153507000000
#1417 defect dread dread ckan-sprint-2011-10-24 closed fixed Browser language detection doesn't work

In Firefox:

  • Edit | Preferences | Content | Languages - put 'de' at top language
  • Clear cookies
  • Load thedatahub.org
  • it should be in German, but it's in the default language - English.
1319539010000000 1319651617000000
#1418 defect dread dread ckan-sprint-2011-10-28 closed duplicate If default language is not 'en' then you can't switch language to 'en'

e.g. cz.ckan.net has lang=cz but when you click on Home page link "English" it flashes message "Language has been set to: English" but the rest of the page is still Czech.

1319539146000000 1319539691000000
#1419 enhancement dread ckan-sprint-2011-11-07 closed invalid Can't log in via OpenID

I couldn't log into theDataHub with OpenID today. I tried both Google ID and MyOpenID. Both times the login on the remote auth server went fine, but when it returns you to theDataHub you get error "Login failed. Bad username or password."

1319543013000000 1319796164000000
#1420 enhancement johnglover johnglover ckan-sprint-2011-11-21 closed fixed Refactor ckanext-qa to work as a celery task

Currently can run as a celery task launched by paster command.

Still have to add a CKAN plugin so that QA will run when a resource is added or URL changed, and then update the extension controller/templates to look for QA data from the task_status table instead of directly on the resource.

1319626138000000 1320843789000000
#1421 defect dread dread ckan-sprint-2011-10-28 closed fixed Tag cloud recipe broken

home page doesn't have c.facets setup.

Only affects 1.5 branch - ok 1.4.3.1. Was broken in cset:05f60f2d26e8.

1319638575000000 1319639311000000
#1422 enhancement dread dread closed fixed Remove proxy & etag caching

Rufus decided this was more trouble than it was worth.

Removes ckan/lib/cache.py

1319639663000000 1319645722000000
#1423 enhancement markbrough ckan-backlog new Edit resources suggestions
  • Description vs Name - Edit Resources view is showing the name of the package rather than the description, and a lot (all?) of the packages before the upgrade don't have names, so might be good to swap this round again, e.g.: http://thedatahub.org/dataset/edit/iati-registry
  • Moving resources - Moving them up or down the list used to be quite useful if you had a lot of resources that you might want to leave on the resources page, but only one or two that were actually current and that you wanted to draw attention to. This doesn't exist any more on CKAN but I think it would be good to add it back in.
1319641906000000 1338203678000000
#1424 enhancement dread ckan-backlog new Openness notice should be clearer

ckan-discuss discussion suggests changes to the 'openness' indicator ( http://lists.okfn.org/pipermail/ckan-discuss/2011-October/001786.html )

Dataset view page:

  • If there is an explicit but non-OKD compliant license, such as CC-BY-NC, then this should be stated explicitly, perhaps: “This dataset is Not Open. License: Creative Commons Attribution Noncommerical. This is not an open license as it does not meet the Open Knowledge Definition.”
  • If the license is marked as “Other::License Not Specified”, then this should be stated explicitly, perhaps: “This dataset is Not Open. It is published without an explicit license, the publisher reserves all rights to the dataset.”
  • 3. If the license field was left empty by the contributor of the Data Hub record, then again this should be stated explicitly, perhaps: “This dataset is Not Open. The license of this dataset is unknown or unspecified. Start an enquiry on IsItOpenData? »
  • There is a bug so that non-open licenses doesn't have an openness notice.
  • If downloadable resources are not available, this should not affect 'openness' - check this has been removed.
1319648089000000 1319648089000000
#1425 enhancement zephod dread closed fixed Dataset delete button

From Jonathan gray:

At the moment to delete a package you have to go to 'state' and change from 'active' to 'deleted'.

For me this was not very obvious, and someone recently wrote to me about this too.

Its a small issue - but unless you know what 'state: active' refers to, its not obvious what this means, nor that this is where to go if you want to delete a package.

From UI perspective this could be replaced by either a trash can or some text that says 'delete', perhaps with some prompt that explains what happens when you delete a package - explaining that non-admins won't be able to see it, and how you purge, etc.

1319649860000000 1330085282000000
#1426 defect dread dread ckan-sprint-2011-10-28 closed invalid Validation errors cause 409 not 400 response

Validation error is incorrect arguments, so should be a "400 Bad Request" response. "409 Conflict" is for when there is a conflict creating two resources with the same identifier. Yet we get 409 for all validation errors.

1319708619000000 1319709236000000
#1427 enhancement dread dread ckan-sprint-2011-10-28 closed fixed Tags validation needs to limit length to 100 characters

The tag name is limited in length in the db table to 100 characters. If you try and create a longer one you get an exception. This should cause a ValidationError? and tell the user this sensibly.

1319708698000000 1319709925000000
#1428 defect dread dread ckan-sprint-2011-11-07 closed fixed config options not used or documented

Need to ensure config options in the deployment.ini_tmpl match up with doc/configuration.rst. Remove any unused ones too.

1319803594000000 1319804773000000
#1429 enhancement rgrp rgrp ckan-backlog new Provide DOIs for datasets in a CKAN instance

DOI = digital object identifier = http://www.doi.org/

As a Publisher I want a DOI for my dataset so that it can be cited by and linked to by others in a standard and easy way.

Details

  • Probably implement as extension rather than core
1319977305000000 1319977305000000
#1430 defect amercader ckan-sprint-2011-11-07 closed fixed Documents get mixed between SOLR cores

On some occasions (apparently random), the documents indexed in a specific SOLR core get mixed with different site_ids.

E.g: We look for all documents in the testing.iatiregistry.org core, faceted by site_id. We would expect all documents to have site_id = iati_testing, but some of them have site_id = iatiregistry.org

http://okfn-solr.fry-it.com:8080/solr/testing.iatiregistry.org/select?indent=on&version=2.2&q=*:*&fq=+state:active&facet=true&facet.field=site_id

<lst name="facet_fields">
<lst name="site_id">
<int name="iati_testing">265</int>
<int name="iatiregistry.org">255</int>
</lst>
</lst>

If we compare one of the records which disappeared from the "iati_testing" site_id in both the production and testing SOLR cores of the server, the records are exactly the same, including the indexed_ts property:

http://okfn-solr.fry-it.com:8080/solr/iatiregistry.org/select?indent=on&version=2.2&q=id:97d1ab0d-b203-4757-8f4e-a0c84d2f759f&facet=true&facet.field=site_id

http://okfn-solr.fry-it.com:8080/solr/testing.iatiregistry.org/select?indent=on&version=2.2&q=id:97d1ab0d-b203-4757-8f4e-a0c84d2f759f&facet=true&facet.field=site_id

Note that the response from the URLs shown may vary, as the testing site could have been reindexed.

1320068076000000 1324033923000000
#1431 defect dread dread ckan-v1.5 closed fixed Captcha field - foreign chars cause exception

During registering a user, the user inputs foreign chars into the captcha field.

URL: http://thedatahub.org/user/register
...
Module ckan.lib.captcha:22 in check_recaptcha
<<                                     remoteip=client_ip_address,
                                          challenge=recaptcha_challenge_field,
                                          response=recaptcha_response_field))
           f = urllib2.urlopen(recaptcha_server_name, params)
           data = f.read()
>>  response=recaptcha_response_field))
Module urllib:1267 in urlencode
<<          for k, v in query:
                   k = quote_plus(str(k))
                   v = quote_plus(str(v))
                   l.append(k + '=' + v)
           else:
>>  v = quote_plus(str(v))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xea' in position 0: ordinal not in range(128)
1320078849000000 1320084104000000
#1432 enhancement rgrp ckan-backlog new [super] Data processing system for CKAN and Webstore

Super ticket: #1190

A data processing system which utilizes the Webstore. One could get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules)

1320142747000000 1339774041000000
#1433 enhancement kindly rgrp ckan-sprint-2011-11-21 closed fixed Support SQLAlchemy 0.7

Why: current stable version of sqlalchemy. geoalchemy stuff required 0.7 and likely that some other things will require it soon.

Probably requires work on vdm https://bitbucket.org/okfn/vdm

NB: should have discussion before making 0.7 the default required version in CKAN core.

1320143453000000 1324472583000000
#1434 enhancement seanh dread ckan-v1.8 closed fixed Add i18n strings from extensions

Decide which extensions are important.

1320145648000000 1341245889000000
#1435 enhancement rgrp ckan-v1.6 closed wontfix Switch to continuous.io for buildbot (?) 1320145831000000 1323283538000000
#1436 defect johnglover johnglover ckan-sprint-2011-11-07 closed fixed Packages with a 'pending' state not listed on groups page 1320155227000000 1320243278000000
#1437 defect dread dread ckan-sprint-2011-11-07 closed fixed JSONP parameter in Action API

Action API needs JSONP support - be able to return responses encapsulated in a function of a supplied name. This is important for remote sites running javascript to interact with a CKAN site.

Specifying the callback parameter is the way we've achieved JSONP with the RESTful and Search APIs. It should work like this:

curl http://test.ckan.net/api/action/package_show?callback=jsoncallback -d '{"id": "fd788e57-dce4-481c-832d-497235bf9f78"}'

Or maybe the callback should be specified in the payload in the context or data_dict?

(My understanding is that CORS is similar - when more browsers support it, can we drop JSONP?)

1320161088000000 1320173795000000
#1438 enhancement dread ckan-future new Action API - parameter discovery/checking

Many actions in the Action API require parameters. What params are needed should be listed and checked. Because currently, if you get them wrong you simply get a useless 500 error.

Currently they are listed in the docs, extracted from the code manually.

So you could GET /action/api/package_list to receive not only the help text, but a list of arguments.

And if you send an extra or missing argument then an intelligent error message can be returned.

implementation

How about some sort of decorator on the action function:

@logic_params(id, offset, limit)
def get_package_list(context, data_dict):
    ...

This would do the param checking, and is there a way to extract these params from the function? Or do a registration of the logic function?

I'd certainly like to keep the list of the list of params for the function with the function, for ease of reading the code.

Another good thing would be to pass in the params named as themselves, rather than having them contained in the data_dict.

1320161890000000 1338205050000000
#1439 enhancement dread ckan-backlog new Action API discoverablility

A good service API needs to be discoverable, so you are not always having to refer to the documentation html.

Maybe /api/action should return a list of actions available? (Currently this returns a 404.)

  • It would be nice to sort these into get/create/update/delete.
  • #1438 Parameters for each of the actions must be discoverable too

/api/action/{action_name} should also return the help text / parameters allowable. (Currently this returns 400 error)

1320161970000000 1325474974000000
#1440 defect dread dread ckan-sprint-2011-11-07 closed fixed 'paster db load' doesn't update search index

paster db load ckan.pgdump will load in the database but not update the search index, no we're using SOLR.

implementation

We can't tack on 'rebuild' to this command, because it needs a 'db upgrade' first. I'm tempted to make 'db load' do the upgrade and rebuild all in one command.

1320232415000000 1320235084000000
#1441 defect dread dread closed fixed Use local ckan logo image

Using local image rather than the same image at assets.okfn.org reduces dependency on another server that may be down, slow or remote. James agrees.

1320234597000000 1320235062000000
#1442 defect dread dread ckan-sprint-2011-11-07 closed fixed You can't GET then POST a dataset as new

If you try to GET a dataset, purge it then POST it again, it fails checking authz for the group:

Module ckan.controllers.api:278 in create
<<                  (register, subregister))
               try:
                   response_data = action(context, data_dict)
                   location = None
                   if "id" in data_dict:
>>  response_data = action(context, data_dict)
Module ckan.logic.action.create:240 in package_create_rest
<<      api = context.get('api_version') or '1'
       
           check_access('package_create_rest', context, data_dict)
       
           dictized_package = package_api_to_dict(data_dict, context)
>>  check_access('package_create_rest', context, data_dict)
Module ckan.logic:129 in check_access
<<          #    log.debug('Valid API key needed to make changes')
               #    raise NotAuthorized
               logic_authorization = new_authz.is_authorized(action, context, data_dict)
               if not logic_authorization['success']:
                   msg = logic_authorization.get('msg','')
>>  logic_authorization = new_authz.is_authorized(action, context, data_dict)
Module ckan.new_authz:16 in is_authorized
<<      auth_function = _get_auth_function(action)
           if auth_function:
               return auth_function(context, data_dict)
           else:
               raise ValueError(_('Authorization function not found: %s' % action))
>>  return auth_function(context, data_dict)
Module ckan.logic.auth.create:115 in package_create_rest
<<          return {'success': False, 'msg': _('Valid API key needed to create a package')}
       
           return package_create(context, data_dict)
       
       def group_create_rest(context, data_dict):
>>  return package_create(context, data_dict)
Module ckan.logic.auth.create:15 in package_create
<<      else:
               
               check2 = check_group_auth(context,data_dict)
               if not check2:
                   return {'success': False, 'msg': _('User %s not authorized to edit these groups') % str(user)}
>>  check2 = check_group_auth(context,data_dict)
Module ckan.logic.auth.create:90 in check_group_auth
<<      groups = set()
           for group_dict in group_dicts:
               id = group_dict.get('id')
               if not id:
                   continue
>>  id = group_dict.get('id')
AttributeError: 'unicode' object has no attribute 'get'

(using RESTful Model API)

The reason is the POST is expecting the group to be expressed as a dict, not the list of names that GET returns.

1320263318000000 1320275623000000
#1443 defect dread dread ckan-sprint-2011-11-07 closed fixed language detection

If language detected doesn't match any of the offered CKAN languages, you get the default language, but see an error flash up for every page, saying:

Could not change language to 'None': IOError: [Errno 2] No translation file found for domain: 'ckan'

Affects 1.5b only.

1320425588000000 1320432511000000
#1444 defect dread dread ckan-future closed wontfix Language detection defaults to en_us?

Using curl you always get English, even if ckan.locale_default=de. Find out why. (1.5b)

Maybe we should disable locale detection for this release if lots of people's browsers don't have it set correctly (e.g. chinese with us internet explorer)

1320432634000000 1340190852000000
#1445 enhancement zephod rgrp ckan-sprint-2011-12-05 closed fixed Resource View page in WUI

Super ticket: #1032

  • Locate at: /dataset/{dataset}/resource/{id}

See: http://wiki.ckan.org/Dataset_View_Page

Implemented in branch feature-1445-resource-view.

Still to do:

  • Add inline data explorer to page
1320665049000000 1330019916000000
#1446 enhancement rgrp rgrp ckan-sprint-2012-01-09 closed fixed Data Explorer v2

Super ticket: #1602 (Data Previewer v2)

We already have first pass of Data Explorer that was released as part of #1357.

Tickets include (* indicates improvement over current explorer)

Est: 10-15d (should be broken down -- partly is in recline issues)

1320665596000000 1326281658000000
#1447 defect kindly dread ckan-backlog assigned disk space leakage

Periodically we see some CKAN servers fall over because they run out of disk space. We need to find out if there is a common cause and fix it.

One problem in the past has been file handles running out when creating lots of tiny files in the data directory.

Another problem has been several enourmous backups being created every day - pdeu on eu25.

1320666843000000 1340727330000000
#1448 enhancement kindly kindly closed fixed Set up nice way to do celery deployment.

Celery is awkward to deply, need to find a way to do it more simply. i.e using celery-pylons and supervisor. A modified version of celery-pylons may be the best solution. 1d

1320666977000000 1325774155000000
#1449 enhancement johnglover johnglover ckan-sprint-2011-11-21 closed fixed Improved resource listing on dataset view page

Should list resources in the form of search results rather than as a table.

1320677693000000 1321873008000000
#1450 enhancement zephod johnglover ckan-sprint-2011-12-05 closed fixed Simplify dataset view page

See http://wiki.ckan.org/Dataset_and_Resource_Pages_Proposal

  • Create a collapsible section for the README
  • Reduce the number of things on the page
  • Tidy up groups and tags in the sidebar
  • Resource view becomes a list of search results

(additional)

  • Resource navbar item links to first resource
  • Resource navbar has a dropdown menu of all resources in the dataset
  • Edit and Authz navbar items should be in a separate logical group to the right

Very rough summary: create more info section and move stuff out (relayout page e.g. README, tags etc)

1320677788000000 1330019974000000
#1451 enhancement johnglover johnglover ckan-sprint-2012-01-09 closed fixed Reintegrate download stats on dataset and resource view page - 0.5d
  • css class: resource-url-analytics
  • Assign to the link tag (a) everywhere we want to count (dataset, resource view) - 0.25d
  • Display counts in same place ...

And deploy on http://thedatahub.org/ - 0.25d

Possible: Also move analytics extension into core (decided not to).

1320677859000000 1324401792000000
#1452 enhancement dread dread closed wontfix Offer detected browser language, rather than auto-switch

There are issues with setting the site's language according to the browser detection:

  • Many browsers have the language setting incorrectly set. e.g. people who download US versions of Mozilla, pirated US copies in the far east.
  • Although thedatahub.org has a worldwide audience, many CKAN instances are designed for a particular country's citizens. So Open Berlin should probably be in German as default for all.

I suggest the site should have a default language saved in the config. The browser language *should* be detected, and that prompts a flash message offering to change to that language. And if you change language that is saved in the cookie (as we currently do).

Then do some testing to see if this suits people.

1320682652000000 1340190587000000
#1453 enhancement icmurray icmurray ckan-sprint-2011-12-05 closed fixed Flexible tag names

Allowing more flexible tag names:

  • allowing spaces
  • allow capital letters (search is case in-sensitive)
  • allow all punctuation except for commas and double-quotes '"'
  • allow unicode
  • commas delimit tag names in the package create/edit form

Effects:

  • package creation/edit form.
  • /tag/{tagname} uri
  • search action
  • api controller (search/package-create/edit)
  • web controller (search/package-create/edit)
  • search api documentation
  • autocomplete for tag names
1320771850000000 1329395697000000
#1454 enhancement dread dread ckan-sprint-2011-11-21 closed fixed postgres search code/data removal

We made SOLR the main and only supported way to do CKAN search three months ago (#1275), but I notice that we still have remnants of the postgres full-text search hanging around:

  • model code (ckan/model/search_index.py)
  • table in the database (package_search)

The presence of these still create sqlalchemy warnings and add to our db, (& db backups etc), so it would be good to get rid of them if we are not using them. John says this can probably be removed

1320838999000000 1320849527000000
#1455 defect johnglover dread ckan-sprint-2011-12-05 closed fixed Search results when 'all_fields' don't include 'extra' fields

When you do a search like this:

http://thedatahub.org/api/search/package?q=tauberer+census&all_fields=1

the "extra" fields (e.g. "triples", "shortname") get missed off the results. The docs say it should be a "full record" and I don't see any reason why this is missed off.

This is a problem because search all_fields is the only way for clients and front-ends to get packages in bulk. They end up (like lodcloud) doing thousands of requests to get packages individually.

The full record is:

http://thedatahub.org/api/rest/dataset/2000-us-census-rdf
{"count": 1, "results": [{"res_description": ["Download", "XML Sitemap", "SPARQL enpdoint", "Example (RDF/XML)"], "name": "2000-us-census-rdf", "license": "Non-OKD Compliant::Creative Commons Non-Commercial (Any)", "author": "Joshua Tauberer", "author_email": "http://razor.occams.info/", "ckan_url": "http://thedatahub.org/dataset/2000-us-census-rdf", "notes": "2000 U.S. Census converted into over a billion RDF triples.\n\nPopulation statistics at various geographic levels, from the U.S. as a whole, down through states, counties, sub-counties (roughly, cities and incorporated towns)\n\nNotes: also found in the of SPARQL Endpoints.\n\nFrom home page:\n\n> * For the detailed Census statistics, you'll have to download the raw Census data files from the Census Bureau, my Perl script and the patch file below and run it yourself because the files are too big for me to offer as a download!\n> \n> * The data and scripts can be reused under Creative Commons Attribution-NonCommercial-ShareAlike.\n", "entity_type": "package", "site_id": "www.ckan.net", "download_url": "http://www.rdfabout.com/demo/census/", "indexed_ts": "2011-11-01T12:52:36.034Z", "url": "http://www.rdfabout.com/demo/census/", "state": "active", "title": "2000 U.S. Census in RDF (rdfabout.com)", "groups": ["lod", "lodcloud"], "res_format": ["", "meta/sitemap", "api/sparql", "example/rdf+xml"], "license_id": "cc-nc", "revision_id": "fcbad0de-79ea-41bd-8e01-eb832a05b732", "res_url": ["http://www.rdfabout.com/demo/census/", "http://www.rdfabout.com/sitemap.xml", "http://www.rdfabout.com/sparql", "http://www.rdfabout.com/rdf/usgov/geo/us/ny"], "id": "551ec435-f198-4d52-9b56-ec0b0be6aec9", "tags": ["census", "data", "demographics", "deref-vocab", "format-dc", "format-geonames", "format-politico", "format-rdf", "geographic", "linkeddata", "lod", "lodcloud.nolinks", "no-license-metadata", "no-provenance-metadata", "no-vocab-mappings", "population", "published-by-third-party", "rdf", "statistics", "us"]}]}
1320858265000000 1324474466000000
#1456 enhancement amercader amercader ckan-sprint-2011-11-21 closed fixed Use resource description instead of name if both are present

If a resource has both description and name the name is used. Descriptions are generally more, well, descriptive, so let's use those.

1320862619000000 1324472178000000
#1457 defect jilly.mathews ckan-future new Bug with DataNL instance

"when logging into http://register.data.overheid.nl/ with OpenID, the /user/me page gives a 404. CKAN version 1.3.4."

n the manual it says an API key kan be created via http://test.ckan.net/user/me /. However, when I try the corresponding http://register.data.overheid.nl/user/me, I get a 404 error (not found).

1320925041000000 1320925041000000
#1458 enhancement rgrp rgrp ckan-future assigned Support previewing kml files in data viewer

Super ticket: #1151 (viewing geo data)

E.g. preview of http://thedatahub.org/dataset/louisvillecrime should bring up a map

1320936488000000 1340632932000000
#1459 enhancement rgrp rgrp ckan-backlog new Featured Dataset feature

Provide way to mark a dataset as featured. Featured database show up on the front page.

TODO: detail this more.

1321113012000000 1321113012000000
#1460 enhancement rgrp assigned Improve extensions documentation

Current extensions documentation needs some work: http://docs.ckan.org/en/latest/plugins.html

  • Queue extension section may now be out of date (?)
  • Think about how it integrates with https://github.com/okfn/ckanext-example (especially tutorial and example extension)
  • Document all plugin points (auto-extract from CKAN source??)
1321114523000000 1340191001000000
#1461 defect pudo closed fixed CkanClient doesn't submit auth headers for GET requests

e.g. package_register_get.

1321354037000000 1321359503000000
#1462 defect amercader thejimmyg ckan-sprint-2011-12-19 closed fixed Multi-Instance CKAN Solr Search Issues

With the 1.5 package release, CKAN instances over-write each other's indexes. Adria has a fix.

I'd like to see a new CKAN 1.5.1 release with the Solr changes merged and the Solr site ID set up as part of the install ckan.site_id = dgu-os.okfn.org. I'd also like the ability to search on ID and to have the ID directly visible on the package page.

We need to decide how to do this upgrade without affecting existing CKAN solr indexes.

1321375470000000 1330083671000000
#1463 defect thejimmyg ckan-sprint-2011-11-21 closed wontfix QA extension no-longer works with packaged CKAN 1.5

The extension needs upgrading:

No module named webstore.database

The other option is to delay this functionality and merge with the publisher dashboards after the group refactor.

John, David Raznick, David Read, Ira, what do you think?

1321375614000000 1321874259000000
#1464 enhancement thejimmyg ckan-v1.6 closed wontfix Replace RabbitMQ with Celeryd to support running multiple instances

The current harvesting implementation can only have one instance per server.

We could either:

  • Accept this
  • Change the RabbitMQ code to support multiple instances
  • Switch the entire harvesting to use Celery
  • Wait for James's redis based live feedback based system
1321375709000000 1328529042000000
#1465 enhancement thejimmyg ckan-v1.7 closed fixed Upgrade the harvester to support publishers properly

At the moment most of the harvesting functionality is only available to sysadmins. This is fine for most cases but going forward we will need to grant each publisher access to their own sources, harvest objects and harvest object errors. This will probably be done based on the groups refactor.

One more pressing concern is giving DGU publishers the ability to make calls to /harvest/object/id. We want them to be able to see documents that faild validation and also to see *all* errors in each document.

1321375923000000 1338203858000000
#1466 enhancement thejimmyg ckan-backlog new Need to support https login for multiple instances as part of the CKAN package install 1321375978000000 1328529062000000
#1467 defect thejimmyg thejimmyg ckan-sprint-2012-01-09 closed worksforme CKAN dumps dgu miss certain publisher information

Pawel knows about this so David Read, Pawel and I need to find time to discuss it.

1321376042000000 1326120319000000
#1468 defect johnglover thejimmyg ckan-sprint-2011-12-05 closed fixed Update the search index for individual packages

When I delete a package from the pylons shell I need to remove that package's entry from the Solr search index. That doesn't happen at the moment.

It would be useful if the search-index rebuild command could take a list of package IDs to rebuild.

1321376195000000 1322591997000000
#1469 enhancement amercader thejimmyg ckan-sprint-2012-03-05 closed fixed Geo-spatial search on the front end

Now that we have a geospatial search API, we should build a front-end that integrates with the existing search for displaying results.

1321376324000000 1330102878000000
#1470 defect dread amercader ckan-sprint-2011-11-21 closed fixed Check user name in the profile form 1321446143000000 1324473955000000
#1471 enhancement seanh dread ckan-v1.8 closed fixed Be compatible with new ubuntu release in April

Nils flagged up that Ubuntu launches another LTS in April which we should probably move to. This ticket is to get the ball rolling with what we do. We can always include in our package the exact package versions of deps that we want, but it is good to be in line with the release if possible.

Here's how to see the packages included with it: http://packages.ubuntu.com/search?keywords=python-pylons&searchon=names&suite=all&section=all

Pylons 0.9.7 -> 1.0-2 Requires a small amount of work SQLAlchemy 0.7.3-2 OK Babel 0.9.6-1 OK Genshi 0.6-2 Perfect

Another alternative is to move to another web framework, especially minding that Pylons is not being further developed.

1321552345000000 1340810270000000
#1472 enhancement rgrp rgrp ckan-sprint-2011-11-21 closed fixed Convert ckanclient to git and github from mercurial 1321624905000000 1321721874000000
#1473 defect dread dread ckan-sprint-2011-11-21 closed fixed max name length validators missing

Max length of package name and version field is 100 characters, but the validators seem to enforce this, although in the web interface the javascript limits typing in these fields, so it just affects the API really.

1321631452000000 1321872717000000
#1474 enhancement kindly ckan-sprint-2011-11-21 closed fixed fix up navl tests

navl tests are being skipped unskip them!

1321825892000000 1321826753000000
#1475 enhancement kindly kindly ckan-sprint-2012-01-23 closed fixed Create documentation showing how to make an extension that use celery. 1321826681000000 1327576068000000
#1477 enhancement kindly kindly ckan-v1.6 closed fixed [Super] Group refactor

This is the implementation of the final draft in

http://wiki.ckan.org/Groups_Refactor

Steps to complete this will follow (somewhat) in the following order.

Change package_group table to membership. (#1478) 3d

  • model migration. (done)
  • simplify revisioning (done)
  • Remove any sqlalchemy releations from groups. (done)
  • change logic functions (done)

Single type per group (#1531)

  • migration add type to group table. (done) 1d

Add capacity table (#1522)

  • model migration. (done)
  • add to logic functions.

Allow other objects to be added to groups. (#1531)

  • change logic functions (done)

Adapt thedatahub.org to use new model.

  • move auth groups over to groups/members
  • change user_object roles to use groups? keep user object roles for time being.
  • make create update package/group put the correct items in the group/members/capacity tables.
  • work out how ui works.

--not doing the below as part of this super ticker now #1669.

Create model for dgu.

  • adapt auth functions to use auth.
  • adapt package_create/update group_create/update to fill in the correct capacties/members.

Adapt Iati, and potentially other custom instances.

Sub-tickets:

  • #1279 Deprecate Authorization Groups
  • #1198 Publisher hierarchy
1321959320000000 1328016209000000
#1478 enhancement kindly kindly ckan-sprint-2011-12-05 closed fixed change package_group to members table

This will comprise of the following steps.

  • model migration.
  • simplify revisioning
  • Remove any sqlalchemy relations from groups.
  • change logic functions

estimate 2d

1321959973000000 1323172654000000
#1479 defect dread dread ckan-sprint-2011-12-05 closed fixed Can't edit a user with a unicode email address
  1. Register User with an email address with a unicode char (e.g. u'\u044e')
  2. View the User in the UI (/user/) or with 'user_show' Action API

Exception:

Module ckan.controllers.user:98 in read
<<          try:
                   user_dict = get_action('user_show')(context,data_dict)
               except NotFound:
                   h.redirect_to(controller='user', action='login', id=None)
>>  user_dict = get_action('user_show')(context,data_dict)
Module ckan.logic.action.get:488 in user_show
<<      check_access('user_show',context, data_dict)
       
           user_dict = user_dictize(user_obj,context)
       
           if not (Authorizer().is_sysadmin(unicode(user)) or user == user_obj.name):
>>  user_dict = user_dictize(user_obj,context)
Module ckan.lib.dictization.model_dictize:189 in user_dictize
<<      
           result_dict['display_name'] = user.display_name
           result_dict['email_hash'] = user.email_hash
           result_dict['number_of_edits'] = user.number_of_edits()
           result_dict['number_administered_packages'] = user.number_administered_packages()
>>  result_dict['email_hash'] = user.email_hash
Module ckan.model.user:59 in email_hash
<<          if self.email:
                   e = self.email.strip().lower()
               return hashlib.md5(e).hexdigest()
               
           def get_reference_preferred_for_uri(self):
>>  return hashlib.md5(e).hexdigest()
UnicodeEncodeError: 'ascii' codec can't encode character u'\u044e' in position 17: ordinal not in range(128)
1321960486000000 1321961592000000
#1480 enhancement dread dread ckan-sprint-2011-12-05 closed fixed Munge functions in util

munge_title_to_name, munge_tag v. useful for client software importing things into CKAN. I want it for ONS.

1321965837000000 1321978546000000
#1481 defect johnglover dread ckan-sprint-2011-12-05 closed fixed License not shown if no resources

The license is not displayed any more, unless there are resources e.g. http://test.ckan.net/dataset/dtest#section-basic-information is OGL, but that's not shown.

1321983266000000 1323169343000000
#1482 defect johnglover zephod ckan-sprint-2011-12-05 closed fixed Minor JS bugs

Three JS bugs have appeared:

  • The slug editor is not updating the text field again. [Fixed; commit:fadda17383]
  • The View Dataset page will stop executing javascript if there are no resources. [Fixed; commit:e54b089f6e]
  • The Add-Dataset and Add-Group URL editor shows the wrong URL for datasets and groups. Broken in this commit:

https://github.com/okfn/ckan/commit/5a01e67 by 'map.explicit=true'. JohnG: Can you look into this? [Fixed; commit:88e88380f6]

1321991091000000 1322062576000000
#1483 defect johnglover johnglover ckan-sprint-2011-12-19 closed fixed Tasks sometimes fail to update task_status table when trying to report a failure

The error reports do not have a "value". The original task in the task status table is not made and it tries to make a new task, but can't.

Issue: why is the original task_status entry not made?

kindly: there is a chance of course that the task is quicker than the plugin to get to the point where it needs to update the table.

kindly: so we may have to find a way to update the task status before we run the task.

kindly: I think you can supply a task_id to celery. i.e we make our own guid as the task id. So we can use that one to populate the task status.

1322050605000000 1323279151000000
#1484 defect dread dread ckan-sprint-2011-12-05 closed fixed Creating package with duplicate tags causes exception

e.g.

curl http://localhost:5000/api/action/package_create -d '{"name":"test5", "tags":[{"name": "tag6"}, {"name": "tag6"}]}' -H 'Authorization:tester'

Also broken in Web UI (adding a package).

Affects API and Web UI in 1.5 and I think all versions from 1.4.1

1322052046000000 1322059006000000
#1485 enhancement kindly kindly closed fixed Package/Group form extension mechanism so you can add forms for particular package_types

We want to be able to change form depending on package type or group type.

This is dependent on a type field being added to the Package and the Group.

1322059169000000 1340034422000000
#1486 defect seanh seanh closed fixed "allow_partial_update" option does not work with resource lists of packages
  1. Call ckan.logic.action.update:package_update()
  2. Pass in a context dict with "allow_partial_update":True
  3. Pass in a partial data dict containing only the package's ID and those fields that you want to update (e.g. title).

=> All of the package's resource will have their statuses changed to "deleted".

The "allow_partial_update" option should apply to the resource lists of packages, but it does not.

1322061318000000 1323172027000000
#1487 enhancement kindly ckan-sprint-2011-12-05 closed fixed Fix group ordering on homepage

ordering on homepage by name instead of group count

1322094280000000 1324474147000000
#1488 enhancement dread dread ckan-sprint-2011-11-21 closed fixed List of installed extensions - add to the status_show

Add the list of extensions install (config's ckan.plugins) to status_show logic action (See #1087)

1322130555000000 1322241966000000
#1489 enhancement dread ckan-backlog assigned Updating example theme/extension

ckanext-example needs updating for CKAN 1.5:

  • theme changes
  • new forms

About: 'ckanext-exampletheme' was created in Spring 2011 as an example CKAN extension that showed how to customise the look & operation of CKAN. This moved to github and renamed 'ckanext-example'.

1322137920000000 1324292384000000
#1490 enhancement amercader amercader ckan-sprint-2011-12-05 closed fixed Standardize output from package listings coming from the logic layer

Right now, the two logic functions that return a list of packages (package_search [1] and group_packages_list [2])use custom functions to generate the output dict. That's suboptimal because:

  • The template function that renders the package listing (package_list_from_dict) could end up receiving different dicts.
  • Not all package properties are available at the template level (e.g extras, tags...). These can be really useful to customize the listing from an extension.

In general only the functions present in lib/dictization/model_dictize.py should be used to build the output of a logic function, in that case package_dictize. If necessary, they can be modified to include missing properties, like on this particular case the "isopen" property, needed by the template renderer.

[1] https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L685 [2] https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L442

1322225239000000 1324033557000000
#1491 defect seanh dread closed fixed Visible strings need internationalisation

Sean spotted that some strings need internationalising, such as flash messages. Generally all strings that appear on the web front-end should be internationalised.

Particular places that need i18n:

  • ckan/controllers/admin
  • ckan/lib/base

Note: there are some exceptions, such as in i18n.py, very obscure error messages and stuff that only appears on the API.

NB: there is a cost in making a string internationalisable (all our volunteers have to translate it), so we should not be too zealous. We should also look at the i18n/ckan.pot to see if any of the existing strings can be reused.

1322481000000000 1330020630000000
#1492 defect kindly dread closed wontfix Interference between extra and relationship fields in API

From Jan Kučera <elquenor@…>

I have also tried to use extras fields to store information about relationships between datasets. I found out that if you first add relationship using CKAN API into dataset relationships attribute and than you add extras field for the same relationship with key in format "parent_of:dataset_name" or "child_of:dataset_name", storing this extras field will clear all relationships in relationships attribute stored earlier.

I might be some kind of bug but I suppose you will know better. Is it possible to use relationships attribute and extras field for relationships at the same time?

1322481496000000 1340034528000000
#1493 defect dread dread ckan-sprint-2011-12-05 closed fixed 'search-index rebuild/clear' doesn't work if no ckan.site_id

You can't delete things from the SOLR search index if the ckan.site_id and ckan.site_url are blank.

Should assert that one of these are set up.

1322484422000000 1324474360000000
#1494 enhancement seanh seanh ckan-sprint-2012-01-09 closed wontfix API call for getting a user's public activity stream as rendered rtext

This could be implemented as a separate API call, or the rendered text versions of the activities could be added into the JSON returned by the existing API call.

This requires setting up templates for rendering activity streams items and detail items as nice, human-readable text.

There are some open questions, e.g.: Do we want the entire activity stream rendered as a block of plain text? As HTML? Or do we want a list of JSON objects, where each object contains its textual and/or HTML representations as fields?

Activity stream items and their related detail items are separate objects that each have their own textual representations.

For a mockup of the kind of text messages we want, see:

http://datahub.pudo.org/pudo

but note that this ticket is just for creating the text snippets themselves, not rendering then in an HTML page or RSS feed. Also the mockup only show activity items and not their detail items.

1322495447000000 1326109757000000
#1495 enhancement kindly seanh ckan-sprint-2012-01-23 closed fixed API call for getting a user's public activity stream

An API call that returns a user's public activity stream in JSON format.

1322495549000000 1327576099000000
#1496 enhancement seanh seanh ckan-v1.7 closed fixed Render a user's activity stream on her user page 1322495588000000 1334582338000000
#1497 enhancement dread dread ckan-sprint-2011-12-05 closed fixed Suppress benign warnings

We really shouldn't have lots of benign stuff printed on stderr when we run every command. It confuses users and hides important stuff.

1322569640000000 1322569805000000
#1498 enhancement amercader amercader ckan-sprint-2011-12-05 closed fixed Support multiple SOLR schema versions

At some point the SOLR schema used by CKAN will need to be changed and the changes won't be always backward compatible, so we need a mechanism to support instances that use older versions of the schema. This process will involve:

  • Better documentation on how to deploy SOLR, as single instance or multicore
  • A convention on how to tag schema versions and SOLR endpoints to ensure that CKAN instances point to the correct SOLR URL.
  • (If possible) Checks in CKAN core that deal with different schema versions.
1322585264000000 1323165876000000
#1499 enhancement dread dread ckan-sprint-2011-12-05 closed fixed list/read/delete packages on the command-line

It would be useful to be able to manipulate packages using the paster command-line interface. In particular listing, reading and deleting packages. Seeing which packages were deleted too.

1322591656000000 1322591888000000
#1500 defect dread dread ckan-sprint-2011-12-05 closed fixed Slug API calls tidy up
  • API call documented but don't exist any more: dataset create_slug
  • API call not documented but exists: is_slug_valid
  • API call not documented, badly sited in API and using custom code in controller (not logic layer): /package/autocomplete
1322600285000000 1323166918000000
#1501 defect johnglover amercader ckan-sprint-2011-12-05 closed fixed Pagination links broken

Most of the different pagination links are broken and point to wrong locations. e.g:

  • Dataset listings in group page:

http://test.ckan.net/group/bibliographic

  • Tag letters in tag listing:

http://test.ckan.net/tag

  • User listing:

http://test.ckan.net/user

  • Revision listing:

http://test.ckan.net/revision

This is most probably related with:

https://github.com/okfn/ckan/commit/1772a5c

1322646397000000 1322684411000000
#1502 defect johnglover johnglover ckan-sprint-2011-12-05 closed fixed Group package list is ordered by revision timestamp instead of alphabetically 1322680312000000 1324480415000000
#1503 task dread markbrough ckan-sprint-2011-12-05 closed fixed Move from Bitbucket to Github

Reasons: more people familiar with Git, bigger Open Source community on Github.

Keeping track of the move here: https://docs.google.com/spreadsheet/ccc?key=0AvCDqUH8jVN8dGNVNEpXLWk3UTlJZXhFMjBPbXVtRmc&hl=en_GB#gid=0

1322745084000000 1323793662000000
#1504 defect dread dread ckan-sprint-2011-12-05 closed fixed Action API given blank data causes exception

This is what caused the exception:

curl http://localhost:5000/api/action/package_search -d='{"q": "osm"}'

The problem is the content-type is formencoding, the '=' is the first character of the request, and so internally the data_dict is set to ' ', which needs to be a dictionary.

1322756748000000 1324035928000000
#1505 defect dread dread ckan-sprint-2011-12-05 closed fixed SearchError and SearchQueryError cause exception in Action API

This query caused ckan to except because ckan/controllers/api.py doesn't catch SearchError? and SearchQueryError?:

curl http://localhost:5000/api/action/package_search -d '{"sort": "metadata_modified"}'
1322758968000000 1324474577000000
#1506 enhancement zephod rgrp ckan-sprint-2012-03-05 closed fixed [super] UX improvements to dataset edit, create and search + group + user page (Dec 2011)

Historical Parent Tickets

Step Zero

  • COMPLETELY REDESIGN resource edit process

Dataset Edit-Resources

  • Force format field to be lowercase
    • Modify backend to serve it up in that format
  • Subticket #978: Add extra fields to resources in Web UI
  • Fix display of large datasets (http://test.ckan.net/dataset/europeana-lod)
  • When submitting forms with errors, open the correct resource pane.
  • No hint is given that the time format for a resource is in ISO, eg 2005-01-01T10:10:10
  • Resource description field to become the markdown editor we use everywhere else.

Dataset Settings

  • Subticket #1648: Clarify edit terminology
  • Merge Groups and Tags into main Settings section (no point having them on their own and most people will miss this).
  • Fix (i.e. allow / show) name editing on dataset edit.
    • Fix the slugify so it does not overwrite if i change title after fixing name.
    • Make slugify encourage people to use shorter names. Either by cutting off (not so good) or showing a warning and saying something like 'hey you may want to cut this down. Hit edit to customize the dataset name now'
  • Autocomplete the author/maintainer box (requires API extension?) [WONTFIX what? it's not like that field is linked to users of the site or anything]

Dataset View

  • Fix animation glitch in expanding description
  • Can we make the list of files look more like a table? Lose space between files? Lose rounded corners?
  • Verify that there are no more (none) placeholder <td> objects (i believe they've been refactored out)

Dataset Create

  • "Upload or link data files, APIs and other materials related to your dataset."
  • Include the description markdown editor again (keep it small)
  • Licence should maybe be restored
    • "(Don't worry if you don't know what your licence is)"
  • Show the log message field
    • Beautify
  • Pre-populate the log message field?
  • ? Show Home Page field when creating dataset [WONTFIX don't push it]
  • ? Do some Guerilla UX to see whether we need to Redirect to Resources directly after dataset creation? [WONTFIX doesnt make sense with new resource editor awesomeness]

User pages

  • Improvements to user page UX - 1d - #1396
  • Broken links for non-Gravatar use icons - 0.25d - #1719

Etc

  • Take before and after screenshots
1323088306000000 1330702859000000
#1507 enhancement zephod rgrp ckan-backlog assigned Minor fixes to dataset add on Group edit form - 0.5d

Group edit dataset add form needs some work

  • Dataset name is not cleared when you add
  • No way to remove item from list of datasets to be added if I make a mistake
  • (2nd Apr 2012) It now seems that option to add multiple datasets at once has disappeared (perhaps during the CSS/HTML refactor ...)
1323088429000000 1338205220000000
#1508 enhancement ross rgrp ckan-v1.6 closed fixed [super] Finalize and deploy some of our major (new) features as of Dec 2011

This ticket is about finalizing, then deploying (to http://thedatahub.org/), and finally announcing some of recent major new features such as:

  • Deploy wordpresser extension for thedatahub (so we can start blogging) - #1533 - WONTFIX
  • (Deploy new Resource and Dataset pages (invite comments))
  • Get thedatahub geo-ified i.e. ckanext-spatial - #1601 (?)
    • Any additional dev that needs to be done? (e.g. geo in search ...?)
  • (Deploy / re-enable ckanext-googleanalytics on thedatahub - #1451 (more stuff #1566))
    • Rename the extension to ckanext-analytics?
  • Custom forms (and use of 'dataset types') - Ticket (??)
  • (Get QA deployed and running - #1582)

It would be best if this was done by someone *unfamiliar* with the extensions so feedback could be provided on quality of install docs etc etc.

1323088817000000 1328000871000000
#1509 defect dread assigned Mis-dated old revisions of datasets

e.g. http://thedatahub.org/dataset/osm%402011-07-12T12%3A16%3A47.590358 gives:

This is the current revision of this dataset, as edited at 2011-07-12 12:16.

but it should say the date 2011-07-12 (which is the data being displayed).

The problem is looking at PackageRevision?, when it might be a tag or extra.

1323090398000000 1340190814000000
#1510 enhancement seanh seanh ckan-v1.7 closed wontfix Migrate existing data into activity streams database tables

See '### Task 2' in this etherpad: http://ckan.okfnpad.org/notifications

I have no idea yet about how to do this or how long it might take me.

1323093901000000 1339677620000000
#1511 enhancement kindly seanh ckan-sprint-2012-01-23 closed fixed Logic function for getting a user's public activity stream

Add a function to logic.action.get that returns a user's public activity stream as a list of dicts.

1323094207000000 1327576134000000
#1512 defect ross rgrp ckan-v1.8 closed fixed Modifying user name makes you semi-log-out

If you modify your user name, it looks like you are logged out, but you are actually logged in and therefore can't login. (Reported by Pablo)

As a work around, you need to logout and log back in.

implementation

  • We should force this by auto-logging you out (DR: maybe!)
1323100214000000 1342781122000000
#1513 defect dread dread ckan-sprint-2012-02-06 closed fixed Logged in/out problems

Richard Cyganiak: I often see a problem where half the system thinks that I'm logged in, while the other half insists I'm not. Logout then login again may fix it.

David Read: I think I've seen it several times that the "My Account" link is given (suggesting you're logged in) but when you click it it asks you to log in, which means you weren't.

1323100353000000 1328205099000000
#1514 defect dread ckan-v1.6 closed duplicate Modifying user name loses connection with revisions

If you edit your user name, the number of revisions you made returns to 0. This is because in the Revision object, the user's name is stored, rather than the user's ID.

rgrp: We can reconnect that pretty easily (and have a longer term solution that involves not using the usernames but the userids in in the Revision objects so we don't hvae this problem in future!)

1323100659000000 1323279018000000
#1515 enhancement seanh seanh ckan-v1.7 closed fixed [super] Activity Streams

We want to add "activity streams" (chronological lists of activities) to CKAN.

Activity streams standard: http://activitystrea.ms/

Super branch where the feature was initially developed (this has now been merged into master): https://github.com/okfn/ckan/tree/feature-1515-activity-streams

Etherpad: http://ckan.okfnpad.org/notifications (most relevant parts inlined here)

Friedrich's mockup: http://datahub.pudo.org/pudo

Tasks

Strikethrough means the task is done.

  • Migrate existing data into activity streams model [3.5d] #1510
  • Generate activities when objects are saved [4d] #1298
  • Get the activity stream for a given user as a JSONifiable list of dictionaries [1d] #1511
  • API call for getting a user's public activity stream as JSON [0.5d] #1495
  • API call for getting a user's public activity stream as rendered text, HTML, etc. [5d] #1494
  • Render a user's activity stream on her user page [3d] #1496 (needs review/page design work)
  • Generate activities for other types of objects besides users (e.g. packages, groups, tags, extras, and save them in the database.
  • Logic functions for getting activity streams for packages, groups, and users.
  • API calls for getting activity streams of packages, groups, and users as JSON.
  • Rendering activity streams for packages and groups etc. into their pages. (done but disabled pending page design work)
  • API calls for inserting events into activity streams. (Still need to handle activity details in the API, ticketed).
  • Conform to activitystrea.ms standard
  • Enable users to subscribe to and unsubscribe from activity streams, see an aggregated activity stream on their dashboard page.
  • Generate aggregated activity streams for arbitrary sets of multiple activity sources/streams
  • Email notifications
  • Extension point for extensions to register themselves as renderers for different types of activity
  • Deploy to UAT [0.5d]
  • Docs and announce [1d]
1323105048000000 1338204149000000
#1516 task amercader amercader ckan-sprint-2011-12-19 closed fixed Update SOLR schema after 1.5.1 release and set up multicore SOLR instances if needed

CKAN 1.5.1 will introduce changes in the SOLR schema and functions to support multiple schema versions. Unfortunately the changes in the schema will be backwards incompatible so either both CKAN and SOLR are upgraded, or SOLR is configured as multicore, with one core for each schema. The solr cores would look like:

http://<solr-server>/solr/schema-<version>

Different scenarios:

  • Single SOLR instance used by only one CKAN site (e.g. SOLR running on the same machine):
    • Update CKAN
    • Update SOLR schema (symlink to suitable version in CKAN source)
    • Rebuild search index
  • Single SOLR instance used by multiple CKAN sites.
    • Update CKAN source
    • Configure SOLR as multicore, one core for version 1.2 and another for 1.3 of the schema (symlink to appropriate version in CKAN source)
    • Update solr_url on each of the CKAN sites with the suitable core
    • Rebuild search index

These are the SOLR and CKAN instances that need to be updated (Please add any missing ones):

  • Same machine as the CKAN site:
    • test.ckan.net
    • iati.test.ckan.net
    • data.gov.uk/ catalogue.data.gov.uk (confirm)
  • s046.okserver.org
    • iatiregistry.org
  • s052.okserver.org
    • datacatalogs.org
  • s004.okserver.org / eu4.okfn.org / solr.okfn.org
    • thedatahub.org (=www.ckan.net)
    • datagm.org.uk
    • publicdata.eu
    • hri.fi (dev.fvh.fi?)
    • it.ckan.net
    • ie.ckan.net
    • cz.ckan.net
    • register.data.overheid.nl (nl.ckan.net?)
    • no.ckan.net / datakilder.no
    • br.ckan.net
    • colorado.ckan.net
    • at.ckan.net

TODO: which SOLR server are using these instances?

  • data.norge.no
  • nederland.ckan.net
  • lt.ckan.net
  • pl.ckan.net
  • datadotmontreal.ca/
  • ca.ckan.net / datadotgc.com
1323108431000000 1323362689000000
Note: See TracReports for help on using and creating reports.