Ticket #1516 (closed task: fixed) — at Version 7

Opened 2 years ago

Last modified 2 years ago

Update SOLR schema after 1.5.1 release and set up multicore SOLR instances if needed

Reported by: amercader Owned by: amercader
Priority: critical Milestone: ckan-sprint-2011-12-19
Component: ckan Keywords: search solr
Cc: johnglover, kindly, rgrp, dread, nils.toedtmann@… Repository: ckan
Theme: migrations

Description (last modified by amercader) (diff)

CKAN 1.5.1 will introduce changes in the SOLR schema and functions to support multiple schema versions. Unfortunately the changes in the schema will be backwards incompatible so either both CKAN and SOLR are upgraded, or SOLR is configured as multicore, with one core for each schema. The solr cores would look like:

http://<solr-server>/solr/schema-<version>

Different scenarios:

  • Single SOLR instance used by only one CKAN site (e.g. SOLR running on the same machine):
    • Update CKAN
    • Update SOLR schema (symlink to suitable version in CKAN source)
    • Rebuild search index
  • Single SOLR instance used by multiple CKAN sites.
    • Update CKAN source
    • Configure SOLR as multicore, one core for version 1.2 and another for 1.3 of the schema (symlink to appropriate version in CKAN source)
    • Update solr_url on each of the CKAN sites with the suitable core
    • Rebuild search index

These are the SOLR and CKAN instances that need to be updated (Please add any missing ones):

  • Same machine as the CKAN site:
    • test.ckan.net
    • iati.test.ckan.net
    • data.gov.uk/ catalogue.data.gov.uk (confirm)
  • s046.okserver.org
    • iatiregistry.org
  • s052.okserver.org
    • datacatalogs.org
  • s004.okserver.org / eu4.okfn.org / solr.okfn.org
    • thedatahub.org (=www.ckan.net)
    • datagm.org.uk
    • publicdata.eu
    • hri.fi (dev.fvh.fi?)
    • it.ckan.net
    • ie.ckan.net
    • cz.ckan.net
    • register.data.overheid.nl (nl.ckan.net?)
    • no.ckan.net / datakilder.no
    • br.ckan.net
    • colorado.ckan.net
    • at.ckan.net

TODO: which SOLR server are using these instances?

  • data.norge.no
  • nederland.ckan.net
  • lt.ckan.net
  • pl.ckan.net
  • datadotmontreal.ca/
  • ca.ckan.net / datadotgc.com

Change History

comment:1 Changed 2 years ago by amercader

  • Summary changed from SOLR schema after 1.5.1 release and set up multicore SOLR instances if needed to Update SOLR schema after 1.5.1 release and set up multicore SOLR instances if needed

comment:2 Changed 2 years ago by johnglover

  • Description modified (diff)

comment:3 Changed 2 years ago by amercader

  • Milestone set to current-ckan-sprint-2011-12-19

comment:4 Changed 2 years ago by amercader

  • Description modified (diff)

comment:5 Changed 2 years ago by amercader

  • Description modified (diff)

comment:6 Changed 2 years ago by amercader

  • Description modified (diff)

comment:7 Changed 2 years ago by amercader

  • Status changed from new to closed
  • Resolution set to fixed
  • Description modified (diff)

Closing as the main Solr servers are ready to support different Solr schemas. I.e they have two different cores:

to which CKAN instances can point to.

The CKAN instances that have not been updated (the ones under s004) are pointing to a Solr core with an old version of the schema, so they can wait until upgraded to 1.5.1 to update the solr_url property and rebuild the index.

Data.gov.uk can be dealt with during the next deployment.

It's not clear which Solr server are using the rest of the instances, but they can be updated as necessary when they upgrade their CKAN instance.

Note: See TracTickets for help on using tickets.