CKAN: Ticket Query

#728: CSW Harvesting shall be optimised in respect of reharvesting only records that have changed

johnbywater — Thu, 21 Oct 2010 15:35:40 GMT

Hi Will, this is important again because some CSW servers we use have over 300 documents in. Could you take a look at modifying the filter please?

#794: Investigate reconciling UKLP Publisher and Provider with DGU

johnbywater — Mon, 08 Nov 2010 14:50:11 GMT

This needs more analysis, but the GEMINI2 attribute "metadata point of contact" must be reconciled with the registered publisher (or agent).

This might also be used to filter records harvested from a CSW source, but filtering also needs more analysis, as does distinction between agent and provider.

#1152: True support for generic CSW servers

amercader — Mon, 23 May 2011 09:02:14 GMT

The CSW harvesters implemented at the moment were developed with the DGU project in mind, and they assume all remote CSW servers to implement the Gemini 2 specification. Gemini 2 is the profile defined in the UK for INSPIRE complying metadata, so obviously catalogs from other countries or non-INSPIRE complying ones won't be able to be harvested.

The changes needed to support generic CSW servers (i.e. those implementing the ISO 19139 profile) are:

Handling the validators (right now are hardcoded in the harvester

code). This probably involves issues discussed in the CREP 3 (ticket #1134)

Changes in the model to adapt the specification to ISO 19139

Renaming objects and classes which are now Gemini-centric

List of CSW servers tested:

https://spreadsheets.google.com/spreadsheet/ccc?key=0Atp3cZFjuIOAdDBVQWRINnlfN1d0b2lleHVEdjBSb2c&hl=en_US&authkey=CNu4hsEB#gid=0

#1382: Deleted resources are present for harvested package

thejimmyg — Tue, 11 Oct 2011 12:58:09 GMT

Perhaps the importer deletes them before re-importing. We shouldn't have deleted resources, so let's investigate.

#2268: UKLP Use same schema as form for harvested records

amercader — Fri, 30 Mar 2012 11:56:46 GMT

Datasets created via the harvesting (UKLP ones) and the ones created via the form use a different schema. That could lead to problems. We need to discuss the best approach.