Ticket #191 (closed enhancement: fixed)

Opened 4 years ago

Last modified 2 years ago

Searching by modification date

Reported by: dread Owned by: johnglover
Priority: minor Milestone: ckan-sprint-2011-12-19
Component: ckan Keywords:
Cc: amercader Repository: ckan
Theme: none


Cost - 2 days

Search interface has new options to filter and sort the results by the date the package has been last modified in ckan. Search options are included in both Web UI and Search API.

The filter specifies a range of dates. The results can be sorted by ascending or descending dates. The last modification date is surfaced in the package.

Example search parameters:

modification-range=5/4/09- Exclude packages last modified earlier than 5/4/09
modification-range=5/4/09-5/12/09 Exclude packages last modified outside of 5/4/09-5/12/09
order_by=mod Sort by metadata modification. Defaults to newest first.
order_by=mod-newest Sort by metadata modification, newest first.
order_by=mod-oldest Sort by metadata modification, oldest first.

Change History

comment:1 Changed 3 years ago by dread

  • Repository set to ckan
  • Theme set to none

Will's suggestion is to have "modified_since" param, just as we have for Revision Search.

Maybe we don't need a range. 'Order by modification' should be on by default for queries with 'modified_since' param.

comment:2 Changed 3 years ago by dread

  • Priority changed from awaiting triage to critical

Do this after refactor #1129

comment:3 Changed 3 years ago by dread

  • Owner rgrp deleted
  • Status changed from new to assigned

comment:4 Changed 3 years ago by dread

Maybe allow searching by package creation date too?

Suggestion from kindly: when indexing a package, have the search backend also store the metadata_modified value, to make it easy to search on it.

comment:5 Changed 3 years ago by dread

  • Milestone set to ckan-backlog

comment:6 Changed 2 years ago by dread

So you can do this sort of search:

curl http://thedatahub.org/api/action/package_search -d '{"q": "groups:lodcloud", "sort": "metadata_modified asc"}'

but it doesn't work because solr doesn't store the metadata_modified field yet.

comment:7 Changed 2 years ago by shevski

  • Owner set to johnglover

comment:8 Changed 2 years ago by thejimmyg

  • Cc amercader added
  • Priority changed from critical to minor
  • Milestone changed from ckan-backlog to current-ckan-sprint-2011-12-19

John will just check that the API version 3 does support modified after the new solr schema handled by Adria is in place.

comment:9 Changed 2 years ago by johnglover

  • Status changed from assigned to closed
  • Resolution set to fixed

working in current master and 1.5.1

comment:10 Changed 2 years ago by dread

  • Status changed from closed to reopened
  • Resolution fixed deleted

A test for this would be great, and maybe add as an example in the search docs too?

comment:11 Changed 2 years ago by dread

Added a fix for the problems caused by the #1546 fix. SOLR indexing was excepting for new packages (tests such as ckan/tests/functional/api/model/test_package.py:TestPackagesUnversioned.test_entity_update_indexerror failed). The problem was (somehow) related to last_modified function using a new connection causing problems for the modified_metadata in finding the revision & package table entries created during the commit.

Changeset: [release-v1.5.1c 2c595ae] and cherry picked to [origin/defect-191-modification-date f98a4b2]

comment:12 Changed 2 years ago by dread

I put the sort syntax example in the apiv3.rst table and moved the full curl example to http://wiki.ckan.org/Searching_CKAN

I added tests for metadata_modified and prepared this branch for closure.

Changeset: [origin/defect-191-modification-date bebf7a5]

John, please review and close/merge if you think it is all ok.

comment:13 Changed 2 years ago by johnglover

Thanks for this David. This one in particular was causing me some grief: https://github.com/okfn/ckan/commit/f98a4b2a5f5013fa4aed475bd8b3237bb7847fcc Good spot. All looks good to me.

comment:14 Changed 2 years ago by johnglover

  • Status changed from reopened to closed
  • Resolution set to fixed

comment:15 follow-up: ↓ 16 Changed 2 years ago by dread

Went into CKAN 1.6

comment:16 in reply to: ↑ 15 Changed 2 years ago by dread

Replying to dread:

Went into CKAN 1.6

Sorry, I meant 1.5.1

Note: See TracTickets for help on using tickets.