Ticket #336 (closed defect: fixed)

Opened 4 years ago

Last modified 4 years ago

Resource Search API

Reported by: donovanhide Owned by: dread
Priority: blocker Milestone: v1.1
Component: ckan Keywords: rest resource
Cc: Repository:
Theme:

Description (last modified by dread) (diff)

As a

CKAN client such as ScraperWiki

I want to

search for Package Resources, either by URL or other field, or just get them all. I want to be able to get all the resource's fields, such as URL.

Proposed implementation

Add resource search API at:

/api/search/resource

AND resource added to model API at:

api/rest/resource

(see ticket:358)

Functional differences from the ScraperWiki suggested patch:

  • URL is not normalised

  • URLs are not grouped
  • All fields of the resource object are returned, not just the URL
  • Package is identified by its ID, not name or full URL. (This is for consistency in the API - you can simple prepend 'http://ckan.net/package/' to the package ID)

This is to make our API more general, simple and consistent. It means that the ScraperWiki client has to do a bit more processing to get exactly what it needs. Is this ok?

Example search

POST

{"url": "scraperwiki.com/", "all_fields": 1}

to: /api/2/search/resource

returns JSON:

 [{"id": "a3dd8f64-9078-4f04-845c-e3f047125028",
   "package_id": "b8a325c8-af2a-43f3-8245-9db7d73dfbfe",
   "URL": "http://scraperwiki.com/lincolnshire-councillors", 
   "format": "CSV", 
   "Description": "Scrape of www.lincs.gov/councillors.pdf by ScraperWiki.",
   "hash": "", 
   "position": 2
 }]

Note use of package_id instead of package_name is something we're moving towards in the API, since names can change. When we've done ticket:341 then ckan.net/package/lincs-councillors will be a synonym of ckan.net/package/b8a325c8-af2a-43f3-8245-9db7d73dfbfe

Search Parameters

Key:  q
Description: Search all resource fields for the value

Key: url / description / format / 
Description: Search particular field for the value

Key: all_fields
Value: 0 or 1 (0 is default)
Description: If 1 (true), the full record of the package resource
(and it's package reference) are returned, rather than just the
PackageResource ID.

May also choose to introduce 'offset' and 'limit' to page through a large number of results.

JSONP achieved through API-wide parameter - see ticket:342

Search is case insensitive.

Original request

Hi, have attached a patch for adding a resource list api call. Have also added a JSONP compatible callback section, along the lines of #388.

Could also add a search version. Not sure what the best url would be for that though.

Haven't written a test as the structure seems to follow a functional spec. Is that document around somewhere?

Donovan

Attachments

resource.patch (11.7 KB) - added by donovanhide 4 years ago.

Change History

Changed 4 years ago by donovanhide

comment:1 Changed 4 years ago by dread

  • Owner set to dread
  • Description modified (diff)

comment:2 Changed 4 years ago by dread

  • Description modified (diff)

comment:3 Changed 4 years ago by dread

  • Description modified (diff)

comment:4 Changed 4 years ago by dread

  • Description modified (diff)

comment:5 Changed 4 years ago by dread

  • Description modified (diff)

comment:6 Changed 4 years ago by dread

  • Description modified (diff)

comment:7 Changed 4 years ago by dread

  • Description modified (diff)

comment:8 Changed 4 years ago by dread

  • Description modified (diff)

comment:9 Changed 4 years ago by dread

  • Summary changed from Resource extension to REST API to Resource Search API

comment:10 Changed 4 years ago by dread

  • Status changed from new to closed
  • Resolution set to fixed

Done in cset:742adebb707c and cset:1748e6554e77.

comment:11 Changed 4 years ago by dread

Took 0.75 days.

comment:12 Changed 4 years ago by Donny

  • Priority changed from awaiting triage to blocker
  • Status changed from closed to reopened
  • Resolution fixed deleted
  • Type changed from enhancement to defect

http://ckan.net/api/search/resource?url=http://scraperwiki.com&all_fields=1&callback=ckantest

yields

Bad search option: Field "callback" not recognised in Resource search.

comment:13 Changed 4 years ago by dread

  • Status changed from reopened to closed
  • Resolution set to fixed

Fixed in cset:e719f449bc74

Note: See TracTickets for help on using tickets.