Ticket #2485 (new defect) — at Initial Version

Opened 2 years ago

Last modified 23 months ago

Encourage leak containment by limiting the number of requests a CKAN process serves

Reported by: nils.toedtmann Owned by: icmurray
Priority: awaiting triage Milestone: ckan-v1.9
Component: ckan Keywords:
Cc: pudo, amercader, seanhammond, kindly Repository: ckan
Theme: none

Description

CKAN has memory leaks. They can be contained by limiting the time-to-live of a ckan process. An easy way to achieve this is to limit the number of requests a ckan server process can serve before it gets killed and replaced.

One should ...

  • research ways to limit requests-per-process with the different web servers (see below for a start);
  • explain these safeguards in the CKAN documentation and encourage users to apply them;
  • consider the helper script ckan-create-instance to generate Apache configs with WSGIDaemonProcess ... maximum-requests=XY being active instead of commented out. Or at least let it warn the user to use MaxRequestsPerChild;
  • Investigate current CKAN deployments whether they suffer from mem leaks, and if so contain them.

How to limit requests-per-process

  • Apache:
    • Use WSGIDaemonProcess with maximum-requests=50 or whatever limit is appropriate. We did this sucessfully on datahub.io and the ckan farm
    • (This need to be verified) Without WSGIDaemonProcess, MaxRequestsPerChild 50 should achieve the same.
  • nginx/supervisord: to be researched.
Note: See TracTickets for help on using tickets.