Ticket #2485 (new defect)

Opened 2 years ago

Last modified 23 months ago

Encourage leak containment by limiting the number of requests a CKAN process serves

Reported by: nils.toedtmann Owned by: icmurray
Priority: awaiting triage Milestone: ckan-v1.9
Component: ckan Keywords:
Cc: pudo, amercader, seanhammond, kindly Repository: ckan
Theme: none

Description (last modified by nils.toedtmann) (diff)

CKAN has memory leaks. They can be contained by limiting the time-to-live of a ckan process. An easy way to achieve this is to limit the number of requests a ckan server process can serve before it gets killed and replaced.

One should ...

  • research ways to limit requests-per-process with the different web servers (see below for a start);
  • explain these safeguards in the CKAN documentation and encourage users to apply them;
  • consider the helper script ckan-create-instance to generate Apache configs with WSGIDaemonProcess ... maximum-requests=XY being active instead of commented out. Or at least let it warn the user to use MaxRequestsPerChild;
  • Investigate current CKAN deployments whether they suffer from mem leaks, and if so contain them.

How to limit requests-per-process

  • Apache:
    • Use WSGIDaemonProcess with maximum-requests=50 or whatever limit is appropriate. We did this sucessfully on datahub.io and the ckan farm
    • (This need to be verified) Without WSGIDaemonProcess, MaxRequestsPerChild 50 should achieve the same.
  • nginx/supervisord: to be researched.

Change History

comment:1 Changed 2 years ago by nils.toedtmann

  • Description modified (diff)

comment:2 Changed 23 months ago by icmurray

  • Milestone set to ckan-v1.8

comment:3 Changed 23 months ago by icmurray

  • Milestone changed from ckan-v1.8 to ckan-v1.9
Note: See TracTickets for help on using tickets.