Ticket #1345 (closed enhancement: fixed)
Investigate possible memory leak
| Reported by: | nils.toedtmann | Owned by: | kindly |
|---|---|---|---|
| Priority: | awaiting triage | Milestone: | |
| Component: | ckan | Keywords: | |
| Cc: | rgrp, pudo | Repository: | ckan |
| Theme: | none |
Description
There is some evidence pointing to CKAN handling memory inefficiently or even leaking under certain conditions:
When we migrated ckan.net/thedatahub.org from eu7.okfn.org (32bit) to s053.okserver.org (64bit) (ticket) we experienced extraordinary memory usage peaks (ticket). Here are the observed value with Apache default settings:
- eu7, mpm-prefork: base level ~0.6GB, peaks up to 2GB
- s055, mpm-prefork: base level ~1GB, peaks up to 4GB
- s055, mpm-worker: base level ~1.5GB, peaks up to 6GB
William reduced the life-time of a WSGI CKAN process from 500 requests down to 25 requests (changeset). This (together with two other tweaks) changed the situation drastically:
- s055, mpm-event: base level ~1.4GB, no peaks
This suggests that the more requests a CKAN processes serves over time, the more memory it consumes, aka bad memory management or a leak.
To prove this theory, one could reduce the total number of WSGI CKAN processes as much as possible without killing the performance (e.g. down to processes=3), and then observing the relation between maximum-requests=25...500 and memory consumption.
On 14/09/11 17:49, David Read wrote:
Someone to do a bit of top-down memory-use profiling would be very useful. Also useful would be something in the tests that reported what test cases use lots of memory - this could be in the nose plugin.
+1
This becomes an actual problem on the CKAN farm s057, see http://trac.okfn.org/ticket/1245