OOM manager project

[Jim Gettys]

OLPC needs a OOM governor, so that the "right" process gets shot when we
run low on RAM, and that processes that might get shot know enough to
save state for restart.  As you know, various problems appear if the
wrong process is killed, usually resulting in needing a restart.

Note that the kernel has to be able to recover memory when it needs it,
or it will deadlock: this is a situation where the kernel must be in
control, but user space could cooperate much better than it does today,
by providing appropriate hints.  So don't say: "the kernel shouldn't
kill processes: user space should"; that design doesn't fly.

Here's Kimmo Hämäläinen description of the (current) kernel OOM killer.

The OOM killer selects a process to kill by assigning a score to each
process; the process with the highest score is the lucky winner that
will be killed. The current OOM score for
a process is visible in proc. The entry is in /proc/PID/oom score. The
starting point of the score is the amount of memory consumed by the
process and its children. This value is adjusted as follows:
• It is set to zero if the process has no memory management or if the
process has a negative
nice value (this can be used for protecting processes from the killer).
• Divided by the square root of the CPU time consumed by the process.
• Divided by the square root of the square root of the run time of the
process.
• Multiplied by 2 if it is a process with a positive nice value.
• Divided by 4 if it is a superuser process.
• Divided by 4 if it is a process with direct hardware access.
• Finally, the value is adjusted (shifted either left or right) by the
oom adj value. It is shifted left in case the value is positive and
right in case the value is negative.
This means that a negative oom adj value will decrease the score and
also decrease the risk that the particular process will be killed. A
positive value will have the opposite effect. The value should be no
smaller than -16 and no larger than 15.

Please note that you can set the oom adj value in the proc file system.
It is located at /proc/PID/oom_adj. For more information about how the
OOM killer behaves, see the Linux kernel source code, mm/oom kill.c in
particular.

So we need an OOM killer helper.  

We have the ability to provide the kernel with much of the 
information it needs for much better behavior, if we choose.

I see this project evolving through the following incremental
improvements (and incremental difficulty) as set out below:

1) start by setting the oom_adj appropriately so that the processes we
really care about don't get shot.

2) make this a window manager plug in (plug in, as people including us
may end up using other window managers) that uses the stacking order on
the screen to rank order the activities that are running.

3) provide a mechanism by which applications may be given a hint that
they might find it good to save enough state for a checkpoint restart,
because they are likely a good candidate for shooting.

4) use the XRes facilities in X (and/or modify X) to provide the kernel
with the pixmap usage on a process ID basis, for local
applications/activities.

5) see if there are better OOM algorithms that Linux presently has.

Discussion?  Anyone want to take on this project, or parts of this
project?
                                        - Jim

-- 
Jim Gettys
One Laptop Per Child