No surprise on memory

[Tomeu Vizoso]

On Sat, Dec 20, 2008 at 23:57, Albert Cahalan <acahalan at gmail.com> wrote:
> [multiple people]
>
>> I recently learned a few very important things about Linux memory
>> management (I'm speaking about how its supposed to work, irrespective
>> of any bugs).  Operating systems experts already know all of this,
>> but I did not.
>
> This is a good reminder for those of us who tend to assume that
> anyone joining these discussions is an OS expert.
>
>> Conclusion: no magic get-out-of-jail-free card.
>
> There certainly isn't anything that can work with perfect
> reliability, even if policy was to disable overcommit and
> check malloc everywhere.
>
> Pay particular attention to how every proposed solution meets
> the real goals, remembering that nearly all activities save the
> user's data via a non-atomic process that requires memory.
> Simply put, it is never acceptable to force a well-bahaved
> activity to die or live without the memory it demands.

It's great that you have such high quality standards, but I hope that
in case we fail to reach those, we find a way to at least improve in
some way what we already have.

>>> It may be interesting to adjust the OOM score of some applications.
>>> This way it should be possible to protect the core applications
>>> (sugar-shell, journal, X, ...) from being killed in an OOM situation.
>>
>> I'm with Benjamin here, if the OOM killer kicked in soon enough and
>> activities were clearly marked as first candidates to be killed,
>> stability would be much much better.
>
> No way.
>
> The core applications only exist for the desired activity. If that
> desired activity must die, you might as well power off the laptop.
> The only processes slightly worth saving are klogd and syslogd,
> allowing developers to figure out what just happened.

I would agree with you if our users ran only one activity at a time,
but I think that's not the case. If it was, we rarely would run out of
memory.

What is happening right now is that the user launches several
activities even if may not need all of them, and when the system runs
out of memory the whole system dead locks, losing the data of the
active activity and having to restart the system.

What I suggested would cause one of the background activities to die,
and it would have already saved its state to the journal when it went
to the background. The user would keep working and in order to resume
the work on the killed activity would only need to go to the journal
and click on one of the icons at the top of the list.

Powering off the system sounds to me as less convenient to the user.

>> And if background activities were killed before the active one,
>> we would avoid data loss.
>
> Background activities can contain valuable unsaved state too.

All python activities are requested to save their state when they go
to the background. Non-python activities could chose that event to
trigger an auto save if they wanted.

> Of course, this is somewhat theoretical because kids do not
> intentionally have background activities. The ten activities running
> in the background on a typical kid's XO are a big contributer to
> these memory problems.

Then I don't understand why you said above that killing one of the
background (probably non-intentional) activities is as bad as powering
off the laptop.

>> Combine that with Mac OS (pre X) style "estimated memory allocation"
>> metadata for each activity and the user experience could perhaps even
>> work.
>
> This is key. Until the UI absolutely refuses to let the user start
> a set of 2+ activities that could run out of memory, memory problems
> are a given. For activities with unbounded memory usage, this means
> they get the machine exclusively.

I certainly cannot imagine activity authors properly filling that
field, given that most activities we have today have a wrong version
field. Also, as a child used the old MacOS quite a bit and remember
having to mess with those fields quite often.

Regards,

Tomeu