fixing etoys

[Albert Cahalan]

Here are some ideas that might help you fix some of the problems
with start-up performance, shut-down performance, open source,
and software engineering practices.

You're trying to do a persistant system image on an OS that wasn't
really designed for it. If you were on an exotic system with a
persistant image (like EROS, KeyKOS, or OS/400), you might be able
to cut the power at any time without losing more than a few seconds
of work. The key here is that the persistant system image OSes
continuously write out changes to disk. They do this atomicly,
and in small chunks, rather than overwriting everything at once in
a dangerous non-atomic operation.

You're trying to do a persistant system image on an OS that wasn't
really designed for it. If you were on an exotic system with a
persistant image (like EROS or OS/400), you'd be able to cut the
power at any time without losing more than a few seconds of work.
The key here is that the persistant system image OSes continuously
write out changes to disk. They do this atomicly, and in small
chunks, rather than overwriting everything at once in a dangerous
non-atomic operation.

Start with that. Break each object out into a separate file.
Your database becomes a directory rather than a big blob file.
When an in-memory object changes, you write a copy to a fresh
new file on disk. You keep the old on-disk copy around until
the new one has been synched. (fsync or sync probably, but be
very careful to avoid doing this more than once every few
seconds) You may need subdirectories for better performance;
it is very unwise to rely on Reiserfs-like performance when
having large directories.

On-demand loading is required for start-up performance.

Inheritance from a read-only image will cut per-instance disk size.
On a Debian system, install the read-only image under /usr/share
and the per-instance changes somewhere under $HOME. On the XO,
the read-only image stays in the activity and the per-instance
part goes into the journal. For older XO system software which
does not support grouping multiple files into single Journal entries,
you'll have to do it yourself with a standard archive format.
There are three to choose from: tar, cpio, pax.

To make things maintainable outside of the walled garden, store the
objects in text form. Make them be like nicely formatted source code.
Be permissive in parsing them, and try to preserve any comments that
might be added by users with regular text editors. Try to maximize
compatibility with GNU Smalltalk.

If you have bulk data that would not reasonably be editable with
a text editor, such as PNG images, then leave that part in binary.

To cut down on dirty anonymous pages in memory, and thus greatly
improve the situation on low-memory swapless systems like the XO,
you could do mmap(...,PROT_WRITE|PROT_READ,MAP_PRIVATE,...) on the
binary blobs at startup. This should just be things like PNG images.