[OLPC-devel] Flash driver status.

[David Woodhouse]

On Sat, 2006-05-13 at 04:29 +0100, David Woodhouse wrote:
> With help from Tom Sylla, I've got this down to about 8½ seconds. I've
> also got the bad block stuff working, albeit slowly. It takes a while to
> start up with a virgin chip, because it scans the _whole_ chip instead
> of just looking for the two byte markers. That's fixable with the
> attached patch, but I haven't committed that because I want to fix it
> properly in the generic code if I can -- which involves working out why
> Thomas was so paranoid in the first place :)

That's fixed now -- we've changed the core code not to be so paranoid,
and now even starting up with a virgin chip is relatively fast. The code
for this is in the 2.6.17-rc4-mm1 kernel and also in my MTD git tree at
git://git.infradead.org/mtd-2.6 -- it'll be sent to Linus as soon as
he's released the 2.6.17 kernel, so it will be in 2.6.18.

The next trick is to reduce JFFS2 memory usage. The first step in that
is to optimise the Linux red-black tree implementation so that it
doesn't waste a whole 'int' for storing a single bit of colour
information -- since the pointers are always going to be aligned, we can
just abuse the lower bits of the 'rb_parent' pointer, and thus shrink
the struct rb_node from 16 bytes to 12.

The Linux red-black trees are used in both JFFS2 'struct
jffs2_node_frag' and the Linux 'struct vm_area_struct' We have one of
the former for every physical node of every file which is open (that's
at least one for every 4KiB of every open file), and we have about five
of the latter for every open library in every running process. That's a
few tens of thousands times four bytes which gets saved.  This is also
in the 2.6.17-rc4-mm1 kernel and going to Linus after 2.6.17.

The next trick I'm working on is to reduce the size of the 'struct
jffs2_raw_node_ref', which we keep for _every_ node on the file system
(not just those belonging to open files). My OLPC board has its flash
about 22% full at the moment, and has 100,000 of these objects. With a
totally full file system, you could have half a million or so.

Each raw_node_ref is currently 16 bytes, and I plan to drop that to 8.
For a start, it contains a 'total length' field, which is largely
redundant because its length can usually be inferred by subtracting its
offset from the offset of the _next_ node on the flash. That works in
99% of cases and we just need to fix the node handling that it works in
_100%_ of cases, then we can drop the 'totlen' field.

Also, each raw_node_ref contains a pointer to the raw_node_ref for the
next physical node on the flash. By storing these in an array instead of
a linked list, we can drop that too. 

Finally, I think I'm going to increase the maximum range of data which
can be covered by one physical node from 4KiB to about 16KiB. That
should reduce the number of nodes too, which will also help keep memory
usage down.

-- 
dwmw2