[OLPC-devel] Flash driver status.

[David Woodhouse]

On Tue, 2006-05-16 at 00:46 +0100, David Woodhouse wrote:
> The next trick I'm working on is to reduce the size of the 'struct
> jffs2_raw_node_ref', which we keep for _every_ node on the file system
> (not just those belonging to open files). My OLPC board has its flash
> about 22% full at the moment, and has 100,000 of these objects. With a
> totally full file system, you could have half a million or so.
> 
> Each raw_node_ref is currently 16 bytes, and I plan to drop that to 8.
> For a start, it contains a 'total length' field, which is largely
> redundant because its length can usually be inferred by subtracting its
> offset from the offset of the _next_ node on the flash. That works in
> 99% of cases and we just need to fix the node handling that it works in
> _100%_ of cases, then we can drop the 'totlen' field.

That much is done in git://git.infradead.org/jffs2-devel-2.6.git

I'll be pulling that into the main mtd-2.6.git tree when it's had a
little bit more testing.

> Also, each raw_node_ref contains a pointer to the raw_node_ref for the
> next physical node on the flash. By storing these in an array instead of
> a linked list, we can drop that too. 

That's next...

> Finally, I think I'm going to increase the maximum range of data which
> can be covered by one physical node from 4KiB to about 16KiB. That
> should reduce the number of nodes too, which will also help keep memory
> usage down. 

... then that, which should also improve compression by about 11% in
some cases -- if we're compressing 16KiB blocks at a time instead of
only 4KiB we do much better.

Taking /lib/libc-2.4.so from my workstation, for example...
The file itself is 1528292 bytes.

That's 374 blocks of 4KiB, which compress to an average of 1989 bytes --
total 744228 bytes. (48.7% of original size).

Or it's 94 blocks of 16KiB which compress to an average of 7193 bytes --
total 676142 bytes (44.2% of original size).

That's just the data payload -- you gain a little bit more because you
have a 68-byte node header on each data node. Counting the node headers
too, the size taken on the flash for the 4KiB case will be ~769660 bytes
and the size on the flash for the 16KiB case will be ~682534 bytes -- a
saving of 11% or so by switching to 16KiB.

In practice I don't think the _overall_ benefit will be quite that much
-- you'll only get the best of it when you're writing _full_ 16KiB data
nodes, which doesn't happen on small files or at the end of a file.
Files which were under 4KiB in the first place aren't going to be
affected at all, for example.

Some files might get better than 11% improvement, if they compress
better than PPC binary code does. I suspect that'll be relatively minor
though.

-- 
dwmw2