Alternate proposed logic for fs-update unused blocks optimisation

[James Cameron]

On Tue, Apr 19, 2011 at 06:09:09PM +0100, Daniel Drake wrote:
> Maybe dependent on the possibility of an extX-specific optimization in
> the .zd file, ...

Here's another optimisation for review as well.  It isn't quite
extX-specific though.

On a 253MB .zd4 file the fs-update took three minutes on Toshiba Class 4
SD, compared to the usual 15 minutes.  It took four minutes on the usual
SanDisk SD.

This was a test build of 10.2 containing only Sugar and six activities.
This five-fold reduction in fs-update would not occur for your current
builds, but there would be some considerable reduction anyway,
especially on larger SD card images where the ratio of unused blocks to
used blocks is higher.

The algorithm is:

- create a psuedo-random marker block of 4096-bytes, called a fill
  pattern,

- instead of creating a sparse image file, create the image file by
  appending enough of the fill pattern blocks to reach the requested
  size, ... on a fast system this is an extra few seconds,

- during zhashfs, skip processing of zblocks that contain only the fill
  pattern, and for those that contain the fill pattern in only part of
  the zblock, zero it so that the fill pattern is not used as input to
  the compression.

The visual effect during fs-update is wonderful.  The display skips
forward, leaving skipped blocks grey, and begins writing green blocks.
It clearly shows how the space is allocated.

There is one small problem left to fix; Q3A64 fs-update displays an
error at the end because the total number of blocks written has not
agreed with the number of blocks specified at the top of the file.  I
propose to fix that somehow, but I've yet to decide on exactly how.  It
was introduced by -r2186, where on your request Mitch changed the check
from last block written to number of blocks written.  It does not happen
with Q3A62.

The written filesystem images pass "e2fsck -f" when run from a USB
booted Linux, and the laptop boots and runs normally.  An "rpm -Va"
looks normal, when the olpc-os-builder timestamp changes are filtered
out.

The attached patch is to olpc-os-builder, unfortunately to an instance I
had lying around gathering dust.  Good enough for testing the idea.
It's from 27th July 2010, but the change looks easy to apply to master.

-- 
James Cameron
http://quozl.linux.org.au/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: olpc-os-builder-fs-update-steroids-2.patch
Type: text/x-diff
Size: 4222 bytes
Desc: not available
URL: <http://lists.laptop.org/pipermail/devel/attachments/20110420/87c11c25/attachment.patch>