SD cards and DATA_STAT_AFTER_ERASE

[John Gilmore]

> Currently, when installing software, OLPC firmware erases the entire
> disk then writes the entire disk contents, even if most of that is
> zeroes. I am looking at an optimization where we can simply avoid
> writing those 0 blocks, greatly speeding up the flashing process. In
> my test case of 1 SD card, DATA_STAT_AFTER_ERASE=0 and the vendor is
> not lying about this, so I managed to cut down the time needed to
> install an OS image by more than 50%. Hopefully this can apply on a
> wider basis...

I suggest writing paranoia into your code.  Check the flag bit if you
like, but also, read back an erased block and see if it's 1's or 0's.
Hmm!  You can make it perfectly symmetrical: Erase the drive, read
back a block, then compare each block (that you consider writing), to
that block which you read back.  If it erases to all ones, you won't
write any all-one blocks to the drive.  If it erases to all zeroes,
you won't write any all-zero blocks to the drive.

(Of course, when doing these writes, don't do one-block writes to the
drive; accumulate a bunch of them into a single larger write.  If you
know the erase block size, the code could seek to do writes of that
size, aligned on that boundary - particularly after recovering from a
series of blocks it doesn't need to write to.  But some cards may be
able to erase several blocks simultaneously, and may thus prefer a
write of N erase blocks.  Or cards may or may not care whether your
writes are aligned (since they're remapping the blocks anyway through
the flash translation layer), and may just prefer that you always
write one or more erase-blocks' worth of data, no matter what the
alignment.

Since reading flash is much faster than writing (and since one of the
nasty aspects of flash is that writing block X can screw up the
contents of block X-1 or X+1), would you consider reading back the
entire drive after you're done, making sure the whole thing checksums
properly?  (And also making sure that your checksum detects substitutions
of entire all-1 blocks for entire all-0 blocks and vice verse!)

	John