zhashfs: write first block last

[Mitch Bradley]

On 5/5/2011 9:08 AM, Daniel Drake wrote:
> On 5 May 2011 08:57, James Cameron<quozl at laptop.org>  wrote:
>> Q3A65 removed this blanking, because it interfered with NANDblaster,
>> which stores the received blocks of the .zd file in the SD card before
>> fs-update begins.  On extending the blanking to the whole device, the
>> .zd file being read by fs-update was also erased from about line 68
>> of os860.zd generating "/nb-updater:68: Missing newline after zdata".
>>
>> So I think writing block 0 as all-zero will be needed.
>
> Does nandblaster temporarily store the image on disk starting at block
> 0? If so, block 0 already will be lacking a boot sector.

The image is stored at the end, so that when it is later expanded it 
won't overwrite not-yet-expanded parts of the image.  A delicate little 
dance.

>
> Is it possible to fix the erasing? Perhaps nandblaster could do it
> once it has confirmed that there's a server transmitting, before
> starting the receive process.

It would be necessary to erase the entire device, rather than just the 
number of blocks specified by the zblocks: line, because that number is 
not known until very late in the process.

Erasing everything is probably not a bad idea.

> It has a few advantages:
> - nicely solves this issue :)
>
> - Second-guessing how cards actually work, I expect having less SD
> card sectors occupied means less work for the housekeeping routines of
> the embedded flash controller
>
> - Guesswork again, writing to erased sectors is probably quicker than
> writing to occupied sectors. Flash memory needs erase-before-write,
> right? Hopefully not if its already erased.
>
> - I'm hoping that moving to lazy inode table initialization means that
> we can implement quick on-the-fly resizing to utilise all SD card
> space in future. The one danger with this is block group descriptors
> being reused from a previous filesystem which has some remnants left
> on disk:
>
>      The fact that parts of the inode table are
>      uninitialized is not a problem so long as the block group descriptors,
>      which contain information regarding how much of the inode table has
>      been initialized, has not been corrupted However, if the block group
>      checksums are not valid, e2fsck must scan the entire inode table, and
>      the the old, uninitialized data could potentially cause e2fsck to
>      report false problems.
>
> Erasing the disk in advance means that there's no danger of old
> descriptors being there.
>
> - Around 4% of the writes of a sparse zd file are zero-blocks. I'm
> still hoping to persuade someone to implement the logic I originally
> proposed (in addition to the sparse thing) to further speed up
> fs-update in the common case. The disk needs to be pre-erased for that
> to work.
>
> - It's nice to have for the privacy aspect
>
>> Do you have a corresponding patch to zdextract in bios-crypto?  Your
>> change to zhashfs has broken zdextract.  We don't use zdextract in the
>> tool chain, but it was a surprise to find it no longer works correctly.
>
> Sorry, wasn't aware of this tool. Perhaps you could fix it for sparse
> first, and if it's not trivial to fix it for block-zero-last files at
> the same time, I can work on that afterwards.
>
> Thanks,
> Daniel