Treatise on Formatting FLASH Storage Devices

[david at lang.hm]

On Wed, 4 Feb 2009, Benjamin M. Schwartz wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> david at lang.hm wrote:
>> On Wed, 4 Feb 2009, Mitch Bradley wrote:
>>
>>> http://wiki.laptop.org/go/How_to_Damage_a_FLASH_Storage_Device
>>>
>>> Read it and weep.
>>
>> this completely ignores wear leveling, which is very nessasary for just
>> about any filesystem, but especially for FAT (which appear to be the only
>> filesystems this author is familiar with)
>
> Umm, what?
>
> "To alleviate the "wear out" problems, the FTL must move data around so
> that repeated writes to a given sector don't cause too many writes to the
> same NAND page."
>
> Mitch is describing "FLASH" devices like SD cards.  All such devices have
> a built-in microcontroller (the FTL) that performs wear-leveling.
> Layering additional wear-leveling filesystems like JFFS2 or UBIFS on top
> of the FTL requires a reverse translation (block device->MTD) and is not
> recommended.  e.g.  From http://www.linux-mtd.infradead.org/doc/ubifs.html :
>
> "UBIFS was designed to work on top of raw flash, which has nothing to do
> with block devices. This is why UBIFS does not work on MMC cards and the
> like - they look like block devices to the outside world because they
> implement FTL (Flash Translation Layer) support in hardware, which simply
> speaking emulates a block device"

so if the device is performing wear leveling, then the fact that your FAT 
is on the same eraseblock as your partition table should not matter in the 
least, since the wear leveling will avoid stressing any particlar part of 
the flash.

as such I see no point in worrying about the partition table being on the 
same eraseblock as a frequently written item.

as for the block boundry not being an eraseblock boundry if the partition 
starts at block 1

if you use 1k blocks and have 256k eraseblocks, then 1 out of every 256 
writes will generate two erases instead of one

worst case is you use 4k blocks and have 128k eraseblocks, at which point 
1 out of every 32 writes will generate two erases instead of one.

to use the intel terminology, these result in write amplification factors 
of approximatly 1.005 and 1.03 respectivly.

neither of these qualify as a 'flash killer' in my mind.

now, if a FAT or superblock happens to span an eraseblock, then you will 
have a much more significant issue, but nothing that is said in this 
document refers to this problem (and in fact, it indicates that things 
like this follow the start of the partition very closely, which implies 
that unless the partition starts very close to the end of an eraseblock 
it's highly unlikely that these will span eraseblocks)

so I still see this as crying wolf.

as for ubifs, that is designed for when you have access to the raw flash, 
which is not the case for any device where you have a flash translation 
layer in place, so it is really only useful on embedded system, not on 
commercially available flash drives of any type.

> As for the author only being familiar with FAT, that is hilarious.  Mitch
> implemented JFFS2 support in OFW, and wrote this page to explain how he
> produced optimal ext2 formatting of FTL FLASH.  Indeed, that is the
> subject of
> http://wiki.laptop.org/go/How_to_Damage_a_FLASH_Storage_Device#Screwed-Up_Formatting

I didn't read carefully enough before I made that comment. my apologies.

David Lang