ext2 vs ext4 vs exFAT for XO content SD cards?

[Arnd Bergmann]

On Saturday 15 August 2015 21:05:48 Adam Holt wrote:
> What filesystem would people recommend for ~128GB SD cards inserted into XO
> laptops or XSCE servers heading far afield, to insert very large
> (evolving?) digital libraries just like Internet-in-a-Box?
> 
> Multiyear reliability for all this content would be important, but some say
> ext4 (slow journaled filesystems?) have their own problems on SD?

ext4 is better than most file systems in this regard (much better than ext3,
also better than btrfs).

> Is wear-leveling of modern SD cards (Flash memory) fully taken care of by
> the largest manufacturers/drivers already, or should we seek out particular
> filesystems/drivers?  And unmount / power off carefully etc?

Some cards handle it better than others. Traditionally, Kingston cards
were completely unusable after writing data on the order of the card size,
while Samsung and Sandisk did much better (I have not checked in a few years
though, so YMMV). Most other manufacturers have multiple sources for the
embedded controllers on the cards, so it's a bit of hit and miss. The
controllers have all evolved a lot in the past few years, the simple
controllers that used to be on 8GB cards can no longer cope with the
increased block sizes on today's cards (32GB and larger).

> In-country copying and eventually in-country remixing of SD cards
> (containing local-language, local-vid cultural jewels) would be Absolutely
> Wonderful -- if we can achieve that by using a filesystem that works
> instantly (exFAT? NTFS?) across all Windows and Mac computers ideally?  For
> all static content anyway, on 1 partition of several?  Accelerating dup'ing
> of SD cards as a bonus?!
> 
> Many paths possible for sure, but most important: what
> interop/longevity/other issues are we overlooking??

SDXC cards are mandated to be using exFAT (just like SDHC cards have
to use VFAT, and indeed this is the only difference between the two)
by the SD card standard. If you don't use this, you are strictly speaking
in violation of the standard and the cards might not behave as designed.
In particular, the card is allowed to only do efficient garbage collection
for the access patterns that you get with a single exFAT partition that
spans the entire card and has all its metadata aligned exactly in the
way that the spec defines.
In practice, things tend to work mostly ok with other file systems, but
if you use NTFS or ext3 (rather than ext4), you are usually asking for
trouble.

The best longevity would be provided by f2fs, which is designed to work
fine on most SD cards. The downside is that it only works on relatively
modern Linux kernels (3.x or higher).

I would expect that cards today use only dynamic wear leveling, not
static wear leveling as real SSDs do. This means that content on a
read-only partition will decay with the normal life of the card
(several years, but depending on the quality of the card and the
environmental conditions, e.g. not too hot), independent of the
presence of partitions you write to.

Dynamic wear leveling works best if you have a lot of unused blocks,
so a good strategy for long life would be to leave a whole partition
(e.g. 20% of the size of the writable partition, the more you have,
the longer the card will survive) that never gets written after
manufacturing, or at least gets erased using the fitrim ioctl
command after the initial imaging.

For a 128 GB card with 115GB of actual space, you could then use something
like:

* 80GB zisofs/cramfs/squashfs for static data
* 30GB f2fs/ext4 for writable data
* 5GB unused space for wear leveling

	Arnd