ext2 vs ext4 vs exFAT for XO content SD cards?

[Adam Holt]

Thanks You Arnd, we have many months work ahead of us and this will be a
big help!

On Thu, Aug 20, 2015 at 4:55 PM, Arnd Bergmann <arnd at arndb.de> wrote:

> On Saturday 15 August 2015 21:05:48 Adam Holt wrote:
> > What filesystem would people recommend for ~128GB SD cards inserted into
> XO
> > laptops or XSCE servers heading far afield, to insert very large
> > (evolving?) digital libraries just like Internet-in-a-Box?
> >
> > Multiyear reliability for all this content would be important, but some
> say
> > ext4 (slow journaled filesystems?) have their own problems on SD?
>
> ext4 is better than most file systems in this regard (much better than
> ext3,
> also better than btrfs).
>
> > Is wear-leveling of modern SD cards (Flash memory) fully taken care of by
> > the largest manufacturers/drivers already, or should we seek out
> particular
> > filesystems/drivers?  And unmount / power off carefully etc?
>
> Some cards handle it better than others. Traditionally, Kingston cards
> were completely unusable after writing data on the order of the card size,
> while Samsung and Sandisk did much better (I have not checked in a few
> years
> though, so YMMV). Most other manufacturers have multiple sources for the
> embedded controllers on the cards, so it's a bit of hit and miss. The
> controllers have all evolved a lot in the past few years, the simple
> controllers that used to be on 8GB cards can no longer cope with the
> increased block sizes on today's cards (32GB and larger).
>
> > In-country copying and eventually in-country remixing of SD cards
> > (containing local-language, local-vid cultural jewels) would be
> Absolutely
> > Wonderful -- if we can achieve that by using a filesystem that works
> > instantly (exFAT? NTFS?) across all Windows and Mac computers ideally?
> For
> > all static content anyway, on 1 partition of several?  Accelerating
> dup'ing
> > of SD cards as a bonus?!
> >
> > Many paths possible for sure, but most important: what
> > interop/longevity/other issues are we overlooking??
>
> SDXC cards are mandated to be using exFAT (just like SDHC cards have
> to use VFAT, and indeed this is the only difference between the two)
> by the SD card standard. If you don't use this, you are strictly speaking
> in violation of the standard and the cards might not behave as designed.
> In particular, the card is allowed to only do efficient garbage collection
> for the access patterns that you get with a single exFAT partition that
> spans the entire card and has all its metadata aligned exactly in the
> way that the spec defines.
> In practice, things tend to work mostly ok with other file systems, but
> if you use NTFS or ext3 (rather than ext4), you are usually asking for
> trouble.
>
> The best longevity would be provided by f2fs, which is designed to work
> fine on most SD cards. The downside is that it only works on relatively
> modern Linux kernels (3.x or higher).
>
> I would expect that cards today use only dynamic wear leveling, not
> static wear leveling as real SSDs do. This means that content on a
> read-only partition will decay with the normal life of the card
> (several years, but depending on the quality of the card and the
> environmental conditions, e.g. not too hot), independent of the
> presence of partitions you write to.
>
> Dynamic wear leveling works best if you have a lot of unused blocks,
> so a good strategy for long life would be to leave a whole partition
> (e.g. 20% of the size of the writable partition, the more you have,
> the longer the card will survive) that never gets written after
> manufacturing, or at least gets erased using the fitrim ioctl
> command after the initial imaging.
>
> For a 128 GB card with 115GB of actual space, you could then use something
> like:
>
> * 80GB zisofs/cramfs/squashfs for static data
> * 30GB f2fs/ext4 for writable data
> * 5GB unused space for wear leveling
>
>         Arnd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.laptop.org/pipermail/devel/attachments/20150820/5ca5f142/attachment.html>