[Code-review] [PATCH] Maintain a metadata copy outside the index.

[Benjamin M. Schwartz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin Langhoff wrote:
| On Wed, Jun 25, 2008 at 6:55 PM, Benjamin M. Schwartz
| <bmschwar at fas.harvard.edu> wrote:
|> I agree that it's impractical to hack atomicity into the current system,
|> but any future datastore should _definitely_ provide strong
|> atomicity/checkpoint guarantees.  Designing this in should not be
|> difficult, and even if it is, it is worth doing.
|
| This *is* difficult. A FS that performs great on NAND flash devices
| and provides atomicity and checkpointing is not trivial. Keep in mind
| that activities do a wide range of FS operations - including capture
| and playback of sound and video and large files - so mmap and all the
| relevant posix ops need to be there, and be efficient.

I fully agree.  That's why I don't think the datastore should be
implemented as a File System*.

Instead, I think we should follow a push/pull model.  Activities can write
their files directly into a standard file system, and then atomically push
those files into the datastore.  Activities can also request objects from
the data store (also atomically), in which case they should be provided as
standard files in the filesystem. At no time should Activities be able to
modify the datastore's internal originals.

Implementing such behavior by simply making temporary copies is trivial.
If making temporary copies is seen as unpalatable, then a similar effect
may be achieved by manipulating permissions so that the datastore provides
read-only files to the Activities, and each activity is responsible for
making copies before writing.  A kernel-level CoW system might also be
convenient here, if available, but it is not a prerequisite.

- --Ben

*: In some abstract sense, a filing system and a data store are two
phrases for the same thing.  In this context, a File System refers to a
very specific sort of program that is built to the Linux VFS and is
accessed through various kernel-level mechanisms.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkhjGC8ACgkQUJT6e6HFtqRkJQCeKBdVelDLJt3KRq7WZjSGgWRA
Mp8An3etFQop6CskmrZGRZHWApR6pH53
=r1Pf
-----END PGP SIGNATURE-----