very simple datastore reimplementation

[Martin Langhoff]

On Sat, May 10, 2008 at 2:15 AM, Jim Gettys <jg at laptop.org> wrote:
> FUSE is great, but...
>
>  It means interoperability must be an explicit planned-in-advance action:

Exactly. That's also my beef with the FUSE approach. The things we can
do via FUSE we can do in the regular FS. So let's avoid FUSE, and use
a *simple* convention on how to store things in the FS. Something
roughly along the lines of:

 - Make a directory per "document", name it
   <user-picked-name>-<10-char-hash>

 - Inside that directory, metadata.txt has simple metadata in a nice,
human-readable format.

 - If there are versions for this file, perhaps they are in a .git
directory, or similar (.svn anyone? ;-) )

 - Otherwise, the files in the directory are whatever the activity or
application considers to be "the document"

 - We use this same format on external media and on local media

 - Jornal can maintain a hidden "documents metadata" database at the
root of each FS. This is merely a cached index to have faster mount
times (avoiding a rebuild of said index if we can deem it safe).

 - We need a reasonable failsafe behaviour if we find that
metadata.txt is b0rken and unreadable.

and that's it. It's a trivial thing, will work on any fs, on any OS,
no magic tricks needed. We can do fast searches on based on the
"documents metadata", and the only "slow" op is mounting a device
where the documents metadata is stale or missing.

IMO, jg is hinting towards something like this. If something like this
is done in a way that it uses atomic ops most of the time (using an
idiom of "write to tmpfile, close and then mv" instead of
write-in-place) then we have something pretty much bulletproof,
failsafe, and fast. Or at least as fast as the kernel-level FS
implementation is.

And it's bound to be a small simple thing too :-)

cheers,



m
-- 
 martin.langhoff at gmail.com
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff