very simple datastore reimplementation

[Eben Eliason]

On Fri, May 9, 2008 at 2:26 PM, Joshua N Pritikin <jpritikin at pobox.com> wrote:
> On Fri, May 09, 2008 at 12:10:07PM -0600, Jameson Chema Quinn wrote:
>> To be more clear about this use case: I think that there should definitely
>> be a way for the onboard datastore to store the metadata for an absent file,
>> with hints about what place(s) to find that file (networked backup, sd
>> cards, usb devices) and how to recognize it when you do. This should include
>> the possibility for offloading old intermediate versions. Then, even when
>> you do not have access to the backup storage, you can see what you are
>> missing. This makes the result of suddenly yanking the SD card out more
>> well-defined (assuming no filesystem corruption), and means you do not ever
>> have to merge/separate two indexes (there is just one index).
>
> I was surprised to read this. My opinion is that the index should only
> include files which are available on local storage. Otherwise the index
> can fill up with broken links, and it will be difficult to explain why
> the broken links don't work. Access to backups is a good idea, but not
> via such a by-default mechanism.

Actually, I quite like the idea that the record of history is retained
locally, even when files are not. This is the type of system we've
been designing the Journal to support, should we want to, for some
time.  The notion that the Journal serves as a log of everything
you've done, regardless of whether or not the file is presently
available, is an interesting idea.  Of course, its absolutely
necessary to make it quite obvious when the links are broken so that
the distinction is clear.  Another option, of course, would be to keep
the single index, but only represent the "broken links" when it is
possible to recover them (eg. the backup service is currently
reachable).

- Eben