very simple datastore reimplementation

[Tomeu Vizoso]

On 5/8/08, Jim Gettys <jg at laptop.org> wrote:
> On Thu, 2008-05-08 at 16:06 +0200, Tomeu Vizoso wrote:
> > On 5/8/08, Jim Gettys <jg at laptop.org> wrote:
> > > On Thu, 2008-05-08 at 13:09 +0000, Tomeu Vizoso wrote:
> > >
> > > > I'm having trouble understanding what you are requesting and what
> > > > could be done about that.
> > > >
> > > > Can you please enumerate the requirements that affect the internal
> > > > file layout and any other view that we may be able to provide?
> > >
> > > That there is *some* hope of finding a file by a human in a raw file
> > > system, that can be done with software already present on the system....
> >
> > With the proposed metadata text file, there's already that hope. You
> > think it's not enough and you may very well be right. What I'm asking
> > is: how big an effort are we willing to devote to this and until which
> > point we want to compromise on robustness and simplicity?
>
> Until we know what the tradeoffs really are, we need to explore in this
> direction.  Names only as hashes has proved to be a major headache in
> practice in the field.

Ok, as I'm still confused about which are the requirements, I'm going
to have a try at writing them down and I ask you to complete it:

Requirements that affect naming of files _inside_ the DS:

- Debuggability in the field. The user has some problem and wants to
know what is going on in the internal structures of the DS.

- Robustness. Any component that does any magic regarding the naming
of files can fail. If there was no magic going on, the system as a
whole would become more robust.

- Access the internal structures in a different system with no custom software.

Requirements that affect naming of files _as exposed_ by the DS:

- Compatibility with legacy apps. The current D-Bus API doesn't care
about filenames, but there's interest in that unmodified applications
can have some degree of interaction with the DS. For this, we should
have some kind of POSIX API that allows unix apps to access the DS as
any other file system. This includes the use of cp, tar, zip, etc to
take files out from the DS.

> > > This may be to enable manual retrieval in backups (without having to
> > > restore everything) as well as interoperability.  Just having a name
> > > that is a hash makes this tough...
> >
> > Is it ok if only the last version of this file is available in that
> > way? Past versions may be stored as reverse deltas
>
>
> I don't see what the fact the file might contain a delta would have to
> do with with the naming, though, certainly being able to easily
> distinguish the most recent file from older deltas by the name is
> something humans would like to be able to do.

If we internally store deltas, some kind of magic will need to happen
so the user can access other than the last version.

Thanks,

Tomeu