olpcfs next steps (was very simple datastore reimplementation)

[Tomeu Vizoso]

On 5/8/08, C. Scott Ananian <cscott at laptop.org> wrote:
> On Thu, May 8, 2008 at 3:11 PM, Marco Pesenti Gritti <mpgritti at gmail.com> wrote:
> > On Thu, May 8, 2008 at 8:43 PM, C. Scott Ananian <cscott at laptop.org> wrote:
> >  >  Again, in the olpcfs design we provide a vfat-like metadata structure
> >  >  (mounted via FUSE) so that the filesystem on the USB key appears to
> >  >  the journal code just like the flash does.  No FUSE is necessary to
> >  >  see the files on the Windows box, although if you want to see all the
> >  >  metadata then you need some tool.  If you're transferring the file to
> >  >  another XO, then the metadata comes along with you.
> >
> >  These are all very good points in favor of olpcfs to me.
> >
> >  Personally I'm pretty much sold on the design. And I'm all for
> >  experimenting with it on a branch (try to port journal and a few
> >  activities), even if it's not a priority for August. Are you aware of
> >  any showstopper in this regard? Feature wise it seem to be pretty much
> >  there already.
>
> The current implementation of indexes is incomplete, and there are
> some design questions regarding how search results for certain key
> types ought to be presented using the POSIX API.  These are not
> showstoppers, but I/we should finish the implementation before Journal
> search would be expected to work.

Have you already thought about using something like Xapian for all the
search needs? Eben will correct me, but I think that we want pretty
advanced "Google like" search. Implementing all this on top of a
berkeley db may mean duplicating a lot of work.

I think that with a couple dozens of new lines of code, I can bring
the DS rewrite to the functionality of the old DS. That will appear
(hopefully soon) in:

http://dev.laptop.org/git?p=users/tomeu/datastore;a=blob;f=src/olpc/datastore/indexstore.py

> I also haven't implement version
> tagging or version gc, but these wouldn't be required to start playing
> with things.

Agreed.

> There are some journal integration questions, for
> example w.r.t. grouping objects by action -- if a number of files have
> 'action_id' set to (say) 1, is there a file named '1' with mime-type
> 'text/sugar-action' somewhere with more details about the action?
> That sort of thing is best figured out by actually hacking up some
> code and figuring out what extra information needs to be stored.

Yes, we have some unfortunate uncertainty there. But how this affect olpcfs?

Thanks,

Tomeu