olpcfs next steps (was very simple datastore reimplementation)

[C. Scott Ananian]

On Fri, May 9, 2008 at 6:09 AM, Tomeu Vizoso <tomeu at tomeuvizoso.net> wrote:
>  > The current implementation of indexes is incomplete, and there are
>  > some design questions regarding how search results for certain key
>  > types ought to be presented using the POSIX API.  These are not
>  > showstoppers, but I/we should finish the implementation before Journal
>  > search would be expected to work.
>
>  Have you already thought about using something like Xapian for all the
>  search needs? Eben will correct me, but I think that we want pretty
>  advanced "Google like" search. Implementing all this on top of a
>  berkeley db may mean duplicating a lot of work.

I think "google like" search may be a separate functionality on top of
the filesystem.  I'll think about this more.  The current search
functionality is based on exact searches, and it uses the same basic
datastructures we use for data storage in the filesystem.  "Google
like" searches seem like a very different thing, and I'm not sure it's
wise to mix them.  You really want something like Beagle, strigi, or
http://en.wikipedia.org/wiki/List_of_search_engines#Desktop_search_engines
for that, since you want to take apart a large number of
application-specific data formats to extract keywords and index those.
 olpcfs is concentrating on indexing explicit filesystem metadata,
like tags, not on the larger task of extracting and indexing full-text
from documents.

For "not reinventing the wheel" reasons, I'd prefer to see full-text
search be based on an existing desktop search engine.

>  > There are some journal integration questions, for
>  > example w.r.t. grouping objects by action -- if a number of files have
>  > 'action_id' set to (say) 1, is there a file named '1' with mime-type
>  > 'text/sugar-action' somewhere with more details about the action?
>  > That sort of thing is best figured out by actually hacking up some
>  > code and figuring out what extra information needs to be stored.
>
>  Yes, we have some unfortunate uncertainty there. But how this affect olpcfs?

It doesn't, directly.  I was just trying to enumerate the list of
"things to do".
 --scott

-- 
 ( http://cscott.net/ )