[sugar] DS possibilities

[Tomeu Vizoso]

On Thu, Apr 24, 2008 at 1:36 AM, Martin Langhoff
<martin.langhoff at gmail.com> wrote:
> On Thu, Apr 24, 2008 at 7:42 AM, Benjamin M. Schwartz
>  <bmschwar at fas.harvard.edu> wrote:
>  >  Reports from the field, especially from Carla and Bryan, have indicated
>  >  that the datastore can get into a corrupted state from which it cannot
>  >  recover.  The corruption persists over a reboot.  After corruption,
>
>  Can we get copies of the corrupt DS files?

You have one here:

http://dev.laptop.org/ticket/6864

>  As per the irc discussion today, I think we can
>
>   - teach the DS backend how to recover gracefully if it finds a corrupt entry
>   - tweak the most vulnerable points of the DS code to be more atomic

Agreed. My only doubt is if we should refactor or rewrite.

One thing that I'd like to make clear is that I have never seen a
corrupted Xapian index, nor have I heard of anything that points to
this.

The problems we have had with uninitializable datastores were due to
trying to open an index with a too old release and with the 'config'
file corrupted (http://dev.laptop.org/ticket/5494 and
http://dev.laptop.org/ticket/6864). That file is not part of xapian
but of the high-level python bindings (secore), and we are using a
quite old copy of that.

>  A good trick is to start yanking the power (or kill-9'ing the DS
>  process) while something is being saved.
>
>  Most of the times I've seen this kind of corruption, changing some
>  open calls to make a copy beforehand was enough. If one document is
>  lost due to powerloss, tough, the important thing is that on reboot
>  everything still works, even if we lost that document.
>
>
>  >  subsequent datastore calls usually raise exceptions, which are not handled
>  >  by the Activities (including the Journal) and so no Activities will load,
>  >  and Sugar is unusable.
>
>  Activities should handle errors returned by the DS :-/ those are
>  activity bugs. But we assume a generally working DS for a working
>  Sugar environment.

Agreed.

Thanks,

Tomeu