[PATCH] Maintain a metadata copy outside the index (was Re: Datastore & backup - request for help)

[Tomeu Vizoso]

On Fri, May 23, 2008 at 10:19 AM, Tomeu Vizoso <tomeu at tomeuvizoso.net> wrote:
> On Fri, May 23, 2008 at 6:27 AM, Martin Langhoff
> <martin.langhoff at gmail.com> wrote:
>> On Fri, May 23, 2008 at 3:25 PM, Martin Langhoff
>> <martin.langhoff at gmail.com> wrote:
>>> I'll diagnose a bit more and post back.
>>
>> I'm baffled. I initially suspected PEBKAC but it keeps happening. With
>> the patch the DS gets renamed, and journal never starts. If the
>> datastore code gets to complain somewhere I haven't been able to find
>> it.
>>
>> If I back the patch out, and rename the datastore back to "datastore",
>> I get a new, empty datastore on reboot. (Data files are there,
>> though).
>>
>> Nothing in the patch looks suspicious so I am a bit lost. Might still
>> be a problem with something silly. I am testing this on a 703 image
>> where I moved /usr/lib/python2.5/site-packages/olpc/datastore aside,
>> and replaced it with a symlink to my git checkout. The git checkout
>> was identical to the files from the 703 image, and I applied your
>> patch.
>>
>> I've now merged in your latest commit, but it's just a version
>> identifier change, so bah.
>
> Ouch, sorry, I should have been more explicit in that this patch
> introduces a dependency on cjson. I'm going to send next another patch
> that falls back to simplejson (slower) if cjson is not available.

Attached, I'm not sure this is the right approach for json.

When we care about performance and not about interchanging complex
data with other platforms, may be better to do our text-based format
for storing the metadata.

So we should perhaps drop cjson in this patch, and move to a custom
format when the DS starts reading the metadata for those files.

If you have any other problem with the DS:

- check the shell.log file in ~/.sugar/default/logs
- check datastore.log
- start the DS manually by 'datastore-service -m'

Regards,

Tomeu
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Maintain-a-metadata-copy-outside-the-index.patch
Type: text/x-patch
Size: 7670 bytes
Desc: not available
URL: <http://lists.laptop.org/pipermail/devel/attachments/20080523/e2afde54/attachment.bin>