[Sugar-devel] Technique to extract all records from xapian DB?

[Martin Langhoff]

On Wed, Dec 16, 2009 at 9:46 PM, Tomeu Vizoso <tomeu at sugarlabs.org> wrote:
> On Wed, Dec 16, 2009 at 17:48, Martin Langhoff
> <martin.langhoff at gmail.com> wrote:
>> On Fri, Dec 11, 2009 at 2:35 PM, Tomeu Vizoso <tomeu at sugarlabs.org> wrote:
>> Yeah. It's a very complex beast that, after following the callstack
>> hither and yon, only makes a v simple mapping. I've written a 20 line
>> mini-reader class using xapian directly. Instead of following the DS
>> code, I used the CLI tools that xapian includes, and it is a trivial
>> thing. Needs a bit more time but it'll work.
>
> Sounds good, if you need later to move to python, we can do something
> easily if we know the prefixes for each field.

The code is all python using the xapian bindings. I used the CLI tools
to figure out the data layout...  because I hadn't spotted the mapping
array you've just given me :-)

> I think it should be stable for the fields we care about, which is a fixed set:
>
> http://git.sugarlabs.org/projects/sugar-datastore/repos/mainline/blobs/sucrose-0.82/src/olpc/datastore/model.py#line379

Will use that and stop guessing. They still have odd prefixes which I
have to strip out, some do, some don't... the whole thing is rather
strange. I don't know if I want to know the hysterical raisins behind
it...


m
-- 
 martin.langhoff at gmail.com
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff