Sample large datastore

[Martin Langhoff]

On Mon, May 19, 2008 at 9:26 PM, Tomeu Vizoso <tomeu at tomeuvizoso.net> wrote:
> On Mon, May 19, 2008 at 10:30 AM, Tomeu Vizoso <tomeu at tomeuvizoso.net> wrote:
>>
>>    results, count = data_store.find({}, ['uid'])
>
> You may want to limit the amount of results returned by
>
> results, count = data_store.find({'limit': 20}, ['uid'])
>
> as trying to get too much info from the DS in one go can make things
> very slow and ultimately OOM.

Ah. Uhmmm... Does the "data object" in the datastore contain only the
metadata? Or do we get a copy of the file stashed in a variable?

  ... this is worrying... though at least it would explain in one
simple blow the OOM and perf problems.

cheers,


m
-- 
 martin.langhoff at gmail.com
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff