[Sugar-devel] poor man's mmap "sliding window" on Python 2.5.x

[Martin Langhoff]

On Tue, Jul 7, 2009 at 9:23 AM, Martin
Langhoff<martin.langhoff at gmail.com> wrote:
> Thanks for looking into this. I'll post the (trivial, really) repro
> code I have later (I'm on a gruelling >35hr trip at the moment).

Had some time to retest this on the plane, and I think it was
mis-diagnosis. The original code I was testing is lost. In re-testing
this I find that the problem is more nuanced, and I may have been
wrong: looking at 'top', the kernel does not appear very eager to
discard old mapped pages.

The process is doing a linear read through the file, and is slow
enough that it appears only to grow. But if I run another process that
allocates a lot of memory, then the kernel does discard pages pages.

A good way of monitoring this seems to be:

   watch --differences grep -A8 <filename> /proc/<pid>/smaps

So the mmap does the right thing. ACCESS_READ doesn't seem to make any
difference.

#!/usr/bin/python

import mmap
import sys

def mmap_to_death(fpath):
    fh = open(fpath, 'r+')
    mm = mmap.mmap(fh.fileno(), 0, access=mmap.ACCESS_READ)

    l = len(mm)
    c = 0
    buf = ''

    while c < l:
        buf = mm[c]
        c = c+1

    mm.close()
    fh.close()

mmap_to_death(sys.argv[1])



m
-- 
 martin.langhoff at gmail.com
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff