Python memory usage (Was: The "iGoogle bug")
bernie at codewiz.org
Wed Sep 19 10:29:22 EDT 2007
Ivan Krstić wrote:
> It's how memory is normally managed for Python modules that's
> problematic; security has little to do with it. There are some
> efforts around making Python's memory management more friendly to
> embedded(ish) platforms, and we need to see if we stand to benefit
> from this kind of work.
I also looked into it, and after some research I still think we could
get back some memory by mmap()'ing bytecode, but not much.
Many modules we import are just bindings, which have a tendency to
contain a myriad of very short functions. A lot of memory apparently
goes in the metadata overhead to describe function and class definitions.
Finally, I've been explained that some smarter bindings employ a lazy
technique to greatly reduce their memory usage, on the basis that
applications typically only ever use a small set of functions and
classes from the APIs they bind to.
PyQt works like this, and it's a huge win because Qt contains hundreds
of classes. I hear the same trick was attempted with PyGTK, but it
required binary incompatible (?) changes and eventually the work halted.
PyGTK is responsible for a huge 0.9 seconds of load time and probably
quite a lot of much memory. So I guess it would be an interesting
// Bernardo Innocenti - http://www.codewiz.org/
\X/ One Laptop Per Child - http://www.laptop.org/
More information about the Devel