[sugar] Re: [OLPC-devel] Re: pygtk performance issue

[Ian Bicking]

Johan Dahlin wrote:
>> There was also the issue that Mike Heam brought up about the linking
>> overhead of importing _gtk.so in particular.  Here I must profess that I
>> am largely ignorant of the issues, but I'll speculate anyway.  Perhaps
>> just splitting up _gtk.so into separate modules would be helpful,
> 
> Splitting _gtk.so into several pieces would be benefit, especially combined
> with the dynamic namespace patch[1] which is currently disabled (see
> the bug for more information).
> A problem is that it can only really be split into two pieces, one for
> gtk and one for gdk, the others does not provide any benefit.
> The gtk<>gdk dependencies inside PyGTK are a little troublesome making
> the split non-trivial.
> 
> Most of the gains are going to be made by:
> 
> 1) Avoiding to register the classes for /all/ types, enums, flags, until
> they're actually created: gtk & gdk modules.
> 2) Creating function wrappers on demand, might be done for methods too.
> 3) Importing modules on demand, applies to atk, pango, gdk, cairo
> 
> All of these mostly solved but depends on splitting gtk & gdk for maximum
> benefits.

Of course, it only helps if an application then *never* imports these 
other modules.  If it just means that the modules are lazily imported, 
but in a typical application they still get imported eventually, then 
all that's been improved is perceived startup time.  But then startup 
time just gets shifted into warmup time, which isn't a big benefit.

And if using fork as an optimization, then you actually want to preload 
everything anyway, just speculating that later processes will use it, 
and you definitely don't want lazy imports.  So moving things around 
might not be useful.  Forking solves many problems at once.

>> particularly if there are really different kinds of functionality so
>> that some might never be needed by an application (just switching to
>> lazy imports only stretches out the performance problem, and I'm not
>> sure that's a good solution here).  But even then, the memory overhead
>> is quite substantial (8Mb?) so his suggestion to fork an
> 
> I wonder where this 8M number comes from
> PyGTK uses around 5-6M including GTK+ and Python.

I was remembering the total memory after doing "import gtk", which 
included overhead beyond PyGTK.  Here's what I currently see, from the 
(Py)GTK that sugar-jhbuild builds:

just running python (no imports):
RSS:          2764 kb total
               1668 kb shared

after "import gtk":
RSS:          8500 kb total
               5196 kb shared

Which maybe isn't so bad, just 3M unshared, 2M more than Python on its 
own.  If I do a fork I get:

RSS:          8508 kb total
               8268 kb shared

Which is only 240k, so a substantial improvement.  But it's 
copy-on-write, and I don't know how much writing will happen after the 
fork.  A really dumb test I did only showed another 200k becoming 
unshared after running some simple gtk routines after a fork (but doing 
the fork after everything involved was imported).


-- 
Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org