[sugar] python activities startup

[Guido van Rossum]

It looks like you all are on top of this, more or less. Putting off
imports until they are needed can definitely be a big help (but note
that doing an import over and over, even though subsequent times it is
ineffective, still costs a bit since it takes a few lookups and calls
to decide that it is redundant).

In the past I've seen that one cause of slow imports at start-up is a
slow filesystem combined with a long sys.path list. For each module
imported we do a couple of stat() calls in each directory on sys.path
until we find it; the names tried are something like x.py, x.pyc,
x.so, xmodule.so, and x (the latter in case it's a subpackage). You
should be able to test this theory quickly using strace(). To fix
this, rewrite __import__ (it can be done in Python!) or write a PEP
302 import hook, to cache the contents of the directories on sys.path,
so that instead of stat() it does a dict lookup.

I believe someone is working on delayed import, but probably not
before Python 3000, which means you'll have to wait years before you
can run that (all your dependencies must be ported first).

--Guido

On Feb 6, 2008 5:43 AM, Tomeu Vizoso <tomeu at tomeuvizoso.net> wrote:
> Hi,
>
> as we all know, activities that use the python API (most of them) start
> up very slow in the XO, a trivial one launching in 7 seconds.
>
> http://dev.laptop.org/ticket/5228
>
> I don't know yet if performance work will land in update.2 or in
> update.3, but now may be a good moment to summarize what we know and see
> which are our options.
>
> By the data in #5228, looks like more than 50% of time is spent
> importing modules. dbus, telepathy and pygtk make for more than 30% of
> _total_ startup time.
>
> In that ticket is attached a file that lists the 161 modules imported at
> startup. I doubt most of those modules are actually used during startup,
> and many won't be used neither during the activity lifetime, they are
> just imported because one module we actually use *may* need it at some
> point.
>
> It's clear that we would prefer to not pay this price upfront at
> startup, but rather that the needed initializations happened during
> runtime as (if) needed.
>
> I would like to take the chance to ask to Guido what's his opinion on
> this, as I'm sure this has been discussed in the python community at
> some point. How could we move all these initializations from import time
> to use time?
>
> I see the following options:
>
> - Reduce the number of things that are done at startup. We probably
> could write to the datastore, initialize sharing and creating the D-Bus
> service at a later point after startup. These changes look to me as
> quite invasive and somewhat risky. The benefit from the user point of
> view is not clear, as we would like those things to happen as soon as
> possible after the activity window has been brought up.
>
> - Modify the modules that do significant initializations at import time
> so that code is executed lazily as needed. Those modules will be inside
> python itself, dbus, pygtk, telepathy, sugar, etc. The drawback of this
> approach is that we'll need to maintain forks for some time.
>
> - Hack the import logic in python so that the module-level code is
> executed only when some function or method from that module is called or
> a variable is set. I'm not sure if this is possible by just using the
> existing hooks or if we would need to patch python. In case this brings
> some incompatibilities, we could activate this "fast mode" through some
> flag.
>
> I wonder too if python 2.6 and 3.0 will bring some improvement in these
> areas?
>
> Thanks,
>
> Tomeu
>
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)