Python pyc/pyo (Was: Inflation)

[Mike C. Fletcher]

Bernardo Innocenti wrote:
...
> No other dynamic language needs that pre-tokenized junk.
> Is Python so slow at parsing?  Has anybody measured the
> hit?
>   
There is some minor improvement in speed from the cached byte code.  
That said, there's no reason to *ship* them, you can run a compileall 
script on installation (doesn't help with filesystem space usage 
though).  You only need one of pyc or pyo, choose whether we run with 
optimisation on or not and use that version of the compiled cache.  IIRC 
there are flags to the Python executable that can skip generating the 
byte code if you wanted to test it.  I'm guessing you'll see a 
noticeable but not huge slowdown in startup time for activities.  
Alternately you can just try deleting all of the pyc and pyos and seeing 
if the activities slow down in starting up.  You'd also want to time the 
whole-system boot, since Sugar is written in Python as well.
> Someone told me that Python 2.5 can now also read
> library files from an archive, and this is supposed
> to reduce startup time a lot.  (I don't see why it
> should, on any decent filesystem).
>   
My understanding is that slowdown is seen due to large number of stats 
during module import.  Python does run-time lookup of each module, and 
with more involved PYTHONPATHs that can be 90 or 120 stats per import 
(you have to check for py, pyc, pyo, so and pyd versions (as well as 
un-decorated names for packages) for each name in the import and paths 
of 30 directories can happen readily).  Multiply by a few thousand 
imports  in a really big application and it can start to add up.  In 
contrast, the zip importer has loaded the zip directory table already on 
the top-level resolution, so can do simple in-memory hash lookups for 
each name from that point on.  That's about the same effort required as 
a second import of the same module name.

HTH,
Mike

-- 
________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://www.vrplumber.com
  http://blog.vrplumber.com