[OLPC-devel] OLPC theme benchmarked

[Manu Cornet]

Hi !

I have been coding a small "torture" application for GTK themes [1],
which mainly does two things: 1) crash test the engine by calling all
the GTK paint functions with unsual parameters and 2) measure the time
the theme takes to create/map/expose/resize/destroy widgets (with help
from Federico's widget profiler's infrastructure).

While fixing the OLPC theme to pass my crash test [2], I noticed that it
was quite a bit slower than I expected, so I thought that precise time
measures would be interesting. I hope this is useful, it will definitely
help me in the next few days :)


The full report is available as a spreadsheet [3]. Before writing a
short analysis, a few disclaimers:

* The main point of this benchmarking is to have a record of the current
speed (so that we can see how it evolves when some more work has been
done). The theme is a work in progress, and therefore understandably
still very slow compared to more polished ones.

* I tried my best to make the testing conditions as stable as possible:
same computer (of course), same programs running in the background,
tests performed within a small time window, etc.

* The different time measures correspond to the few important events in
a widget's life: create, map, expose, destroy. The "expose with resize"
corresponds to resizing the widget a lot of times (from a very small
size to a very big size) and measuring the expose time for each size.

* Each cell of the document is an amount of time (in seconds) averaged
from about 80 iterations of the same event, on the same type of widget.

* These are only observations, not explanations. Make the slow parts
faster will be the next job!

* Absolute time values aren't that useful, but ratios are more
interesting.


Okay, so a few observations:

* Create : ratios vary from 0.92 to 1.07, showing that create time is
pretty much the same for all themes, for each type of widget.

* Mapping a widget takes the OLPC theme about twice as much time as
Clearlooks (or HighContrast).

* Destroy is always slightly (30 %) faster on OLPC than on Clearlooks,
but slower than HighContrast (about 1.4 ratio). Destroy time are pretty
much the same between OLPC and Human (Ubuntu's theme) -- ratio ~ 1.

* Expose is a bit slower (1.9 ratio, and sometimes a little faster) with
OLPC than with Clearlooks or HighContrast, but faster than Human.

* Expose with resize is the most CPU-consuming task, and therefore
probably yields the most significant results. And the OLPC theme doesn't
do well with this one: it is from 30 to 130 times slower than Clearlooks
(50 times on average), and 35 times slower than Human.

I'll spend the next few weeks (along with the other developers of the
theme) trying to bring those numbers down while still aiming towards
Diana's great mockups :o)

Cheers,
Manu

[1]
http://wiki.laptop.org/go/GTK_for_OLPC#GTK_theme.2Fengine_torturer_and_crash_tester

[2]  See the report at
http://www.manucornet.net/pub/olpc/log_olpc_theme_torture.txt

[3] http://www.manucornet.net/pub/Themes_performance.ods