Webcam support in Squeak (for VOLPC)

[Dan Williams]

On Wed, 2007-03-07 at 20:07 +0100, salsaman wrote:
> Dan Williams wrote:
> 
> >On Wed, 2007-03-07 at 19:38 +0100, salsaman wrote:
> >  
> >
> >>Dan Williams wrote:
> >>
> >>    
> >>
> >>>If you send X a pixmap that's not in the native framebuffer format
> >>>(which happens to be 565 right now) then X will have to format convert
> >>>that to 565 for you, which ends up being really slow, especially for
> >>>large pixmaps.  The moral of the story is, don't format convert, and
> >>>keep everything you want to be fast in native framebuffer (ie 565)
> >>>format.
> >>> 
> >>>
> >>>      
> >>>
> >>Well, that could be a big problem for LiVES (and for any other 
> >>application using GTK+/GDK). LiVES uses RGB24 as its internal format. 
> >>Why the heck use 16bit colour for the X server ? Why not use 24bit like 
> >>the rest of the modern world ;-) ?
> >>    
> >>
> >
> >Because then you have to push around a lot more data, and you loose
> >about 6MB _more_ of memory to VRAM than we already loose.  Plus, it's
> >unclear whether 888 is really a win.  You gain because you don't have to
> >format convert, but you loose because the pixmaps are much larger.  We
> >don't have a ton of memory bandwidth, and we're memory limited anyway.
> >
> >  
> >
> The pixmaps will be the same size anyway, because GdkPixbuf uses RGB24.
> 
> 
> >We have explored going to 888 but there wasn't time to do real
> >performance measurements to make sure that 888 wasn't a regression in
> >the general case.
> >
> >We also explored going to 4444 or 1555 to make format conversions
> >unecessary when alpha was being used, but that will take some work in X
> >and cairo that we don't have time to do.
> >
> >Simply put, the Geode GX processor is not a very fast processor, and the
> >GPU has known limitations.  Using the GX was a choice that was made
> >based on tradeoffs of price vs. performance and we may just have to live
> >with the limitations.
> >
> >If existing applications don't respect the limitations of OLPC, then
> >either they must be fixed to do so, or they will not be performant.
> >
> >Dan
> >
> >
> >
> >
> >  
> >
> "Must be fixed to do so" !?

OLPC doesn't cater to any specific existing application.  It's a piece
of hardware with constraints, and all software running on that hardware
must respect those constraints if they want optimal performance.  This
is the case with all hardware.  It just happens that desktop computers
where LiVES probably already runs are fast enough to make this not a
problem.

> It's not possible to use RGB565 in LiVES for a variety of reasons: it 
> uses GDK, which only supports 24/32 bit colour; and RGB565 is not a 
> supported palette in any of the modern video processing standards (the 
> only application which I know supports it internally is MOB, and I 
> believe is dropping it soon, or may already have).

Yeah, we do suffer from some of that right now with Sugar.  However, we
are using Cairo which does support 565.  X also supports 565 fairly
well.  You simply cannot depend on 888 (ie, 24/32 bit) being fast enough
here to do what you probably want on the OLPC machines for various
reasons.

> We (video developers) decided about 2 years ago to drop RGB565 support 
> in Livido (the now becoming common effect standard) 
> http://livido.dyne.org/ and hence it is not present either in Weed 
> (LiVES' own effect standard which is based on Livido).

That's nice, but dropping support means you are limiting your options
for projects like OLPC that are not using the latest video hardware.
It's a tradeoff; to meet $100, we cannot use the latest hardware.

> In short, supporting RGB565 in LiVES would require several man-years of 
> work to make a hacked version of GDK, followed by several months of work 
> to alter LiVES, plus rewriting all of the effect standards. Even then, 
> since most effects process in RGB24, it would require a lot of palette 
> conversions internally.
> 
> I am not even sure if LiVES will run on a 16 bit display, AFAIK this has 
> never been tested.

That said, it wouldn't hurt to try and see what the performance is.  X
will automatically convert anything you draw in 888 to 565 for you.  So
it's likely that LiVES will run, but more slowly.

> A really don't see what you lose going to a 24 bit Xserver. Sure, it 
> needs a bit more memory, but for a 1024x768 screen resolution, you are 
> only talking about 800KB more.

We run a 1200x900 framebuffer.  You also need EXA acceleration memory
and more memory for RandR rotation.  Moving to 24-bit would mean the
loss of an additional 6MB of system memory on top of the existing 4 or 6
we already lose to framebuffer.

This also means that all pixmaps _everywhere_ will be 1/3 larger than
they are today.  That means applications take up more memory on an
already limited 128MB system.  That's life.  Is the tradeoff here to
allow 24-bit only applications to work worth the loss in memory and
performance because you're shoving around 1/3 more data on every
graphics operation?  I don't know.

Furthermore, LCD panels don't give you all 8 bits per channel of color
fidelity.  Most panels will only actually display 666 in the best case.
So using 888 is essentially loosing performance for something that the
user will _never_ ever see.

> I think a lot of applications will have problems if you stick to 16 bit 
> colour. Just my opinion though.

16 bit color isn't all that bad, since our LCD panel can only display
666 anyway.  But it's a fact of life that the OLPC machine is not the
fastest machine out there, and there are limitations that it has to get
to the $100 price point.  Those limitation include graphics that are not
as fast, nor as flexible as a desktop with even an entry-level graphics
card sold today.  That's life.

Dan