Webcam support in Squeak (for VOLPC)

[Dan Williams]

On Wed, 2007-03-07 at 19:52 +0100, Bert Freudenberg wrote:
> On Mar 7, 2007, at 18:32 , Dan Williams wrote:
> 
> > On Tue, 2007-03-06 at 17:14 -0700, Jonathan Corbet wrote:
> >> If there's any way you could get Squeak to work with
> >> rgb565 and cut out both conversions, I suspect you would be a lot
> >> happier.
> >
> > Yes, format conversions really hurt.  Thou Shalt Not Format Convert.
> > While the machine does have MMX, which helps somewhat, it's not a  
> > magic
> > bullet.  First, the GPU can only accelerate blends to the framebuffer
> > when the src is the same format as the dest.  The GPU can accelerate
> > format conversions, but the pipe back to main memory is really slow.
> > Therefore, it's not a win to do hardware-accelerated format  
> > conversions
> > in GPU memory unless you're going to blit them directly to the
> > framebuffer using GPU acceleration too.
> 
> So if I hand a 555 to X to be drawn on the screen, the GPU would  
> accelerate it?

No, since the native format is _565_ with no alpha channel.  You'll have
to go through a format conversion in X to convert 555 -> 565, which can
then be blitted to the framebuffer.

Remember, to be as fast as possible, hand X visuals that are the same
depth as the framebuffer visual.  That means 565.

> >   This means that all format
> > conversions are best done on the Geode itself, which isn't going to be
> > fast because it's a software format conversion.
> >
> > If you send X a pixmap that's not in the native framebuffer format
> > (which happens to be 565 right now) then X will have to format convert
> > that to 565 for you, which ends up being really slow, especially for
> > large pixmaps.
> 
> Why doesn't it use the GPU? Provided I'm drawing to the display, that  
> is.

Because the GPU sucks, more or less.  If the visual you hand to X is not
in the same format as the framebuffer (which is 565), then X must format
convert to 565 format.  That costs CPU.

> > The moral of the story is, don't format convert, and
> > keep everything you want to be fast in native framebuffer (ie 565)
> > format.
> 
> What about switching to 555 in general? Seems a much more logical  
> format.

We have discussed that option, yes.  There hasn't been time to test
that.  The other alternative is 444, which allows for 4 bits of alpha,
something Jim pointed out they do on the iPaq.

If we switched to 555 or 444, then we need to go back and write MMX
acceleration routines for X and cairo that accelerate 555 and 444
operations.  We also need to make sure that the EXA driver properly
accelerates these blits.  I wrote the 8888->0565 MMX accelerated blit
function and got that into X and cairo, but doing 555 or 444 requires
writing multiple acceleration functions and I don't have the expertise
in MMX to do that.

We also need to add some stuff to cairo to use 555, which is not a
format that cairo works natively with right now.

In short, anything besides 888 or 565 requires work in X, cairo, and
possibly the EXA driver which we don't have people to do right now, and
we also don't know how much gain we have.  We are fundamentally limited
by the GX's GPU itself, which does suck in many ways.  It's questionable
how much return we will get for any effort we put into this work due to
the nature of the GX.

That said, investigating 555 would be helpful if anyone has the time to
do it and write some X and cairo testcases that demonstrate that 555 is
faster than what we do now.

Dan