Webcam support in Squeak (for VOLPC)

[Dan Williams]

On Tue, 2007-03-06 at 17:14 -0700, Jonathan Corbet wrote:
> Bert Freudenberg <bert at freudenbergs.de> wrote:
> 
> > > I'm curious as to why that is.  The read() interface is not the
> > > preferred way to go, but it should not perform notably worse than  using
> > > mmap().
> > 
> > When you mmap, the capturing does not block but works in parallel to  the
> > processing, which should be significantly more effective, no?
> 
> When you read, the driver goes into "speculative read" mode and
> continues to acquire frames for a short period.  This will continue for
> as long as the application continues to consume the frames in a timely
> manner.  There is very little difference in efficiency between read()
> and mmap() as long as the application is keeping up with the frame rate.
> 
> > Diego found more than 90% of time is spent in kernel land (not  captured by
> > gprof). We now reach about 16 fps at 320x240. This  includes capturing,
> > converting to 5-5-5 RGB which is Squeak's native  format, bitblt to
> > Squeak's framebuffer, convert to 5-6-5 to match X,  then XShmPutImage. If
> > the profiling is correct, then the first and  last step take 90%.
> 
> Something is wrong with the profiling, the kernel doesn't do anywhere
> near that much work.  A B1 system can sustain 640x480 at 30fps in
> mplayer; if things are coded right the system can handle the video
> stream out of the camera.  
> 
> My suspicion is that all the format conversions are killing the
> performance.  It might be possible to get rgb555 directly out of the
> sensor, if that would be helpful.  But that still leaves you with one
> conversion, and the OLPC just doesn't have a lot of spare horsepower for
> that kind of work.  If there's any way you could get Squeak to work with
> rgb565 and cut out both conversions, I suspect you would be a lot
> happier.

Yes, format conversions really hurt.  Thou Shalt Not Format Convert.
While the machine does have MMX, which helps somewhat, it's not a magic
bullet.  First, the GPU can only accelerate blends to the framebuffer
when the src is the same format as the dest.  The GPU can accelerate
format conversions, but the pipe back to main memory is really slow.
Therefore, it's not a win to do hardware-accelerated format conversions
in GPU memory unless you're going to blit them directly to the
framebuffer using GPU acceleration too.  This means that all format
conversions are best done on the Geode itself, which isn't going to be
fast because it's a software format conversion.

If you send X a pixmap that's not in the native framebuffer format
(which happens to be 565 right now) then X will have to format convert
that to 565 for you, which ends up being really slow, especially for
large pixmaps.  The moral of the story is, don't format convert, and
keep everything you want to be fast in native framebuffer (ie 565)
format.

Or, use the Geode LX.

Dan