[OLPC-Games] vision processing / juggling

[Tim Wintle]

> I think they want to wait on GSoC to finish before merging back into
> the trunk.  In any case, it's going to release with pygame 1.9, which
> is the next one after 1.8.1 comes out in a few days.

Great news - can't wait to be able to integrate it into my games.

> Nirav
> 
> On Tue, Jul 15, 2008 at 5:19 AM, Tim Wintle <tim.wintle at teamrubber.com> wrote:
> > Nariv - this looks great - any idea when the camera module may get
> > merged back into the pygame trunk?
> >
> > Tim Wintle
> >
> > On Mon, 2008-07-14 at 08:21 +0900, Nirav Patel wrote:
> >> > Nirav Patel's GSoC project is vision processing for the XO (I believe
> >> > using a subset of the OpenCV library)... and rather good at colored
> >> > object / face recognition. (Eric, try the
> >> > http://wiki.laptop.org/go/Colors! activity on an XO when you get a
> >> > chance... it can use the green power cord (or really any light in a
> >> > dark room) as drawing input)
> >>
> >> The original plan was to use OpenCV, but I switched to adding vision
> >> stuff to pygame instead.
> >>
> >> > I was thinking ... a learn to juggle activity may make use of this
> >> > processing to track progress and give tips (practice 1/2 balls again!,
> >> > etc). Greg -- you've done some vision processing and you are a
> >> > juggling guru... you might want to consider applying for a laptop at
> >> > http://projectdb.olpc.at/ to play with this sort of thing. :)
> >> >
> >> > I wonder what best color / pattern could go on the balls to be easy to
> >> > track... possibly to track spin? :)
> >>
> >> Any uniformly and brightly colored objects would work well.  For
> >> convenience of coding, its probably best to use three different
> >> colored balls.  Tracking spin would involve optical flow, which I do
> >> plan on writing, but I'm not sure its feasible to run realtime on the
> >> XO.  There is basically a two step process for color based object
> >> tracking in the vision stuff I'm writing.
> >>
> >> The first step is finding the color of the object.  This involves the
> >> user holding up an object so it fills a Rect being overlaid on images
> >> being captured by the camera.  You would want to set up the Camera
> >> object so that it captures images in HSV with the following:
> >> camera.Camera("/dev/video0", size, "HSV")  It would then involve
> >> looping something like the following until the user hits a key, where
> >> cam is the Camera object, display is the display surface, and ccolor
> >> is the average color you want:
> >>
> >>        snapshot = cam.get_image(snapshot)
> >>        display.blit(snapshot, (0,0))
> >>        crect = pygame.draw.rect(self.display, (255,0,0), (300,220,40,40), 4)
> >>        ccolor = pygame.transform.average_color(snapshot, crect)
> >>        pygame.display.flip()
> >>
> >> The second step is tracking the object.  It involves looping through
> >> the following, where thresh is a surface the size and depth of
> >> snapshot, cent[0] is the (x,y) coordinate of the object, and cent[1]
> >> is the size in pixels:
> >>
> >>        snapshot = cam.get_image(snapshot)
> >>        pygame.transform.threshold(thresh, snapshot, ccolor,
> >> (20,25,25), (0,0,0), True)
> >>        mask = pygame.mask.from_surface(thresh)
> >>        cc = mask.connected_component()
> >>        cent = cc.centroid()
> >>
> >> Note that if you're running this on the XO, you're going to want to
> >> capture images as "RGB" at (320, 240), pygame.transform.scale them
> >> down to (160, 120), convert the colorspace to "HSV", and then do all
> >> the thresholding and mask stuff.  Note also that Camera.get_image()
> >> returns 24 bit Surfaces, while the XO displays 16 bit surfaces.  So
> >> blitting a Surface captured by the camera to the display involves a
> >> strangely cpu expensive operation that makes it impossible to capture
> >> and blit 640x480 images at 30 fps.  This may be something I can find a
> >> workaround for, but for now, stick with 320x240 or smaller.
> >>
> >> There are a few example scripts in
> >> http://git.n0r.org/?p=pygame-nrp;a=tree;f=examples/camera though none
> >> of them are OLPC specific.
> >>
> >> Nirav
> >>
> >> > Brian
> >> >
> >> _______________________________________________
> >> Games mailing list
> >> Games at lists.laptop.org
> >> http://lists.laptop.org/listinfo/games
> >
> > _______________________________________________
> > Games mailing list
> > Games at lists.laptop.org
> > http://lists.laptop.org/listinfo/games
> >
> _______________________________________________
> Games mailing list
> Games at lists.laptop.org
> http://lists.laptop.org/listinfo/games