[Testing] Testing w/ kids - starting this Sat.

[Joseph A. Feinstein]

When I proposed the idea of getting in touch with children (and I 
didn't even think about the MIT Museum - Mel decided to give it a 
try), I didn't mean the implementation to be really formal; moreover, 
when I learned about the proposed Museum's event, I specifically 
insisted it to be non-formal; I suggested it to be observational at most.

Joe
----------
At 05:16 PM 10/13/2008, Seth Woodworth wrote:


>On Mon, Oct 13, 2008 at 10:29 AM, Mel Chua 
><<mailto:mel at laptop.org>mel at laptop.org> wrote:
>Thanks for the writeup, Frances! Ccing the testing list in case 
>anyone's curious about how things went. (Also, the kid-testing was 
>Joe's idea; I just got it to happen at the Museum this Saturday.)
>
>
>There were a number of volunteers at the scene
>
>Note - the Boston-area OLPC group (MassXO) runs this station at the 
>Museum every Saturday from 1-4, so we were really piggybacking off 
>their event. ;)
>
>
>As a volunteer with MassXO (and not as an OLPC'er) I've also been 
>attending the Museum for the past few months.
>
>
>
>The questionnaire was much too long and detailed. We never even got 
>anyone using it as we decided early on that this particular day 
>should be our observation day where we see what the foot traffic is 
>like, get a sense of what the students like and see how the testing 
>may work in this given environment...our "baseline" in a sense.
>
>That's good to know too - I really didn't expect the questionnaire 
>to work as planned; it was to give us a strawman to try out things 
>against, and something to do/observe by default if nothing else was 
>going on (clearly something else did happen). Hopefully now we have 
>a better sense of what kind of questions we can ask experts who do 
>this kind of testing for a living; they can help us design and tune 
>these tests (and others) to get us the kind of data that we want. 
>Speaking of which...
>
>Frances - since we now know what kind of information we couldn't 
>capture (with the procedures as they stand), what kinds of 
>information do you think we could (or did) capture at this kind of 
>location? Once we know this, we can decide whether it's worth trying 
>this again (if it's info that we want and warrants the effort we're expending).
>
>
>I'm pretty convinced that this particular setup with museum is not 
>going to give us an useful feedback or testing data.  See below:
>
>
>
>Its a great environment for introducing our program to folks who may 
>or may not have heard of us, but may not be the testing ground we 
>were hoping for.
>
>
>My hope is that whatever we work out for this kind of setting won't 
>be reliant on one person or another being there, but that we'll have 
>instructions that a group of people (perhaps with a particular 
>background) can set up and run themselves - that way other groups in 
>other places can eventually do this too.
>
>
>This is a good idea, providing more strawman structures for testing 
>groups to follow, tear-down, or rebuild is a an excellent 
>one.  Especially helping testing groups to try things in new 
>locations and in new ways.
>
>
>One arrangement that might work better (thinking off the top of my 
>head here) is to keep the XO demo table as a free and open space 
>with intro volunteers helping people play with the laptops, as they 
>are now - and then have a separate table in the corner set off as a 
>"testing lab," with 1-2 XOs and a volunteer that's been trained 
>beforehand on some basic UI test cases (and ask researchers from a 
>UI test lab to help us figure out some simple cases they can run).
>
>
>This might work a bit better, anything we do at the museum is going 
>to have to be very off-to-the-side.
>
>
>Have a sign-up sheet on the "XO fun" table where people can sign up 
>for, say, 15min test slots in "the lab," where they'll be guided 
>through the process; they'll be dedicated to being in "test mode" 
>since that's what they signed up for, and a single researcher that's 
>trained on how to carry out this experiment beforehand can get a lot 
>more focused data that we want in a planned, timed session than 
>people passing by the table will be inclined to write down.
>
>Thoughts?
>
>-Mel
>
>
>
>This is the layout/arrangement at the MIT Museum.  In the main lobby 
>of the museum, there are a number of interactive exhibits, including 
>underwater submarines, live fish, and a city-car simulation.  This 
>is a fairly small space that people wonder through and then make 
>their way to the longer-term exhibits upstairs.
>
>Parents, grandparents and children wonder through this space, and 
>see that there is an active table and chairs with people behind it, 
>and cute little laptops.  People are moderately interested in the 
>program, grab a tri-fold brochure, look at the laptop a minute, and 
>then move to other exhibits.  Some fraction of those people stick 
>around for longer and really want to hear about OLPC and about the 
>XO.  They want to know how it's different from a 'real laptop' they 
>want to know how the collaboration works, and they want to know what 
>software it comes with.  They stick around to ask a few more 
>questions and then they leave.  If their kids are interested in the 
>laptops while their parents talk to the nice volunteers then the 
>kids stick around.
>
>You're able to engage the children for at-most 10 minutes, while 
>answering questions to their parents.  It's just not the right 
>environment to ask people to do anything formal or to restrict what 
>they're doing.
>
>Also, for the most part fairly-well-off kids at a nice museum in the 
>US are not our target market.  Trying to make any sort of assumption 
>based on what these children do in a short amount of time would be 
>even less informative.
>
>And lastly trying to have these weekly information sessions serve 
>OLPC feels really tacky and likely to hurt our relationship with the 
>MIT Museum.  They are in the business of fostering knowledge and 
>information sharing.  It just so happens that they like OLPC and 
>want to spread knowledge about what we're doing.  If they start 
>trying to get their customers to do formal and (for children) boring 
>testing, they're going to lose customers.
>
>This idea needs some rethinking before we drop it on the Museum 
>staff and volunteers.
>
>--Seth
>
>