[Testing] Testing w/ kids - starting this Sat.

[Seth Woodworth]

On Mon, Oct 13, 2008 at 10:29 AM, Mel Chua <mel at laptop.org> wrote:

> Thanks for the writeup, Frances! Ccing the testing list in case anyone's
> curious about how things went. (Also, the kid-testing was Joe's idea; I just
> got it to happen at the Museum this Saturday.)
>
>>
>> There were a number of volunteers at the scene
>>
> Note - the Boston-area OLPC group (MassXO) runs this station at the Museum
> every Saturday from 1-4, so we were really piggybacking off their event. ;)


As a volunteer with MassXO (and not as an OLPC'er) I've also been attending
the Museum for the past few months.



>
>  The questionnaire was much too long and detailed. We never even got anyone
>> using it as we decided early on that this particular day should be our
>> observation day where we see what the foot traffic is like, get a sense of
>> what the students like and see how the testing may work in this given
>> environment...our "baseline" in a sense.
>>
> That's good to know too - I really didn't expect the questionnaire to work
> as planned; it was to give us a strawman to try out things against, and
> something to do/observe by default if nothing else was going on (clearly
> something else did happen). Hopefully now we have a better sense of what
> kind of questions we can ask experts who do this kind of testing for a
> living; they can help us design and tune these tests (and others) to get us
> the kind of data that we want. Speaking of which...
>
> Frances - since we now know what kind of information we couldn't capture
> (with the procedures as they stand), what kinds of information do you think
> we could (or did) capture at this kind of location? Once we know this, we
> can decide whether it's worth trying this again (if it's info that we want
> and warrants the effort we're expending).
>

I'm pretty convinced that this particular setup with museum is not going to
give us an useful feedback or testing data.  See below:



>
>  Its a great environment for introducing our program to folks who may or
>> may not have heard of us, but may not be the testing ground we were hoping
>> for.
>>
>
> My hope is that whatever we work out for this kind of setting won't be
> reliant on one person or another being there, but that we'll have
> instructions that a group of people (perhaps with a particular background)
> can set up and run themselves - that way other groups in other places can
> eventually do this too.
>

This is a good idea, providing more strawman structures for testing groups
to follow, tear-down, or rebuild is a an excellent one.  Especially helping
testing groups to try things in new locations and in new ways.


>
> One arrangement that might work better (thinking off the top of my head
> here) is to keep the XO demo table as a free and open space with intro
> volunteers helping people play with the laptops, as they are now - and then
> have a separate table in the corner set off as a "testing lab," with 1-2 XOs
> and a volunteer that's been trained beforehand on some basic UI test cases
> (and ask researchers from a UI test lab to help us figure out some simple
> cases they can run).
>

This might work a bit better, anything we do at the museum is going to have
to be very off-to-the-side.


>
> Have a sign-up sheet on the "XO fun" table where people can sign up for,
> say, 15min test slots in "the lab," where they'll be guided through the
> process; they'll be dedicated to being in "test mode" since that's what they
> signed up for, and a single researcher that's trained on how to carry out
> this experiment beforehand can get a lot more focused data that we want in a
> planned, timed session than people passing by the table will be inclined to
> write down.
>
> Thoughts?
>
> -Mel
>


This is the layout/arrangement at the MIT Museum.  In the main lobby of the
museum, there are a number of interactive exhibits, including underwater
submarines, live fish, and a city-car simulation.  This is a fairly small
space that people wonder through and then make their way to the longer-term
exhibits upstairs.

Parents, grandparents and children wonder through this space, and see that
there is an active table and chairs with people behind it, and cute little
laptops.  People are moderately interested in the program, grab a tri-fold
brochure, look at the laptop a minute, and then move to other exhibits.
Some fraction of those people stick around for longer and really want to
hear about OLPC and about the XO.  They want to know how it's different from
a 'real laptop' they want to know how the collaboration works, and they want
to know what software it comes with.  They stick around to ask a few more
questions and then they leave.  If their kids are interested in the laptops
while their parents talk to the nice volunteers then the kids stick around.

You're able to engage the children for at-most 10 minutes, while answering
questions to their parents.  It's just not the right environment to ask
people to do anything formal or to restrict what they're doing.

Also, for the most part fairly-well-off kids at a nice museum in the US are
not our target market.  Trying to make any sort of assumption based on what
*these* children do in a short amount of time would be even less
informative.

And lastly trying to have these weekly information sessions serve OLPC feels
really tacky and likely to hurt our relationship with the MIT Museum.  They
are in the business of fostering knowledge and information sharing.  It just
so happens that they like OLPC and want to spread knowledge about what we're
doing.  If they start trying to get their customers to do formal and (for
children) boring testing, they're going to lose customers.

This idea needs some rethinking before we drop it on the Museum staff and
volunteers.

--Seth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.laptop.org/pipermail/testing/attachments/20081013/3cd16518/attachment.htm