Usability testing

[Patrick Dubroy]

(CC'ed to the sugar ml. For context, see
http://lists.laptop.org/pipermail/devel/2008-April/012674.html)

Like Carol, when I first got involved with OLPC, I was surprised to
find that so little user testing had been done. I was told by someone
that to their knowledge, no formal user testing had been done *at
all*. But, as the same person explained, the entire system has been
developed by a small number of people under immense time pressure,
which naturally makes it difficult to do iterative design.

I think everyone is in agreement that it would be great if more user
testing could be done on the Sugar interface. But, that doesn't mean
everyone agrees on the methods.

On Fri, Apr 11, 2008 at 10:40 AM, Benjamin M. Schwartz
<bmschwar at fas.harvard.edu> wrote:
>  Carol Lerche wrote:
>  | A good example is the rococco color picking widget.  According to my
>  | observation this is very difficult for small children to use, and to learn.
>
>  Perhaps you would care to look at
>  http://wiki.laptop.org/go/Designs/Toolbars#11
>
>  A glance through the Designs section of the wiki will answer your question
>  about why there isn't a bigger push for usability testing: because Sugar
>  isn't anywhere close to implementing its design.  Personally, I expect
>  that usability testing will serve almost exclusively to tell us what we
>  already know is wrong with the current implementation.  Once the new
>  designs are implemented, then it will be a good time to test usability to
>  search for further improvements.

Ben, you're right that it will be easier to do testing once the
designs have been implemented. But we don't need to fully implement
the interface before it has been implemented -- user testing could be
done on the existing designs using paper prototyping
(http://en.wikipedia.org/wiki/Paper_prototyping) or other lightweight
usability methods. This makes it easier to iterate on the designs
without wasting a lot of development time.

In the end, nothing beats putting a working interface in front of the
actual users, but given the constraints of this project, that kind of
testing is difficult. For one thing, it's tough for most of the
developers to get access to children in developing countries.

However, doing some kind of testing is better than nothing. I'd
encourage people to do any kind of user testing they can do -- paper
prototypes, informal think-alouds with friends and family, etc. Simply
putting your design in front of someone who's not familiar with the
project will teach you lots. In fact, I just completed a study like
this on the Pen Tablet user interface:
http://wiki.laptop.org/go/Pen_Tablet_UI/User_Study

If there's one conclusion we can make here, it's that we could do a
better job in coordinating our usability efforts. In the next few
days, I'll try to set up a central place on the wiki that can use to
do this. Anyone else who is interested in this can feel free to do so,
of simply get in touch with me to let me know you're interested.

Pat
--
Patrick Dubroy
http://dubroy.com/blog - on programming, usability, and hci
http://wiki.laptop.org/go/User:Pdubroy