Usability testing

[Greg Smith (gregmsmi)]

Hi Tomeuz et al,

I have done a few usability tests and they are a lot of work and not
easy to turn in to code later.

The main thing you can learn is if something makes no sense to the user.
You can often discover that so usability test is best when you have a
design and want a final sanity check on it.

More valuable are user - development relationships to understand users
work flow. A good way is what Bryan suggest: sit in class for a few
weeks. Still we need feedback on lots of questions over a long period of
time. For that we need a group of users who can give feedback quickly
and regularly.

Building that group is not going to be easy! We have to help people
accomplish their daily activities to build trust and value. Then we have
to learn from users and explain what questions development tries to
answer (AKA ask them to help you with your daily activities :-).

Its great that Walter and others have a lot of direct experience
introducing the XO to new people. Until we hear more directly from "the
field" they speak for the user. 

Beyond that we have a few data points:

The report by Carol's daughter: (see
http://wiki.laptop.org/go/User_talk:Gregorio#User_experience.2C_input.2C
_ideas_and_blogs South Bronx Teacher Feedback link). 

One key idea there is that kids wont wait for an activity to load. The
activity icon blinks but the kids didn't get that. Maybe an animated GIF
or a mini-animation would help. Or maybe paint the activity window right
away, then fill it in slowly. Downside of that is you are tied to
activity even if it never loads. Two ideas but we need more user
feedback that its important issue before I would suggest it's a
development priority.

After my last post on UI changes, I contacted a teacher in Peru. I asked
how much time it takes to learn the cursor and mouse pad. I also asked
if future changes there will frustrate teachers or require re-training.

He said it only took two hours to learn the touch pad. In terms of
future changes, he said it wont be frustrating or need retraining,
assuming these changes will be innovative. See
http://lists.laptop.org/pipermail/peru/ April [OLPC-Peru] Experiencias
personales con laptop XO.

In short he agreed with Walter and Ben, not me from our last exchange
:-) One more data point. 

Interestingly, the "tensest, worrisome and emotional part" of the
training was learning how to take the computer apart!

I sent a similar question to a teacher trainer in Uruguay linking to new
sugar design ideas:
http://wiki.laptop.org/go/Designs/Activity_Management I asked if it will
be hard for users to change to that. No answer yet. I'll let you know
what I hear.

These exchanges were useful but not quite the co-design level of
interaction I'm hoping for. Maybe users will feel more empowered over
time.

I can try to get feedback on more questions. 

Starting with this one from Walter:
>And there are certain features where we are far from reaching
consensus, most notably the behavior of the Frame
appearance/disappearance. I would love to see a "usability" study of
this feature within the context of the new design.

What do you want to know about that? Does this feature solve a problem
that users care about? Tell me what this feature is supposed to do for
users and if you have an example of the new design (preferably
interactive) and I'll ask about it.

I press the X0 F3 key to task switch so I could ask how people move from
one activity to another. A use case for switching activities would help.
E.g we could ask: "what do you do when you get an e-mail and want to
paste some text from it in to write?"

Line up a list of questions or open issues and I'll work to get you
feedback. A good question which is closely related to what a user wants
to do is the first step.

It took a dozen e-mails, two phone calls and several months before I
understood what is hard about using public blogging tools. See the
answer in first two sections at:
http://wiki.laptop.org/go/Blogs_Knight_Challenge#Educational_blogger_pro
ject

That and a bug in the current browse activity which makes it impossible
to post to Blogger.com from build 656 :-). (bug details to follow when I
get it isolated from Firefox on Windows).

Give me a few easy questions or open issues and I'll try to get you
first hand answers.

Thanks,

Greg S

***************
Message: 6
Date: Sun, 13 Apr 2008 10:56:10 +0200
From: "Tomeu Vizoso" <tomeu at tomeuvizoso.net>
Subject: Re: Usability testing
To: "Patrick Dubroy" <pdubroy at gmail.com>
Cc: bens at alum.mit.edu, devel at lists.laptop.org, sugar at lists.laptop.org
Message-ID:
	<242851610804130156q5204ab02ga5ef16b6f17d4c0c at mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

On Sun, Apr 13, 2008 at 2:37 AM, Patrick Dubroy <pdubroy at gmail.com>
wrote:
>  If there's one conclusion we can make here, it's that we could do a
>  better job in coordinating our usability efforts. In the next few
>  days, I'll try to set up a central place on the wiki that can use to
>  do this. Anyone else who is interested in this can feel free to do
so,
>  of simply get in touch with me to let me know you're interested.

Thank you very much, the developers have no means to organize such
things, so the only way I see is the community stepping up and
organizing themselves.

I have heard discussions inside OLPC about the need of using usability
testing in order to gather a better understanding of how the UI
decisions work when a kid is finally put in front of Sugar. I think
we'll see at some point OLPC resources dedicated to this task, but I
don't think it's wise to wait for that to happen.

In my opinion, things would work better if an OLPC employee/contractor
is later integrated into an existing wider community effort for
usability improvement, rather than people waiting for things to happen
from OLPC.

Thanks again,

Tomeu