Demon Castle Tutorials - Usability Test [Was: Re: [Etoys] Demon Castle Tutorials for eToys]

Mon Oct 23 02:42:34 EDT 2006

Hi Takashi,

This weekend, we ran a usability test on your tutorial. The way I setup the 
environment is I asked 3 girls (2 of my daughters, both age 15, and their 
friend, same age) to do, separately:
	- Start each of the 3 projects of the tutorial (I did the start/stop)
	- Be in a room with a computer by herself, with no help
	- Call me in if she gets stuck, BUT only after 2-3 minutes of trying. I 
helped only the minimum necessary in such case.

All was repeated with each. All 3 finished all 3 tutorials, some help was 
needed, Jacqueline only needed help once, to "repair the script".

I realize the ages are probably older then the target age. Also my girls did 
use eToys quite a bit before (up to about 2 years ago, when they used it 
last .. I noticed they did a _very_ good job forgetting it :) ). Their friend 
never used eToys; but all used computers to a fairly large degree.

I basically decided to note (and report here) things on which they get stuck 
too long or surprised by. The report below is sort of a summary of the 3 of 
them. First, let me say that you would be happy to see (and hear) their 
comments: 
	- Wow, this is pretty cool
	- Ah, I can use it
	- What i cute demon  :)
	- etc

Please do not take the length of this "report" as a negative sign - the girls 
did love the tutorial, and all got to the end with some help, Jacqueline 
almost did it all by herself!

So the report:
===========

A. Generic Findings:
================
	- Kids are impatient :) they cannot wait 2 minutes without asking for help, 
but get better over time.

	- When they get lost, they make things worse by just clicking around, often 
trying things they learned in previous lessons.  

	- The "Try Again" button was valuable, but it would be nice to add a note 
(bold) "You can always click on the "Try Again" button". Also is it possible 
for the "Try Again" to completely restore the state, including deleting 
copies, restoring destroyed scripts etc? For example, sometimes they ended up 
with many copies of the key (as they decided to try the "duplicate"), and 
things became confusing. Also, Joelle deleted the scriptor, and that was the 
end of her lesson until I restarted it. So restoring the page state 
completely on "Try Again" would be great.

	- We had a few MNU, but they did figure to just click the "x" on the left, 
and kept playing. This is funny, for a developer it is a mojor event. For a 
child, as long as they know what to do, just get rid of it and keep going...

	- Not sure how hard this is - is it possible to "lock up" the instructions, 
in the sense kids cannot write to it, a few times they got confused and 
started dragging out images from the instructions

B1. Handles/Hallo Tutorial:
=============

	- Clicking on the black parts of the "key morph" does not act on in. I 
realize this is etoys behaviour, but it confused one of them, brought 
confusion about how the halo is brought up, and I had to explain. I think for 
the initial tutorial, the key morph should have a black background, so it is 
sensitive to click anywhere on it or close to it.

	- Apart from that, pretty smooth sailing!

B2. Paint Tool Tutorial:
=================

	- I think the "auto-painter" confused more then helped, overall. I would 
probably remove it, caused much confusion for everyone. (Also, once painted, 
the painted morph stays around on all following pages it seems?).

	- On the following few "Paint my face" instruction pages: In the 
instructions, perhaps instead of pasting the icons, better would be to start 
with hallo up, and put the "Paint Box" close, or inside, of the 
instructions?. The girls thought that they should click on the icons inside 
the instructions, instead on the PaintBox all the way right. Eventually they 
ploughed through it though :)

B3. Scripting Tutorial (hardest, understandably, but excellent):
=====================================

	- a movie would be ideal (Broken scriptor: hardest to get right)

	- The "Try Again" comment from A. Generic Findings: applies.

	- The girl who did not use eToys, was confused and kept asking "what is 
scriptor"?, "what is a tile"? - We should define and 
describe "scriptor", "tile", "exclamation", and "clock", in more detail, 
perhaps a more text and arrows would help. It is hard to describe though...

	- The arrow "Try the  exclamation button" should point to "!" (points to 
scriptor)

	- Page "Is the scriptor broken" page: Perhaps could add more description 
like: "Once the scriptor is repaired, click the clock as in the previous 
page". This is where the girl who did not use eToys started to get stuck.

	- Page "Let me explain a bit about a scriptor here.": - What do you think 
about moving this page  up front, maybe simplifying it (as noted 3 items 
above)

5. At the end:
==========

This was funny, where the demon says "I am sorry I captured you. But I just 
want a human friend. I am a lonely demon.", the girls all noted "why does he 
ask about a human friend, it should be a cat friend?" I never noticed it at 
all...

Beautiful stuff - please let me know if we can help. We noticed a few typos 
and formulation potential changes - if you'd like, we can summarize them.

Thanks Milan, Jacqueline, Joelle and Najiba

On 2006 October 20 11:08, Takashi Yamamiya wrote:
> Hi Andreas, Milan,
>
> I have updated handles tutorial.
> http://tinlizzie.org/olpc/tutorial/DemonCastle1.005.pr
>
> Milan Zimmermann wrote:
> > Takashi,
> >
> > I remember one thing I got stuck on (almost) completely, was when I
> > started resizing the "BIG KEY". When you try to make it big and small
> > again in a few directions without letting go (and it goes
> > LARGE/Small/OK"), it becomes almost impossible to hit the OK size. Maybe
> > it should be more generous with the range of OK (it seems to use a
> > relative measure perhaps). I will have more notes over the weekend,
>
> Aha! everybody who tried it complained this issue, but I just forgot...
> I made it easier, thank you!
>
> > On 2006 October 19 23:58, Andreas Raab wrote:
> >>> http://tinlizzie.org/olpc/tutorial/DemonCastle1.004.pr (handles)
> >>
> >> * On the first page where it says "Click THIS arrow" I'd move the text
> >> closer to the arrow. I was kinda looking for "another" arrow because the
> >> one you mean is quite far away from the text.
>
> Oops, you are right. That was bad.
>
> >> * On the third page (with the first key) I would change the text from
> >> "Go ahead you can move the key" to something like "Go ahead, drag the
> >> key on the keyhole" (to emphasize that you mean click and drag)
>
> Make sense,
>
> >> * On the "resize handle" page it was very hard to find quite the right
> >> size for the key. It seems as if there is only a small range in which it
> >> will fit and it was quite tricky to get that "just right". I wonder if
> >> there is an easy way to increase that range?
>
> I am going to fix other two projects later.
> Thank you so much!
>
> Takashi