[Testing] Community testing meeting minutes, 2008-11-13

[Mel Chua]

Thanks to all who came - garycmartin, tabitha_, fhop, kevix, anna_bham, 
Caryl, bjordan, adricnet, gregdek, aly, martin_xsa, marcopg - especially 
in the middle of the G1G1 preparation rush.

Log: http://meeting.laptop.org/olpc-meeting.log.20081113_1659.html

Notes (includes links to log): 
http://wiki.laptop.org/go/Community_testing_meetings/2008-11-13

Next week's agenda: 
http://wiki.laptop.org/go/Community_testing_meetings/2008-11-20

NOTE: We won't have a 2008-11-27 meeting due to the U.S. holiday of 
Thanksgiving (which affects many of our current testers), unless at 
least 4 people email me before the 2008-11-20 meeting and say they'd 
like one and that they would be there. Our next meeting after 2008-11-20 
will be on 2008-12-04.

Highlights:

* Our first short-term goal (proposed thanks to Caryl): All 28 
G1G1-2008-shipped Activities tested by Dec. 25, 2008. Definition of 
"tested" well on its way to being solid - this time next week we'll have 
a metric-o-riffic progress report! See 
http://wiki.laptop.org/go/Community_testing for our current progress and 
measurements of such.) Longer-term plan: this will enable us to 
stress-test a good community testing participation framework so that we 
can recruit Lots Of New Volunteers (G1G1 donors?) starting in January. 
Resources: http://wiki.laptop.org/go/How_to_test_an_Activity, 
http://wiki.laptop.org/go/Reporting_test_results

* Tabitha and Alastair from the NZ Wellington OLPC community test group 
introduced themselves and their work - they have great procedures, 
months of experience testing Activities and are going to lead the charge 
in putting up oracles and reaching out to Activity developers. Their 
group meets again tomorrow. A warm welcome to all our NZ testers - and 
thank you to Martin for the introduction!

* Many thanks to Anna from the Birmingham deployment for her work on 
http://wiki.laptop.org/go/Dsh on the XS - looking forward to hearing 
more about her + Kevix's experiments with the school server!

* Ben, Gary, Kevix, and Mel have drafted an 
http://wiki.laptop.org/go/Activity_testing_automation design, and 
experimental implementation has begin in parallel with a design review, 
and Gary has begun to work on Sugarbot-on-XO implementation with 
Sugarbot's original developer, Zach.

* More participants in meeting leading to more exciting - but also 
noisier, faster, and more chaotic - meetings. Still ok, but maybe time 
to think of ways to scale?

* Suggested test completion metric: Ponies, supplemented by pony-eating 
dragons. (You... had to be there.) 
http://www.berlinwallpaper.com/PrissPrints/images/Stickups/MLP.jpg

Action items (due next meeting):

* '''bjordan, gregdek, Caryl''': Continue focusing on making 
http://wiki.laptop.org/go/How_to_test_an_Activity a good set of 
instructions through testing it with your Activities (Speaak, Paint, and 
TamTamEdit respectively) and giving feedback - Caryl with a focus on 
newbie testers, Greg with a focus on metrics.

* '''garycmartin, kevix, gregdek, mchua''': 1-2 hour sprint to implement 
a first draft of a subjective 'complete' metric (and a 'participation' 
metric) to show the group next week, along with the "by our metrics, we 
are X% done with our G1G1-yay! goal" (Mel to see if skierpage would like 
to join us.)

* '''martin_xsa, marcopg''': choose date for a testing meeting to devote 
mainly to developer-tester relationships, probably after Sugarcamp

* '''tabitha_, aly''': pick some Activities for the Wellington test 
group to tackle, go through the 
http://wiki.laptop.org/go/How_to_test_Activities "oracle-making" section 
for them (optional: go through your 2 previous test notes and put those 
in the oracles of the Activities you tested in the past).

* '''tabitha_, aly''': with the Wellington test group, explore how we 
can build excellent test/developer relationships. This is a very 
open-ended charter - come back next week and tell us what we should do / 
what you're doing / what the plan is for this. (Might want to sync up 
with martin_xsa and marcopg on having that conversation with developers.)

* '''garycmartin''': continue working on sugarbot with zach, post 
updates to the testing mailing list (side note: marcopg and mchua can be 
dragged in to join the sugarbot party when sugarcamp/G1G1 passes a bit.)

* '''anna_bham''': Documenting dsh/ssh-to-remote-XO work - mchua to help 
figure out how to fit this into http://wiki.laptop.org/go/Testing_tools.

* '''anna_bham, mchua''': Gather to brainstorm out a plan for XS 
community testing; we should see if we can make it fit in with our G1G1 
Activity testing goal somehow (maybe to set up large #s of XOs for test?)