[Testing] Community-based testing of OLPC activities

[Kim Quirk]

Hi Greg,
Thanks for putting this together. I think the matrix of test plans is
a good idea. We had tried to create something like this at
http://wiki.laptop.org/go/User_Stories_Testing site. Yours is a more
friendly and probably easier to use, so it would be great if you can
incorporate your look/style with the test plans and info that are
there.

My biggest issue with a wiki based test case management system is that
you can't easily track results. Test cases are rarely run once -- but
need to be run many times after pretty much any code changes that
might affect it. If there is some way to hook this 'result' page into
a test case management database, then we would get the best of both
worlds. Easy update, easy run and good reporting on a wiki page.

The kind of reporting we need is: date/timestamp for each run; tester;
pass/fail results; link to bugs

Do you think that can be done in the wiki?

Another idea that people have had -- but no time to implement -- is a
Test Activity. This activity could be loaded onto the laptop or
simulator and run from there to provide you with all the test cases.
You pick a test case, run it, add your pass/fail, comments and or bug
tracking numbers and when you hit 'submit' it sends the results back
to our testcase mgmt server.

To me this would be ideal... but it seems very difficult to pull
together all the pieces. Someone is working on the log viewer,
gatherer (it is important to have the logs whenever there is a
problem); others have thought about a Test Activity; and some have
looked into a test case mgmt database, but have not found something we
can use.

Thanks for your thoughts and ideas as well. I have copied the
'testing' mailing list. A group of people interested in the testing
problem. Can I add your email to this list?

Regards,
Kim



On Nov 7, 2007 12:41 PM, Greg DeKoenigsberg <gdk at redhat.com> wrote:
> On Tue, 6 Nov 2007, Samuel Klein wrote:
>
> > Greg, you might take a look at these pages to get your bearing:
> >
> > http://wiki.laptop.org/go/Tests
> > http://wiki.laptop.org/go/Test_process_sprint
>
> Hi again.  I know you guys are in crunch time, so I'll try to make it
> brief.
>
> My goal is to bring community folks -- Red Hat and otherwise -- to
> participate in OLPC in a way that, for each participant, is (a) simple,
> (b) achievable, (c) rewarding/fun, and (d) adds value to the project.
>
> The first idea, and the one that is most immediately achievable:
> community-based QA of activites.  Maybe they will be activities in the
> critical path; maybe they won't be; that depends on your confidence.
> From my perspective, it doesn't matter.  Any progress is good, and any
> community "stickiness" is doubly good.
>
> The following page is a first step:
>
> http://wiki.gregdek.org/index.php?title=OLPC_Community_QA
>
> It's on my own mediawiki, so if you like, I can move it over.  In many
> ways, it's a re-encapsulation of some of the test matrix work you've
> already done -- but it focuses much more strongly on *ownership of small
> tasks*.  Identifying people who are willing to take responsibility for
> small, achievable tasks, and connecting them with one another.
>
> The second step will be to actually get activities into this matrix and
> get test plans from the developers.  From my perspective, the most
> important tasks here are to:
>
> * Encourage developers to put their activities into the grid;
> * Encourage testers to claim activities from the grid;
> * Put developers and testers in direct contact with one another.
>
> So.  If you think I'm on the right track, I'm happy to keep going with
> this work and make it semi-official.  This includes tracking down
> activity owners and getting their buy-in, and recruiting QA folks
> from the various communities.
>
> Please take a look and give me your thoughts as soon as you can.
>
> Thanks for sparing a few minutes to look this over.
>
> --g
>
> --
> Greg DeKoenigsberg
> Community Development Manager
> Red Hat, Inc. :: 1-919-754-4255
> "To whomsoever much hath been given...
> ...from him much shall be asked"
>