[olpc-nz] [Testing] Improving the reporting of test results

[Tom Parker]

On Wed, 2010-12-22 at 17:14 +1100, Sridhar Dhanapalan wrote:
The discussion is useful, and I'm by no means proposing that it be
> abolished in lieu of sterile bug reports. Perhaps at the end of each
> comment should be a link to the relevant bug report. Then the report
> can be updated as the discussion progresses.

I make tickets during the Auckland testing. It's quite hard work as I'm
fairly constantly interrupted by other testers. I tend to only do it for
those issues which I can properly write up as a bug report with steps to
reproduce and/or detailed information. I add the bug numbers to the test
report (it's a collaboratively written email). Don't underestimate the
overhead of creating a bug report, it often takes 20 or 30 minutes
during which I have to ignore everyone, so I only do this when I feel
there is something valuable to report (which is why it takes so long
which is why I only do it ....).

For things like "it seems slow", I don't produce a formal bug report
unless I can quantify how slow, for example I would need to run an old
build with the old wikipedia and compare to the new one before reporting
that the new one is actually slower (that said, I see Martin has some
performance related changes upstream).

Tabby used to copy in developers but stopped due to workload at the
testing (finding who the developers are is hard work) and has always
posted to the testing list.

If people want follow up, we generally re-image the laptops at the
beginning of each Saturday, so if they get in touch before we do this,
we can re-test or provide more information (such as the logs).

It would be nice if the log activity had an easy way to collect logs
onto a USB or even submit a bug report, as this is something that only a
few of us can do un-aided and then a few more can do with help. Most of
the testers don't know their way around the terminal or if they do, they
don't know where the logs are in sugar.

10 out of 35 tickets I've raised have been closed, and 15 of the 25
still open are "new".

I raise tickets at sugarlabs unless it's obvious they should go
elsewhere.