[Testing] Improving the reporting of test results

[Sridhar Dhanapalan]

We at OLPC Australia have been the recipient of some really good
testing feedback (especially from NZ) on the lists. However, I also
perceive a major weakness in the process. My central question is, how
do we ensure that the feedback actually gets to the relevant people in
a useful way?

The executive summary is that all bugs/issues should be reported in an
issues tracking system (redmine, trac, bugzilla, etc.), so that the
developers can easily tend to and manage them. I'll explain...

The basic develop-test-report cycle goes like this:

  1. software/hardware is developed
  2. testers give it a spin
  3. testers note their results
  4. developer easily sees the feedback
  5. developer can easily manage the feedback
  6. based on the feedback, GOTO 1

What I see is that steps 1-3 are being handled quite well.

There appears to be a big hole between steps 3 and 4. This is because
the testing feedback is only posted as prose on the mailing lists.
There's nothing to be sure that the relevant developers are seeing
those messages. Given the sheer volume of list messages, they probably
aren't.

Even if the developer does see the feedback, how does (s)he manage it?
This is what issues tracking systems are for. If an issue is properly
reported, it can be properly managed along with the other tasks
required for the project. Statuses, priorities and owners can be
assigned. Now that it's in the system, it won't get lost. This is all
neatly described at http://wiki.laptop.org/go/Reporting_bugs

If step 4 occurs but not step 5 (i.e. prosaic feedback is received,
not input into an issues tracking system), it is up to the developer
to turn the prose into a bug report. This is time consuming and
detracts from the act of development (step 6).

In summary, I strongly urge testers to fully report their findings in
the appropriate issues tracking system. In doing so, you make sure
that the developers see your findings (thus making your testing
worthwhile) and can easily act upon them.

Examples of tracking systems to submit to:

  OLPC: http://dev.laptop.org/
  Sugar Labs: http://bugs.sugarlabs.org/
  OLPC Australia: http://dev.laptop.org.au/


Cheers,
Sridhar


Sridhar Dhanapalan
Technical Manager
One Laptop per Child Australia
M: +61 425 239 701
E: sridhar at laptop.org.au
A: G.P.O. Box 731
     Sydney, NSW 2001
W: www.laptop.org.au