[Testing] Today's/Tonight's QA Phone Call Conclusions & Unofficial Minutes
simon at laptop.org
Wed Feb 24 10:55:25 EST 2010
thanks for your thoughts!
On 02/23/2010 12:05 AM, Tim McNamara wrote:
> Hi Simon -
> Some quick thoughts:
> On Tue, Feb 23, 2010 at 7:00 AM, Simon Schampijer<simon at laptop.org> wrote:
>> * Chamindra (Sri Lanka Testing Lead) explains the great streamlining
>>> power of:
>>> Writing test cases and following them are a snap -- once someone
>>> installs this PHP site?
>>> As you noted the new Sugar release is scheduled for the end of march .
>> With another month of planned bug fix builds.
>> We need to do a lot of testing *now* :) A first attempt was the testing day
>> . We handed out a Sugar on a Stick image that people could use for
>> testing. This was not menat to work on the XO. One could either put it on a
>> USB-Stick and boot from another machine or run it in simulation.
>> To be honest, the outcome of this day was rather 'ok'. I have asked myself
>> a few questions how we could improve that, and I am happy to get feedback
>> from the actual people doing testing and QA.
>> * How does people want to work together? Asynchronously? Synchronosouly
>> during a meeting? (If) How often should we do meetings? Weekly?
> Planning around an asynch model is probably the best. We can do events etc
> periodically, but my hunch is that asynch will lower barriers to entry. And,
> it may be easiest to have lots of little testing sessions than a smaller
> number of much larger testing sessions.
Sounds good to me. The reason I think testing sessions can be fruitful
is that it might be easier to get into the community and reach better
results. For example to clear questions like: "How to write a good bug
report?" Once people are introduced to the topic, of course working
asynchronously (especially when most of the people are volunteers and
have other jobs and family etc) makes sense.
> One of the things that this ad-hoc QA team has been discussing is utilising
> a platform designed for test suites called Testlink (http://testlink.org).
The current url seem to be: http://www.teamst.org/
> That way, project managers& test analysts can write up specific test cases
> and/or migrate the test cases currently on wiki.laptop.org. Volunteers&
> teams that come in simply pick the next test at the top of the list.
> Ideally, this will avoid duplication and miscommunication across all testing
When we did QA at olpc we did use a wiki and a template (like you said
above). That worked ok for a start. It wasn't really a good tool to
track the success of a test for example. And it was not integrated with
the bug tracker. I did not know teamst until you pointed me to. If
someone is already familiar with it and think it will bring us forward,
I found this link with some more open source testing tools:
>> * Do people only want to test XO-images?
> My impression is that people will test what is useful& easy. Scratching own
> itch, etc. Therefore, if people are sent XOs, they will test XOs. If there
> is 'central' direction, I'm sure people will be open to testing.
Of course testing Sugar on the XO makes sense to find hardware related
issues. New Sugar features can be tested using Sugar on a Stick. As the
olpc deployments want to ship Sugar 0.84 many people will want to test
this, I guess. The current development version 0.88 needs testing, too -
but might not be on everyone's radar as it is not scratching directly
the own itch in the near future :)
> Despite the disjointed way things are, I personally feel that testing is an
> area where OLPC, Sugarlabs& activity developers need to have a coordinated
It's often difficult to divorce Activities, Sugar, XOs... I
> personally would feel it would be best if there was a central place to
> report test results& have developers take their leads from there. At the
> moment, Sugar& OLPC have their own trackers.
I think having those two trackers makes sense. You have to see here olpc
as a downstream project, and Sugar Labs as the upstream one. There are
hardware related issues for example, that should go into the olpc bug
tracker. Of course a ticket can be opened in the sugarlabs bug tracker
and being referenced there. We fought strongly for Sugar being a real
upstream project and I think we already made progress towards that
direction. Of course it is not always directly obvious ;p
Will testlink make things
Depends on how well it is integrated. Testlink could be, as you said, an
effort where all the parts are contributing to.
>> From my perspective as the release manager, I of course can announce new
>> images, with notes about what has been fixed, and what new features did
>> land. Would there be people willing to do testing from there? Or do I have
>> to run meetings etc?
> I don't know if more meetings will be beneficial at this stage. The testing
> efforts seem to be centred around small groups that are very geographically
> isolated. Mailing lists are probably sufficient.
Ok, which mailing lists should I announce it to? Where do most of the
testers read? How do we gather the testing results? Directly file bug
reports. If we have good guidelines how to file a bug report this might
More information about the Testing