[olpc-nz] Today's/Tonight's QA Phone Call Conclusions & Unofficial Minutes

[Tom Parker]

On Sat, February 27, 2010 8:31 am, Tim McNamara wrote:

> Something as "simple" as a tracker would work. If you want something
>> tested, raise a ticket. Developers can look at the ticket to see what
>> the status of your test request is. Testers can look at the list of open
>> tickets to see what is ready for testing. Every request has a date and a
>> log of what has been done. This is how the test groups work in the
>> software development houses I've worked with.
>>
>
> This is worth investigating. Properly written test cases & test requests
> will solve these issues.

Test requirements are a different issue. "What do we test" is currently
answered by http://laptop.org.nz/test-request which isn't really working.
How to test can be in the ticket, or elsewhere.

It looks like Tabitha has already tried the ticker approach on the
laptop.org tracker (see first paragraph of the laptop.org.nz link above).
I wasn't able to construct a query there to find just those tickets -- is
there a more advanced search function in trac than the single search box?
I'd like to exclude closed tickets for a start.

> What do we test today? What has been taken up by
> other groups when that email came through to x, y, z list? What
> methodology
> do Activity developers want used?

If you want to test something, assign the ticket it to yourself (or your
group).

> Bug/ticket triaging becomes significant.. because these reports are not
> bugs.

Problems arising out of test requests should be filed as regular bugs. I'm
used to jira, where we have different types of tickets, so you can raise a
bug, an improvement or a support request. Looking at trac.sugarlabs.org, I
don't see anything similar.

Is it possible to create a "test request" ticket against the specific
component under test and write a query to find all the open test request
tickets against all components? I guess a tag would work, does trac
support tags or labels? If we use the full-text search, then the tag would
have to be somewhat obscure to avoid matching unrelated sentences.

> I sense this may place unnecessary burden on developers over time. First
> of
> all, it creates noise in the tracker. Secondly, devs will need to create a
> new ticket for the next version of whatever component, but the test cases
> remain the same.

The test request has to be duplicated for each release (otherwise we
wouldn't know that new testing is required) but the details on how to do
the testing shouldn't be included in the test request ticket itself. They
should be on a wiki, or in a test management software somewhere.

> I like the fact that we don't need much more infrastructure, but don't
> know
> how practical things will be in the medium term.

It will work fine, a few dozen extra tickets in the tracker won't hurt
anyone, especially if they have their own component/class/what-have-you.