[Testing] Actions: OLPC QA Call with Sri Lanka's Professional Volunteers

[James Cameron]

On Wed, Mar 17, 2010 at 08:34:46PM -0400, Holt wrote:
> Updates & fixed "Test Cases" wiki links below!
> 
> Both above links now fixed -- James Cameron can now help provide a sample test
> case for 10.2.0 below:
> 
>                     http://wiki.laptop.org/go/Test_cases_8.2.2
>                     http://wiki.laptop.org/go/F11_for_XO-1#How_to_help_test
>                     http://wiki.laptop.org/go/Test_cases_10.2.0

I've had a good think about this.  In summary, I withdraw the suggestion
to use the links above, and you should use either
http://wiki.laptop.org/go/1_hour_smoke_test or
http://wiki.laptop.org/go/Category:Test_cases for self-directed
testing, and http://dev.laptop.org/report/42 for ticket-specific test
cases.

The reasons for my recommendation are:

1.  we are not following the same release process,

2.  we do not necessarily know the next release number,

3.  we have adequate test cases for self-directed testing,

4.  we have ticketed test cases for specific problems.


Further details below.

1.  we are not following the same release process.

The format chosen for the Wiki pages in releases 8.2.0 and 8.2.1 was
based on a release planning process that is not being followed at the
moment.  Or I've missed it.  I don't think the release planning
process previously followed should be followed though, due to resource
constraints.

What used to happen was that the release plan identified tickets that
would be fixed before a release, and based on those tickets the test
cases were selected in a per-release test cases Wiki page.
Test_cases_8.2.1 is an example of such a page.

Not having release plans prevents this process, and besides for the
XO-1.5 we're changing quite a bit.

2.  we do not necessarily know the next release number.

Critical to the naming of the page was knowledge of the next release
number.

Regarding XO-1.  OLPC OS 8.2.1 was released long ago.  OLPC OS 8.2.2
is in preparation.  Martin Langhoff is the release engineer.

I'm not aware of a formal plan for this release, though I think it is
fair to say that all the test cases on the Wiki should be considered.

Regarding XO-1.5.  OLPC OS 10.1.0 was released on 19th December,
consisting of build os64.  We don't yet know whether the next release
will be marked 10.1.1 or 10.2.0.  Chris Ball is the release engineer.

I'm not aware of a formal plan for this release, though I think it is
fair to say that all the test cases on the Wiki should be considered,
except for those that test SimpleMesh, or do reinstallation of NAND.

3.  we have adequate test cases for self-directed testing,

http://wiki.laptop.org/go/1_hour_smoke_test
is an adequate initial test that
verifies most required functionality is present.  If a laptop fails
the smoke test, then advice should be sought on whether the laptop,
the software, or the smoke test is faulty.

http://wiki.laptop.org/go/Category:Test_cases is the collection of
test cases from which the smoke test and release test plans were
derived.  This should be used for self-directed testing.

4.  we have ticketed test cases for specific problems.

In the current build testing, we test on a per-ticket basis.

Development builds are in two streams; a stable stream (os200, os201)
and an unstable stream (os101 through to os114).

The developers are very interested in testing being done on the latest
unstable version, since this testing feeds the development process,
either confirming fixes or generating work that must be fixed before
release.

Some tickets have a clear test case, either marked as such or implied
by the ticket symptom.

-- 
James Cameron
http://quozl.linux.org.au/