[Testing] Fwd: 11.2.0 Test cases

[Samuel Greenfeld]

What we need to figure out in the short term is what test case storage
system we are going to use, as well as what sort of test execution
planning & reporting system we are going to use.  While I've since
gotten a Nitrate install (which does both) to mostly work so I can
discuss it at FUDcon, we do not know when Fedora is going to formally
put theirs online and what it will look like.

In general I know what test cases I want.  But I don't know what system
to put the test cases in.  These test cases are not necessarily unit
test cases; for instance there may be more than one way to execute a
collaboration test, and the comments/system has to note that with the
test run.  And the number of test cases which need to be put somewhere
is potentially large; if we create just 5 test cases for the ~40 G1G1
XO-1.5 activities that's ~200 test cases & items to be run, not
including any for the base system or enhancements.  Moving them all
later could be a pain.

It looks like the current testcase Wiki template that Fedora is using is
a subset of ours (although many Fedora test cases don't use it).
Therefore presuming they make a converter to handle that, they likely
can handle our existing test case template similarly.  Nitrate and other
TCMS's I've looked at use an inline HTML editor instead of Wiki code, so
a converter would have to be used or lots of manual work to go back and
forth.


I'm a bit more concerned about where to put the test execution plan, as
in some ways that controls how the test cases should be written.  To
simplify a wiki-based approach, we could pair it with another project
with a "developer's/tester's" database of all activities' functionality,
and how this varies over time.  The advantage of this is that it means I
have to create relatively few wiki pages, and the database probably
could be an alternative view of the activities.sugarlabs.org data with a
few extra database columns.

Every activity basically needs the same sort of things tested:
Open/close, save/open journal entries, supported MIME types,
collaboration, etc.  So a standard test case with a database containing
the specifics makes sense.  The advantage is we don't have to worry
about creating lots of Wiki pages, and we have accurate documentation on
how activities work outside of the test case/run system(s).  The problem
with this two-system approach is that I don't think it ports to any
other test case management system cleanly if at all.



On 01/26/11 20:33, Samuel Klein wrote:
> On Wed, Jan 26, 2011 at 8:23 PM, S Page <skierpage at gmail.com> wrote:
>> I'm in violent agreement with everything you say :)
>>
>> *If* you have useful information in w.l.o pages, other pages can query
>> and show it.  That's all.
>>
>> I don't know anyone else who's managed test results in a wiki.
> 
> Ditto.  Syncing with Fedora's process make sense to me.
>