[Testing] "Test Activity" Update

[Michael Stone]

On Sun, Oct 21, 2007 at 08:30:28PM -0400, Kim Quirk wrote:
> I would love to get an update of the Test Activity -- sending logs to a
> central server; what might be the next or a parallel step we can take.

My conclusion at the moment is that it is a significant design flaw that
our definition of "activity" doesn't include a scripting interface and a
better tracing mechanism that automatically tries to push logged data
upstream.

We probably can't fix the scripting interface issue right now, but the
data-packaging and upload is something that can be fixed
straight-forwardly with a tiny modification to sugar.

In short, sugar could easily add an optional "activity-wrapper" that
gets invoked with the same arguments as "sugar-activity" but that gets
to do arbitrary snooping before, during, and after it triggers the real
activity launch.

In the short term, the wrapper will just turn on logging, launch the
real activity in the normal fashion, wait for the activity to die, and
then package and attempt to upload the logs.

In the long term, we may find more creative uses for such a wrapper -
for example, it might learn how to talk to the Test Activity and it
might learn how to talk to the activity it's wrapping. :)

(Incidentally, Rainbow already uses this idea to good effect when it
runs activities under strace. More to come here, shortly.)

Anyway, after we implement something like this, then the Test Activity
basically turns into a thin wrapper around this API and a prepackaged
view of the test case db you mention below, along with a form for human
feedback.

> I would also like to discuss the test case mgmt system. I think these two
> activities need to be linked -- the test cases are in the trac database, and
> the reporting back from the laptops out in the world woud be to this
> database.

Agreed. Thus far, I know much less about how this should work.

Michael