Automated testing of activities

[Kent Quirk]

Please don't read this as me objecting to the concepts of automated 
testing or accessibility support -- I'm generally in favor of both. But 
they can have pretty major implications, especially on schedules.

Many questions:

When you say "accessibility" you mean "support for vision-impaired 
users"? Are there other accessibility requirements?
What in particular are the requirements? Support for a screen reader? 
Large fonts? Zoomable screen? Will there be a screen reader application 
for the system? Where is it coming from, and how will it interact with 
activities?
How does the accessibility interact with the intent to support 
localizable or localization-free activities, in large part by leaving 
text out of the interface entirely? What is a textless application 
supposed to do in this environment?
What parts of the system are going to have to comply with this 
requirement? The mesh view, for example? The clipboard? What about, say, 
drawing activities? The record application? Games?

Is automated testing intended for more than just battery life testing? 
If not, is it really necessary for every activity to support it? If so, 
what do you expect to accomplish? Will it actually save more than the 
amount of time taken to implement it for a given activity?

What are the time constraints?

The potential scope is huge...it would be nice to understand the actual 
requirements.

   Kent



Jim Gettys wrote:
> I want to give everyone a heads up to start thinking about the following
> issues:
>
> 1) we need to be able to do basic automated smoke testing of activities.
> 2) we need to support accessibility in activities.  Some of you may not
> be aware of it, but this is a non-optional requirement of some
> jurisdiction, most of which roughly follow US government laws in this
> area.
> 3) we need to be able to quantify improvements/regressions in activity's
> energy usage, as we optimize various code in our system.
> 4) we need to be able to automate testing of realistic workloads (or
> play loads :-), of our systems that roughly simulate the use of a child
> in school, so we can see how we're doing when we change various knobs
> that we have for controlling power usage, from backlight, to use of the
> dcon, to blanking the screen, to suspending aggressively, etc.
> Applications adding hints in key locations that suspending might be a
> good thing to do are also becoming possible, as our power management
> infrastructure improves.
>
> But if we can't reproduce results, we'll be in fog, unable to see what
> direction to go.
>
> We'll therefore need to be able to script applications.  So long as
> we're on an XO-1 with its resolution screen *and* you don't change the
> UI, it's not all that hard.  But we expect all of you to want to tune
> your UI's, and also we need to ensure accessibility needs get met.
> Future systems our software runs on may have different screens; this
> model will break down pretty quickly. 
>
> Note that the hooks required for accessibility hooks makes it possible
> to script activities by name, rather than by X/Y coordinates on the
> screen, and wait for results, and that this technology therefore can
> remove the screen resolution dependence of such scripting.  Custom
> widgets you build will need such hooks, in any case.
>
> We'll shortly be setting up a battery life testing infrastructure in a
> revived Tinderbox; with the machine we have with instrumented with > 20
> measurement points we can gather great amounts of useful information on
> the behavior of our system and of individual activities.
>
> At some point, we'll start asking you to generate a workload for an
> activity, which should be able to address many of the issues above.
> More when the infrastructure work is further along.
>
>                                - Jim
>
>   


-- 
------------------------------------------------------------
Kent Quirk           I'm making a game about global warming.
Game Architect                        Track the progress at:
CogniToy                http://www.cognitoy.com/meltingpoint