[Grassroots-l] pilot project working group

[Sameer Verma]

Edward Cherlin wrote:
> On Feb 3, 2008 11:45 PM, Alexander Todorov <alexx.todorov at gmail.com> wrote:
>
>   
>> We (olpc-bg) 've had a discussion on how do we measure success of a
>> pilot. We didn't come to a clear conclusion.
>> One of the proposals was:
>> - measure computer literacy as is done now (for grown ups). That's not
>> applicable to children because they don't need office software skills. A
>> child used to computers will pick up the necessary skill very quickly if
>> needed.
>>
>> - measure achievements in education: how? We don't have a clear idea
>> what exactly to measure because it hasn't been done so(for small
>> children) in the country.
>>     
>
>   
Typically there are two broad perspectives - development and use. The 
development perspective will deal with developer issues such as lines of 
code, bug fixes, suspend/resume etc. and can be measured using data from 
logs (such as Richard is doing with logbat) and we can get a good 
understanding of how things work in the context of software, hardware, 
and network.

Use perspective is a bit more complicated. Here, we need to measure the 
user's perception of the system. Assessing that perception at a young 
age is difficult because we cannot ask 6 year olds to fill out a 37 item 
questionnaire with Likert scale items :-)
> You need to get the experts involved in your experiment design. Invite
> Alan Kay, Seymour Papert, Jerome Bruner, and Benjamin Mako hill, who
> are in OLPC in one way or another, or close to it, and ask them whom
> else to bring in.
>
>   

 From an experimental design perspective, we need to measure "before" 
and "after" and assess if the delta between the two is significant 
enough to be of any value. Anecdotal evidence is supportive, but not 
conclusive from an experimental design view. Assessment will require 
both qualitative and quantitative approaches. We also need to keep in 
mind that the results of any study will be limited by the extent to 
which we can apply results to the rest of the world (external validity).

> One line of attack on the problem would be to read the great
> educational researchers such as Maria Montessori and Caleb Gattegno to
> find out how they observed children learning, and how they verified
> the hypotheses they came up with. The education profession has still
> not caught up with their discoveries. Unfortunately, some in the
> profession have turned their advances into a new orthodoxy, complete
> with schisms and anathemas.
>
> Another line of attack is to ask the children what they are learning
> and how to measure it. They know more than you think, so don't dismiss
> what they say out of hand. But you can't follow blindly what they say
> either.
>
>   
Indeed. My daughter (28 months old) can open the XO and turn it on in 10 
seconds flat. It takes a lot longer for it to boot into something 
meaningful for her, so she loses interest at that point. But that's 
another story. Many grown-ups cannot figure out how to open the laptop 
even after seeing the technique a few times. Of course, she wouldn't be 
able to express that procedure to me. I would have to observe it and 
take notes, photos and time her actions.

Sameer

-- 
Dr. Sameer Verma, Ph.D.
Associate Professor of Information Systems
San Francisco State University
San Francisco CA 94132 USA
http://verma.sfsu.edu/
http://opensource.sfsu.edu/


> Another key question is: What are the children _not_ learning now,
> that they could learn? Such as how to learn a subject on their own.
>
>   
>> Greetings,
>> Alexander.
>>     
>
>