[Testing] ssh-controlled testbeds with dsh

[Gary C Martin]

Hi Mel,

On 7 Nov 2008, at 23:57, Mel Chua wrote:

> Following up on Gary's emails and Michael's suggestion from last  
> week, I was playing around this morning with...
>
> http://wiki.laptop.org/go/Dsh
>
> ...and life looks pretty glorious.
>
> Why is this cool? Well, say you wanted to run the ps command on all  
> the machines in your /etc/dsh/machines file, which looks like this...
>
> olpc at 18.85.49.113
> olpc at 18.85.49.114
>
> All you have to do is this:
>
> mchua at tumtum-tree:~$ dsh -Mac ps
>
> And you'll get this.
>
> olpc at 18.85.49.113:  PID TTY          TIME CMD
> olpc at 18.85.49.113: 1166 ?        00:00:00 startx
> olpc at 18.85.49.113: 1185 ?        00:00:00 xinit
> olpc at 18.85.49.113: 1211 ?        00:00:02 ck-xinit-sessio
> <...more entries from 18.85.49.113 go here>
> olpc at 18.85.49.114:  PID TTY          TIME CMD
> olpc at 18.85.49.114: 1549 ?        00:00:00 startx
> olpc at 18.85.49.114: 1566 ?        00:00:00 xinit
> olpc at 18.85.49.114: 1577 ?        00:01:31 python
> <...more entries from 18.85.49.114 go here>
>
> To answer Gary's previous question, we're trying to do the third  
> option - option 2 as a special case of option 1. The example on the  
> dsh wikipage is a good simple example (running ps).
>
>>
>> 1). Want to run some admin script on every XO. Perhaps you want to  
>> wipe the datastore and reboot, perhaps you want to trigger suspend  
>> and use administration ping to wake them up again repeatedly, etc.
>>
>> 2). Want to collect a set of data from every XO and look for  
>> anomalies/failures. This could be lists of buddies each is  
>> currently seeing, or AP each is seeing, cpu loads, free memory,  
>> dropped packets, packet collisions, etc.
>
>> You could see option 2 as a specific case of option 1, where you  
>> write a nice batch that collects and cleanly reformats all the  
>> spurious outputs you get from the various tools.

I've updated the below wiki page with some of the bash tricks/scripts  
I've been using. No 3rd party tools needed, all it assumes is that  
you've made a public/private key pair and copied the public key over  
to all the XOs to automate authentication:

	http://wiki.laptop.org/go/Scripts_for_testing_multiple_XOs#Example_code

The last example there shows a slightly more complicated script  
collecting a bunch of nicely formatted data for each XO. If you can  
tell me the numbers/fields you actually want to collect, I could make  
that a more useful script for you...

Regards,
--Gary