[Server-devel] Reflashing over a network

[James Cameron]

(another composite reply)

On Thu, Aug 29, 2013 at 10:45:29AM +0200, Tony Anderson wrote:
> I believe the 'boot net' would require only moments more per laptop
> than the 4-key approach.

Yes, about 10 seconds, but the laptop would have to be unlocked.

Also an added benefit of testing the b, o, o, t, space, n, e, and
enter keys on the laptop.

> The real advantage of wireless 'flashing' could be that a roomful of
> laptops could be flashed concurrently as in NandBlast. I assume that
> NandBlast is using the broadcast capability of the network. This is
> getting beyond my understanding of networking, but wouldn't it be
> possible to implement the 'sender' side of NandBlast on the server
> and have it start broadcasting the image.

Yes.  However, there's no protocol for asking for the broadcast, the
broadcast has to be already happening.  If you consume one wireless
channel at the site for this broadcast, that's one less channel you
can use for collaboration and other client services.

On Thu, Aug 29, 2013 at 10:45:29AM -0400, Richard A. Smith wrote:
> Again I'm hazy on the current state of nandblaster so someone else
> will have to tell you how much effort is needed in the XO firmware
> to make that something that might be compatible with an XS
> broadcaster.

Could probably be done for XO-1.5, XO-1.75 and XO-4.  The code is in
C, in the multicast-nand git repository.  It would need a week or two.
I really don't think it is a good idea, because I can't imagine
wanting to consume that server connected radio spectrum for something
that rare.

(A NANDblaster party can happen away from the server connected access
points, using one of the laptops being deployed.)

On Thu, Aug 29, 2013 at 05:26:51AM -0500, Jerry Vonau wrote:
> On Thu, 2013-08-29 at 18:19 +1000, James Cameron wrote:
> > On Wed, Aug 28, 2013 at 03:52:46PM +0200, Tony Anderson wrote:
> > > This is getting interesting but I am still not sure I understand
> > > the process.
> > > [...]
> > > There is an alternative via
> > > 
> > > boot net
> > > 
> > > which invokes a different firmware process that is working in
> > > all XO models. This process uses tftp. The server would respond
> > > to a tftp request from the XO by sending the fs.zip and .img
> > > files.
> 
> Is tftp used just to retrieve the fs.zip file?

No.  For "boot net", the firmware opens the network, then uses TFTP to GET
a file named OLPC.  This is a Forth script.  It can do what it likes.

I guess you might use a Forth script inside an fs.zip file, but I
don't see why that would be necessary.

Typically, the Forth script would then use HTTP, FTP, CIFS, or NFS for
the next step.

> Is there a speed advantage over http with tftp?

Given the data fetched with TFTP is tiny, the size of a script, I
don't understand why you ask.  Perhaps you thought the bulk of the
data would be fetched using TFTP, but it would not necessarily be so.

TFTP is faster than HTTP for small files, because the number of
network packets exchanged is smaller.

HTTP is faster than TFTP for large files, because there can be
multiple packets in flight.

TFTP might not saturate a shared network as much as HTTP.

Open Firmware uses TFTP for "boot net" because of requirements prior
to OLPC.

> > No point, 'cause the time you waste getting a Forth program into
> > the laptop is better spent getting the install data into the
> > laptop.
> 
> The old recipe called for NN PP tags to be added that to OFW, not
> needed if you use OLPCOFW now.

I'm confused by the context.

In the absence of a USB ethernet adapter, and the absence of NN and PP
tags, the firmware uses the SSID of OLPCOFW.  Otherwise it uses the
SSID from the NN tag, and an optional WPA passphrase from the PP tag.

> > Yes.  It works here, I just tried it with Q4D34JE with the web
> > server on 172.18.0.1 loaded with 32013o2.zd and fs2.zip.
> 
> 172.18.0.1 would need to be available/aliased on the server for this
> as that is coded as the default for the http transfer?

Yes.  In try-fs-update.  Either DHCP needs to hand out addresses in
172.18.x.x, or 172.18.0.1 needs to be routable by the host that owns
the default route.  I added an alias to the host that is my DHCP server.

> In contrast tftp would rely on the DHCP server for the server to
> contact and doesn't have this limitation.

That's one way to put it.  Another way is: Use of TFTP requires a DHCP
server and a TFTP server.  Using an HTTP server configured for
172.18.0.1 doesn't have this limitation.

> > > suppose if we craft an olpc.fth script to setup the wifi
> > > networking that could be used in place of the 4 button boot?
> > 
> > I don't understand the question, sorry.
> > 
> 
> I'm was just wondering if we can override what is defined in
> firmware by booting a usb flashdrive with the same commands that you
> would use for initializing the wireless for example flashing of
> firmware. I mean setting of the SSID(MM), wep/wpa(PP), that sort of
> thing. Much easier to just configure an open access point with
> OLPCOFW.

Yes, you might boot a USB drive with an olpc.fth file on it which
temporarily configures SSID, WPA, then flashes firmware from a network
resource.

\ my boot script
" dudes" $essid
" correctbatteryhorsestaple" $wpa
boot http:\\schoolserver\xkcd\cartoons\install.fth


For unlocked laptops, this can be in boot/olpc.fth.

For locked laptops, this can be in fs.zip in the same way that the
collection stick is constructed, signed with deployment keys.

> > Here's what happens if the four game key boot method is used:
> > 
> > In the absence of USB drive, SD card, or a nearby NANDblaster, the
> > firmware will open the network, which means one of either:
> > 
> > - initialise the USB ethernet device, or
> > 
> > - associate with either OLPCOFW or the SSID identified by the NN
> >   tag,
> > 
> > and then send an HTTP GET to 172.18.0.1 asking for the file
> > fsN.zip, which it then validates against the security system,
> > before executing it as normal as if it had been found on USB
> > drive.
> > 
> 
> I take it this would be for signed build only or can the image be
> unsigned if the XO is unlocked?

When the four game key method is used on an unlocked laptop, security
is temporarily enabled, so an fs.zip from any source has to pass the
signing checks.

On Thu, Aug 29, 2013 at 09:25:26AM -0400, Tim Moody wrote:
> with the Norton Ghost provisioning server the server waits for all
> expected clients to connect and then broadcasts a single copy of the
> image for all to install.  is that what is happening here?

No.  An infinite number of copies is broadcast, until you're sick of
it and turn off the sender.  This would usually happen once the number
of laptops yet to be reflashed approaches a minimum that can be redone
using USB drive.

See http://wiki.laptop.org/go/NANDblaster and the linked pages.

> > > There is an alternative via
> > >
> > > boot net
> > >
> >
> > (Reminder: this is the manual "boot net" scenario, not the
> > automatic four game key boot scenario).
> >
> > > to be clear, this command must be issued by the user after a 4
> > > button boot lands in the firmware?
> >
> > No.
> 
> This seems like a contradiction.  What am I missing?

I don't see any contradiction.  Perhaps you didn't know that a four
game key boot doesn't get you the ok prompt?  You must use the escape
key for that.  See http://wiki.laptop.org/go/Ok

On Thu, Aug 29, 2013 at 09:45:40AM -0700, David Farning wrote:
> An interesting caveat is the expectations of the person doing the
> flashing. In a nandblast or other automatic situation the technician
> doing the work expects it to work automatically... flawlessly.

Ridiculous.  No technician should expect something to work flawlessly.
Broken antennas, twisted antenna coax, RF noise, dead zones, flat
batteries, will all render some receiving laptops unable to complete
in a reasonable time.

The technician would simply reflash problematic laptops with a USB
drive, rather than mindlessly follow a procedure.

The same expectation of flaws can be said of USB drives.  Some ports
are damaged, some USB drive connectors become inconsiderate, and
plastic shells break.

If I was in Tony's scenario with 100+ laptops, I would:

- get two or more USB drives, prepare them with the .zip and .zd files
  (3 min per drive),

- grab one laptop and set it up as sender (25 secs),

- for each remaining USB drive start it reflashing a laptop (5 secs
  per drive),

- for each remaining laptop start it receiving the broadcast (2 secs
  per laptop),

- look busy, monitor progress, and wait for the next USB drive to
  become free (5 mins later),

- select a receiving laptop that is displaying data loss and restart
  it with the USB drive (5 secs),

- if there is a shortage of USB drives and some laptops showing data
  loss, move them closer to the sender,

- repeat last three steps until all laptops are reflashed (typically
  within the hour).

Then there's the charging, keyboard and display checks, and so on.  ;-)

-- 
James Cameron
http://quozl.linux.org.au/