[Server-devel] [XSCE] Reflashing over a network

[Richard A. Smith]

On 08/29/2013 04:45 AM, Tony Anderson wrote:

> The real advantage of wireless 'flashing' could be that a roomful of
> laptops could be flashed concurrently as in NandBlast. I assume that
> NandBlast is using the broadcast capability of the network. This is
> getting beyond my understanding of networking, but wouldn't it be
> possible to implement the 'sender' side of NandBlast on the server and
> have it start broadcasting the image.

Nandblaster is much more complex than a simple broadcast.  It does use 
"broadcast" but it does it in as low level and as raw of a format 
possible.  The reason is that wireless is lossy and broadcast packets 
have no means of ack from the clients.  To compensate nandblaster uses 
forward error correction where it sends extra information with each 
block(s) of data. You can sort of think of it as RAID 5 for wireless. 
The actual amount of data is configurable but and example would be that 
for every 2 blocks of data 1 extra block is sent that allows you to 
reconstruct the 2 blocks of data from any of the 3 blocks.  In that case 
you could sustain a 33% data loss and still get 100% of the data.  But 
you have also decreased your throughput by that same 33%.

In order to make this scheme work you have to turn off a whole host of 
things that are stock with wireless.   The wireless card in XO-1 allowed 
operation at a very low level.  The cards in XO-1.5 and beyond had 
varying degrees of that low level functionality making it more difficult 
to make nandblaster TX work.

Over time I think some of those limitations were dealt with.  I've lost 
touch of the complete state of nandblaster.  James can perhaps update 
with exactly what works and what doesn't.

Getting a good nandblaster in a noisy wireless environment can be very 
tricky.  Unless you get 100% data the client has to wait an entire cycle 
of the image to pick up the missing blocks.  That can be a long time. 
But if you crank up the error correction so your 100% reception rate 
goes up the overall programming time still suffers because for 99% of 
the image data you sent error correction that was not needed.  You can 
quickly end up sending double the original image size.

When the image size increased in XO-1.5. Mitch Bradley and I spent a lot 
of time working on a wired version of nandblaster for the factory 
production line. (At the factory XO's were flashed via USB wired network 
adapters)  While it worked it was a bit twitchy based on what sort of 
equipment was in between the server and the XO clients.  Broadcast and 
multi-cast packets are not handled optimally by a lot of equipment.

We were able to get a lower programming time by using fiber-optics to 
the programming rack and a server that could handle the IO requirements 
of many stock http: connections sending the same file.  That eventually 
gave way to a gang programmer for XO-1.5 SD cards and then for XO-1.75 
and XO-4 the factory now preps a bunch of USB flash drives and uses those.

USB flash drives have the unique ability to scale perfectly with the the 
number of XOs you want to program at once.  It actually turns out to be 
cheaper than a bunch high performance equipment too.

Somewhere in the repositories we have the code (linux) that will 
generate a image file with the forward error correction and a server 
that will broadcast that image onto any network that can do multi-cast. 
  I think that should work with any wireless setup as long as its 
configured to allow multi-cast ranges.

The XO (OFW) client would night need some love as its not been used 
since 1.5.  IIRC the changes to stock nandblaster were not that 
extensive.  One could also make a tiny linux version of the client that 
you could netboot and do it that way although net booting in the face of 
massive multi-cast traffic might be problematic.  Perhaps you could load 
the tiny-linux client via USB and then let it join the multi-cast network.

If you have a lot of XOs to program and you don't require speed then a 
nandblaster type solution is very workable.  Its slow but requires very 
few resources.

Again I'm hazy on the current state of nandblaster so someone else will 
have to tell you how much effort is needed in the XO firmware to make 
that something that might be compatible with an XS broadcaster.

-- 
Richard A. Smith  <richard at laptop.org>
Former One Laptop per Child