XO in-field upgrades

[Christopher Blizzard]

[ Fixing Alex's address. ]

On Mon, 2007-06-25 at 14:58 -0400, C. Scott Ananian wrote:
> On 6/25/07, Christopher Blizzard <blizzard at redhat.com> wrote:
> > The broadcast just contains a product/version ID - doesn't have to
> > include the entire update.  No more expensive than the presence stuff we
> > have today.
> 
> You're misunderstanding me.  My concern is with waking machines up by
> broadcasting this information.  We don't wake up on presence, but we
> do want to wake up on (some, urgent) upgrades.

That's going to be interesting, yeah.  You would need to teach the
wireless firmware about it?  How about just checking on wakeup?  Some
kind of wake-on-lan signal?

> 
> > I think that what Alex here can do it either way.  You could use the
> > vserver copy on write stuff if you want to use the hammer of a
> 
> Again, you're misunderstanding me slightly.  Vserver has *very odd*
> semantics for its copy-on-write and for writing inside containers.  We
> need to sync up on that & come up with a plan, or we run the risk of
> creating a useless tool.

Can you explain how they are odd?  It sure would help everyone.

> 
> Binary diffs are also bidirectional, FWIW.

Yeah, but you always need both sets of information to be able to
generate them.  So you have to host full file + diff data if you want to
host an update.  The nice thing about Alex's system is that you only
have to host the file data that you're using on your system instead of
file + diff data.  You end up using less space that way.  If you want to
downgrade, you have to get the files or use the vserver versions (maybe
you could just use the old files handled by the CoW stuff to drive the
update system - that might work pretty well!)

> 
> > When you hear "complete blobs" can you describe what you mean?  I
> > suspect that you're thinking of something different than what Alex has
> > actually implemented.
> 
> Quoting from https://hosted.fedoraproject.org/projects/updatinator/ :
> ---
> Each computer running Updatinator tracks a specific image id, and runs
> a specific version of that image. Whenever it finds a manifest for a
> later version of that image id (and that manifest is signed by the
> right gpg key) that manifest and the required blobs for updating to it
> is automatically downloaded. When the manifest and all required blobs
> are downloaded Updatinator sends a dbus signal so that the system may
> let the user apply the update (e.g. automatically, or by showing some
> ui asking the user if he wants to update).
> ---
> 
> My next post will give some concrete #s justifing the use of binary diffs.
>  --scott
> 

Keep in mind that those "blobs" he's talking about are just files.  The
only place where we would add binary diffs would be for individual
files, not entire trees.  So what we're downloading today is only the
changed files, largely for the sake of expediency and what I describe
above for the space savings.

Speaking for myself I've never been opposed to use binary diffs.  To be
sure, all of my original ideas included them.  But if I have to choose
between having something that works today with full files and saves some
space and adding the complexity of binary diffs later, I will use the
former.  I would love to hear what you have to say about numbers for
binary diffs.  (I believe that in earlier discussions with people who
tried these systems before that any diff system that didn't understand
elf was doomed to failure, but I am ready to be shown the money.)

I think that both Alex and I are happy to listen to ideas here (esp
about the vserver stuff that you mention!) but it's June 25th and we
have to be pretty practical about what we can do between now and when we
have to ship.  The update system needs to be very simple, easy to test
and easy to validate + understand.  The key to that is using a simple
design and simple implementation.  Added complexity has a high bar for
inclusion here.

--Chris