update mechanism for new releases

[Daniel Drake]

Hi,

We have a bug open here which is a difficult question, in my opinion:
http://dev.laptop.org/ticket/9349

For the upcoming XO-1.5 software build (which will hopefully make it
onto XO-1 in the near future), do we want to ship olpc-update or use
standard Fedora technologies?


My unconclusive thoughts and questions:


olpc-update has some features which aren't available elsewhere, such as
the ability to switch between 2 OS builds installed on the same disk.
However, I've never seen or heard of this being used in the field.

olpc-update is a "deployment quality" system in that it has covered
everything - not only from being an updater, it has olpc-update-query
which implements logic to figure out when to ask for updates and where
to ask for them. It's accompanied by the theft deterrence protocol and 3
different server-side implementations of that. It's been implemented in
OLPC deployments.

olpc-update creates a huge mass of hardlinks on disk, one for every file
on the main OS. It then rsyncs in the updated files on a copy-on-write
basis. Therefore you end up with a hell of a lot of hardlinks, and 2
complete copies of every file that has changed. (this means it is not
great for situations when a lot of files have changed, e.g. changing
between major Fedora releases)

olpc-update presents challenges for deployments that customise their
images. Namely, to produce a build that it is possible to olpc-update to
involves quite a bit of hassle (inserting security keys into the
firmware of each XO, setting up an OATS server, etc). We did manage to
figure this out in Paraguay (thanks to the internet connection, since it
wouldn't be safe to put the OATS server in the schools) and I improved
some tools accordingly.

We've switched to dracut for our initramfs, meaning that the
initramfs-level stuff for the olpc-update system needs to be
reimplemented. Basically we have to wrangle with the strange /versions
filesystem layout and "frob" the /versions/current symlink if the user
is switching between versions. I have reimplemented this logic in a
dracut module but it is completely untested.



The option for using "Standard Fedora technologies" as far as I am aware
is to basically use "yum update" in a fashion somehow suited for
deployments.

Before F11, this would have meant a lot of downloading as each RPM
update was downloaded in full even if only a little bit changed. As of
F11, Fedora is now joining the "only download the bits that change"
movement with yum-presto and deltarpms: http://lwn.net/Articles/329484/

The deltarpm description on that lwn article leaves a little to be
desired -- it wouldn't work with big packages (is the quoted 200mb the
compressed or uncompressed size?), the reconstruction from deltas is a
CPU intensive task and we have a slow CPU, and it's all done in memory
and we don't have a huge amount of memory especially on XO-1.
olpc-update has an overhead of about 5 minutes of checksumming files.

Even though olpc-update isn't great for doing big distro updates
(because of storing 2 copies of changed files, in this case almost all
of them), it worked in those situations. I've never attempted an
RPM-based update from e.g. Fedora 10 to Fedora 11. How well does that
work out for regular Fedora users?

"yum update" always seems to use a surprising amount of bandwidth,
redownloading entire package databases just to see if anything new is
available. olpc-update was much more efficient in this respect, sending
only a 128-byte hash of the filesystem contents file to the OATS server.
For rpms, is there a more efficient alternative for updates-checking in
situations where there is only going to be e.g. 1 update per month?

"yum update" has historically not worked very well on the XO. It hits
OOM, it fills up yum's cache and then aborts, it gets confused between
i586 and i686, etc.

We would have to tweak yum's behaviour quite heavily.. I don't think we
want an rpm cache, or for it to keep the .rpm files at all.

It introduces new questions of security, signing, etc. Deployments will
want to sign their rpm updates that they push out, so we now need a
mechanism for getting that specific RPM signing public key onto all the
laptops in a deployment so that they can trust the updates server.
We had this nailed down for olpc-update: deployments can insert "local"
public keys into the manufacturing through keyjector firmwares for
existing laptops, and Quanta can now manufacture laptops with these keys
already in place for future orders. olpc-update and the bitfrost code
used these keys to verify updates.

For the XO-1 possibility it raises the question of how existing laptops
could be migrated to this new system, without losing their user data.

Using "yum update" would be a large benefit of sharing technologies with
other communities and not having to maintain our own systems and tools.
However olpc-update would be easier in the short term because all the
components have been implemented and deployed, and there aren't really
any open questions.


Any other thoughts/opinions one way or the other?

Daniel