updatinator benchmarking (Was: rsync benchmarking)

[Alexander Larsson]

On Wed, 2007-06-27 at 16:02 -0400, C. Scott Ananian wrote:

> Anyway, here are some quick stats, for upgrades from build 464 to 465 to 466:

I've been doing some more work on updatinator. Now it stores the
"release" info as a simple key-value file that contains the manifest
sha1 value, and the actual manifest is just a blob like the other. This
way we automatically get gzip and bsdiff support for the manifest files
(which tend to be around 2 meg for a full olpc devel image).

Here are some stats using bsdiffs and gzip. I don't have any byte
counter in the app, so i just looked at interface RX/TX stats. It counts
the full packet size instead of the tcp stream byte count, and it might
have data from other apps running on the system, but it should give a
somewhat correct maximum limit at least.

> 464-to-465:
> Full contents of tree: 429,302,610 bytes
> rsync --whole-file: sent 32,034 bytes; received 13,153,813 bytes.
> Standard rsync: sent 96,720 bytes; received 10,047,524 bytes. 9.0s
> user, 64.9s real
> Rsync-with-manifest: sent 79,192 bytes; received 9,139,515 bytes. 4.3
> user, 11.5s real.

Updatinator:
sent 596,681 bytes
recieved 7,545,281 bytes

> 465-to-466:
> Full contents of tree: 458,828,206 bytes
> rsync --whole-file: sent 21,990 bytes; received 25,826,516 bytes.
> Standard rsync: sent 141,948 bytes; received 23,981,423 bytes. 14.0s
> user, 68.5s real
> Rsync-with-manifest: sent 125,158 bytes; received 23,171,430 bytes.
> 10.4 user, 28.5s real.

Updatinator:
sent 1,263,885 bytes
recieved 20,718,744 bytes

I haven't looked into why we're sending so much data, perhaps its due to
other traffic on my box, or perhaps the http accesses are just too
chatty. But even taking that into consideration we're doing 1.2 meg less
total network traffic in the 465-466 case than the best rsync case (and
we're not doing a lot of local disk i/o).

I'm also creating an updatinator repository at 
http://olpc1.redhat.com/updatinator/
so that people can start test it, and eventually use it to easily update
their XOs.

Another interesting piece of stats here is the size of this repository.
A repository with all the ext3 and jffs2 developer images from version
400 to 450 has a blobs directory of 1.3 gigs. The estimated size for
just the tarballs of these revisions is around 12 gigs. This is a pretty
good compression ratio. This is good for the school server disk use and
means it can easily store all old releases.