boot-anim

Fri Jun 13 12:00:54 EDT 2008

On Fri, Jun 13, 2008 at 6:30 AM, Neil Graham <lerc at screamingduck.com> wrote:
> As it stands now there seems to be no 100% reliable way to judge the
> compressed size of things on jffs2.
>
> I  cast my eye to the boot-anim,  uncompressed it comes to about 60 Meg, is
> there space being wasted there?
>
> http://lists.laptop.org/pipermail/library/2007-July/000070.html says
>> JFFS2 compresses 4K chunks using zlib.  So it's not just per-file
>> compression, it's compressing bits of a file.  It doesn't compress files
>> where compression doesn't help.  And there's a 68 byte overhead per
>> compressed chunk.  Plus probably some fixed overhead per file.
>
> doing some tests using mkfs.jffs2 on a filesystem with a single file of zeros
> at varying sizes gave me.
>
>  409600 --> 11656
>  819200 --> 23256
> 1228800 --> 34856
> 1638400 --> 46456
>
> each growth of 100 4k blocks added 11600 bytes.  I'll go out on a limb and say
> the best case on jffs2 is turning a 4k block into 116 bytes,  about 35:1
> compression, or down to 2.8% if you prefer
>
> This is why the boot-anim bothers me.  If it were all zeros it should be
> taking up just over 1.5 meg.  It compresses really well, but it can't go
> beyond that limit.
>
> Running the jffs2size.py script on /usr/share/boot-anim reveals.
>
>  no compression : 60480336, 60480K
>  estimated jffs2: 1983630, 1983K (3%)
>  mkfs.jffs2     : 2344068, 2344K (3%)
>  zipped         : 389259, 389K (0%)
>  .tar.gz        : 399360, 399K (0%)
>  .tar.bz2       : 184320, 184K (0%)
>
> It looks like there's easily a meg to be had there, two meg looks doable.
>
> Where to go from here is another issue.  total flexibility and 90% of the size
> gain could be had by using pngs (not sure if there is a speed issue there)
>
> If there were to be a simple runlengh encoding,  the bulk of the size would go
> and jffs2 would crunch the remainder.   wadeb whipped up a tiny rle
> compressor/decompressor pair.
>
> gzipping at a file level removes the block overhead and the size drops to
> about 380k.
>
> For smallest size, lzma compressed rle files. appear to be the best.  lzma
> isn't in the default install. It is small at 90k and has potential other user
> applications.
>
>   6325  frame00.565.gz
>   2283  frame00.565.lzma
>  15236  frame00.565.rle
>   1841  frame00.565.rle.gz
>   1451  frame00.565.rle.lzma
>
>  28097  ul_warning.565.lzma
>  30594  ul_warning.565.rle.gz
>  22935  ul_warning.565.rle.lzma
>
> The final compression method is colour-of-the-bikeshed stuff,  the important
> thing here is freeing up the 2 meg.  Whether the actual saving can be1.9MB or
> 2.2MB is less important.

Thank you for the thorough quantitative data!  Nothing helps convince
me more than numbers. =)

With the attached 4-line patch to ppmto565.py, we generate
bzip2-compressed files with a total (non-jffs2) size of 98,939 bytes.
mkfs.jffs2 size is 107,176 bytes.

By changing two lines (see attached gzip.path), we generate
zlib-compressed files.  Total non-jffs size: 210,096 bytes (!).
mkfs.jffs2 size: 177,544 bytes.

I suspect zlib is available in the initramfs, but bzip is probably not
(and lzma is definitely not).  Even though it costs 70k, I think I'd
go with zlib here.

I'm about to leave for 10 days vacation; bonus points to anyone who
can take the 4 line change to ppmto565.py and turn this into a 'real'
patch for 8.2.  Source code is at:
      http://dev.laptop.org/git?p=users/cscott/act-gui;a=summary
Tasks are:

 a) patch fbutil.c and/or pyfb.pyx to properly load both compressed
and uncompressed images (since some countries have customized the boot
sequence and fielded .565 files we should continue to support:
http://wiki.laptop.org/go/Tweaking_the_boot_animation )

 b) patch boot-anim-start and/or pyfb.pyx to allow *un*loading boot
animation frames.  It doesn't matter when the frames are straight
mmaps from disk, because the kernel knows it can through those pages
out of the page cache if it needs extra memory.  When you're doing
computation on the mmapped data, then the pages are dirty and get
stuck in memory, so we probably need to free the uncompressed frames
we're no longer using.

 c) double check that this doesn't have a significant performance
impact on the XO, probably by writing a simple script which loads &
displays the images similar to pytest.py and timing the script on an
XO both before and after the changes.

For extra credit, you could insert your code into the initramfs by
rebuilding it as described at the bottom of
http://wiki.laptop.org/go/Kernel_Building, and double check that it
doesn't break (ie, that we really do have zlib in the initramfs).  If
we don't, then it's probably reasonable to leave the activation and
first frame parts of the animation in uncompressed form, and only
compress frames 01-25, which are displayed after we've left the
initramfs.

Happy hacking!
 --scott

-- 
 ( http://cscott.net/ )
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bzip.patch
Type: text/x-diff
Size: 1051 bytes
Desc: not available
URL: <http://lists.laptop.org/pipermail/devel/attachments/20080613/4dc94e37/attachment.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gzip.patch
Type: text/x-diff
Size: 1048 bytes
Desc: not available
URL: <http://lists.laptop.org/pipermail/devel/attachments/20080613/4dc94e37/attachment-0001.patch>