LZO support

[NoiseEHC]

Currently I am profiling/rewriting the LZO decompression code.

The speed result (from memory, not too reliable):
LZO compression: ~9.5 MB/sec
LZO compression ASM version: ~9.3 MB/sec
LZO decompression: ~25MB/sec
LZO decompression ASM version: ~40 MB/sec
Here the ASM version is the version from the original LZO library.
The test were made with a 40MB file which was created cat-ing data from 
/lib (as I remember...), it is half .so and half text, compression is to 
50-60%.

See, the ASM "optimized" version is slower than the kernel's one 
(lzo1x_compress.c). I am shooting at ~80-100 MB/sec, it would make it 
worthwhile to include in the kernel. Note that if this succeeds then 
loading will become IO limited instead of CPU limited.

Could somebody make an LZO compressed image? Unfortunately I am not 
qualified for this work.

ps:
This work stalled a little bit since it seems the people who converted 
minilzo to be compilable with the kernel messed up the code.

Erik Garrison wrote:
> On Fri, Jul 11, 2008 at 06:33:41PM +0100, Gary C Martin wrote:
>   
>> One concern I have with auto saving state before powering off is the  
>> potential corruption of journal data. How robust is the Journal if  
>> power off happens half way through an ongoing auto state save – do you  
>> loose both the new journal entry and the original entry you had  
>> resumed from (partially overwritten)?
>>
>> The worst case just now seems to be EToys, it's really great for  
>> tinkering, but creating even simple scripts seems to require a massive  
>> amount of time to save (example 5+min!! to close & save a script with  
>> five polygon space invaders with 2 frames of animation and a few lines  
>> of movement code). Add to that that EToys seems to be almost as power  
>> intensive as video recording (my impression only, I've not power  
>> tested). I find I'm the habit of plugging into mains before I try and  
>> close an EToys instance I've made more than a few modifications to.
>>
>>     
>
> Based on some very provisional testing, I believe that the extremely
> long save times we're seeing in EToys are at least partly related to our
> use of zlib compression for data nodes in the filesystem.  We could
> switch to LZO compression for a 500% speedup in compression (write), a
> 50% speedup in decompression (read), and a slight (but significant)
> decrease in compression ratio.
>
> While working with Record I noticed that we cannot write very high
> resolution video (>320x240 @ 10fps) to the filesystem because of this
> issue.  It would help to know what other applications are suffering
> because of poor fs write speed.
>
> There is an existing ticket to address this issue
> (http://dev.laptop.org/ticket/2072).  This ticket suggests that we could
> add LZO support to OFW in order to implement LZO-compressed data nodes
> in the filesystem.
>
> In this scheme OFW would require LZO support to boot a kernel which
> resides on an LZO-compressed fs.  But an alternative is to implement a
> partitioning scheme in which user data resides on a LZO-compressed
> partition and the system data resides on a zlib-compressed partition.
> This would provide us with good compression for system components
> (saving space) but would also allow users to write data much more
> quickly.  That said, implementing LZO compression across the entire
> system might improve system responsiveness by a noticeable margin, so it
> might be worthwhile.  Testing is required.
>
> Erik
> _______________________________________________
> Devel mailing list
> Devel at lists.laptop.org
> http://lists.laptop.org/listinfo/devel
>