Datastore profiling

[Tomeu Vizoso]

On Wed, Aug 13, 2008 at 6:33 PM, Erik Garrison <erik at laptop.org> wrote:
> On Wed, Aug 13, 2008 at 06:06:03PM +0200, Tomeu Vizoso wrote:
>> On Wed, Aug 13, 2008 at 5:51 PM, Erik Garrison <erik at laptop.org> wrote:
>> > On Wed, Aug 13, 2008 at 11:41:38AM +0200, Tomeu Vizoso wrote:
>> >> In most cases, the files that activities put into the DS are moved
>> >> instead of copied, because of the long time that takes writing to
>> >> jffs2 due to compression. That will change the profiling results a
>> >> lot.
>> >
>> > I'm working on the jffs2 compression issue.  Compression algorithm
>> > support could greatly improve system i/o performance.
>>
>> That looks interesting, but from my POV, would be better for Sugar
>> performance if compression was disabled by default, and was mostly
>> used just by pilgrim when creating the jffs2 image file.
>
> In July, NoiseEHC reported results from his testing of lzo and zlib
> compression.  To best estimate the results we should expect from
> implementing filesystem-wide compression, he compiled the kernel zlib
> and lzo in userspace to test their performance.  Please refer to
> http://lists.laptop.org/pipermail/devel/2008-July/016956.html
>
> I do not understand why we wouldn't want to accept a >500% increase in
> write performance and a >300% read performance boost at the cost of a
> ~25% increase in the size of compressed data.  Please clarify your POV.

Well, if switching from zlib to lzo is so much better, I don't have
any problem with that.

My point is that most data written by the user is already compressed:
png, jpeg, ogg, pdf, odt, zip, etc so by automatically compressing it
again we only slow things down a lot.

That's why I think that by making writes uncompressed by default and
only compress when explicitly requested (for example when pilgrim
installs the rpms) we would fit much better our users needs. Later, as
time permits, we could teach the datastore to request compression for
files in uncompressed formats.

Makes any sense now?

Regards,

Tomeu