Datastore profiling

[Erik Garrison]

On Wed, Aug 13, 2008 at 07:51:08PM +0200, Tomeu Vizoso wrote:
> On Wed, Aug 13, 2008 at 6:33 PM, Erik Garrison <erik at laptop.org> wrote:
> > On Wed, Aug 13, 2008 at 06:06:03PM +0200, Tomeu Vizoso wrote:
> >> On Wed, Aug 13, 2008 at 5:51 PM, Erik Garrison <erik at laptop.org> wrote:
> >> > On Wed, Aug 13, 2008 at 11:41:38AM +0200, Tomeu Vizoso wrote:
> >> >> In most cases, the files that activities put into the DS are moved
> >> >> instead of copied, because of the long time that takes writing to
> >> >> jffs2 due to compression. That will change the profiling results a
> >> >> lot.
> >> >
> >> > I'm working on the jffs2 compression issue.  Compression algorithm
> >> > support could greatly improve system i/o performance.
> >>
> >> That looks interesting, but from my POV, would be better for Sugar
> >> performance if compression was disabled by default, and was mostly
> >> used just by pilgrim when creating the jffs2 image file.
> >
> > In July, NoiseEHC reported results from his testing of lzo and zlib
> > compression.  To best estimate the results we should expect from
> > implementing filesystem-wide compression, he compiled the kernel zlib
> > and lzo in userspace to test their performance.  Please refer to
> > http://lists.laptop.org/pipermail/devel/2008-July/016956.html
> >
> > I do not understand why we wouldn't want to accept a >500% increase in
> > write performance and a >300% read performance boost at the cost of a
> > ~25% increase in the size of compressed data.  Please clarify your POV.
> 
> Well, if switching from zlib to lzo is so much better, I don't have
> any problem with that.
> 
> My point is that most data written by the user is already compressed:
> png, jpeg, ogg, pdf, odt, zip, etc so by automatically compressing it
> again we only slow things down a lot.
> 
> That's why I think that by making writes uncompressed by default and
> only compress when explicitly requested (for example when pilgrim
> installs the rpms) we would fit much better our users needs. Later, as
> time permits, we could teach the datastore to request compression for
> files in uncompressed formats.
> 
> Makes any sense now?

Thank you for clarifying.

Erik