[linux-mm-cc] First experience of compressed cache

[John McCabe-Dansted]

On Tue, Apr 1, 2008 at 4:53 PM, Nitin Gupta <nitingupta910 at gmail.com> wrote:
>  Compressing page-cache(filesystem backed) pages, is highly desirable
>  but I don't yet have
>  a way of cleanly implementing this without intrusive kernel patching.
>  If you want an implementation for pagecache+swapcache compression, you
>  can refer patches
>  for 2.6.x at:
>
>  http://linuxcompressed.sourceforge.net/

I think squashfs also does this. A pity squashfs doesn't support
writes. However have the filesystem handle the compression means you
don't have to decompress them onto disk, so it seems a better
solution. Also, if you are going to be compressing disk pages it would
be good to get space savings as well. The only downside I can imagine
might be reduced performance for random reads.

On Tue, Apr 1, 2008 at 4:59 PM, Nitin Gupta <nitingupta910 at gmail.com> wrote:
>  Pages in compcache are no part of any LRU lists. So, they are "pinned" till
>  some other page is swapped to same swap slot.

This  presumably also means that compcache stores complete junk under
normal circumstances, i.e. compressed pages that have been deallocated
or modified. It was thinking of writing a userspace daemon that
compares /proc/swaps and /proc/compcache to determine if compcache is
holding onto junk. If so it would iteratively calloc memory and
suggest to the kernel that it be swapped out (for files this can be
done with fsync/posix_fadvise. Not too sure how to do this with
anonymous pages).

Another approach might be to use the swapwriter feature that hibernate
uses to write zeros wherever possible. Either way zeros should
compress to negligible size.

Ideally the kernel would tell compcache when to free memory, but that
is more likely to require patching the kernel.

-- 
John C. McCabe-Dansted
PhD Student
University of Western Australia