[linux-mm-cc] Hi All

[Nitin Gupta]

Hi,

On 9/25/07, IKEDA, Munehiro <m-ikeda at ds.jp.nec.com> wrote:
>
> Hello,
>
> Nitin Gupta wrote:
> >>> We are interested in knowing the current stage of the project. Could
> you
> >> refer us to the latest documentation and/or code-base of the project?
> >> Anything else which you think we should look at ?
> >>
> >> I suspect that you can get this information from Nitin. Actually, I
> >> believe we need more tests on latest Ikeda's patches
> >> which implemented a shrink operation to CCache. You can find this mail
> >> thread here:
> >> http://lists.laptop.org/pipermail/linux-mm-cc/2007-July/000165.html
> >
> >
> > There are few reasons why I am not very much willing to extend/improve
> > existing 2.6.x patches:
>
> I agree that we shouldn't tweak existing code much further and I
> probably understand what Nittin wants to do.
> So please consider my descriptions blow as just FYI to clarify things.
>
>
> > 1) Existing approach has big metadata overhead. Ikeda's patches
> increases it
> > further (though, I think, it was necessary for his dynamic ccache
> resizing
> > work).
>
> Actually it is necessary for dynamic ccache resizing.  But it is not
> only for it, but also for preventing compressed page cache data
> (fs-backed-ccache) from beeing stored eternally in ccache.
> We can't free fs-backed-ccache without address-space-object-pointer
> (page->mapping) unless accessing.  Thus, chunk_head without mapping
> member trigers a kind of memory-leaking.  It is a BUG as long as ccache
> handles fs-backed pages.


Agree.

It is needeless to say that mapping member is unnecessary for anonymous
> pages, because it is always swapper_space.





> 2)  It compresses both page-cache and swap-cache pages. Anderson and co.
> > have shown in paper (link below) that for embedded devices only
> swap-cache
> > pages need to be compressed. This can result in significant
> simplifications
> > in existing code. Also, I think ccahing is mainly useful for embedded
> > devices only and not much for desktop/server systems.
> > 3) Its storage structure for compressed pages has some problems (large
> > metada, poor locality for compressed data chunks, etc.)
>
> I agree.



One more point: In previous design we made separate allocations for storing
metadata also (struct chunk_head and struct chunk) this further increased
space overhead because of additional slab overhead due to these two. Now,
all metadata is embedded with data itself so we are also avoiding this
additional slab overhead.

> 4) It decompresses page in context where we cannot sleep in general (like
> in
> > find_get_page() and friends). This is a serious drawback since we need
> to
> > allocate some temporary memory in decompression path also which we
> cannot
> > allow to sleep and thus has very high chances of failing - If
> decompression
> > path fails, we loose data and what not!
>
> Why cannot we sleep?  The context is basically same as swap-in, which
> may sleep the context.
> In fact, my patch can make the context sleep and works well.


I don't really remember what exactly failed when trying to sleep while
decompressing. But you can easily find that out - in cc_readpage() do:

-    comp_page = alloc_page(GFP_ATOMIC);
+    comp_page = alloc_page(GFP_KERNEL);

and you will see kernel panic/oops because you are sleeping in wrong
context. Basically, we cannot determine in find_get_page() and friends as to
whether we can sleep or not.

So, initial idea was to have some reserved page pool in case atomic
allocation fails. But then, I think it became difficult to keep refilling
that pool if too many atomic allocs fail (which is very common for ccache as
it works essentially in low mem conditions).


Well, I found 2 bugs after my last submitting.
> If, only if, you want to evaluate 2.6.21-based ccache, please download
> patches from the link
> "ccache_avoid-oom_v3.tar.bz2"
> on
> http://tree.celinuxforum.org/CelfPubWiki/JapanTechnicalJamboree16
>
> I'm glad if that helps.



I think Anderson was working on  testing your patches. I am sure to merge
your work if this patch series continues. Thanks a Lot for your great work
on improving this!



--------------
Are you interested in the new implementation which is now under design?
Probably, we can all meet up on IRC and discuss new plan?
For all interested in this, please let me know time suitable for you (with
time zone)

For me: 9:00 am (GMT +5:30) on 25/9 works.
We can have 1/2-1 hour discussion.

If things go well, we can have weekly 15 mins sessions on IRC to discuss
status and this will help us stay on track.


Cheers,
Nitin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.laptop.org/pipermail/linux-mm-cc/attachments/20070925/c7338f37/attachment.htm