Hi,<br><br><div><span class="gmail_quote">On 9/25/07, <b class="gmail_sendername">IKEDA, Munehiro</b> <<a href="mailto:m-ikeda@ds.jp.nec.com">m-ikeda@ds.jp.nec.com</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hello,<br><br>Nitin Gupta wrote:<br>>>> We are interested in knowing the current stage of the project. Could you<br>>> refer us to the latest documentation and/or code-base of the project?<br>>> Anything else which you think we should look at ?
<br>>><br>>> I suspect that you can get this information from Nitin. Actually, I<br>>> believe we need more tests on latest Ikeda's patches<br>>> which implemented a shrink operation to CCache. You can find this mail
<br>>> thread here:<br>>> <a href="http://lists.laptop.org/pipermail/linux-mm-cc/2007-July/000165.html">http://lists.laptop.org/pipermail/linux-mm-cc/2007-July/000165.html</a><br>><br>><br>> There are few reasons why I am not very much willing to extend/improve
<br>> existing 2.6.x patches:<br><br>I agree that we shouldn't tweak existing code much further and I<br>probably understand what Nittin wants to do.<br>So please consider my descriptions blow as just FYI to clarify things.
<br><br><br>> 1) Existing approach has big metadata overhead. Ikeda's patches increases it<br>> further (though, I think, it was necessary for his dynamic ccache resizing<br>> work).<br><br>Actually it is necessary for dynamic ccache resizing. But it is not
<br>only for it, but also for preventing compressed page cache data<br>(fs-backed-ccache) from beeing stored eternally in ccache.<br>We can't free fs-backed-ccache without address-space-object-pointer<br>(page->mapping) unless accessing. Thus, chunk_head without mapping
<br>member trigers a kind of memory-leaking. It is a BUG as long as ccache<br>handles fs-backed pages.</blockquote><div><br>Agree. <br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
It is needeless to say that mapping member is unnecessary for anonymous<br>pages, because it is always swapper_space.</blockquote><div><br><br> </div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
> 2) It compresses both page-cache and swap-cache pages. Anderson and co.<br>> have shown in paper (link below) that for embedded devices only swap-cache<br>> pages need to be compressed. This can result in significant simplifications
<br>> in existing code. Also, I think ccahing is mainly useful for embedded<br>> devices only and not much for desktop/server systems.<br>> 3) Its storage structure for compressed pages has some problems (large<br>
> metada, poor locality for compressed data chunks, etc.)<br><br>I agree.</blockquote><div><br><br>One more point: In previous design we made separate allocations for storing metadata also (struct chunk_head and struct chunk) this further increased space overhead because of additional slab overhead due to these two. Now, all metadata is embedded with data itself so we are also avoiding this additional slab overhead.
<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">> 4) It decompresses page in context where we cannot sleep in general (like in
<br>> find_get_page() and friends). This is a serious drawback since we need to<br>> allocate some temporary memory in decompression path also which we cannot<br>> allow to sleep and thus has very high chances of failing - If decompression
<br>> path fails, we loose data and what not!<br><br>Why cannot we sleep? The context is basically same as swap-in, which<br>may sleep the context.<br>In fact, my patch can make the context sleep and works well.</blockquote>
<div><br>I don't really remember what exactly failed when trying to sleep while decompressing. But you can easily find that out - in cc_readpage() do:<br><br>- comp_page = alloc_page(GFP_ATOMIC);<br>+ comp_page = alloc_page(GFP_KERNEL);
<br><br>and you will see kernel panic/oops because you are sleeping in wrong context. Basically, we cannot determine in find_get_page() and friends as to whether we can sleep or not.<br><br>So, initial idea was to have some reserved page pool in case atomic allocation fails. But then, I think it became difficult to keep refilling that pool if too many atomic allocs fail (which is very common for ccache as it works essentially in low mem conditions).
<br> </div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Well, I found 2 bugs after my last submitting.<br>If, only if, you want to evaluate
2.6.21-based ccache, please download<br>patches from the link<br>"ccache_avoid-oom_v3.tar.bz2"<br>on<br><a href="http://tree.celinuxforum.org/CelfPubWiki/JapanTechnicalJamboree16">http://tree.celinuxforum.org/CelfPubWiki/JapanTechnicalJamboree16
</a><br><br>I'm glad if that helps.</blockquote><div><br><br>I think Anderson was working on testing your patches. I am sure to merge your work if this patch series continues. Thanks a Lot for your great work on improving this!
<br><br><br><br>--------------<br>Are you interested in the new implementation which is now under design? Probably, we can all meet up on IRC and discuss new plan?<br>For all interested in this, please let me know time suitable for you (with time zone)
<br><br>For me: 9:00 am (GMT +5:30) on 25/9 works.<br></div></div>We can have 1/2-1 hour discussion.<br><br>If things go well, we can have weekly 15 mins sessions on IRC to discuss status and this will help us stay on track.
<br><br><br>Cheers,<br>Nitin<br><br>