[linux-mm-cc] Queries about Compcache allocator

Leonidas . leonidas137 at gmail.com
Thu Oct 22 07:39:23 EDT 2009


On Thu, Oct 22, 2009 at 4:18 AM, Nitin Gupta <ngupta at vflare.org> wrote:
> Hi,
>
> On Thu, Oct 22, 2009 at 12:21 PM, Leonidas . <leonidas137 at gmail.com> wrote:
>>
>> I am new here, stumbled across Compcache while using VMware player, got
>> fascinated by the idea, currently going through the documentation part.
>> Have following questions:
>>
>> 1. xvMalloc is a pool based allocator. Can xvMalloc be used as a
>> general purpose allocator replacing kmalloc
>> in case a kernel driver does many smaller but mostly similar sized
>> allocations? I have gone through the doc
>> which compares xvMalloc with SLOB and SLUB, I think it fits the bill in case.
>>
>
> SLOB is hopeless. Please see this:
> http://lkml.indiana.edu/hypermail/linux/kernel/0903.2/01240.html
>
> For SLUB vs xvmalloc, you should consider that xvmalloc is, in its
> current state,
> not scalable at all while SLUB is much better in this aspect.
>
> Also, xvmalloc was designed to work well under memory pressure (it always
> grows pool with single pages etc) while SLUB internally allocates
> higher order pages
> to reduce fragmentation. This constraint might not be applicable for
> your driver, so
> cannot be counted as negative point for SLUB.
>
> Lastly, if all the allocation are of (nearly) the same size then SLUB
> might be same
> as xvmalloc in terms of fragmentation since this is the case SLUB is
> designed for.
>
>
>> 2. Does xvMalloc take care of adjusting memory allocations from
>> appropriate zones depending on the memory
>>  pressure? E.g. allocate from zone high mem only if zone normal is
>> depleted etc?
>>
>
> It always requests highmem pages. So, the underlying page allocator
> might supply highmem or "normal" page depending on what's available.
>
Yes, this makes lot of sense.

>> 3. Out of curiosity, I was trying to compare Jemalloc Linux port and
>> xvMalloc user port. Jemalloc claims very good
>>  performance in most cases and guarantees very less fragmentation in
>> heavily multithreaded/multicore environment.
>>  Has it been considered? Any idea why it can/can not be used for
>> compcache purpose.
>
> Various allocators were considered for compcache (not jemalloc though) but
> none of the alternatives could satisfy all the requirements of compcache. Some
> of which are:
>  1- Mem pool must be expanded using 0-order (single) pages since we need
> to allocate under memory pressure (its a swap device). Under these conditions,
> higher order allocation are almost guaranteed to fail.
>     1.1 Many allocators result in large fragmentation when this limit
> is imposed (including SLUB).
>  2- We should be able to allocate from "highmem" -- this thing is
> applicable only
> for 32-bit systems with >1G of RAM (neither SLUB or SLOB allows this).
>
>
> I have never gone though jemalloc details but if it satisfies above requirements
> and also gives low fragmentation under these constraints then it can surely be
> used for compcache.
>
>
> Thanks,
> Nitin
>

I am not sure whether you would be able to have a look at Jemalloc performance
numbers and suitability for the project immediately. In case you have a look and
you find this as a possible candidate please let me know, I would be more than
willing to port it to kernel space.

-Leo.


More information about the linux-mm-cc mailing list