[Server-devel] Squid tuning recommendations for OLPC School Server tuning...

[Adrian Chadd]

2008/9/23 Martin Langhoff <martin.langhoff at gmail.com>:

> Any way we can kludge our way around it for the time being? Does squid
> take any signal that gets it to shed its index?

It'd be pretty trivial to write a few cachemgr hooks to implement that
kind of behaviour. 'flush memory cache', 'flush disk cache entirely',
etc.

The trouble is that the index is -required- at the moment for the disk
cache. if you flush the index you flush the disk cache entirely.

>> There's no "hard limit" for squid and squid (any version) handles
>> memory allocation failures very very poorly (read: crashes.)
>
> Is it relatively sane to run it with a tight rlimit and restart it
> often? Or just monitor it and restart it?

It probably won't like that very much if you decide to also use disk caching.

>> You can limit the amount of cache_mem which limits the memory cache
>> size; you could probably modify the squid codebase to start purging
>> objects at a certain object count rather than based on the disk+memory
>> storage size. That wouldn't be difficult.
>
> Any chance of having patches that do this?

I could probably do that in a week or so once I've finished my upcoming travel.
Someone could try beating me to it..

>
>> The big problem: you won't get Squid down to 24meg of RAM with the
>> current tuning parameters. Well, I couldn't; and I'm playing around
>
> Hmmm...
>
>> with Squid on OLPC-like hardware (SBC with 500mhz geode, 256/512mb
>> RAM.) Its something which will require quite a bit of development to
>> "slim" some of the internals down to scale better with restricted
>> memory footprints. Its on my personal TODO list (as it mostly is in
>> line with a bunch of performance work I'm slowly working towards) but
>> as the bulk of that is happening in my spare time, I do not have a
>> fixed timeframe at the moment.
>
> Thanks for that -- at whatever pace, progress is progress. I'll stay
> tuned. I'm not on squid-devel, but generally interested in any news on
> this track; I'll be thankful if you CC me or rope me into relevant
> threads.

Ok.

> Is there interest within the squid dev team in moving towards a memory
> allocation model that is more tunable and/or relies more on the
> abilities of modern kernels to do memory mgmt? Or an alternative
> approach to handle scalability (both down to small devices and up to
> huge kit) more dynamically and predictably?

You'll generally find the squid dev team happy to move in whatever
directions make sense. The problem isn't direction as so much as the
coding to make it happen. Making Squid operate well in small memory
footprints turns out to be quite relevant to higher performance and
scalability; the problem is in the "doing".

I'm hoping to start work on some stuff to reduce the memory footprint
in my squid-2 branch (cacheboy) once the current round of IPv6
preparation is completed and stable. The developers working on Squid-3
are talking about similar stuff.


Adrian