[Server-devel] [XSCE] Has anyone else seen squid swap.state bloat?

[James Cameron]

You don't say why a separate partition is impractical.

The Squid intent from antiquity was that the filesystem be separate,
which in those days was only practical and performant with separate
partition or disk, with a filesystem tuned for many small files.

To use a separate filesystem yet avoid separate partitions, use a
filesystem on a loopback block device, backed by a large file.  You
will find the performance equivalent.  For example;

	dd if=/dev/zero of=/squid-cache.ext4 bs=1M count=256
	mkfs.ext4 -F /squid-cache.ext4
	mount /squid-cache.ext4 /var/cache/squid

Experimentation with parameters to get around a Squid misbehaviour is
impractical; given your deployment feedback cycle, it could take
years; only real-world experience is valid.

Or set up a big red button "clear the cache" that is tested to work
even with a full disk.

Using a cron job to destroy the cache if disk space is short might be
a better first option; but design and test it to not require disk
space to run.

On Wed, May 11, 2016 at 04:17:06AM +0000, Tim Moody wrote:
> I don't think it is practical to have a separate partition and while [1]http://
> wiki.squid-cache.org/SquidFaq/ConfiguringSquid#
> What_.27.27cache_dir.27.27_size_should_I_use.3F talks about a separate
> partition, it and other docs make  it clear that the entire partition should
> not and need not  be used.
> 
> We need to experiment with [2]http://www.squid-cache.org/Doc/config/cache_dir/
>  parameters to find something that works.  If that fails then we should create
> a cron job.
> 
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> Date: Wed, 11 May 2016 08:23:56 +0530
> Subject: Re: [XSCE] Has anyone else seen squid swap.state bloat?
> From: anishmg at umich.edu
> To: me at jvonau.ca
> CC: xsce-devel at googlegroups.com; server-devel at lists.laptop.org
> 
> I think this is serious enough to warrant atleast a doc note to have the squid
> cache in a different partition as it makes the machine completely unusable
> otherwise. I don't think it is maybe the best idea to have it in library (but
> rather a separate partition).
> 
> Or, as you say, a cron job - but in my case the bloat filled up the hdd in less
> than a week.
> 
> On Sun, May 8, 2016 at 6:11 PM, Jerry Vonau <[3]me at jvonau.ca> wrote:
> 
>    
> 
>     > On May 5, 2016 at 11:20 PM Anish Mangal <[4]anishmg at umich.edu> wrote:
>     >
>     >
>     > Yesterday, an interesting issue happened. On an older XSCE install, the
>     > squid cache bloated over time to create a 300GB swap file and made the
>     > server unusable. I was wondering if anyone else may have come across this
>     > before?
>     >
> 
>     Yes, squid's model assumes that the cache that lives on a partition all by
>     itself with no real consideration for other programs on that partition.
>     Think that was the original reason for /library in the first place, maybe
>     we should rethink where the content/databases should reside.
>    
>     > Attached is the munin screenshot showing the increasing disk usage over
>     > time. I deleted the file and things are back to normal - for now.
>     >
> 
>     Weekly cron job to do that unattended?
> 
>     > The xsce git hash is 9a072ead3da91b84e6db896554c50bfa37e7b4e6
>     >
>     > --
>     > Anish
> 
>     Jerry
> 
> --
> Anish
> 
> References:
> 
> [1] http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid#What_.27.27cache_dir.27.27_size_should_I_use.3F
> [2] http://www.squid-cache.org/Doc/config/cache_dir/
> [3] mailto:me at jvonau.ca
> [4] mailto:anishmg at umich.edu

> _______________________________________________
> Server-devel mailing list
> Server-devel at lists.laptop.org
> http://lists.laptop.org/listinfo/server-devel


-- 
James Cameron
http://quozl.netrek.org/