[Server-devel] [XSCE] Has anyone else seen squid swap.state bloat?

[Anish Mangal]

I would say given the severity of this issue (of making the server
*unusable* with regular use), we should have atleast some fix, or to
disable squid (which it is by default?) while better/practical approaches
are discussed.

Also, we do have monit. Maybe there is a way to get it to monitor squid
cache bloat.

Just my two cents :-)

On Wed, May 11, 2016 at 10:22 AM, James Cameron <quozl at laptop.org> wrote:

> You don't say why a separate partition is impractical.
>
> The Squid intent from antiquity was that the filesystem be separate,
> which in those days was only practical and performant with separate
> partition or disk, with a filesystem tuned for many small files.
>
> To use a separate filesystem yet avoid separate partitions, use a
> filesystem on a loopback block device, backed by a large file.  You
> will find the performance equivalent.  For example;
>
>         dd if=/dev/zero of=/squid-cache.ext4 bs=1M count=256
>         mkfs.ext4 -F /squid-cache.ext4
>         mount /squid-cache.ext4 /var/cache/squid
>
> Experimentation with parameters to get around a Squid misbehaviour is
> impractical; given your deployment feedback cycle, it could take
> years; only real-world experience is valid.
>
> Or set up a big red button "clear the cache" that is tested to work
> even with a full disk.
>
> Using a cron job to destroy the cache if disk space is short might be
> a better first option; but design and test it to not require disk
> space to run.
>
> On Wed, May 11, 2016 at 04:17:06AM +0000, Tim Moody wrote:
> > I don't think it is practical to have a separate partition and while
> [1]http://
> > wiki.squid-cache.org/SquidFaq/ConfiguringSquid#
> > What_.27.27cache_dir.27.27_size_should_I_use.3F talks about a separate
> > partition, it and other docs make  it clear that the entire partition
> should
> > not and need not  be used.
> >
> > We need to experiment with [2]
> http://www.squid-cache.org/Doc/config/cache_dir/
> >  parameters to find something that works.  If that fails then we should
> create
> > a cron job.
> >
> >
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> > Date: Wed, 11 May 2016 08:23:56 +0530
> > Subject: Re: [XSCE] Has anyone else seen squid swap.state bloat?
> > From: anishmg at umich.edu
> > To: me at jvonau.ca
> > CC: xsce-devel at googlegroups.com; server-devel at lists.laptop.org
> >
> > I think this is serious enough to warrant atleast a doc note to have the
> squid
> > cache in a different partition as it makes the machine completely
> unusable
> > otherwise. I don't think it is maybe the best idea to have it in library
> (but
> > rather a separate partition).
> >
> > Or, as you say, a cron job - but in my case the bloat filled up the hdd
> in less
> > than a week.
> >
> > On Sun, May 8, 2016 at 6:11 PM, Jerry Vonau <[3]me at jvonau.ca> wrote:
> >
> >
> >
> >     > On May 5, 2016 at 11:20 PM Anish Mangal <[4]anishmg at umich.edu>
> wrote:
> >     >
> >     >
> >     > Yesterday, an interesting issue happened. On an older XSCE
> install, the
> >     > squid cache bloated over time to create a 300GB swap file and made
> the
> >     > server unusable. I was wondering if anyone else may have come
> across this
> >     > before?
> >     >
> >
> >     Yes, squid's model assumes that the cache that lives on a partition
> all by
> >     itself with no real consideration for other programs on that
> partition.
> >     Think that was the original reason for /library in the first place,
> maybe
> >     we should rethink where the content/databases should reside.
> >
> >     > Attached is the munin screenshot showing the increasing disk usage
> over
> >     > time. I deleted the file and things are back to normal - for now.
> >     >
> >
> >     Weekly cron job to do that unattended?
> >
> >     > The xsce git hash is 9a072ead3da91b84e6db896554c50bfa37e7b4e6
> >     >
> >     > --
> >     > Anish
> >
> >     Jerry
> >
> > --
> > Anish
> >
> > References:
> >
> > [1]
> http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid#What_.27.27cache_dir.27.27_size_should_I_use.3F
> > [2] http://www.squid-cache.org/Doc/config/cache_dir/
> > [3] mailto:me at jvonau.ca
> > [4] mailto:anishmg at umich.edu
>
> > _______________________________________________
> > Server-devel mailing list
> > Server-devel at lists.laptop.org
> > http://lists.laptop.org/listinfo/server-devel
>
>
> --
> James Cameron
> http://quozl.netrek.org/
> _______________________________________________
> Server-devel mailing list
> Server-devel at lists.laptop.org
> http://lists.laptop.org/listinfo/server-devel
>



-- 
Anish
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.laptop.org/pipermail/server-devel/attachments/20160511/b7a5a699/attachment-0001.html>