<div dir="ltr"><div>I would say given the severity of this issue (of making the server *unusable* with regular use), we should have atleast some fix, or to disable squid (which it is by default?) while better/practical approaches are discussed. <br><br></div><div>Also, we do have monit. Maybe there is a way to get it to monitor squid cache bloat.<br></div><div><br></div>Just my two cents :-) <br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 11, 2016 at 10:22 AM, James Cameron <span dir="ltr"><<a href="mailto:quozl@laptop.org" target="_blank">quozl@laptop.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">You don't say why a separate partition is impractical.<br>
<br>
The Squid intent from antiquity was that the filesystem be separate,<br>
which in those days was only practical and performant with separate<br>
partition or disk, with a filesystem tuned for many small files.<br>
<br>
To use a separate filesystem yet avoid separate partitions, use a<br>
filesystem on a loopback block device, backed by a large file. You<br>
will find the performance equivalent. For example;<br>
<br>
dd if=/dev/zero of=/squid-cache.ext4 bs=1M count=256<br>
mkfs.ext4 -F /squid-cache.ext4<br>
mount /squid-cache.ext4 /var/cache/squid<br>
<br>
Experimentation with parameters to get around a Squid misbehaviour is<br>
impractical; given your deployment feedback cycle, it could take<br>
years; only real-world experience is valid.<br>
<br>
Or set up a big red button "clear the cache" that is tested to work<br>
even with a full disk.<br>
<br>
Using a cron job to destroy the cache if disk space is short might be<br>
a better first option; but design and test it to not require disk<br>
space to run.<br>
<br>
On Wed, May 11, 2016 at 04:17:06AM +0000, Tim Moody wrote:<br>
> I don't think it is practical to have a separate partition and while [1]http://<br>
> <a href="http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid#" rel="noreferrer" target="_blank">wiki.squid-cache.org/SquidFaq/ConfiguringSquid#</a><br>
> What_.27.27cache_dir.27.27_size_should_I_use.3F talks about a separate<br>
<span class="">> partition, it and other docs make it clear that the entire partition should<br>
> not and need not be used.<br>
><br>
</span>> We need to experiment with [2]<a href="http://www.squid-cache.org/Doc/config/cache_dir/" rel="noreferrer" target="_blank">http://www.squid-cache.org/Doc/config/cache_dir/</a><br>
<span class="">> parameters to find something that works. If that fails then we should create<br>
> a cron job.<br>
><br>
</span>> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br>
<span class="">> Date: Wed, 11 May 2016 08:23:56 +0530<br>
> Subject: Re: [XSCE] Has anyone else seen squid swap.state bloat?<br>
> From: <a href="mailto:anishmg@umich.edu">anishmg@umich.edu</a><br>
> To: <a href="mailto:me@jvonau.ca">me@jvonau.ca</a><br>
> CC: <a href="mailto:xsce-devel@googlegroups.com">xsce-devel@googlegroups.com</a>; <a href="mailto:server-devel@lists.laptop.org">server-devel@lists.laptop.org</a><br>
><br>
> I think this is serious enough to warrant atleast a doc note to have the squid<br>
> cache in a different partition as it makes the machine completely unusable<br>
> otherwise. I don't think it is maybe the best idea to have it in library (but<br>
> rather a separate partition).<br>
><br>
> Or, as you say, a cron job - but in my case the bloat filled up the hdd in less<br>
> than a week.<br>
><br>
</span>> On Sun, May 8, 2016 at 6:11 PM, Jerry Vonau <[3]<a href="mailto:me@jvonau.ca">me@jvonau.ca</a>> wrote:<br>
<span class="">><br>
><br>
><br>
> > On May 5, 2016 at 11:20 PM Anish Mangal <[4]<a href="mailto:anishmg@umich.edu">anishmg@umich.edu</a>> wrote:<br>
> ><br>
> ><br>
> > Yesterday, an interesting issue happened. On an older XSCE install, the<br>
> > squid cache bloated over time to create a 300GB swap file and made the<br>
> > server unusable. I was wondering if anyone else may have come across this<br>
> > before?<br>
> ><br>
><br>
> Yes, squid's model assumes that the cache that lives on a partition all by<br>
> itself with no real consideration for other programs on that partition.<br>
> Think that was the original reason for /library in the first place, maybe<br>
> we should rethink where the content/databases should reside.<br>
><br>
> > Attached is the munin screenshot showing the increasing disk usage over<br>
> > time. I deleted the file and things are back to normal - for now.<br>
> ><br>
><br>
> Weekly cron job to do that unattended?<br>
><br>
> > The xsce git hash is 9a072ead3da91b84e6db896554c50bfa37e7b4e6<br>
> ><br>
> > --<br>
> > Anish<br>
><br>
> Jerry<br>
><br>
> --<br>
> Anish<br>
><br>
</span>> References:<br>
><br>
> [1] <a href="http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid#What_.27.27cache_dir.27.27_size_should_I_use.3F" rel="noreferrer" target="_blank">http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid#What_.27.27cache_dir.27.27_size_should_I_use.3F</a><br>
> [2] <a href="http://www.squid-cache.org/Doc/config/cache_dir/" rel="noreferrer" target="_blank">http://www.squid-cache.org/Doc/config/cache_dir/</a><br>
> [3] mailto:<a href="mailto:me@jvonau.ca">me@jvonau.ca</a><br>
> [4] mailto:<a href="mailto:anishmg@umich.edu">anishmg@umich.edu</a><br>
<div class="HOEnZb"><div class="h5"><br>
> _______________________________________________<br>
> Server-devel mailing list<br>
> <a href="mailto:Server-devel@lists.laptop.org">Server-devel@lists.laptop.org</a><br>
> <a href="http://lists.laptop.org/listinfo/server-devel" rel="noreferrer" target="_blank">http://lists.laptop.org/listinfo/server-devel</a><br>
<br>
<br>
--<br>
James Cameron<br>
<a href="http://quozl.netrek.org/" rel="noreferrer" target="_blank">http://quozl.netrek.org/</a><br>
_______________________________________________<br>
Server-devel mailing list<br>
<a href="mailto:Server-devel@lists.laptop.org">Server-devel@lists.laptop.org</a><br>
<a href="http://lists.laptop.org/listinfo/server-devel" rel="noreferrer" target="_blank">http://lists.laptop.org/listinfo/server-devel</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature"><div dir="ltr"><div>Anish<br></div><div><br><br></div></div></div>
</div>