[Server-devel] Ejabberd CPU/RAM Spike -> Crashes

[Martin Langhoff]

On Thu, Dec 17, 2009 at 11:35 AM, Devon Connolly <devcon at gmail.com> wrote:
> XS Version: 0.6
> 1 GB Physical Ram, 2GB Swap

Ok - the RAM is on the low side for an XS but should handle 150 ok.

> # ejabberdctl connected-users
...
I counted 12 lines in the output of connected-users. That should not
cause trouble.

> After leaving it on all night, load averages hit 30....

 - Did you also leave XOs running connected to it, or were XOs
completely disconnected?

 - Are you perhaps using an AP that does its own DHCP? One way to
check for certain is to connect an XO, and then grep /var/lib/dhcpd/
(or is it /var/spool/dhcpd/ ?) for the MAC address of the XO....

> {error_logger,{{2009,12,17},{10,0,25}},"Protocol: ~p: register error:

That crash dump is because it cannot spawn the new thread/process --
there's no hint in it of who/what is hogging them.

Seems that ejabberd is consuming all resources (network handles, RAM)
over time, even with no usage or very light usage. This is unexpected.
We did a lot of load-testing of ejabberd, with many clients
connecting, sending msgs, disconnecting over a period of time and we
never saw such resource leaks.

What we saw was memory usage growing a bit with connects/disconnects,
and a GC trimming it down periodically. Memory & cpu use was
reasonably stable over time, within that see-saw.

Is there anything else that could be odd or non-standard in your
setup? Are you in a VM? Is eth0 on the XS configured via dhcp with a
short lease? Is there anything in the network between the XOs and the
XS?

cheers,



m
-- 
 martin.langhoff at gmail.com
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff