[Server-devel] School networks and electrical equipment damage

[Jon Nettleton]

On Thu, Jun 6, 2013 at 9:58 PM, Daniel Drake <dsd at laptop.org> wrote:
> Hi,
>
> Those of us familiar with setting up school networks (server + switch
> + APs) in some of our deployments will be familiar with  the
> occasional loss of hardware, due to surges in the low quality
> electrical supply or whatever, even when the system is protected by a
> cheap UPS which supposedly offers some protection.
>
> This has often been the case in Nicaragua, so the group is now buying
> more expensive UPSes, PoE switches, and PoE access points for new
> schools. This means that the server and switch are connected to mains
> power via a UPS which hopefully protects them, and none of the APs are
> connected directly to the mains (instead they get Power over Ethernet)
> which hopefully offers some isolation from bad electrical conditions.
>
> This equipment is expensive, especially in places like Nicaragua where
> lots of import taxes are applied. But the hope is that the investment
> pays off in that the equipment doesn't get zapped.
>
> However, one week after deploying this equipment in the first school,
> we are left with a server that doesn't boot, 3 out of 4 access points
> broken with a nice burning electronics smell, and a broken switch with
> a lot of visible damage to the electronics.
>
> And the most surprising thing - we had not even turned on the network
> yet, pending some electrical work. Everything was connected up except
> one crucial link - the UPS was not plugged into mains power. So all of
> this damage happened without any of the devices having a connection to
> the mains.
>
> Connectivity-wise, the setup was:
> WAN: Phone line - ADSL modem - XS
> LAN: XS - Switch - 4 APs
>
> And power connections: the XS, ADSL modem and switch were connected to
> the UPS. The APs were connected to the switch over ethernet for both
> power and data. Again, since the battery was not connected to mains
> power, none of the devices had a power source.
>
> The connectivity engineer's best bet is that a lightening bolt landed
> at the school or nearby, and that this caused a power surge on the
> phone line. This surge passed through the ADSL modem, server, switch,
> and 4 APs, destroying everything in its path (except 1 AP that was
> connected over a longer cable than the rest).
>
> I figured this is a story worth sharing, for any other projects
> considering splashing out on more expensive equipment...
>
> Also, I'm wondering if anyone has any advice/experience here. Would
> others expect this more expensive setup to be more resilient to bad
> electrical conditions than a cheaper setup - will the investment pay
> off?
>
> I figure that the case of a lightening bolt might be a bit extreme,
> but electrical storms are a nightly occurance here almost daily during
> the 6 month rainy season.
>
> I have seen that some UPSs (unfortunately not these ones) allow a
> phone line to be passed through them, supposedly offering some
> protection. Would such a system protect against a lightening bolt,
> assuming thats what happened here?

What is the grounding of the electrical setup there?  You may want to
invest in having a separate grounding rod installed specifically for
the circuit the network equipment is on and possibly a lighting rod.
When we built out a small data center in Alewife we actually had the
building install a lightning rod on the roof with a dedicated ground
circuit to help protect our circuits in the building.

In general I have found that more expensive network equipment handles
dirty power a bit better than the cheap stuff.  As for lightning and
other larger power surges, all of it fries about the same.  For POE
WAP's I would suggest looking at the Ubiquiti lineup.  I have a couple
of Picostation2's and a Nanostation M2 and have very impressed with
their coverage and stability for the price.  They are also
indoor/outdoor certified.

-Jon