From NFO
Re: Chicago issues
Jan 14 2024 11:48:10 AM PT The issues in Chicago are actually due to components in our routers overheating, due to a major datacenter outage with their cooling system. We are unable to stop the packet loss for now as a result. We are actively monitoring for updates from the facility on a fix.
Upstream link issue
FYI. Might see some network P/L today..
- captainsnarf
- Posts: 2714
- Joined: Tue Jul 06, 2021 1:51 pm
- Location: Washington
- Server Sponsor: Yes
- Server Admin: Yes
- Contact:
Re: FYI. Might see some network P/L today..
HOC, Landords, and Omni are currently down. SoL is still up, come play
- pooty
- Posts: 4537
- Joined: Sat Apr 03, 2021 10:22 am
- Location: Michigan
- Server Sponsor: Yes
- Server Admin: Yes
Re: FYI. Might see some network P/L today..
Omni is up... I was just loading a new map, so its up and running, but maybe the network is playing games.
- McLovin
- Posts: 1194
- Joined: Sat Apr 03, 2021 12:54 pm
- Location: Salt Lake City, Utah
- Server Sponsor: Yes
- Server Admin: Yes
Re: FYI. Might see some network P/L today..
1800 MST - Unplayable
Re: FYI. Might see some network P/L today..
routers overheating ?? call LEON the overheating expert
Re: FYI. Might see some network P/L today..
Put it outdoor. It's - 4 in Chicago right now.
- California
- Posts: 17
- Joined: Mon Jul 10, 2023 4:36 am
- Location: Youngstown,Ohio
Re: FYI. Might see some network P/L today..
During this time of crisis we should come together as a community and watch Videodrome and perhaps play Overwatch. Stay safe out there kings.
Nah bro I’m sweet.
- pooty
- Posts: 4537
- Joined: Sat Apr 03, 2021 10:22 am
- Location: Michigan
- Server Sponsor: Yes
- Server Admin: Yes
Re: FYI. Might see some network P/L today..
Lolz!routers overheating ?? call LEON the overheating expert
I could connect but won't stay connected.
From NFO
Code: Select all
The issues in Chicago are actually due to components in our routers overheating, due to a major datacenter outage with their cooling system. We are unable to stop the packet loss for now as a result. We are actively monitoring for updates from the facility on a fix.
Update @ 3:48pm CST: The facility (Equinix) reports that they are continuing to try to recover from multiple failed chillers, which went down due to the low temperatures. Currently, temperatures are still rising in the facility, wreaking havoc on our equipment, including our routers. Our last look at our main router showed an internal case temperature of around 160 degrees F.
We have tried workarounds, but both of our redundant routers are seeing serious problems, including failed power supplies, due to the heat issue, and all of our upstream connections are intermittently offline because the network cards and cables are so hot. Only when they have brought temperatures down again will we be able to assess what is permanently broken and adapt to it.
Update @ 6:31pm CST: The facility still has not brought down temperatures, though they are trying with fans. Our primary router seems to have entirely failed, likely because it overheated; we are transitioning to exclusive use of our secondary router, but do not know how long it will hold out. Its upstream links are also still frequently going down.
Update @ 6:53pm CST: As the facility continues to have temperature rises, we are seeing switches and machines fail. We will have to assess these when the cooling is restored. We hope that there are no permanent hardware failures.
No shit, just open a window its so cold out (and dry) it'd cool that right down. See Leon has it handled!Put it outdoor. It's - 4 in Chicago right now.