Skip to content
Register

Cooling failure at 2AM, DC temp hit 38C before we got paged

Datacenter Talk by nate_pad 6 replies 525 views
nate_pad

Member OP

#1

Hi everyone, sorry total noob here

I am trying to understand if this is normal. We have 12 racks at a DC (not sure if I can name them? Its a smaller place). Our contract says 15 minute alert SLA for environmental. Last night at 2:17 AM the CRAC unit failed (both? The tech said "control board" but I dont really understand). Temp went from 22C to 38C in about 40 minutes. We got paged at 3:47 AM. Thats 90 minutes right?

Our gear did not shut down (no thermal shutdown I guess?) but some drives are throwing SMART errors now. The DC says "within SLA" because they "detected at 15 minutes" but "escalation took longer." Is that a real thing? Sorry if dumb question.

Also what is a "shared control board" for redundant chillers? I thought redundant meant two of everything

PacketpunkUma

Member

#2

Redundant chillers sharing one control board is not redundant!!! That is like saying you have redundant servers with one motherboard!!! The word you are looking for is "single point of failure" and your DC has it!!!

90 minutes to page is not 15 minutes!!! SLA means nothing if the penalty is less than your hardware cost!!!

I would be asking for every drive replacement and migrating out!!!

RAJ

Member

4 #3

Guys this is why I dont do small dc anymore, any leads on wholesale space with real 2N cooling? Margins are thin enough without eating drive replacements

@PacketpunkUma what region you in bro? I got a guy at HostHatch but their cooling is N+1 not 2N, recieved mixed reviews

your margin is my opportunity
tifiucem

Member

🇹🇷 cheap vps hunter

#4

Bro this is classic bait and switch. "redundant" but not really redundant. Same control board means same failure domain.

I dont trust any dc under 50k sqft anymore. Too many cowboys. We moved to Vultr last year, 11 days for power feed but at least the chillers have separate brains. 0.08/kWh though.

Actually wait, @nate_pad did you get anything in writing about the "escalation" thing? That sounds like they made it up after!

çok ucuz, çok hızlı
SE

Member

#5

90 min page with a 15 min SLA? That contract worth the paper it's printed on?

Zurich1984

Member

#6

Single control board calling itself redundant is comedy

/24 for sale. No lowballs.
svendeal

Member

Deal Sniper

#7

Which DC was this

world record: 4min Arch install

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft