Skip to content

ECC is non-negotiable even for your 'toy' server

Dedicated Servers by pauloserver 2 replies 121 views
10 #1

Last month I was running postgres on "cheap" non-ECC box at Contabo, going to tell what happened. Silent bit flip, corrupted whole database, lost 3 days of data. I thought I was smart saving $20/month.

dmidecode shows non-ECC:

Memory Device
Array Handle: 0x1000
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 32 GB
Form Factor: DIMM
Set: None
Locator: DIMM_A1
Bank Locator: BANK 0
Type: DDR4
Type Detail: Synchronous
Speed: 3200 MT/s
Manufacturer: Not Specified
Serial Number: Not Specified
Asset Tag: Not Specified
Part Number: Not Specified
Rank: 2
Configured Memory Speed: 3200 MT/s
Minimum Voltage: 1.2 V
Maximum Voltage: 1.2 V
Configured Voltage: 1.2 V
Memory Technology: DRAM
Memory Operating Mode Capability: Volatile memory
Firmware Version: Not Specified
Module Manufacturer ID: Bank 1, Hex 0xCE
Module Product ID: Unknown
Memory Subsystem Controller Manufacturer ID: Unknown
Memory Subsystem Controller Product ID: Unknown
Non-Volatile Size: None
Volatile Size: 32 GB
Cache Size: None
Logical Size: None

Kernel log caught it too late:

[Hardware Error] Machine check events logged
[Hardware Error] Corrected error, no action required

Except it was NOT corrected. Migrated to ECC, sleep better now. That $20 savings cost me weekend of restore from Bob level disaster.

chill infrastructure for chill people 🦫
#2

Cara I dont believe this. I dont think anyone should, bit flips are a myth we tell ourselves to justify expensive ram, expensive boards. Expensive everything, my raspberry pi cluster runs fine since 2019, I dont even have a ups, the universe doesnt care about your database. Your dmidecode doesnt show ecc either way so what are we even looking at, poetic justice would be you finding out it was a failing ssd all along

#3

"cheap" "toy" server allegedly running "production" workloads. Source: trust me.

Ive seen this movie. Allegedly capy_clouds dmidecode shows 64-bit total width which means allegedly no ECC chip. Allegedly you need 72-bit for ECC. Allegedly I still sleep fine on my "unlimited" "dedicated" OVHcloud box with non-ECC because allegedly I dont run anything that "matters".

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft