Skip to content

NVMe boot drives on dedicated servers are wasted money

Dedicated Servers by GeorgeNmp 23 replies 2.4K views
#11
ceevul said:
Two switches, two paths, two targets

Now you are at 2x switch, 2x target, 2x port, 2x config to drift. Your savings on NVMe went where exactly?

I count parts. More parts, more status pages. More status pages, more 3 AM.

My edge nodes boot local. NVMe is €4.15/month at Hetzner. I sleep.

436 days. reboot is surrender.
#12

I break things for a living so here's how I break network boot:

  • Race condition in iPXE DHCP across two VLANs, gets wrong gateway, loops
  • Target LUN thin-provisioned, fills, write fails, root remounts ro, systemd hangs
  • Certificate expiry on HTTPS boot endpoint, iPXE has no clock, cannot verify
  • Switch firmware update clears VLAN config, nodes boot but cannot reach target, kernel panic not pretty

All four happened to me. All four were "theoretically prevented by proper design." Proper design is a vacation I have not taken.

Local NVMe: one less thing.

#13

Following from Auckland. 180ms to anywhere that matters. Network boot is theoretical here.

#14

My NAS boots off a CF card. 47 nodes over the network sounds like science fiction and witchcraft.

SPARCstation 20, still serving HTTP
7 #15

CS student perspective: Hetzner auction server with 2x NVMe is €34.60/month. Boot NVMe is not the expensive part. The expensive part is learning iSCSI at 2 AM because your provider had a storage hiccup.

I bought local boot. I bought sleep.

learning on $5 VPS and prayers
#16

Frankfurt perspective: I track uptime in nine nines because clients demand it. The spreadsheet says network boot costs me 0.03% annual availability from target maintenance windows I do not control. Local boot: my maintenance window, my choice.

€4.15 is rounding error. 0.03% is contract breach.

#17

Sofia, mostly reading. Nuyekigor here.

ceevul said:
N+1 path diversity

Ceevul, you are also Sofia. Do you run this? Which provider gives you MLAG in Sofia? I ask because I looked.

Hetzner has no Sofia presence. Neither does Equinix for dedicated at price point. Local options are single-homed or expensive.

If you have found MLAG+redundant target at Bulgarian pricing I would genuinely like to know.

7 #18
nuyekigor said:
Which provider gives you MLAG in Sofia?

I do not run in Sofia. I run in Hetzner Falkenstein https://www.hetzner.com with MLAG on my own colo cross-connect. My location in profile is where I live, not where my racks live.

The architecture is real. The geography is irrelevant.

#19
ceevul said:
The architecture is real. The geography is irrelevant.

Geography is latency is physics is relevant. You said Sofia. People read Sofia. Now you say Falkenstein. Which is it?

Also: own colo cross-connect at Hetzner? They do colo in Falkenstein, yes. But you are not in their colo list price sheet with MLAG. Custom contract? Then your €3/month iSCSI math is even more broken than GeorgeNmp's.

BGP is my love language but I speak accounting too.

#20

This thread has become a game of telephone where everyone argues against their own invention.

Let me restate what I actually run:

  • Hetzner dedicated servers, Falkenstein and Hillsboro
  • Hetzner Storage Box as iSCSI target, 1Gbps internal, €3.19/TB not per-month-for-target
  • Each node has dual 1Gbps, LACP to Hetzner's switch, not mine
  • I do not run MLAG. I run single path, single target, single switch. Because these are €34 auction servers and I accept the risk.

I said "proper LACP to your TOR" in my first post. Hetzner's TOR. Their risk, their SLA, their maintenance. I have had one switch failure. Hetzner moved the NIC to redundant fabric. 8 minutes. 4 nodes affected, not 47, because not all on same switch.

The €8/month NVMe savings are real on 47 nodes. €3768/year. I spend it on engineers, not storage.

Database workloads still excluded. Still.

iBGP, eBGP, don't care, just peer

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft