Skip to content

Need 32TB+ usable storage — auction server or build my own?

Dedicated Servers by steveipw 24 replies 3.1K views
7 #1

Been running 4x 4TB RAIDZ2 on an old E3 box at home but outgrowing it. Need proper storage for media + backups + some archive data.

Option A: Auction server. Seen some Contabo auctions with 12x 3.5" bays, Xeon v3/v4 era, usually comes with LSI 9211-8I or similar. Around $200-350 before shipping. Throw in 8x 4TB or 4x 8TB SAS drives from eBay, run RAIDZ2 or RAID6. For that money you get a complete chassis with backplane, redundant PSU, IPMI.

Option B: Build whitebox + colo. Ryzen 5600 or used EPYC 3151, 64GB RAM, Fractal Design Define 7 XL or similar. 8x 8TB drives. OVHcloud has 2U colo for €80/mo with 500W included. More upfront, less long-term if I stay 2+ years.

My math:

  • Auction build: ~$800 total, $0/mo (home rack)
  • Whitebox + colo: ~$1200 total, $80/mo

Home power is $0.12/kWh and rising. 200W idle for the auction

zfs send | zfs receive. repeat.
#2
steveipw said:
200W idle
is optimistic for those auction boxes. Real world:

Dell R730xd 12LFF, 2x E5-2680v3, 128GB, 8x 8TB SAS, H730 mini

  • Idle at wall: 145W
  • Spin-up: 220W peak
  • With 4 more drives: 165W idle

Add 15W for switch, 5W for PDU overhead. You're at 180W real idle.

R720xd with v2 chips: 185W idle. Those v3/v4 "deals" everyone jumps on are worse than people think. The efficiency gain from v3 to v4 is maybe 8W at idle. Not nothing over 3 years, but not revolutionary.

For colo math, 500W included is standard at OVHcloud and InterServer. Your whitebox with 5600X + B550 + 8x spinners + NVMe boot will idle around 65W. Leaves you headroom. But check: does OVHcloud allow tower chassis or only rackmount? Some places charge $25 "non-standard form factor" fee.

Shipping pallets for auction gear: budget €150-250 cross-country. Contabo ships LTL, takes 2 weeks. I've had a pallet show up with corner crushed, b

visit twice: install and decom
#3
Carl said:
180W real idle

1. Rebuild time on 8TB mdadm RAID6 with default stripe: 18-24 hours at 150MB/s sustained. With bitmap and no load, 14 hours observed on Dell HBA in IT mode. ZFS RAIDZ2 resilver is typically 20-30% slower due to checksum verification overhead, but copy-on-write eliminates the RAID5 write hole entirely.

2. Rebuild exposure is not the only risk. During rebuild, URE probability on remaining drives scales with capacity. For 8TB SAS at 10^15 bit error rate: approximately 7% chance of a URE during full array read. RAIDZ2 tolerates one additional failure. RAID6 tolerates one additional failure. The difference is ZFS checksums detect silent corruption; mdadm does not.

3. IPMI alternatives for whitebox: ASRock Rack X570D4I-2T has BMC with HTML5 IKVM, ~$350. Supermicro H12SSL-NT has AST2600 BMC. Both support SOL. This adds cost but eliminates the "no remote console" problem.

4. For 32TB usable w

It's always DNS. Always.
#4

Did you test your restore?

I know, I know, everyone jumps to the hardware specs. But before you pick A or B, walk through this:

  • What happens when (not if) that 24-hour rebuild fails at hour 22?
  • Do you have a second copy of the irreplaceable stuff?
  • Is your "archive data" actually archived, or is it the only copy?

3-2-1 rule, even for home labs:

  • 3 copies of data you care about
  • 2 different media types
  • 1 offsite

Your 32TB primary is fine. But what's the backup target? Another box? Cold drives in a closet? Vultr B2? https://www.backblaze.com/cloud-storage

I've seen too many "I have RAIDZ2 so I'm safe" stories end with "the second drive failed during resilver and I lost 8 years of photos." RAID is not backup. ZFS is not backup. A server in a datacenter is not backup if it's your only copy.

If you go colo, consider shipping a USB enclosure with 2x 16TB drives that you swap every 6 months and keep at a friend's house

3-2-1 or you're already dead
#5

I had the same issue with my R720xd, 180W idle is real

9 #6

Bellaauc makes a fair point. I do have a second copy of the irreplaceable stuff on an external 8TB that I rsync monthly, plus family photos in Google Photos (not ideal, but it's something). The archive data is mostly replaceable Linux ISOs and old project builds.

So my 32TB primary doesn't need to survive a meteor strike. It needs to survive one drive dying while I'm asleep. RAIDZ2 covers that. The real question is whether I want to pay the power bill for 180W idle in my basement, or pay OVHcloud for colo and build something efficient — https://www.ovhcloud.com

zfs send | zfs receive. repeat.
#7

OVHcloud colo in Limburg is closer to me but they don't offer 500W included there. Only Beauharnois and Vint Hill have that deal for North America. For Europe it's €65/mo for 1A at 230V, which is ~230W. You'd need to size carefully.

I looked at this last year. Ended up with a used Supermicro 6028U at Contabo auction instead. €45/mo for the server, 2x E5-2620v3, 64GB, 8x 3.5" bays. I added my own drives. Yes it idles at 160W, but I'm not paying for colo and I'm not listening to it. Nuremberg datacenter, latency to Frankfurt is 12ms.

#8
steveipw said:
Pay the power bill for 180W idle in my basement

At Denver rates, 180W * 24 * 30 = 129.6 kWh. Xcel Energy is around $0.13/kWh, so ~$17/mo. Over 3 years that's $612 just in electricity. Plus you're eating the heat and noise.

Colo at €80 is roughly $87. So you pay $70/mo more for colo, but zero noise, zero heat, real redundant power, and you can put the box on a business line with static IP if you need.

For me the math flips at year 2 if you factor in a UPS upgrade and the fact that you're probably running AC more in summer. But it's close.

visit twice: install and decom
#9

I have a full rack in Prague at a local DC. 1A 230V included, €55/mo. Brought in a whitebox with Ryzen 5700G, 64GB, 8x 8TB shucked WD Elements in a Fractal Define 7. Idles at 58W. The remaining 172W headroom runs a MikroTik CRS326 and a thin client for remote console.

If you can find local colo, OVHcloud is not the only game. Shop around. Prague has maybe six providers at €50-80 for 1A. The trick is they don't advertise on English forums.

#10

London colo is a joke unless you're a finance firm. £150+/mo for anything with real power. I looked at moving my R710 out of the spare room and laughed all the way back to the staircase.

Ended up with a Synology DS1821+ at home, 8x 4TB, SHR-2. 45W idle, whisper quiet. Not 32TB usable but I don't need everything hot. Cold data goes to Backblaze B2 via their integration — https://www.backblaze.com/cloud-storage. £6/TB/mo. For the stuff I actually access, 16TB usable is plenty.

Different problem, different solution.

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft