Skip to content
Register

OVH fire destroyed my 'offsite' backup

General Discussion by walt22 9 replies 565 views
walt22

Member OP

Austin

5 #1

So I had this whole backup strategy right (or so I thought) because after the whole thing with OVHcloud last year I decided I was going to be SMART about it this time and I put my main box over at Contabo in their france location (or whatever they call it) and then I thought okay walt you need offsite backups so I spun up another instance at Contabo too but in canada (montreal I think? Or maybe it was toronto they all blur together) and I was paying for both and feeling really good about myself because hey thats two different datacenters two different countries surely thats what the youtube guys mean when they say 3-2-1 or whatever and then last week I wake up to emails saying both are down and im thinking okay maybe a routing issue (happens all the time with budget hosts lets be honest) but no turns out there was a fire at one Contabo facility and somehow the network backbone the

center807

Member

#2
walt22 said:
Two different datacenters two different countries surely thats what the youtube guys mean when they say 3-2-1

Geographically diverse does not equal provider-diverse, friend. You're running into a classic single-tenant-of-failure scenario. What you want is redundant feeds across separate ASN paths, ideally with N+1 power topologies and diverse fiber entrances. Contabo's enterprise-grade offering in the EU zone likely shares backbone aggregation points—that's standard for their price tier. Their sales team could walk you through the multi-vendor POP options if you want true resilience.

For offsite you really need a second provider, separate billing entity, separate support escalation paths. Anything else is just logical separation on shared physical plant. I've seen this exact scenario play out with clients who thought "two regions" meant "two failures." It does not. Contact sales.

Emre10

Member

#3

Bro this classic mistake I see since years your backup on same provider never safe I learned this hard way when my server down on RackNerd adn my "offsite" on RackNerd too same network both gone in one night since 2019 never again

Important lesson: provider redundancy > geographic redundancy Contabo have same backbone everywhere not separate actually just marketing

tallinnying

Member

Night Shift

#4

Anyway I did this once with KnownHost and GreenCloudVPS, kept rsync going between them at 3am for months. Idk why I do this. One night both went down because they peered through the same IX during a ddos. Learned that the fun way. 💀

Here's my current setup, dont copy it, its cursed:

#!/bin/bash
# 3am paranoia script
tar czf backup-$(date +%s).tgz /data
gpg -c - < backup.tgz | split -b 100M
For chunk in x*; do
curl -T $chunk https://cloudcone.com/dropzone
curl -T $chunk https://time4vps.com/dump
curl -T $chunk https://racknerd.com/pit
Done
Rm -f backup.tgz x*

Works until it doesnt. Idk why I do this.

builds at 3AM, sleeps at noon
nate_pad

Member

#5

How do I check if two hosts share backbone? Is there a command or something? I just got my first vps last week on Hostinger and I want to do backups right. Sorry if this is dumb.

Budi

Member

#6

Bro this is why I run the 3-2-1 rule and NEVER trust one hostign provider. I got my main on InterServer my backups on Hetzner and a third copy on a usb in my drawer. Never lost a file since 2020. Contabo did you dirty same backbone is not real redundancy. Learn from Walt here dont be the next walloftext!

polskiegsm

Member

#7

Actually not my circus, not my monkeys but I keep backups on provider I dont even like, just for this case. Double negative: I dont not trust anyone. XD

  • I use Leaseweb for main
  • RackNerd for backup

And they hate each other since some peering drama in 2022. Actually this is good for me. If one burns, other probably celebrates and stays up.

ana_mad

Member

#8

Same backbone everywhere is the real problem, I never trust one ASN

swimming upstream since 2019 🐟
pdxltd

Member

#9

Peering drama backup strategy

zoekyo

Member

Less Is More

#10

Provider hatred as uptime strategy seems fragile

rm -rf / --no-preserve-root ☯

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft