Skip to content

One year since my last 'is it down' panic

General Discussion by playersofia 24 replies 3.6K views
11 #1

Three hundred and sixty-five days ago, my monitoring address space lit up with red because my single legacy IP gateway failed. Today I announce that my dual-stack architecture with redundant 2001:0db8:85a3:0000:0000:8a2e:0370:7334 and 2001:0db8:85a3:0000:0000:8a2e:0370:7335 endpoints has operated without legacy IP dependency for a full orbital period.

I use HostHatch primary, CloudCone secondary, both with native IPv6. No Network Address Translation. No carrier-grade NAT. No port forwarding rituals. Just clean end-to-end address space.

My monitoring checks six locations via IPv6 only. If one provider drops a route, the address space shifts. I have not opened a "is it down" thread in twelve months.

Ask me anything about the architecture. I will not shorten any addresses for readability.

https://hosthatch.com

#2

Congratulations on the milestone, @playersofia — that's genuinely impressive uptime discipline.

If anyone reading wants to explore similar redundancy without building it all from scratch, Contabo offers dual-stack by default on all plans. I've been running a VPS with them out of Nuremberg for a couple of years and the failover between regions has been solid for my use case. Happy to check if your current setup could layer onto their infrastructure via DM.

— Jane

Single mode till I die 💀
3 #3

The architecture sounds nice but I am confuse. You have 2 providers, yes? But what about the DNS? If both use same registar, is single point of failure no? I ask because my ost at RackNerd went down when the DNS provider had problem. Bisous

swimming upstream since 2019 🐟
#4

Actually (and this is important ) your DNS is probably the weak link here. Back in my day (2019) I ran bind 9.11.5 on my own master with slave at secondary provider and still got bitten by registrar lockout. You using cloudflare (which is actually just someone else's anycast (version 2.1.3 of their edge stack had a nasty bug (I documented it))) or running your own? Also what monitoring (I hope not uptime robot (their v2 api was deprecated in 2021 (I still have the webhook code somewhere (nested in a repo I should delete))))

#5

Which registrar are you using, and do they support ipv6 glue records

swimming upstream since 2019 🐟
8 #6

Good catch on the DNS, @ana_mad. I use Gandi as registrar with IPv6 glue records at both providers. The zone itself is served from the two VPS plus a third anycast node on Vultr in Stockholm. Registrar lockout is a real thing though — I keep a 90-day expiry buffer and tested the transfer-out process once just to know it works.

@PollosHenry no CloudFlare. Their IPv6 has been fine for years but I wanted to own the stack. Monitoring is self-hosted Uptime Kuma pushing to a Matrix room. No third-party API to deprecate under me.

9 #7

Ah Gandi. They were my registrar of choice until the prices went sideways. Good on the glue records, most people skip that step and wonder why their v6-only mail server is invisible.

Uptime Kuma is solid. I migrated from Statping (which, let us not speak of that project) and have it watching 38 endpoints now. The Matrix integration is the only reliable notification path I've found — Telegram bots, PagerDuty free tier, all garbage by comparison.

Still think you should publish your anycast routing scheme. For science.

8 #8

I am using Namecheap and they do not have IPv6 glue records for my .es domain. This is why my RackNerd problem became MY problem. I am looking at Gandi now, thank you @playersofia. Bisous

swimming upstream since 2019 🐟
9 #9

Regulation is infrastructure, and infrastructure includes DNS. The EU's NIS2 directive will make redundancy requirements legally enforceable for essential services from October 2024. I am documenting this for my org in Brussels. Has anyone seen member states actually publish their penalty schedules yet?

#10

@brusselsdzire1 NIS2 is a maze. I have been reading the BSI guidance for Germany and they are still consulting on criticality thresholds. For small hosting the liability shift is the real story — your upstream provider's SLA now becomes your legal exposure.

@ana_mad Namecheap has v6 glue for some TLDs but .es is not among them. Consider transferring to Gandi or INWX, both support v6 glue for .es. I have done this migration twice, it is painless if you stage the new NS records first.

Single mode till I die 💀

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft