Skip to content
Register

100 days since last 'why is my server down' panic

Dedicated Servers by RAJ 28 replies 4.2K views
RAJ

Member OP

#1

Guys I just hit 100 days since my last 3am "is it down" wake up. Used to be weekly thing.

What changed: finally ditched the cheap monitoring (you know the one, recieved too many false positives) and built proper stack. Uptime Kuma on a $5 HostHatch VPS as external watcher, plus self-hosted cachet on main infra. Separate alerting paths: telegram for real stuff, email for summaries.

The peace of mind is unreal. Margins on my little reseller operation actually improved because I'm not panic-renewing their servers at 4am anymore.

Any leads on better self-hosted status page? Cachet feels clunky. Want something I can white-label for clients too.

your margin is my opportunity
factorvet

Member

Ex-Support

#2
RAJ said:
Finally ditched the cheap monitoring

Trusting your monitoring?

I feel you on the 3am panic. Spent 8 years in support. The "is it down" reflex doesn't go away, it just gets better data.

Gently correcting: your 100 days probably means your stack is actually stable, not that you fixed monitoring. The monitoring just confirmed what was already true. Most "dumb" 3am pages are DNS or CDN hiccups that self-resolve. Your external watcher helps distinguish real from phantom.

For status pages: I've heard good things about gatus but haven't run it. The real fix is separate alert tiers: P1 (call me), P2 (morning email), P3 (weekly digest). Most things are P2 or P3.

Carl3

Member

Keyboard Defender

#3

Kiddo 100 days? Thats nothing ive got 4 years on my Leaseweb dedi and I sleep like a baby

Shitf key stuck sorry not sorry

The secret is you dont check. Ever. If its down the clients tell you. Zero alerts = zero panic.

Also whitelabel status pages are a scam kiddo. Just host a plain html page "all good" and update it when someone complains!

Fritz48

Member

Datenschutz First

1 #4

To be precise:

  • The status page itself becomes a critical dependency. If it goes down, your clients cannot verify your transparency.
  • GDPR: if the status page logs visitor IPs or uses third-party resources (JavaScript, fonts), this requires legal basis documentation.
  • White-labeling often involves subdomains. As far as I know, certificate management for these multiplies your operational burden.
I have used Statping-ng and now use a minimal static site generated from my Prometheus alerts. No cookies, no external resources, no JavaScript. This is not as feature-rich as Cachet, but the compliance surface is minimal. For your 100 days milestone: congratulations. The monitoring maturity curve is, as far as I know, longer than most anticipate. You are perhaps at the

Neuland. Aber schnell.
engzur

Member

BGP Dreamer

#5

Zero alerts just means zero visibility

diegoracks

Member

#6

Engzur, friend, zero alerts means you build right. I make same with httping and grafana, separate watcher box is key. HostHatch at $5 is enough for this, actually. For status page I hear good things about white-label options but Cachet still work if you theme it heavy.

RAJ

Member OP

#7
factorvet said:
Most "dumb" 3am pages are DNS or CDN hiccups that self-resolve

This. Exactly this. Before Uptime Kuma I was using one of those "check every 5 minutes from one location" services. Every Cloudflare hiccup in Singapore became my problem. Now I have watchers in three HostHatch regions and I set consecutive failures before alert. Night and day.

Still want that status page though. Gatus looks clean, will spin it up this weekend.

your margin is my opportunity
Carl3

Member

Keyboard Defender

2 #8
engzur said:
Zero alerts just means zero visibility

Visibility is overrated kiddo. You know what clients hate more than downtime? Getting 47 "resolved" emails about a blip they never noticed.

My Leaseweb box in Amsterdam just works. Has for 4 years. I check the invoice once a month, thats the relationship I want.

engzur

Member

BGP Dreamer

#9
Carl3 said:
Zero alerts = zero panic

This is how you get blacklisted by every RBL and don't know until your bounce rate hits 40%. Or your cert expires — https://crt.sh. Or your disk fills because logrotate broke in 2021 and you never checked.

"Clients tell you" means clients already left.

diegoracks

Member

#10
RAJ said:
Three HostHatch regions

Which ones you use? I have mine in Los Angeles and Zurich, thinking to add Singapore for APAC coverage but the $5 plan only allow certain locations I think.

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft