Skip to content
Register

Monitoring 10+ cheap boxes without going crazy

VPS Hosting by MeritBudi 28 replies 1.4K views
MeritBudi

Member OP

3 #1

I already have 14 VPS from different Provider, can or not? Every day I check one by one with ssh, very tired lah. Then I try Uptime Kuma, very easy to setup dong. But 14 nodes become 42 monitor because I want check ping also, http also, certificate also.

Now I add Prometheus with node_exporter. The graphs don't lie, but my eyes already blind from too many dashboard. Who have better way?

My stack now:

  • Uptime Kuma for alert
  • Prometheus for metric
  • Grafana for pretty
  • Alertmanager for noise

Too many lah. Want simpler but not lose data.

kate3

Member

#2

Virtualization tax on those cheap boxes means node_exporter can spike your cgroup limits. OpenVZ especially. I've seen kernel OOM kills from monitoring overhead on oversold nodes.

Consider the virtualization tax before you deploy. On KVM you have proper /proc. On OpenVZ you are guessing.

VictoriaMetrics single-node replaces Prometheus with lower RAM footprint. Worth evaluating if your boxes are memory-constrained.

virsh list --all | wc -l: 47
uma

Member

99.99% or bust

#3

But your alert fatigue is real. I ran 50+ monitors on Uptime Kuma alone before the status page became useless. 99.97% sounds good until you realize that's still two hours monthly per box. Consolidate. One external Uptime Kuma instance doing blackbox checks. One internal Prometheus per region. Aggregate at a third layer. Your phone will thank you. I keep exactly four alert rules. Everything else is dashboard-only.

436 days. reboot is surrender.
WI

Member

#4

Uptime Kuma with 42 monitors on 14 VPS is already too much, but you also run Prometheus on each? In Turkey we have a saying: the watchman needs a watchman. I run one Uptime Kuma on Contabo, it watches everything else. One ping per minute, one HTTP per five. Certificate expiry once per day. Total 14 monitors, not 42. What are you really afraid of losing?

MeritBudi

Member OP

#5

My certificate expired last month on one box, I didn't know until user complain. That's why I add certificate check. But 42 monitor is crazy lah, I know.

haroldgsm

Member

Grumpy Old Sysadmin

#6

Certificate expiry should be one alert rule total, not per-box.

MeritBudi said:
My certificate expired last month on one box
Use a single wildcard or Let's Encrypt with proper renewal hooks. You're monitoring a process failure with infrastructure tools. Fix the process.

IPv4, IRC, and irssi — fight me
kate3

Member

#7

MeritBudi said:
I already have 14 VPS from different Provider
This is the actual problem. Different providers, different panels, different APIs. You built a Rube Goldberg machine because the underlying estate has no consistency. I would rather have 14 from one provider with an API than 14 from everywhere. At least then you could template the monitoring.

virsh list --all | wc -l: 47
hankels

Member

52 VPS and counting

#8

52 VPS here. I use Netdata on each, streaming to one parent. The parent has 16GB RAM and handles it fine. Netdata is not pretty like Grafana but I see everything in one place. The cloud version is free for my count but I self-host because paranoia. You want simple? One agent, one dashboard, done.

seedbox, NAS, tape, and three offsite
prague1983

Member

btw I use Arch

#9

Netdata is bloat. I run 

collectd
with 
rrdtool
on a 512MB Arch box — https://wiki.archlinux.org — 14 nodes, 5 years of data, 40MB RAM for the whole stack. You people with your containers and your Grafana, you forgot that monitoring worked fine in 2010.

oops: 0000 [#1] SMP
StuSpock

Member

NTP fundamentalist

#10

prague1983 said:
You people with your containers
Your clock is probably wrong on that Arch box and you don't even know. NTP is not optional for metrics. I have seen RRD files with timestamps from 2016 because someone thought 
ntpd
was "good enough" without monitoring it.

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft