Skip to content

wireguard hub with 40 peers—CPU bottleneck I didn't expect

VPS Hosting by cleardmitri 15 replies 2K views
#11

cleardmitri said:
CPU dropped to 12% at same throughput

That is a hell of a drop. 85 to 12 means conntrack was eating 73 percentage points of CPU doing basically nothing useful for WireGuard.

I retested on my end with hashsize bumped to 65536. 147 Mbps -> 312 Mbps, CPU 61%. Better but not clean. Raw table rule took me to 940 Mbps line rate, CPU 18%.

Conntrack is a tax.

fio, iperf, geekbench. results or gtfo.
#12

SingaporeRep said:
Conntrack is a tax.

It is worse than a tax. It is a tax with quadratic growth.

I plotted nf_conntrack_count against peer count on my setup. Linear to about 80 peers, then knee at 120, then cliff. The hash collision path is real.

ronwit said:
The fix it is simple! You go into config of kernel and you make the table big!

Bigger table defers problem. Does not solve. With 200 peers I would need 1M entries, that is 256MB RAM just for conntrack, on a 2GB VPS.

Raw table or dedicated conntrack bypass module are only sane paths at scale.

indentation is not optional
2 #13

The RAM it eats! I forget to say! My server it is only 1GB and I use 180MB for conntrack after the fix! This is big!

Before the fix the server it die. After the fix the server it live but hungry (´・ω・`)

I think for malý server the raw table is best. No memory, no problem.

instant noodles, instant deploys
4 #14

ronwit said:
My server it is only 1GB

Same here. I had the same conntrack stall on a 2-core Tokyo Vultr, 1GB RAM. Not WireGuard, OpenVPN, but same kernel path. Nf_conntrack_max at default, hit wall at ~25 clients.

I did not fix with bigger table. I moved to Hetzner Singapore, same specs, half price, fresh install. Problem identical. It is not provider. It is Linux.

Now I use the raw table rule. 3 months, zero stalls.

8-char NTLM found in 4min 32sec
#15
Hana5 said:
Moved to Hetzner Singapore

Closest I can get from Tokyo.

But the point stands. This is kernel behavior, not provider.

For anyone keeping score, my 52 VPS spreadsheet says:

  • Vultr: conntrack hashsize 1024 default, can resize live
  • Hetzner: 4096 default, resize needs reboot
  • GreenCloudVPS Singapore: 8192 default, but their kernel is custom, maybe patched
  • RackNerd: 1024 default, old 5.4 kernels on some nodes, resize broken

The raw table rule is portable. The module param is not.

seedbox, NAS, tape, and three offsite
5 #16

hankels said:
RackNerd: 1024 default, old 5.4 kernels on some nodes

I have RackNerd Amsterdam, kernel 5.4.203. Can confirm resize broken. I set hashsize in sysctl, reboot, back to 1024.

The repo PetraSuper mention, I check it. It is not official RackNerd repo, it is someone on GitHub with RackNerd in name. The module .ko inside, no source, no signature. I do not load this on my server.

Raw table rule is free and safe. Why use mystery module?

çok ucuz, çok hızlı

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft