Skip to content
Register

Compiling the Linux kernel across 4 cheap architectures

VPS Hosting by nerdbann 24 replies 2.9K views
nerdbann

Member OP

DNS Propagator

#1

Compiling linux-6.12 across 4 cheap VPS tiers. Results:

dig @localhost contabo.com TXT
"arm64=847s" "x86_64=1193s" "riscv64=2041s" "x86_vz=1567s"

ARM64 won. Did not expect this. TTL on these results: permanent until I re-run.

Traceroute to the x86 node shows 18ms extra latency at hop 4. Oversold neighbor stealing cycles. Check your resolver, then check your scheduler.

https://contabo.com/en/vps/

engzur

Member

BGP Dreamer

#2

AS64512 announces this prefix but the x86 node is clearly behind a different upstream. RFC 8320 section 4.2: resource isolation is a "SHOULD" not "MUST." Your ARM64 is on their premium AS, x86 is on budget AS. Here's the topology:

[BGP peer]
|
+----+----+
| |
[premium] [budget]
ARM64 x86_64

I have three ASNs. Trust me.

playersofia

Member

2 #3

2001:0db8:0000:0000:0000:0000:0000:0001 responds faster than legacy IP counterpart. Dual-stack is compromise. NAT is theft of address space. Your benchmark uses legacy IP to reach x86 node, of course it suffers. Address space is life, life needs room to breathe. 2001:0db8:0000:0000:0000:0000:0000:0002 for ARM64, same expanded form, same dignity.

irarose

Member

#4

Actually, the ARM64 result is explainable without invoking IPv6 or AS topology. RFC 8256 discusses CPU isolation. The x86 node is likely on a E5-2680v4 or similar Broadwell generation with noisy neighbors. Your ARM64 is probably Ampere Altra Max, which actually has dedicated cores per tenant in most configurations. The 1193s versus 847s is approximately 29% overhead from oversubscription, not architecture inferiority.

nina64

Member

#5
  • Your methodology has gaps I want to address
  • The kernel version matters enormously
    • 6.12 has new scheduler
    • 6.11 would show different patterns
    • 6.10 had the ARM64 regression
  • Your sample size is one compile per architecture
    • Should be minimum three runs
    • Discard outlier
    • Average remaining
  • Missing data I need:
    • Make -j value used
    • Ccache enabled or disabled
    • Whether you ran make clean between attempts
RA

Member

#6

Contabo moved me to another x86 node after ticket, now 892s.

svendeal

Member

Deal Sniper

#7

Which kernel config did you use

world record: 4min Arch install
SP

Member

#8

18ms at one hop doesn't explain 400s gap

sander_ams

Member

#9

Same thing happened to me on same oversold neighbor issue on x86_vz...

nina64

Member

#10
rahul_astra said:
Contabo moved me to another x86 node after ticket, now 892s.

This is the most important data point in the thread. Single node migration, 25% improvement. Your original x86_64 result was contaminated by neighbor load, not architecture.

speed14 said:
18ms at one hop doesn't explain 400s gap

Latency doesn't, but scheduler steal does. Check 

cat /proc/stat | grep steal
on the old node versus new. I bet it was double digits.

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft