Skip to content

Why does my latency to same city vary by 40ms

Networking by kat_fold 19 replies 2K views
#11
ALPHA16 said:
Which router generation was ISP A running

Juniper MX204 on their side, I could tell from the TTL signatures and ICMP quotes. ISP B was harder to fingerprint—looked like a whitebox running FRR, which tracks for a budget reseller.

The MX204 is underrated for mid-density 10G. Runs warm though.

visit twice: install and decom
3 #12
tallinnying said:
I will gist it instead. DM me.

DM sent. Also: 4 months of path data is beautiful. Do you see any correlation with BGP hijack alerts or just normal congestion engineering?

I have been logging RIPE RIS data alongside my own probes and the mismatch is maddening. What they announce versus what actually hits my NIC.

436 days. reboot is surrender.
#13
mikemarian said:
$2/month GreenCloudVPS? NAT means no inbound traceroute

Outbound only, yeah. But for my use case—measuring how my traffic gets routed to them—it works fine. I do not need inbound. The probe initiates, records, ships logs home via wireguard.

If you need full bidirectional, GreenCloud does offer dedicated IPv4 for like $3 more. Still cheap. Their Amsterdam location has been solid for me, though I am pinging from Portland so grain of salt.

my cloud. my rules. my 3AM alerts.
#14
SamAlvi said:
Their Amsterdam location has been solid for me

I looked at GreenCloud once but their UK peering is thin. My smokeping to their Amsterdam from Bristol hits LINX via Telia, which is fine, but their London route goes through Paris for some reason. 18ms versus 4ms.

Ended up on a small UK provider instead. More expensive, actually local.

works on my bench ¯\_(ツ)_/¯
#15
kat_fold said:
Same metro area, maybe 15 miles apart

This is where "same city" gets legally fuzzy. In Canadian telecom regulation, a "local calling area" can span 50km and multiple municipalities. Network topology definitions in provider agreements rarely reference geography at all—they reference POPs, wire centers, and exchange boundaries.

If your contract does not specify a maximum latency to a specific IP or facility, you have no contractual remedy for suboptimal routing. The "same city" framing is marketing, not engineering.

5 #16
torontofields said:
The "same city" framing is marketing, not engineering.

This. I fell for it hard. Both ISPs had "Lisbon" in the datacenter name. One is actually in Amadora, which is fine—still metro. The other routes through Madrid because their "Lisbon" POP is just a switch in a shared facility with no upstream there.

I should have checked the facility codes. Equinix LD1 versus LD2 matters.

~be kind or be gone~
#17

Speaking of facility codes, I just went through this with a client in Portland who wanted "local" backup to a "Portland" datacenter. Turns out the provider's Portland facility is in Hillsboro, which is fine, but their replication target was their Hetzner Ashburn node. 70ms away. Client assumed "same city" meant same latency zone.

I now include a mandatory traceroute clause in my SOWs. Saved me twice this year.

declarative, reproducible, indescribably good
#18

I automated the traceroute clause. Ansible playbook that runs from my Oslo office, tests to every client endpoint, diffs weekly. Alerts if path length increases by more than 2 hops or 10ms.

- name: verify network paths
  shell: "traceroute -n {{ item }} | tail -1"
  loop: "{{ client_endpoints }}"
  register: paths

Mostly catches ISP changes they do not announce. Had one where Telia sold a last-mile segment to a regional player. Latency jumped 15ms. Client never noticed. I did.

push. done. coffee.
#19
SamAlvi said:
GreenCloud does offer dedicated IPv4 for like $3 more

Still no inbound traceroute on the dedicated though. Their filtering drops ICMP time-exceeded from outside. You can ping, you cannot trace back.

Verified this myself last month. Support said it was "DDoS mitigation." Fine for hosting, useless for being a probe target.

Stuck with my BuyVM slice for that.

7 #20

What was the python thing called again? I scrolled up and did not see a link.

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft