Hey folks
We have had some off-topic drift in similar threads, so keeping this focused: what do you actually deploy for stretching L2 between facilities?
My context: two Hetzner pops, 40km apart, need same broadcast domain for legacy VM migration. Currently testing VXLAN over existing IP transit.
Not looking for vendor pitches, looking for war stories
Member
- VXLAN
- evpn control plane
- type-2 routes
- NOTE: flooding issues at scale
---
- alternatives considered
- L2TPv3: WARNING: dead protocol
- GRE: keepalives matter
- GENEVE: NOTE: Vultr only
---
- my deploy
- 3 pops
- 200 VNIs
- no stretched clusters, just mobility
What hardware is running the VXLAN VTEPs?
Member
MikroTik CCR2116 at two pops, third is CHR on Proxmox. Not fancy but gets the job done. The CHR instance is the weak link, crashes under ARP flood if I don't rate-limit.
Looking at EdgeRouter replacements but Ubiquiti stock is fiction in EU right now.
Same here actually. FX2 in Amsterdam, FX3 in Haarlem, both CCR2004. Running ROS 7.12, EVPN since beta. Had to disable fastpath for VXLAN, CPU sits at 40% with 800 Mbps of east-west.
White label nightmare: customer saw 40% on their dashboard and demanded "optimization." Explained three times that's headroom.
How are you handling the MTU? Hetzner transit is 1500 standard, VXLAN overhead eats 50 bytes. I had to negotiate jumbo frames on the dedicated link and they pushed back hard — https://docs.hetzner.com has nothing about this.
Ended up with 1550 on the cross-connect, 9000 inside. Still seeing fragmentation on NFS.
Member
Hetzner will not do jumbo on standard transit. You need their "separate network" option which is 50 EUR/month per port minimum. We abandoned VXLAN for this reason at our Munich pop.
Went with MPLS over L3VPN to a neutral in Frankfurt. Not L2 extension technically but the application team never noticed.
Member
Detroit problems: my "datacenters" are 12km apart and I ran fiber myself through a landlord's conduit. No jumbo frame negotiations, just a crimper and prayer.
MikroTik hEX doing OTV to a Catalyst 3560. Yes, OTV. Found the license on eBay. Cisco doesn't care anymore, neither do I.
OTV? That's still breathing?
We did VXLAN/EVPN between two Vultr locations, Amsterdam and Frankfurt. Worked fine until we tried to migrate a Windows DC and the MAC flapping took down both sides. Type-2 routes propagated faster than the VM booted.
Now we stretch nothing. Shared nothing. Painful but stable.
This is why I check status pages. VXLAN without BUM control is just a fancy way to broadcast your outages.
We run EVPN with ingress replication disabled, PIM-SSM for BUM. Extra complexity but the 14 status pages stay green. Dublin to Cork, 30ms, same broadcast domain for legacy Oracle RAC. Kill me.
