RAJ
Member
OP
- Joined:
- Jul 2024
- Posts:
- 294
- From:
- Mumbai, IN
Guys, curious about your horror stories. Any leads on how the small operators here find out first? Is it the RIR alerts, your NMS, or some random email from a dude in another continent?
I ask because I recieved a notice 6 hours late last month and it already killed my margins for that quarter
Personally my worst was a customer calling to say their site showed a Vultr parking page. Turned out someone leaked my /24 to a friend at a party. No joke
your margin is my opportunity
uma
Member
99.99% or bust
- Joined:
- Jun 2024
- Posts:
- 324
- From:
- Dublin, IE
@pablowild the graphs do not lie but apparently the grandmothers see things before they do
My worst: 99.97% uptime on the status page, green across the board, alert fatigue had me ignoring PagerDuty for 3 hours because "it always recovers"
The actual hijack was visible on every collector. The graphs do not lie. I was just too tired to look. 6 hours of Russian pharma SEO on a /22. Status page showed green because my probes were hitting the hijacker's identical-looking landing page.
Now I check path changes independently. The graphs do not lie but you have to actually graph the right thing.
436 days. reboot is surrender.
rustyrack_grag
Member
- Joined:
- Jun 2024
- Posts:
- 68
- From:
- Tallinn, Estonia
My mom called asking why her knitting blog was in Russian
This is the winner. No contest.
- Human detection beats automated systems
- No TOS violation on the hijacker's part for the content itself
- The notice period was effectively zero
- Your own family became the monitoring layer
I have documented three similar cases. In each instance, the operator had:
- Redundant alerting that was ignored
- A false sense of collector coverage
- No out-of-band page content verification
RR
if it ain't broke, rust it