The backup restore test that saved my sanity

Wesh les gars, incha'allah this helps someone else

Le server حلو until it is not. I run weekly restore tests because paranoia. Last tuesday I picked a random incremental from Vultr and—

ERROR: parent snapshot 0x4a7f not found in chain

The whole incremental chain was toast. Every backup since august 28. I would have found this during a real disaster. I would have cried.

Here is what I do now:

• Full restore test monthly, not quarterly
• Random spot-check of incrementals weekly
• Verify chain integrity with `zfs list -t snapshot` before trusting anything
• Keep two vendors: Vultr primary, Hetzner secondary

The test that "wastes" thirty minutes just saved me six hours of rebuild. Maybe my job.

Anyone else finding this ?

This is a classic missing-parent scenario in copy-on-write filesystems. Your provider's snapshot pruning logic likely has a race condition with replication scheduling.

I have seen similar behavior with providers running ZFS send over congested transit. The BGP communities they use for traffic engineering—specifically 2914:420 for HostHatch, or 1299:50ff for local-pref manipulation on RackNerd—can introduce enough jitter that an incremental lands before its parent finishes garbage collection.

RPKI validation would not help you here, but IRR-maintained route objects would at least let you predict which path your replication traffic takes. Worth checking if your secondary at Hetzner uses a different upstream; diversity in AS-path length is not diversity in AS-path topology.

Run `zfs hold` on your parent snapshots until the incremental confirms. This is not a backup strategy, it is a dependency graph with a missing edge.