Drift'ost no have Singapore DC? I check before, closest is Tokyo. My restore test from Manila take forever, network slow. Better I find something with APAC presence.
Backup provider died with my backups—testing restore was the lesson
I use BuyVM block storage slice, mount with sshfs, test restore to local Proxmox. No need another VPS running 24/7. WireGuard tunnel in, do test, tear down.
I stand by this. But also: test the backup you THINK you have, not the one you wish for. I once had borg running for a year backing up /home. System died. Turns out I needed /etc and /var/lib too. WHOOPS.
3 years of client sites... I was using Duplicati to Backblaze B2. The backup job said "success" every day. Provider died? No, Duplicati corrupted its own database, couldn't list files, restore failed. The files were THERE, just unreachable. Test your RESTORE not your backup.
This is why I rsync to two different providers then borg to a third. Paranoid? I have ASN, I route around damage. But even I only test twice a year. Should be monthly.
Ok this is actually helpful. I have Contabo in Nuremberg, maybe I get their London location for test restores? Same network roughly, different DC. 4 EUR/month I can handle.
Same lor, I got five providers but never test restore from all. Just assume they all work. Maybe my five points of failure become five points of false confidence.
Mail is the worst for this. You think your MX backups work? Try failing over and watch SPF/DKIM break because secondary IP not in DNS. Tested backup, untested routing. Different disaster, same crying.
Lisbon brother, same pain. I use Hetzner storage box for borg, test restore to local machine at home. Fiber fast enough, 100 mbit. Cost 3 EUR for 100 GB, no VPS needed.
I confess that I have been most remiss in my duties. My last test was indeed two years prior. I have now automated a weekly restore to a local Docker container, for which I beg your understanding of my previous negligence.