In Portugal we have a saying: quem não arrisca não petisca. But with backups, you must risk the test. I use Veeam for the work, but for personal Hetzner storage box with borgmatic. The built-in `borgmatic extract` with `--dry-run` is not real test, I learned hard way. Now I restore to old laptop.
Automation question: how do you test your backups actually work?
mediaaustin said:
What language you write the script in?
Python lah, got library for everything. Boto3 for S3, paramiko for SSH, random module pick the backup. 80 lines only, run from my old Pi 4 in corner of room.
The key is destroy after test, don't leave VM running forget and waste money.
Following this thread. Currently my backup strategy is "pray and pay Hetzner monthly". Need to change.
I tried automating restore test with Terraform spin up EC2, restore, run pytest against the DB, destroy. Cost about $2 per test. But AWS billing in USD and my card charges conversion fee, so I stopped. Hetzner billing in EUR same as my other stuff, simpler.
Now I just do manual quarterly restore to local Proxmox. Dyslexic so my runbook has screenshots, can't misread a screenshot.
Has anyone tested Hetzner's own backup service vs Storage Box? I use the automated backup on my CPX instance but wondering if I should switch.
quinnbizz
Member
windsrea said:
Hetzner's own backup service vs Storage Box?
Different things. Automated backup is snapshot of disk, quick restore same datacenter. Storage Box is separate service, you push to it, can be offsite or to different provider entirely.
I use both. Snapshot for "oops deleted file", Storage Box + restic for "datacenter burned down".
...
This. Snapshot is not backup if it's in same account. Your credentials get compromised, attacker deletes snapshot too. Storage Box with different login, or better different provider, is actual backup.
¡Very clear now! I will implement the restic to Storage Box with the check weekly, and the restore test monthly to VM in my local Proxmox. Maybe in future I add the second provider but for now is big improvement.
Thank you all for the help, this community is very good.
hot air, steady hand, magic smoke
What about databases specifically? I do pg_dump before restic backup, but the dump takes 15 minutes and locks tables. Considering pgBackRest instead.
kasiaxus
Member
Super26 said:
I do pg_dump before restic backup, but the dump takes 15 minutes and locks tables.
Use `pg_dump --lock-wait-timeout` or better, run from replica if you have one. PgBackRest is excellent but more complex. For homelab size, dump + restic is fine, just schedule at quiet time.
I do MariaDB with `mariabackup --backup` then restic on the resulting directory. Much faster than SQL dump for large DB. I use https://restic.net for the backup part.
POLISH SERVERS. LOUD FANS. GOOD PRICE.