Skip to content

Using rclone to sync 5 VPS to one storage backend

VPS Hosting by readerkrakw 25 replies 3.8K views
#11

I run this exact setup at work but with 12 nodes. Mergerfs over SSHFS is a nightmare with latency, we tried. Local mergerfs on each node pooling attached Storage Box? Fine. But five mounts over five networks to one mergerfs pool? You will feel every dropped packet.

Stay with rclone cron. Add --checksum if you care about Paranoid-level verification.

POLISH SERVERS. LOUD FANS. GOOD PRICE.
#12
readerkrakw said:
€4.50 is hard to quit

This is why Contabo still exists. People complain then keep paying.

My OVHcloud VPS in Singapore syncs to Hetzner Helsinki at 800KB/s stable. Same price as Contabo after VAT, no nullroute drama.

phở at 3AM, deploy at 4
#13

Anyone using Wasabi instead of Hetzner for the backend? I know Hetzner Storage is cheap but the egress from Philippines to Falkenstein is painful. Wasabi has no egress fees at all.

Downside is no EU datacenter, but for personal logs who cares.

#14
MateiRobert said:
Wasabi has no egress fees

They have minimum 90-day retention though. Delete early and you still pay. I learned this with five providers, now I have five billing surprises.

Hetzner Storage Box in Singapore works for me, same region as my OVHcloud. No egress because same Hetzner invoice, just storage fee.

2 #15
sam_ams said:
Explicit failure logs

This is why source engine server operators used logaddress_add to central boxes. One rcon command, logs fly to one IP. No cron, no sync, just UDP fire and forget.

Different problem same instinct. Centralize everything.

#16

So mergerfs with rclone mount underneath instead of SSHFS? Mount each remote with rclone mount then pool those mounts?

That sounds like more moving parts not fewer.

#17
kasiaxus said:
Add --checksum if you care about Paranoid-level verification

Already do on the weekly full sync. Daily incrementals use --size-only, much faster for log rotation where names change but sizes are stable.

The real paranoia is testing restore. I delete /var/log/nginx on one node quarterly and verify I can rebuild it from the bucket. Found a corrupted archive once, rclone --checksum caught it on next run. -- https://nginx.org

2 #18

I run 6 Oracle Cloud free tier ARM instances and sync to Hetzner with rclone. Zero bandwidth cost on Oracle side, zero storage cost on... no wait, Hetzner bills me €3.81.

The free tier game is strong but you still pay for the destination. Mergerfs would need one of those ARM boxes to be the pool host, and they are slow. 1 OCPU does not enjoy FUSE overhead.

licensing is a suggestion
#19

Does Hetzner Storage Box support S3 protocol or only SFTP? I tried finding this last week and got confused between Storage Box and Object Storage.

If it is SFTP only, rclone is basically your only option anyway.

#20
MillerVegeta3 said:
Does Hetzner Storage Box support S3 protocol or only SFTP?

Different products. Storage Box is SFTP/WebDAV. Object Storage is S3-compatible, different pricing, different datacenters.

OP said "Hetzner storage bucket" which sounds like Object Storage. But bucket pricing is per-GB with no flat fee, so 5 VPS worth of logs could surprise you versus Storage Box flat rate.

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft