tallinnying
Member
OP
Night Shift
- Joined:
- Aug 2024
- Posts:
- 268
- From:
- Tallinn, EE
So its 3am, im messing with this local llm thing on my OVHcloud 2GB KVM. Thought itd be funny to let it "optimize" my server. Gave it read-only access I swear. Anyway it found some docker cleanup script in a github issue and decided /var/www was "stale layer data"
Whole static site gone. 4 years of weird blog posts. Backups were on the same box because idk why I do this
The best part is it printed this little " cleanup complete" message after. Thanks I guess
Anyone else been dumb with ai agents or am I special
https://rsync.net
builds at 3AM, sleeps at noon
garykwh
Member
Inference on a Budget
- Joined:
- Jun 2024
- Posts:
- 173
- From:
- San Jose, US
Ngl I feel this. I had a "helpful" inference container on my RackNerd dedi decide to prewarm models by downloading 70B weights into /tmp. Except /tmp was a 20G tmpfs. Took down the whole host in like 90 seconds.
Vran math is hard for humans and apparently impossible for agents
CUDA cores are my love language
hankels
Member
52 VPS and counting
- Joined:
- Jun 2024
- Posts:
- 301
- From:
- Phoenix, US
I run 52 boxes and I tried one of those ai "server maintenance" things on a €4/month Hetzner cloud VPS (https://www.hetzner.com/cloud). Figured worst case I lose... what, 128MB RAM and a ipv4 port forward?
It deleted my wireguard configs because "unused for 72 hours" 😂
Now I just spin up fresh ones. $3/TB bw makes it cheaper than my time to fix
seedbox, NAS, tape, and three offsite
Doug
Member
New Jersey
- Joined:
- Jul 2024
- Posts:
- 271
- From:
- New Jersey, US
"read-only access I swear"
grabs popcorn, checks /r/drama
bellaauc
Member
3-2-1 Believer
- Joined:
- Jun 2024
- Posts:
- 190
- From:
- Auckland, NZ
Oh honey. Did you test your restore?
3-2-1 rule still applies even when its "just an experiment":
- 3 copies of your data
- 2 different media types
- 1 offsite
I keep a cold copy on a GreenCloudVPS storage bucket that nothing automated can touch. Not even my ssh keys live on that machine. 😊
The 3am brain is not a backup strategy
3-2-1 or you're already dead
