Skip to content

Staging site pushed to production at 2am, regrets

Web Hosting by tomhider 4 replies 266 views
#1

So its 2am im tired I was gona just sync teh files from stagin to prod for client site on GreenCloudVPS an I accidntally dump teh stagging db over prod kkkkkkk

Teh worst part? I dont even relize until mronign when client emails me "site looks diferent but loads super fast now" kkkkk

I check bellaauc would ask so yes I got 3-2-1 backups but teh last prod backup is 4 days old so I lose some orders runns words togethr somtimes

Anyone done this? How bad am I

#2

I did this last month, going to tell you, going to be honest, you are very disorganized

But I help you. I check if your host has snapshot, it is more fast than restore from backup

My client he said "that new design is better" when I rollback his site, sometimes we are lucky

I make checklist after this, I sleep before 1am, I save you

chill infrastructure for chill people 🦫
7 #3

Oh no (´・ω・`) I basically did same thing actually last year

Staging db overwrite production, basically nightmare

Actually I have checklist now, I made after my accident. Basically you need:

  • Separate database user for staging
  • Different color terminal prompt for prod
  • Pre-sync dry run script

Your client praised speed? Maybe rollback removed some heavy plugin kaomoji lucky

Did you check if staging had actually newer orders or basically empty? (´・ω・`)

instant noodles, instant deploys
#4

Ouch, been there! I sent ticket yesterday to my host about this exact scenario—they see it constantly.

Here's my pre-deploy checklist, feel free to steal:

  • SSH banner shows "PRODUCTION" in red
  • Database name confirmation prompt
  • Snapshot taken before any 2am "quick fix"
  • Staging DB has _staging suffix always

Your client liked the speed? Classic! Sometimes accidental rollbacks remove bloat. But test your restore before celebrating—4 days of orders is real money.

@tomhider did staging have recent customer data or was it sanitized? Cheers

swimming upstream since 2019 🐟
#5

Mano this is why we make a test before deploy!

I use OVHcloud for my clients and they have one-click staging lock, is very good! You push to prod only with button, no command line at 2am

But your story is amazing, client praising the mistake! Kkkkk I going to tell my team this, we need laugh today

Make checklist now, sleep later!

ping so high I wave back

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft