garykwh

Try running your inference workloads on that budget. My 3090 pulls 350W and I still tunnel the web frontend through a $3.50 Contabo slice. The GPU does LLM, the VPS does SSL termination and SNI routing. Best of both worlds, nobody hits my home IP directly....

PaperMC with 768M heap is black magic honestly.

Ngl this is why I stick with drift for my inference boxes their gpu scheduling bug last month got patched in like two hours I was shook most providers "escalate to engineering" which means pipe it to the void but when you got llms burning vram at $$$ per hour twenty minutes is actual money saved @an...

Amd embedding microsoft pluton on next gen epyc lol | so my inference boxes are getting an xbox security chip whether I want it or not | ngl this complicates my vram math if I can't trust the firmware boundary | running llama 70b on 8x mi300x already requires every gb accounted for | now I gotta wor...

Ngl I feel this. I had a "helpful" inference container on my RackNerd dedi decide to prewarm models by downloading 70B weights into /tmp. Except /tmp was a 20G tmpfs. Took down the whole host in like 90 seconds.

Vran math is hard for humans and apparently impossible for agents

So ngl I've been running inference monitoring on a shared Hostinger plan ( I know) and need cron every 5 min to check if my LoRA endpoint is warm. Their panel limits to hourly which is useless for my use case—cold start kills my VRAM math, adds like 8 seconds to first token. Looked at Hetzner and Ra...

This is exactly the trap. I almost signed up with them last year and noticed the subscription language was buried under a "manage preferences" link that 404'd at t...

Ngl this started as a joke and became a trauma log. I needed GPU inference nodes, not even training, just LLM hosting for a side project. 24GB VRAM minimum, 48GB preferred.

Host 1: Contabo. "GPU instances coming Q2." Q2 passed. Q3 passed. Ticket response: "the team is excited." I left.

Host 2: OVH...

Ngl yall are talking about blogs like thats the interesting part

The real question is inference per dollar lol. I squeezed llama.cpp onto a 4gb arm box from GreenCloudVPS — https://greencloudvps.com — and it runs... barely. 2 tok/s on q4_0 7B. Same price x86 with avx2 gets you 4 tok/s

For ACTUAL a...

Ngl thats actually wild. Most hosts wouldve sent you a kb link and dipped. But also... running on a decommissioned node as permanent fix? Thats some vintage hardware probably. Hope youre not planning to inference anything on it. How much VRAM even in that thing, 8gb? my $3 VPS has more headro...