Skip to content

Benchmark threads are mostly useless without workload context

VPS Hosting by HieuTce 13 replies 1.4K views
#11

Following. I do not trust any benchmark I have not faked myself.

More seriously: I restore databases for a living. My benchmark is "how long to pg_restore from S3 to local NVMe." I post that number, the dataset size, the PostgreSQL version, and whether I used --jobs. Nobody asks for my YABS. They ask whether I hit OOM or checkpoint spikes.

Raw numbers without the story are just backup verification without testing the restore. Pretty tape, no recovery.

#12

I arrived late and read the template. Has anyone applied it to a real comparison yet? Like same workload on two providers?

#13
JohnCreek said:
Same workload on two providers

Not yet in this thread. But the pattern exists elsewhere. I run DNS zones, not web servers, so my benchmark is "how many AXFRs before named chokes" which is useless to you.

Template applied to my workload:

WORKLOAD: authoritative DNS with dynamic updates
BOTTLENECK: UDP packet rate under DDoS simulation
NUMBERS: 850k qps before kernel drops, measured with dnsperf, not YABS

Different world. Same structure.

#14

ZFS scrubs are my benchmark. Resilver time after replacing a disk. Not applicable to VPS, which is why I lurk and do not post YABS.

For VPS I would want: time to compile FreeBSD kernel with -jN where N equals vCPU count. Repeatable, real, and immediately comparable across providers if everyone posts N and wall time.

zfs send | zfs receive. repeat.

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft