Skip to content

Geekbench vs YABS vs fio: which lie do you prefer?

VPS Hosting by PetraSuper 25 replies 1.8K views
#11
PetraSuper said:
What kernel was GreenCloudVPS running?

This is drifting. The thread is about synthetic vs. real benchmarks, not GreenCloudVPS kernel versions.

That said: if you want to know, check yourself. They don't publish it and we shouldn't guess.

Back on topic: PetraSuper, do you have a repo for this compile script? I would mirror it. Lisbon to Bratislava is fast.

~be kind or be gone~
#12
PetraSuper said:
Maybe fixed `-j2` for all?

No no no. Small server with 1 core, `-j2` okay. Big server with 8 core, `-j2` waste. Better: `-j$(nproc)` but document if throttle happen. User can see "8 core but compile like 2 core" in result.

Also need control for:

  • Ccache disable! Very important. Some user have ccache, result fake fast.
  • LLVM version. 14 vs 17 is big difference.
  • Source download method. `git clone` vs tarball. Network affect if git.

I am make docker image. Same environment everywhere. This am best.

#13
PetraSuper said:
Malý server with good YABS can compile slow

What CPU, what RAM. You never answer.

I am run inference on cheap VPS. Geekbench single-core predict token speed okay. Not perfect, okay. Memory bandwidth matter for big model. YABS disk useless for inference. fio useless. Only thing matter: can model fit in RAM, and how fast CPU feed GPU if any.

For CPU-only inference: yes, compile test maybe correlate. Because LLVM and llama.cpp both memory bound.

CUDA cores are my love language
#14
garykwh said:
Geekbench single-core predict token speed okay

No. Geekbench lie. I test. Same CPU, different motherboard, Geekbench different 15%. Same server, different day, different 8%. Temperature? Background task? Who knows.

Only compile same code, same result. Perkele.

HostHatch in Stockholm. I test. YABS disk 2 GB/s. fio 4k random 200k IOPS. Real world: untar kernel source, 40 MB/s. Why? Compression. CPU bound, not disk. Synthetic never show this.

Time4VPS same. Leaseweb same. All lie.

https://github.com/masonr/yet-another-bench-script

2 #15

My customers run WordPress. They care about TTFB, not LLVM.

I test this for migration. Same WordPress on CloudCone LA vs GreenCloudVPS Amsterdam. CloudCone: PHP slower, but closer to user in US. GreenCloudVPS: PHP faster, latency kill it for US visitor.

Application benchmark means YOUR application. For my people: import 1000 WooCommerce product. That is real. Synthetic? They ask me "is 4 core enough" and I say "test your import."

Montreal to LA, 70ms. Montreal to Amsterdam, 90ms. User feel this.

42U and still growing
#16
Hel said:
Untar kernel source, 40 MB/s. Why? Compression. CPU bound, not disk.

Wesh Hel. This is good point. But decompression am also benchmark! Real world is not single thing.

I am think: we need benchmark SUITE. Not one test. Many test:

  • LLVM compile (CPU + memory)
  • Kernel untar (CPU + disk mixed)
  • Static site build with Hugo or Jekyll (real workload for many)
  • Database import (WordPress people need)

One number never enough. French spacing still important.

RAID 1: because paranoia pays
#17
techmei said:
docker image. Same environment everywhere.

Containers do not solve trust. Docker Hub compromised before. Who builds the image? PetraSuper? Why trust?

I am not saying don't do this. I am saying: verify checksums. Reproducible builds. Nix or Guix, maybe. But even then: who audits?

The real answer: run your own compile. Don't trust spreadsheet. Don't trust docker. Don't trust me.

airgapped, encrypted, faraday'd, still worried
5 #18
marcus_qc said:
Application benchmark means YOUR application

Finally someone gets it.

For mail: I test with GlockApps, Mail-Tester, Postmark seed list. Real inbox placement. No synthetic tell you this. IP warming takes weeks. You can't benchmark warming.

But if you MUST synthetic: YABS network test to single iperf server tells you nothing about mail. fio tells you nothing about mail. Geekbench tells you nothing about mail.

I will say: for mail, check MXToolbox blacklist first — https://mxtoolbox.com/blacklists.aspx. Then worry about speed.

SPF, DKIM, DMARC — holy trinity ✉️
#19

No one talk about price. Brasil, 1 USD = 5.5 BRL. GreenCloudVPS cheap plan, 2 USD, 11 BRL. CloudCone 1.5 USD, 8.25 BRL. But CloudCone only LA. For me in São Paulo, latency bad.

I am use HostHatch. They have São Paulo? No. But they have Chile? No. Closest LA. Same problem.

Real benchmark for me: speedtest to São Paulo. Not YABS to wherever. I use speedtest-cli to Anatel server. This am my real number.

LLVM compile same everywhere. But network, my user in Brasil. Application benchmark need location.

chill infrastructure for chill people 🦫
#20
techmei said:
Ccache disable!

YES. I am forget. Thank you.

pauloserver said:
Speedtest to São Paulo

Also good. I am add location-based test. But this is hard to standardize. Speedtest server change, result change.

New idea: instead of 200 people send result, we make distributed test. You run script, it test compile AND network to three place you choose. But script must be same. Docker maybe solve, but olespete right: who trust docker?

I am think: simple bash script. You read before run. No hidden.

boot anything, anywhere, anytime

Post a reply

You need an account to reply. Log in or register to join the conversation.

Post reply Preview Save draft