Benchmarking a Dedicated Server: fio, sysbench, iperf3

Benchmarking a dedicated server: fio, sysbench, iperf3

Measure disks with fio using direct I/O and a profile close to your real workload, CPU and memory with sysbench, and the uplink with iperf3 against a node in the same country. Run the tests before you migrate the project, on an empty server, and keep the numbers as a baseline: without one, any future complaint that "the server is slow" has no answer.

fio without --direct=1 measures the kernel page cache and prints fantastic numbers unrelated to the disk.
The test file must be at least twice the size of RAM, otherwise the data settles in cache.
Random 4k writes are the single figure that predicts database behaviour.

Preparation: what to do before the first test

Confirm that the array is not rebuilding, the drives are not overheating and nothing else is loading the server. All of that distorts results far more than the difference between drive models.

# Debian/Ubuntu
apt-get install -y fio sysbench iperf3 ioping hdparm
# RHEL, AlmaLinux, Rocky
dnf install -y epel-release
dnf install -y fio sysbench iperf3 ioping hdparm

# array is neither rebuilding nor degraded
cat /proc/mdstat

Disks: fio

Four profiles cover almost every workload. Random 4k reads and writes model a database; sequential profiles model backups and video delivery.

# random read 4k, queue depth 32
fio --name=randread --filename=/mnt/test/fio.tmp --size=32G \
    --rw=randread --bs=4k --iodepth=32 --numjobs=4 --direct=1 \
    --ioengine=libaio --runtime=120 --time_based --group_reporting

# random write 4k, the key number for a database
fio --name=randwrite --filename=/mnt/test/fio.tmp --size=32G \
    --rw=randwrite --bs=4k --iodepth=32 --numjobs=4 --direct=1 \
    --ioengine=libaio --runtime=120 --time_based --group_reporting

# mixed 70/30 workload with a latency report
fio --name=mixed --filename=/mnt/test/fio.tmp --size=32G \
    --rw=randrw --rwmixread=70 --bs=4k --iodepth=16 --numjobs=4 \
    --direct=1 --ioengine=libaio --runtime=180 --time_based \
    --group_reporting --percentile_list=50:95:99:99.9 2>&1 | tee fio-mixed.log

Look at IOPS and latency percentiles rather than average throughput. An average latency of 0.3 ms with a 99th percentile of 40 ms means regular stalls that users notice and the average hides. For a quick idle-latency check use ioping -c 20 /mnt/test.

CPU and memory: sysbench

sysbench measures integer computation and memory bandwidth. On multi-core Xeon parts run as many threads as nproc reports logical cores.

# single thread: per-core performance comparison
sysbench cpu --cpu-max-prime=20000 --threads=1 --time=60 run

# all cores
sysbench cpu --cpu-max-prime=20000 --threads=$(nproc) --time=60 run

# memory write bandwidth
sysbench memory --memory-block-size=1M --memory-total-size=100G \
    --memory-oper=write --threads=$(nproc) run

Uplink: iperf3

Test the uplink against a public iperf3 server in the same country as the site. A single stream almost never fills a gigabit, so run eight or more parallel connections.

# inbound: 8 streams, 30 seconds
iperf3 -c SERVER -P 8 -t 30

# reverse direction, server sending to you
iperf3 -c SERVER -P 8 -t 30 -R

Compare the result with the plan specification: on ZevsHost dedicated servers all German and French configurations run at an unmetered 1 Gbit/s, Dedicated Start US has a gigabit port with a 30 TB quota, and Pro US and Enterprise US run at an unmetered 1 Gbit/s. What unlimited and the 30 TB cap actually mean is explained in the guide on server uplink and traffic.

Which tool measures what

Subsystem	Tool	Key metric	When the number is bad
Disk, random load	fio randread/randwrite 4k	IOPS and 99th percentile latency	NVMe below 50,000 read IOPS
CPU, single thread	sysbench cpu --threads=1	events per second	more than 15% off the baseline
CPU, all cores	sysbench cpu --threads=nproc	events per second	scaling below 60% of the core count
Uplink	iperf3 -P 8	Gbit/s both ways	less than 80% of the stated bandwidth

The --filename option in fio also accepts a block device. Passing --filename=/dev/sdb with a write profile wipes the partitions and data on that disk without asking, and on a mirror the change propagates to both drives instantly. Always test through a file inside a mounted filesystem: mkdir -p /mnt/test and --filename=/mnt/test/fio.tmp. The second source of garbage numbers is testing on top of a running rebuild or live production: before you start, make sure cat /proc/mdstat has no recovery line and vmstat 1 5 shows id near 100. How a rebuild works is described in the guide on replacing a disk in an array.

How to read the results

Read the numbers against a baseline from the same machine.

Random writes collapsing while reads stay normal usually means the controller cache dropped to write-through, or the SSD exhausted its SLC buffer.
Single-thread CPU below baseline while multi-thread looks fine means power saving is on or the firmware caps the frequency.

The most convenient moment to capture these numbers is during dedicated server acceptance, stored in the same file as the hardware inventory. Digging into bottlenecks per subsystem continues in the guide on Linux performance profiling.

Key takeaways

Always run fio with --direct=1 and a file twice the size of RAM.
The disk metric that matters for a database is random 4k write IOPS plus the 99th percentile latency.
A benchmark without a stored baseline is pointless: capture the reference values on an empty server.

Frequently asked questions

Is it necessary to check all three parts — disk, CPU, and network — or does one benchmark cover it?

There's no point stopping at a single measurement: disk, CPU, and network are independent subsystems, and the bottleneck can be in any one of them. A site can slow down from a lack of disk IOPS, a weak single CPU thread, or a link that falls short of its stated bandwidth, so the baseline covers all three at once.

Is it fine to run fio and sysbench while the server is handling real traffic?

Better not to: outside load skews the numbers more than any difference between hardware models, and the result stops being comparable to the baseline you rely on later. Before testing, confirm the array is not rebuilding, the drives are not overheating, and the CPU sits idle, otherwise fio and sysbench measure a fight over resources rather than the server's capability.

What should I do if iperf3 shows throughput noticeably below what the plan promises?

First confirm the test runs across several parallel streams and against an iperf3 server in the same country as the site itself — a single stream almost never fills the whole link. If the number still lags well behind the plan after that, it is worth contacting support with the test log instead of repeating the single-stream measurement again.

How often should the benchmark be repeated after the server is accepted?

There is no fixed rule, but repeating the measurement once the project has moved in under real load makes sense, and again whenever someone reports slowness, comparing the new numbers against the baseline from the empty server. Without that saved original result the comparison is impossible, so the baseline captured at the very start is what makes later tests meaningful.