Benchmarking a dedicated server: fio, sysbench, iperf3

Measure disks with fio using direct I/O and a profile close to your real workload, CPU and memory with sysbench, and the uplink with iperf3 against a node in the same country. Run the tests before you migrate the project, on an empty server, and keep the numbers as a baseline: without one, any future complaint that "the server is slow" has no answer.

fio without --direct=1 measures the kernel page cache and prints fantastic numbers unrelated to the disk.
The test file must be at least twice the size of RAM, otherwise the data settles in cache.
Random 4k writes are the single figure that predicts database behaviour.

Preparation: what to do before the first test

Confirm that the array is not rebuilding, the drives are not overheating and nothing else is loading the server. All of that distorts results far more than the difference between drive models.

# Debian/Ubuntu
apt-get install -y fio sysbench iperf3 ioping hdparm
# RHEL, AlmaLinux, Rocky
dnf install -y epel-release
dnf install -y fio sysbench iperf3 ioping hdparm

# array is neither rebuilding nor degraded
cat /proc/mdstat

Disks: fio

Four profiles cover almost every workload. Random 4k reads and writes model a database; sequential profiles model backups and video delivery.

# random read 4k, queue depth 32
fio --name=randread --filename=/mnt/test/fio.tmp --size=32G \
    --rw=randread --bs=4k --iodepth=32 --numjobs=4 --direct=1 \
    --ioengine=libaio --runtime=120 --time_based --group_reporting

# random write 4k, the key number for a database
fio --name=randwrite --filename=/mnt/test/fio.tmp --size=32G \
    --rw=randwrite --bs=4k --iodepth=32 --numjobs=4 --direct=1 \
    --ioengine=libaio --runtime=120 --time_based --group_reporting

# mixed 70/30 workload with a latency report
fio --name=mixed --filename=/mnt/test/fio.tmp --size=32G \
    --rw=randrw --rwmixread=70 --bs=4k --iodepth=16 --numjobs=4 \
    --direct=1 --ioengine=libaio --runtime=180 --time_based \
    --group_reporting --percentile_list=50:95:99:99.9 2>&1 | tee fio-mixed.log

Look at IOPS and latency percentiles rather than average throughput. An average latency of 0.3 ms with a 99th percentile of 40 ms means regular stalls that users notice and the average hides. For a quick idle-latency check use ioping -c 20 /mnt/test.

CPU and memory: sysbench

sysbench measures integer computation and memory bandwidth. On multi-core Xeon parts run as many threads as nproc reports logical cores.

# single thread: per-core performance comparison
sysbench cpu --cpu-max-prime=20000 --threads=1 --time=60 run

# all cores
sysbench cpu --cpu-max-prime=20000 --threads=$(nproc) --time=60 run

# memory write bandwidth
sysbench memory --memory-block-size=1M --memory-total-size=100G \
    --memory-oper=write --threads=$(nproc) run

Uplink: iperf3

Test the uplink against a public iperf3 server in the same country as the site. A single stream almost never fills a gigabit, so run eight or more parallel connections.

# inbound: 8 streams, 30 seconds
iperf3 -c SERVER -P 8 -t 30

# reverse direction, server sending to you
iperf3 -c SERVER -P 8 -t 30 -R

Compare the result with the plan specification: on ZevsHost dedicated servers the base configurations in Germany run at 100 Mbit/s, in France at 500 Mbit/s, while Pro and Enterprise configurations run at 1 Gbit/s. What unlimited and the 30 TB cap actually mean is explained in the guide on server uplink and traffic.

Which tool measures what

Subsystem	Tool	Key metric	When the number is bad
Disk, random load	fio randread/randwrite 4k	IOPS and 99th percentile latency	NVMe below 50,000 read IOPS
CPU, single thread	sysbench cpu --threads=1	events per second	more than 15% off the baseline
CPU, all cores	sysbench cpu --threads=nproc	events per second	scaling below 60% of the core count
Uplink	iperf3 -P 8	Gbit/s both ways	less than 80% of the stated bandwidth

The --filename option in fio also accepts a block device. Passing --filename=/dev/sdb with a write profile wipes the partitions and data on that disk without asking, and on a mirror the change propagates to both drives instantly. Always test through a file inside a mounted filesystem: mkdir -p /mnt/test and --filename=/mnt/test/fio.tmp. The second source of garbage numbers is testing on top of a running rebuild or live production: before you start, make sure cat /proc/mdstat has no recovery line and vmstat 1 5 shows id near 100. How a rebuild works is described in the guide on replacing a disk in an array.

How to read the results

Read the numbers against a baseline from the same machine.

Random writes collapsing while reads stay normal usually means the controller cache dropped to write-through, or the SSD exhausted its SLC buffer.
Single-thread CPU below baseline while multi-thread looks fine means power saving is on or the firmware caps the frequency.

The most convenient moment to capture these numbers is during dedicated server acceptance, stored in the same file as the hardware inventory. Digging into bottlenecks per subsystem continues in the guide on Linux performance profiling.

Key takeaways

Always run fio with --direct=1 and a file twice the size of RAM.
The disk metric that matters for a database is random 4k write IOPS plus the 99th percentile latency.
A benchmark without a stored baseline is pointless: capture the reference values on an empty server.

Benchmarking a Dedicated Server: fio, sysbench, iperf3