ZFS on a Dedicated Server: Pools, ARC, Snapshots

What ZFS is and how it differs from mdadm and ext4

ZFS is a filesystem and a volume manager at the same time: it combines drives into a pool and handles RAID, in-memory caching, and data checksums in one layer, without a separate RAID controller or mdadm. Every ZFS data block is stored together with a checksum and verified on read, so silent data corruption (bit rot) is caught immediately instead of months later.

On a dedicated server, ZFS is most often installed on Linux via the OpenZFS module or on FreeBSD, where it is part of the base system. Starting from scratch requires experience with RAID and an understanding of how the server disk subsystem works: ZFS does not remove the difference between HDD, SATA SSD, and NVMe, it just manages them differently.

Pool structure: vdev, RAIDZ1/2/3, and mirror

A ZFS pool (zpool) consists of one or more vdevs. Data is written across vdevs in parallel (like RAID 0), while the redundancy level lives inside each vdev:

Creating a mirrored pool from two drives:

zpool create tank mirror /dev/sda /dev/sdb
zpool status tank

A pool can only be expanded by adding a whole new vdev — you cannot grow an existing RAIDZ by one drive without recreating the pool, so plan capacity ahead of time.

ARC and L2ARC: how ZFS uses memory for caching

ZFS keeps its main read cache (ARC, Adaptive Replacement Cache) in server RAM and by default can claim up to half of all RAM for it. This is normal behavior, not a memory leak: ARC releases memory to other processes on demand. On servers with a large volume of cold data, a second cache tier called L2ARC on an NVMe drive is used — it does not store data permanently, it caches what has already been evicted from ARC.

Because ARC does not verify data on the fly as strictly as the checksums stored on disk, ECC memory is especially important for ZFS: without error correction, a corrupted bit in the ARC cache can be written to disk as "correct" data before any check happens.

Snapshots and send/receive for backups

A ZFS snapshot is created instantly and takes up almost no space until data changes:

zfs snapshot tank/data@2026-09-29
zfs list -t snapshot

The difference between snapshots can be streamed to another server without intermediate files:

zfs send -i tank/data@2026-09-01 tank/data@2026-09-29 | \
  ssh backup-host zfs receive tank/data

This is a convenient way to do incremental data transfer for backups: only the changed volume is transferred, not the whole array again, and the receiving side always has a consistent snapshot at the point in time it was taken.

Integrity checks: scrub and monitoring

A scheduled checksum verification of the whole pool is started with:

zpool scrub tank
zpool status -v tank

Scrub reads every block, verifies its checksum, and if redundancy allows, repairs the errors it finds on the fly. On production servers, scrub is scheduled every 1-2 weeks via cron, and pool state is checked for the line state: DEGRADED — it means one drive is already unavailable and the pool is running without redundancy headroom.

Checklist: when ZFS is worth it on a dedicated server

• You need snapshots and fast rollback without stopping the service — in ZFS this is a built-in feature, not a separate tool.
• Protection against silent data corruption matters — ZFS verifies block checksums on every read.
• The server has enough RAM: plan at least 4-8 GB above application needs for ARC.
• ECC memory is installed — otherwise the benefits of ZFS checksums are partly lost at the RAM cache stage.
• Pool expansion is planned ahead in whole vdevs, not individual drives added "as needed".

ZFS is not a cure-all: for simple single-drive web servers its features are overkill, but for file storage and database servers with frequent snapshots it removes tasks that would otherwise require separate utilities on top of ext4 and mdadm.