Choose the layout
Order the server with “No RAID — ZFS-ready” so the drives are left untouched. With large hard drives, use RAID-Z2 (two drives of redundancy): a rebuild of a full 16–22 TB drive takes a day or more, and a second failure during that window must not cost you the pool.
| Server | Layout | Usable |
|---|---|---|
| 8 × 16 TB | one 8-wide RAID-Z2 | ≈ 96 TB |
| 14 × 22 TB | one 14-wide RAID-Z2 (max space) | ≈ 264 TB |
| 14 × 22 TB | two 7-wide RAID-Z2 (faster rebuilds, more IOPS) | ≈ 220 TB |
Create the pool
Always use stable /dev/disk/by-id/ names, never sdX:
$ apt install zfsutils-linux
$ ls -l /dev/disk/by-id/ | grep -v part
$ zpool create -o ashift=12 -O compression=zstd -O atime=off \
tank raidz2 /dev/disk/by-id/ata-DISK1 /dev/disk/by-id/ata-DISK2 … /dev/disk/by-id/ata-DISK8Put metadata on NVMe
A mirrored special vdev on the NVMe drives stores metadata (and optionally small files), which makes directory listings and backups dramatically faster:
$ zpool add tank special mirror /dev/disk/by-id/nvme-A-part4 /dev/disk/by-id/nvme-B-part4
$ zfs set special_small_blocks=64K tankA special vdev is part of the pool: if it is lost, the pool is lost. Always mirror it.
Keep it healthy
- Scrub monthly: Debian and Ubuntu schedule it for you; check with
zpool status. - Stay under 80% full: performance drops sharply beyond.
- Use large records for backups and media:
zfs set recordsize=1M tank/backups. - Encrypt sensitive datasets with
-o encryption=aes-256-gcm -o keyformat=passphrase.