Storage Pool Redundancy¶
ZFS supports following basic RAID vdev types:
- Stripe (RAID0)
- Mirror (RAID1)
- RAIDZ (Single Parity RAIDZ)
- RAIDZ2 (Double Parity RAIDZ)
- RAIDZ3 (Triple Parity RAIDZ)
Note
By combining multiple vdev types in the storage pool, ZFS will automatically do stripping between them.
ZFS supports following nested RAID vdev types:
- Stripped Mirror (RAID10)
- Stripped RAIDZ (Stripped RAIDZ)
- Stripped RAIDZ2 (Stripped RAIDZ2)
- Stripped RAIDZ3 (Stripped RAIDZ3)
Basic RAID vdev Types¶
Stripe (RAID 0)¶
Stripped vdev is equivalent to RAID 0. It does not provide any level of redundancy and it should be used only with a different type of redundancy (e.g. stripped mirror or stripped RAIDZ1/2/3).
| Capacity | N x S(min) |
| Minimal number of drives | 2 |
| Fault tolerance | none |
| Read performance | Nx |
| Write performance | Nx |
(N - number of disks, S - size of the disk, min - smallest disk’s size)
How to create a storage pool with stripped vdevs¶
[root@node01 ~] zpool create storage disk0 disk1
How to add another stripped vdev into a storage pool¶
[root@node01 ~] zpool add storage disk2
Warning
Using striping alone is not recommended for production use!
Mirror (RAID 1)¶
Mirrored vdev is equivalent to RAID 1. ZFS supports the creation of N-way mirrored vdevs. Mirroring provides a good level of redundancy.
| Capacity | 1/N x S(min) |
| Minimal number of drives | 2 |
| Fault tolerance | N - 1 |
| Read performance | Nx |
| Write performance | 1x |
(N - number of disks, S - size of the disk, min - smallest disk’s size)
How to create a storage pool with mirror vdev¶
[root@node01 ~] zpool create storage mirror disk0 disk1
Note
ZFS allows creating mirror vdevs by attaching a new disk to the existing one. This will create a 2-way mirror from a single disk, a 3-way mirror from a 2-way mirror, etc. Only the redundancy level will be increased by attaching a new disk, not the available size within the storage pool.
Warning
When a storage pool has one disk and creating a mirror is required, use zpool attach. Using zpool add will create a stripe between the new disk and the existing in the storage pool. Reverting this action is not possible without copying data off the storage pool!
How to create a mirror in single drive storage pool or increase mirror redundancy¶
[root@node01 ~] zpool attach storage disk0 disk1
Note
It is possible to remove a disk from a mirror vdev by detaching it. The operation is refused if there are no other valid replicas of the data.
How to remove disk from existing mirror in a storage pool¶
[root@node01 ~] zpool detach storage disk0
RAIDZ (Single Parity RAIDZ)¶
RAIDZ, also called RAIDZ1, is a data/parity distribution scheme similar to RAID 5. It protects against RAID 5 write hole and is also faster as is does not have to perform the read-modify-write sequence.
| Capacity | (N - 1) * S(min) |
| Minimal number of drives | 3 |
| Fault tolerance | 1 |
| Read performance | Nx |
| Write performance | 1x |
(N - number of disks, S - size of the disk, min - smallest disk’s size)
How to create a storage pool with a RAIDZ vdev¶
[root@node01 ~] zpool create storage raidz disk0 disk1 disk2
Warning
It is not possible to extend a RAIDZ vdev with additional disks. The storage pool can be extended with a new RAIDZ vdev.
How to extend a storage pool with a RAIDZ vdev¶
[root@node01 ~] zpool add storage raidz disk3 disk4 disk5
RAIDZ2 (Double Parity RAIDZ)¶
RAIDZ2 is similar to RAID 6. The parity is stored twice.
| Capacity | (N - 2) * S(min) |
| Minimal number of drives | 4 |
| Fault tolerance | 2 |
| Read performance | Nx |
| Write performance | 1x |
(N - number of disks, S - size of the disk, min - smallest disk’s size)
How to create a storage pool with a RAIDZ2 vdev¶
[root@node01 ~] zpool create storage raidz2 disk0 disk1 disk2 disk3
Warning
It is not possible to extend a RAIDZ2 vdev with additional disks. The storage pool can be extended with a new RAIDZ2 vdev.
How to extend a storage pool with a RAIDZ2 vdev¶
[root@node01 ~] zpool add storage raidz disk4 disk5 disk6 disk7
RAIDZ3 (Triple Parity RAIDZ)¶
RAIDZ3 uses triple parity and guarantees an excellent level of redundancy. Generally, it should be used with large sized disks.
| Capacity | (N - 3) * S(min) |
| Minimal number of drives | 5 |
| Fault tolerance | 3 |
| Read performance | Nx |
| Write performance | 1x |
(N - number of disks, S - size of the disk, min - smallest disk’s size)
How to create a storage pool with a RAIDZ3 vdev¶
[root@node01 ~] zpool create storage raidz3 disk0 disk1 disk2 disk3 disk4
Warning
It is not possible to extend a RAIDZ3 vdev with additional disks. The storage pool can be extended with a new RAIDZ3 vdev.
How to extend a storage pool with a RAIDZ3 vdev¶
[root@node01 ~] zpool add storage raidz disk4 disk5 disk6 disk7 disk8
Nested RAID vdev Types¶
Stripped mirror (RAID 10)¶
Stripped mirror is a combination a stripping between mirror vdev types. This is the best performing RAID level for small random reads.
| Capacity | G * (1/N * S(min)) |
| Minimal number of drives | 4 |
| Fault tolerance | G x (N - 1) |
| Read performance | G * Nx |
| Write performance | N/Gx |
(N - number of disks in one mirror vdev, S - size of the disk, min - smallest disk’s size, G - number of mirror vdevs in the storage pools)
How to create a storage pool with a stripped mirror vdev¶
[root@node01 ~] zpool create storage mirror disk0 disk1 mirror disk2 disk3
How to extend a storage pool with a mirror vdev and create a stripped mirror vdev¶
[root@node01 ~] zpool add storage mirror disk4 disk5
Stripped RAIDZ (RAIDZ+0)¶
Stripped RAIDZ vdev is stripping across RAIDZ vdev types.
| Capacity | G * ((N - 1) * S(min)) |
| Minimal number of drives | 4 |
| Fault tolerance | G |
| Read performance | G * Nx |
| Write performance | G * 1x |
(N - number of disks in one RAIDZ vdev, S - size of the disk, min - smallest disk’s size, G - number of RAIDZ vdevs in the storage pools)
How to create a storage pool with a stripped RAIDZ vdev¶
[root@node01 ~] zpool create storage \
raidz disk0 disk1 disk2 disk3 disk4 \
raidz disk5 disk6 disk7 disk8 disk9
How to extend a storage pool with a RAIDZ vdev to create a stripped RAIDZ vdev¶
[root@node01 ~] zpool add storage \
raidz disk10 disk11 disk12 disk13 disk14
Stripped RAIDZ2 (RAIDZ2+0)¶
Stripped RAIDZ2 vdev is stripping across RAID22 vdev types. It is a good trade-off between available capacity and redundancy.
| Capacity | G * ((N - 2) * S(min)) |
| Minimal number of drives | 6 |
| Fault tolerance | 2 * G |
| Read performance | G * Nx |
| Write performance | G * 1x |
(N - number of disks in one RAIDZ2 vdev, S - size of the disk, min - smallest disk’s size, G - number of RAIDZ2 vdevs in the storage pools)
How to create a storage pool with a stripped RAIDZ2 vdev¶
[root@node01 ~] zpool create storage \
raidz2 disk0 disk1 disk2 disk3 disk4 \
raidz2 disk5 disk6 disk7 disk8 disk9
How to extend a storage pool with a RAIDZ2 vdev to create a stripped RAIDZ2 vdev¶
[root@node01 ~] zpool add storage \
raidz2 disk10 disk11 disk12 disk13 disk14
Stripped RAIDZ3 (RAID Z3+0)¶
Stripped RAIDZ3 vdev is stripping across RAIDZ3 vdev types.
| Capacity | G * ((N - 3) * S(min)) |
| Minimal number of drives | 8 |
| Fault tolerance | 3 * G |
| Read performance | G * Nx |
| Write performance | G * 1x |
(N - number of disks in one RAIDZ3 vdev, S - size of the disk, min - smallest disk’s size, G - number of RAIDZ3 vdevs in the storage pools)
How to create a storage pool with stripped RAIDZ3 vdevs¶
[root@node01 ~] zpool create storage \
raidz3 disk0 disk1 disk2 disk3 disk4 disk5 disk6 disk7 \
raidz3 disk8 disk9 disk10 disk11 disk12 disk13 disk14 disk15
How to extend a storage pool with a RAIDZ3 vdev to create stripped RAIDZ3 vdevs¶
[root@node01 ~] zpool add storage \
raidz3 disk16 disk17 disk18 disk19 disk20 disk21 disk22 disk23 disk24 disk25
Hot spares (Spare)¶
ZFS allows devices to be associated with pools as “hot spares”. These devices are not actively used in the pool, but when an active device fails, it is automatically replaced by a hot spare. Hot spares can be shared across multiple storage pools.
How to create a storage pool with a hot spare¶
[root@node01 ~] zpool create storage mirror disk0 disk1 spare disk2
How to create a storage pool and extend it with a hot spare¶
[root@node01 ~] zpool create storage mirror disk0 disk1
[root@node01 ~] zpool add storage spare disk2
How to replace a disk manually in a storage pool¶
[root@node01 ~] zpool replace storage disk0
Note
Disk replacement can be done automatically by the system upon inserting a new disk drive. This behavior is controlled by the autoreplace property of the storage pool and is disabled by default. If you want to enable it, execute zpool set autoreplace=on <pool>.
How to remove a disk from a storage pool¶
[root@node01 ~] zpool replace storage disk0 disk2
[root@node01 ~] zpool remove storage disk0
Note
Manual disk replacement should be done when the disk starts to show S.M.A.R.T errors or for a simple replacement for a different model.
Note
It is required to refresh the compute node’s system information after a new storage pool is created. The storage pool will be then available in the Danube Cloud web management.