Because a ZFS datastore in a Proxmox Two-Node cluster operates locally rather than as a shared storage pool, ZFS Pool replication is required to synchronize data between nodes.
To protect running virtual machines against unexpected hardware failures, implementing automated ZFS Pool replication is the key to keep your business running.
Create the ZFS datastore
To proceed with the creation of the ZFS datastore, ensure you have a Proxmox Two-Node cluster in place.
Because ZFS is a local file system where multiple nodes cannot write to the same ZFS Pool simultaneously, unlike a shared SAN or Ceph cluster, Proxmox uses incremental replication to synchronize virtual machine and container data between separate nodes.
Proxmox's built-in GUI replication engine operates strictly per guest (VM or Container) rather than at the whole-pool level. This means every time you create a new VM, you manually have to add a replication job for it.
Create ZFS datastore on Node 1
Login to your Proxmox cluster and select Node 1. Navigate to the Disks section to verify that the disks you want to use for the ZFS Pool are detected.
Access the ZFS section and click Create ZFS.
Enter a Name for the datastore, make sure Add Storage is checked, and choose the desired RAID Level. In this example, the Single Disk option is selected since only one disk will be used. Select the disk and click Create.
Once the ZFS datastore has been created it will also be displayed in the available resources in the selected node.
Create ZFS datastore on Node 2
Select Node 2 > ZFS and click Create ZFS.
Enter the exact same Name given to datastore on Node 1, uncheck Add Storage and choose the required RAID Level (Single Disk in the example). Select the disk to use and click Create.
The ZFS datastore has now been created but is not yet listed as an available resource on Node 2.
Go to Datacenter > Storage and select the ZFS datastore you just created. Click Edit.
From the Nodes drop-down menu, select Node 2 and click OK.
The ZFS datastore is now available for Node 2 as well.
To verify your pools, open the Shell of Node 1 and check the available ZFS Pools.
# zpool list
Do the same for Node 2.
# zpool list
Create the ZFS Pool replication
Create a new virtual machine and choose the ZFS datastore as the target Storage.
To ensure data is available on both nodes in case one fails, you need to set up replication. Go to Datacenter > Replication > Add.
In the CT/VM ID field enter the ID of the VM or container you want to replicate. Specify the Target node and the desired Schedule (every 15 minutes in the example). Click Create.
The replication job has now been created.
Until the replication job is triggered for the first time, the target node will display the VM's disk with a size of 0 bytes.
At scheduled time, the replication runs and the disk data is securely copied to the target node.
After the initial synchronization, Proxmox copies only the incremental changes (deltas) during every subsequent replication run.
Test a Node failure
If the node where the VM is running fails (Node 1 in the example) and you are just using basic ZFS replication without Proxmox HA configured, the VM will not automatically boot up on Node 2. It will remain offline because Node 1 only holds a passive replica disk.
To automate the boot process, you need to configure HA for the protected VM. Select Datacenter > HA and click Add.
Select the VM you want to protect and click Add.
HA for the selected VM is now enabled.
Node failure
Node 1 experiences a sudden power outage.
When HA is triggered by the cluster, the VM is moved to Node 2 and automatically booted.
When Node 1 comes backup online, the VM is migrated back according to the HA settings (the Failback option is enabled).
The VM is now running on Node 1 again.
Keep in mind that when a node fails and the VM is booted to the secondary node, any data written inside the VM between the last completed ZFS Pool replication and the moment node failed will be lost.
