Setting Up Proxmox with ZFS Storage

Setting Up Proxmox with ZFS Storage: A Home Lab Infrastructure Guide

Proxmox combined with ZFS creates a powerful virtualization and storage platform perfect for home labs and small businesses. I've set up multiple Proxmox clusters with ZFS pools, and here's my complete guide to building a reliable, scalable infrastructure.

🎯 Why Proxmox + ZFS?

Proxmox VE is a complete virtualization platform (KVM + LXC containers) with a web-based management interface. ZFS provides enterprise-grade storage features:

Data integrity: Checksums detect and correct corruption
Snapshots: Instant, space-efficient backups
Compression: Save disk space transparently
RAID-like protection: Without hardware RAID controllers
Deduplication: Eliminate duplicate data

Together, they give you enterprise features on consumer hardware.

🖥 Hardware Requirements

Minimum Specs

CPU: 4+ cores (supports virtualization)
RAM: 16GB minimum (32GB+ recommended)
Storage: 2+ drives for ZFS (SSDs preferred)
Network: Gigabit Ethernet

My Setup

CPU: AMD Ryzen 9 9950X3D (16 cores)
RAM: 64GB DDR5
Storage: 4x 2TB NVMe SSDs (ZFS RAID-Z1)
Network: 10GbE for VM traffic

📦 Installation Steps

1. Install Proxmox VE

Download the ISO from [proxmox.com](https://www.proxmox.com) and install:

```bash

Boot from USB

Follow installation wizard

Set root password and network configuration

```

2. Initial Configuration

After installation, access the web UI at `https://your-ip:8006`

```bash

Update system

apt update && apt upgrade -y

Install useful tools

apt install -y vim curl wget git ```

3. Configure ZFS Storage

ZFS is included in Proxmox, but you need to create pools manually:

```bash

List available disks

lsblk

Create ZFS pool (RAID-Z1 = 1 disk parity, needs 3+ disks)

zpool create -f tank raidz1 /dev/sdb /dev/sdc /dev/sdd /dev/sde

Or for mirror (2 disks, 1 redundancy)

zpool create -f tank mirror /dev/sdb /dev/sdc

Set compression (saves space)

zfs set compression=lz4 tank

Enable deduplication (if you have RAM)

zfs set dedup=on tank

Check pool status

zpool status zfs list ```

4. Add ZFS to Proxmox

In the web UI: 1. Go to Datacenter → Storage 2. Click Add → ZFS 3. Select your ZFS pool (e.g., `tank`) 4. Enable Content: Disk image, Container

🏗 Storage Pool Design

I use multiple ZFS pools for different purposes:

Pool Structure

``` tank-vm/ # VM disk images tank-containers/ # LXC container storage tank-backup/ # Backup storage tank-isos/ # ISO images ```

```bash

Create datasets

zfs create tank/vm zfs create tank/containers zfs create tank/backup zfs create tank/isos

Set different properties

zfs set compression=gzip tank/backup # Better compression for backups zfs set recordsize=1M tank/vm # Optimize for large files ```

🔄 ZFS Snapshots

Snapshots are instant and space-efficient:

```bash

Create snapshot

zfs snapshot tank/vm@before-update

List snapshots

zfs list -t snapshot

Rollback to snapshot

zfs rollback tank/vm@before-update

Delete snapshot

zfs destroy tank/vm@before-update ```

Automated Snapshots

Create a cron job for regular snapshots:

```bash

/etc/cron.daily/zfs-snapshots

#!/bin/bash DATE=$(date +%Y%m%d) zfs snapshot tank/vm@daily-$DATE zfs snapshot tank/containers@daily-$DATE

Keep only last 7 days

zfs list -t snapshot -o name | grep "@daily" | head -n -7 | xargs -r zfs destroy ```

🚀 Performance Tuning

ZFS Tuning

```bash

Increase ARC (cache) size (default is 50% of RAM)

echo 'options zfs zfs_arc_max=34359738368' >> /etc/modprobe.d/zfs.conf

Tune for SSDs

zfs set primarycache=all tank zfs set secondarycache=all tank

Disable access time updates (faster)

zfs set atime=off tank ```

Proxmox Tuning

```bash

Increase VM memory ballooning

echo 'vm.swappiness=10' >> /etc/sysctl.conf

Optimize for SSDs

echo 'vm.dirty_ratio=15' >> /etc/sysctl.conf echo 'vm.dirty_background_ratio=5' >> /etc/sysctl.conf ```

💾 Backup Strategy

1. ZFS Replication

Replicate to another server or external drive:

```bash

Send snapshot to remote server

zfs send tank/vm@snapshot1 | ssh remote-server zfs receive backup/vm

Incremental replication

zfs send -i tank/vm@snapshot1 tank/vm@snapshot2 | \ ssh remote-server zfs receive backup/vm ```

2. Proxmox Backup Server

Use Proxmox's built-in backup solution:

```bash

Install PBS on separate machine

Configure in Proxmox UI: Datacenter → Backup

```

3. VM-Level Backups

Use `vzdump` for VM backups:

```bash

Backup single VM

vzdump 100 --storage local --compress zstd

Backup all VMs

vzdump --all --storage local --compress zstd ```

🔒 Security Hardening

1. Firewall Configuration

```bash

Enable firewall in Proxmox UI

Or configure manually:

apt install -y pve-firewall

Allow only necessary ports

8006 (HTTPS), 22 (SSH), 5900-5999 (VNC)

```

2. SSH Hardening

```bash

/etc/ssh/sshd_config

PermitRootLogin no PasswordAuthentication no # Use keys only PubkeyAuthentication yes ```

3. Fail2ban

Protect against brute force:

```bash apt install -y fail2ban systemctl enable fail2ban ```

📊 Monitoring

ZFS Health Monitoring

```bash

Check pool health

zpool status

Monitor I/O

zpool iostat -v 1

Check for errors

zpool status -x ```

Proxmox Monitoring

Use built-in Proxmox metrics (Prometheus-compatible)
Set up Grafana dashboards
Monitor CPU, RAM, disk I/O, network

🐛 Common Issues & Solutions

Issue: Out of Space

```bash

Check space usage

zfs list -o name,used,available,referenced

Clean up old snapshots

zfs list -t snapshot | xargs zfs destroy

Enable compression if not enabled

zfs set compression=lz4 tank ```

Issue: Slow Performance

Check if deduplication is enabled (uses lots of RAM)
Verify ARC size is appropriate
Check for disk errors: `zpool status`
Consider adding more RAM for ARC cache

Issue: Pool Won't Import

```bash

Force import (if disks moved)

zpool import -f tank

Check disk status

zpool status -v ```

🎓 Best Practices

1. Use RAID-Z2 for critical data (2 disk parity) 2. Regular snapshots before major changes 3. Monitor pool health daily 4. Test backups regularly 5. Keep Proxmox updated: `apt update && apt upgrade` 6. Document your setup (pool names, VM purposes)

💡 Advanced: TrueNAS Integration

I run TrueNAS SCALE as a VM in Proxmox for NAS functionality:

1. Create VM with passthrough disks 2. Install TrueNAS SCALE 3. Create ZFS pools in TrueNAS 4. Share via NFS/SMB to Proxmox VMs

This gives you nested ZFS with Proxmox managing VMs and TrueNAS managing storage.

Conclusion

Proxmox + ZFS is a powerful combination that gives you enterprise-grade virtualization and storage on consumer hardware. With proper setup, tuning, and monitoring, you'll have a reliable infrastructure that scales with your needs.

The key is starting simple, monitoring performance, and iterating based on your actual usage patterns.