freenas-proxmox/docs/upgrade-paths.md

Upgrade Paths

Quick reference for every migration scenario. Find your current version and repo source in the table, then follow the link for step-by-step instructions.

---

Where am I, and where do I go?

Current version	Current repo	→ Stay on current major	→ Move to v3.x	→ Move to v4.x
v2.x	Cloudsmith stable	Swap repo to GitHub Pages main	Swap repo + config migration	v2→v3 config migration first, then v3→v4
v2.x	GitHub Pages main	Already on correct repo ✅	Config migration only	v2→v3 config migration first, then v3→v4
v3.x	Cloudsmith stable	Swap repo to GitHub Pages v3	Already on v3 — swap repo only	Swap repo straight to the v4 track
v3.x	GitHub Pages v3	Already on correct repo ✅	Already here ✅	Change dist track to v4
v4.x	GitHub Pages v4	Already on correct repo ✅	—	Already here ✅

Not sure which version you have? Run this on any Proxmox node:

dpkg -l truenas-proxmox freenas-proxmox 2>/dev/null | grep '^ii'

Not sure which repo you're using?

cat /etc/apt/sources.list.d/truenas-proxmox*.list /etc/apt/sources.list.d/freenas-proxmox*.list 2>/dev/null

---

Cloudsmith → GitHub Pages

Cloudsmith is being phased out. GitHub Pages is the permanent home for this project's packages. Your installed package is not affected — this only changes where future updates come from.

Full instructions are in the README — Migrating from Cloudsmith section.

In short:

1. Remove your existing Cloudsmith source file from /etc/apt/sources.list.d/
2. Remove the old Cloudsmith keyring from /usr/share/keyrings/
3. Import the GitHub Pages GPG key to /etc/apt/keyrings/truenas-proxmox.gpg
4. Add the GitHub Pages source pointing at the right dist track (main for v2, v3 for v3, v4 for v4). A v3 Cloudsmith user can go directly to v4. Follow v3.x → v4.x from step 2 onwards.
5. Run apt update

---

What changes between major versions

	v2.x → v3.x	v3.x → v4.x
Breaking change?	Yes — auth model, iSCSI model, package name	Platform only: requires PVE 9. Adds a WebSocket transport, used automatically for SCALE 25.04+. CORE and SCALE < 25.04 stay on REST
In-place upgrade?	No — VM disks must be moved	Yes — switch the dist track, then apt upgrade
Plugin restarts VMs?	No — Move Disk is live	No. Only the PVE daemons restart (pvedaemon, pveproxy, pvestatd, pve-ha-lrm, pve-ha-crm). Running VMs keep their iSCSI connections
Config changes?	Yes — API key replaces username/password	None. storage.cfg is read unchanged
Full guide	migrating-from-v2.md	v3.x → v4.x below

---

v2.x → v3.x

See migrating-from-v2.md for the full step-by-step guide.

High-level steps:

1. Install truenas-proxmox on every Proxmox node (from GitHub Pages v3 dist)
2. Create a TrueNAS API key
3. Add a new v3 storage entry in Proxmox (keep the old v2 entry running alongside)
4. Move VM disks one at a time using Proxmox Move Disk (VMs stay running)
5. Remove the old v2 storage entry once all disks are moved

Do not run apt upgrade from a v2 Cloudsmith install without reading the migration guide first. v3.0 is a breaking change — your storage will need reconfiguration.

---

v3.x → v4.x

v4.0 adds a WebSocket JSON-RPC 2.0 transport for TrueNAS SCALE 25.04 and later. SCALE 25.04 deprecated the REST API, and SCALE 26.x is planned to remove it. The plugin detects the transport per TrueNAS host each time the daemons start:

TrueNAS host	Transport used by v4
CORE 13.x	REST v2.0 (unchanged from v3)
SCALE < 25.04 (e.g. 24.10)	REST v2.0 (unchanged from v3)
SCALE 25.04 and later	WebSocket JSON-RPC 2.0

You choose nothing and configure nothing. A cluster can mix CORE and SCALE hosts, and each storage uses whichever transport its host needs. To see which one was picked:

journalctl -b | grep -o '[^ ]*: detected version=.*' | sort -u
# 172.31.69.90: detected version=TrueNAS-13.0-U6.8 -> rest transport
# 172.31.69.91: detected version=TrueNAS-25.10.6 -> ws transport

The upgrade is an in-place apt upgrade. VM disks are not moved and running VMs are not restarted.

1. Pre-upgrade checks (every node)

• PVE 9 is required. The v4 package installs on PVE 8 without complaint, but PVE 8 refuses to load the plugin (storage API 15 > 11) and every TrueNAS storage on that node goes inactive. Upgrade the node to PVE 9 first (pve8to9 --full). Until then, keep PVE 8 nodes on 3.0.0-1, the last release with storage API 11. Pin it with apt install truenas-proxmox=3.0.0-1 && apt-mark hold truenas-proxmox. Every v3.1+ release also needs PVE 9. The v3 and v4 plugins read the same storage.cfg, so a cluster can run both while you upgrade it node by node.
• No v2 storages left. v2 storages use the zfs: type with iscsiprovider freenas and are not managed by v3 or v4. If this prints anything, finish the v2→v3 migration for those disks first:

  grep -B1 -A8 'iscsiprovider freenas' /etc/pve/storage.cfg
  

• Note any orphan volumes. Volumes left behind by plugin versions ≤ 4.0.1 (see the README troubleshooting entries for #291 and #292) do not block the upgrade. On v4.0.2+ you can delete them normally with pvesm free. Lists disks that no VM config references:

  for s in $(awk '/^truenas(-multipath)?:/{print $2}' /etc/pve/storage.cfg); do
    pvesm list "$s" | awk 'NR>1{print $1}' | while read -r v; do
      grep -rqs -- "${v#*:}" /etc/pve/nodes/*/qemu-server/ || echo "unreferenced: $v"
    done
  done
  

• Check the API key. v4 reads the same truenas_api_key in storage.cfg, or /etc/pve/priv/truenas-<storeid>.key if that file exists (supported since v3.0). If a SCALE host was upgraded to 25.04, make sure its key wasn't revoked. See the README entry API key stops working after upgrading TrueNAS SCALE to 25.04.

2. Switch the dist track to v4 (every node)

If you use the multipath package, keep multipath in Components. Each dist track has its own multipath component, so it moves with the base package.

# deb822 format (PVE 9 / Debian trixie)
cat << 'SOURCES' | sudo tee /etc/apt/sources.list.d/truenas-proxmox.sources
Types: deb
URIs: https://thegrandwazoo.github.io/freenas-proxmox
Suites: v4
Components: main
Signed-By: /etc/apt/keyrings/truenas-proxmox.gpg
SOURCES
# Multipath users: use "Components: main multipath" above instead.

# Remove any leftover one-line source for the v3 track, so apt doesn't see both
sudo rm -f /etc/apt/sources.list.d/truenas-proxmox.list

One-line .list format (if you prefer it over deb822)

echo "deb [signed-by=/etc/apt/keyrings/truenas-proxmox.gpg] \
https://thegrandwazoo.github.io/freenas-proxmox v4 main" \
  | sudo tee /etc/apt/sources.list.d/truenas-proxmox.list
# Multipath users: end the line with "v4 main multipath" instead.

The codename rivendell works in place of v4.

3. Upgrade (every node)

sudo apt update && sudo apt upgrade
dpkg -l truenas-proxmox truenas-proxmox-multipath 2>/dev/null | grep '^ii'   # expect 4.x

apt installs two new dependencies, libprotocol-websocket-perl and libio-socket-ssl-perl. The package postinst then restarts pvedaemon, pveproxy, pvestatd, pve-ha-lrm and pve-ha-crm. Running VMs are untouched, because QEMU holds its own iSCSI connection.

4. Verify

pvesm status | grep -E 'truenas'          # every TrueNAS storage should be "active"
journalctl -b | grep -o '[^ ]*: detected version=.*' | sort -u   # one line per TrueNAS host

Then create a small test disk and remove it on each storage. The Proxmox UI works, or you can use pvesm alloc <storage> <vmid> vm-<vmid>-disk-9 1G followed by pvesm free <storage>:vm-<vmid>-disk-9.

Rollback: set the dist track back to v3 and run apt update. Then downgrade to the version shown by apt-cache policy truenas-proxmox, e.g. apt install truenas-proxmox=3.2.5-1. If you use multipath, add truenas-proxmox-multipath=3.2.5-1 to the same command. Neither storage.cfg nor TrueNAS needs any change.

apt upgrade on the v3 dist track never pulls in a v4 package. You must change the dist track explicitly. This is intentional.

---

v2.x → v4.x

There is no direct path from v2 to v4. v2 uses the stock Proxmox zfs: storage type with patched core files. v3 and v4 use a different storage type and auth model, and VM disks have to be moved onto it. In order:

1. Follow migrating-from-v2.md to install the plugin, add a truenas: storage, and Move Disk every VM off the v2 storage. Install from the v4 track directly if the node is already on PVE 9. The new-storage and Move Disk steps are the same. If the node is still on PVE 8, install from v3 pinned to 3.0.0-1 (apt install truenas-proxmox=3.0.0-1 && apt-mark hold truenas-proxmox).
2. Remove the old zfs: / iscsiprovider freenas storage entries and purge freenas-proxmox.
3. If you installed from v3, upgrade the node to PVE 9, then follow v3.x → v4.x.

---

Dist track reference

Dist track	Codename alias	Contains	Who should use it
main	limelight	v2.x packages only	Users staying on v2.x
v2	limelight	v2.x packages (alias for main)	Same as main — explicit pin
v3	error	v3.x packages only	All current v3 installs
v4	rivendell	v4.x packages only	All new installs and PVE 9 clusters (current stable)

Both the dist track (v3) and its codename alias (error) work in sources.list — they resolve to the same packages.

Packages on one dist track are never promoted to another. Changing major versions always requires an explicit sources.list edit.

Existing v3 users — one-time prompt after v3.2.2 If apt update reports changed its 'Codename' value from 'v3' to 'error', run:

apt update --allow-releaseinfo-change

This is a one-time acknowledgment of the codename being added in v3.2.2. It will not recur.