Multipath tested and confirmed working on TrueNAS CORE 13.x (FreeBSD/ALUA)
and SCALE 24.10 + 25.04 (Linux LIO). Failover and path recovery validated
on both platforms. Closes#256.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If /etc/multipath.conf already had a manual devices{} block before package
install, postinst would append a second identical block, causing multipathd
to emit "duplicate keyword: devices" warnings. Strip any unmanaged devices
block in the append path to keep the file clean.
Fixes#279. Discovered 2026-06-20 on pve01-hq during SCALE 25.04 multipath testing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Old multipath testing debs were never pruned, causing apt to see multiple
3.1.0~beta+<sha> versions and pick whichever sha sorts highest — not newest.
Mirror the existing pool/testing cleanup that already did this correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without a registered JS panel the truenas-multipath type was invisible in
the PVE Datacenter → Storage → Add dropdown. Adds truenas-multipath.js
with all base TrueNAS fields plus the required truenas_portals field, and
wires it into the multipath package install/remove scripts and CI build.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
On install: if /etc/multipath.conf already exists, append a tagged
block rather than skipping silently (old) or overwriting (dangerous).
On upgrade: replace only the tagged block, leaving the rest untouched.
On remove/purge: strip the tagged block; rest of the file preserved.
This means admins who added their own stanzas during manual testing
(e.g. before the package existed) won't lose config on install or
upgrade. Closes the silent "file already exists" footgun.
Also fixes pre-existing SC2015 shellcheck warnings in postrm.
Refs #256
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TrueNAS.pm::qemu_blockdev_options returns an iscsi:// blockdev — wrong for
multipath where path() returns /dev/mapper/<wwid>. Simply removing the override
caused the parent's iscsi driver to be inherited instead.
Fix: explicitly call PVE::Storage::Plugin::qemu_blockdev_options (the
grandparent). It calls path(), sees /dev/mapper/<wwid> starts with '/',
stats it as S_ISBLK, and returns { driver => 'host_device', filename => ... }.
QEMU receives: "driver":"host_device","filename":"/dev/mapper/<wwid>"
Tested: VM 103 started with multipath disk (vm-103-disk-0 on TrueNAS-Scale01-MP).
QEMU confirmed using host_device driver on /dev/mapper/36589cfc000000ad...
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TrueNAS.pm: expose naa field from _find_extent return hash — needed by
TrueNASMultipath.pm to build the /dev/mapper/<wwid> path.
TrueNASMultipath.pm: use multipathd reconfigure + fallback multipath -v0
instead of bare multipath(8) which conflicts with a running multipathd
daemon. Tested end-to-end on pve01-hq against TrueNAS SCALE:
- Two sessions (172.31.69.91 + 192.168.69.91) both established
- /dev/mapper/36589cfc000000eb2adcd24154021d376 appears as block device
- deactivate_volume cleanly flushes and logs out
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PVE::SectionConfig registers properties globally across all plugin types.
Re-declaring the inherited TrueNAS.pm properties (truenas_host, truenas_pool,
etc.) caused a "duplicate property" error at pvedaemon startup when both
plugins are loaded. TrueNASMultipath::properties() now returns only the
new truenas_portals property; all inherited properties remain registered
by TrueNAS.pm.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TrueNASMultipath.pm inherits from TrueNAS.pm which lives in perl5/. The
-I/tmp/pve-stub flag alone doesn't make that path searchable, so the
'use base' failed at compile time. Adding -Iperl5 fixes it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds TrueNASMultipath.pm — a separate PVE storage type 'truenas-multipath'
that inherits all REST API and iSCSI resource management from TrueNAS.pm but
uses iscsiadm + dm-multipath for block device access:
path() → /dev/mapper/<wwid> (NAA from TrueNAS extent)
activate_volume() → iscsiadm login on each portal in truenas_portals;
waits for /dev/mapper device to appear
deactivate_volume()→ multipath flush + iscsiadm logout
qemu_blockdev_options → returns undef (path() used instead)
Packaging:
packaging/DEBIAN-multipath/ — control.j2, postinst, postrm,
changelog.Debian, multipath.conf.example
Distributed via the 'multipath' apt component (ADR-011)
Depends: truenas-proxmox >= 3.1.0, open-iscsi, multipath-tools
CI (build.yml):
- Third staging dir (dist-multipath) and .deb output
- multipath component pool under pool/multipath/v<major>
- Both stable and testing publish steps updated to generate
main + multipath components; Release lists both
- Lint checks TrueNASMultipath.pm and multipath packaging scripts
PoC confirmed 2026-06-07 on pve01-hq against TrueNAS SCALE 24.10:
- Two sessions (172.31.69.91 + 192.168.69.91), both active
- dm-multipath aggregated 3 disks × 2 paths each
- /dev/mapper/mpath* devices appeared correctly
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PVE allocates a vm-<vmid>-state-<snapname> volume when a snapshot is taken
with "Include RAM" selected. parse_volname didn't recognise this pattern,
causing a crash during both setup and cleanup of the state volume.
Added state volume pattern — treated as raw images, same vmid extraction
as disk volumes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Subroutines::ProhibitExplicitReturnUndef — volume_snapshot,
volume_snapshot_delete, volume_snapshot_rollback all used return undef
where bare return is the correct Perl idiom.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements the full PVE storage plugin snapshot interface backed by native
ZFS snapshots on TrueNAS, via the v2.0 REST API:
volume_has_feature — advertises snapshot support for raw volumes
volume_snapshot_needs_fsfreeze — returns 1 (request guest IO freeze)
volume_snapshot — POST /zfs/snapshot {dataset, name}
volume_snapshot_delete — DELETE /zfs/snapshot/id/{uri_escaped_id}
volume_snapshot_rollback — POST /zfs/snapshot/rollback {id, options}
volume_rollback_is_possible — enforces ZFS "most recent snap only" rule
volume_snapshot_info — GET /zfs/snapshot?dataset=... → {name: {id, ctime}}
Snapshot ID format: pool/volname@snapname
DELETE path requires full URI encoding (uri_escape) — unlike the dataset
endpoint, the snapshot API does not accept bare slashes in the path segment.
Tested on pve01-hq (PVE 9.2.3) against TrueNAS CORE 13.0-U6:
- create / list / delete: OK
- rollback_is_possible ordering: correctly rejects non-latest snapshots
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The ACCESS_TOKEN secret was created in 2022 and has expired. Switch both
gh-pages publish steps (stable and testing dist) to github.token, which is
generated fresh per run and never expires. Add permissions: contents: write
to the publish job so GITHUB_TOKEN can push to gh-pages.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
build.yml: testing dist publish condition used 'beta'/'alpha' but the
workflow channel values are 'testing'/'development' — step was always
skipped. Corrected to match actual channel names.
README: badge URLs referenced TheGrandWazoo/truenas-proxmox (repo not yet
renamed); corrected to freenas-proxmox. Workflow badge now includes
branch=release/3.x so it tracks the default branch explicitly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds packaging/changelog.Debian and packaging/DEBIAN-transitional/changelog.Debian
and wires them into the build staging steps. The gzipped changelog lands at
/usr/share/doc/truenas-proxmox/changelog.Debian.gz and
/usr/share/doc/freenas-proxmox/changelog.Debian.gz respectively, satisfying
Debian policy §12.7 and silencing the 'apt changelog' failure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Same sed bug as fixed on gh-pages: was stripping 'Filename: ' label
entirely, leaving a bare path that apt cannot use to download packages.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
build.yml:
- New "Publish to GitHub Pages testing dist" step — runs on beta/alpha
channel builds (pushes to release/3.x and master); replaces pool/testing/
with the latest build so the dist always has exactly one version
- Cloudsmith now receives v2.x stable only; beta/alpha no longer published
to Cloudsmith testing channel
README:
- Replace Cloudsmith testing install instructions with GitHub Pages testing
dist track; same GPG key as stable, just different dist name
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Closes#270. PVE 9 expects APIVER=14; returning 11 caused a "older storage
API, upgrade recommended" warning on every pvedaemon start and qm command.
No new method implementations required — Plugin.pm has no gates between
api() 11 and 14. Bumping VERSION starts the v3.1.0 development cycle.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docs: update ROADMAP.md to reflect v3.0.0 shipped and current state
Migration script: the early-exit check now verifies the dist track matches
the installed major version. A v2 node that had v3 dist track from a test
run now gets corrected rather than skipped.
ROADMAP.md: full update — v3.0.0 moved to Released, GitHub Pages apt repo
added as Released, v3.1.0 scope updated with #272 and multipath redesign,
stale pre-release housekeeping removed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Establishes the pattern for distributing optional packages (starting with
truenas-proxmox-multipath) via apt components within existing dist tracks.
Users add 'multipath' to their sources.list component list to opt in;
standard users are never affected by apt upgrade.
References: #256 (multipath), #272 (testing track)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Stable v3.x and above go to GitHub Pages only. Cloudsmith still receives:
- v2.x stable releases (existing Cloudsmith user base on freenas-proxmox)
- beta/alpha/dev builds (testing channel, until GitHub Pages testing track
ships as a v3.1.0 item)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The limited-scope apt-get update didn't flush the full apt cache, causing
apt-cache policy to miss the new GitHub Pages source on nodes that had a
beta build installed from the testing channel (pve02-hq: 3.0.0~beta).
Full apt update ensures the cache is consistent before the verification check.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
scripts/migrate-repo-to-github-pages.sh:
- Detects installed version and picks the correct dist track automatically
- Finds Cloudsmith sources by URL content (not filename) to handle the
varied naming conventions users have in sources.list.d/
- Removes old keyrings, imports GitHub Pages key, writes new source
- Confirms the package is visible from the new repo before exiting
build.yml publish job:
- Add empty binary-amd64 and binary-arm64 Packages files alongside
binary-all, and set Architectures: all amd64 arm64 in Release
- Silences the "doesn't support architecture 'amd64'" warning on PVE
hosts without duplicating package entries
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- README: new "Migrating from Cloudsmith" section with v2 and v3
command sequences and a one-line reason (Cloudsmith being phased out)
- docs/upgrade-paths.md: full matrix of every starting version × repo
combination with links to the right guide; v4.x rows stubbed with
placeholder instructions so users know what's coming and where to look
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
README: v2.x Prerequisites relabeled legacy (not 'current stable');
condensed to three items. v3.0 does not require SSH.
architecture.md: remove incorrect claim that the Add Storage wizard
requires SSH via ZFSPoolPlugin.pm. Verified in truenas-storage.js:
truenas_pool is a plain text field. Pool is validated at runtime via
GET /pool/dataset, not SSH.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v3.0 requires no SSH — QEMU connects directly via iscsi://. README "How It
Works" and troubleshooting sections had v2.x SSH carryover. Compatibility
matrix now shows v3.x as stable with PVE 9 confirmed.
All install instructions updated from Cloudsmith to the GitHub Pages apt
repo (thegrandwazoo.github.io/freenas-proxmox, v3 dist track). Cloudsmith
testing channel retained for beta builds only.
architecture.md updated to reflect ADR-010 and the live GitHub Pages publish
step in CI.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements ADR-010: stable releases are now published to
https://thegrandwazoo.github.io/freenas-proxmox/ in addition to Cloudsmith.
Dist tracks:
v3 main — v3.x releases (new installs)
main main — v2.x only (backward-compat, never auto-promoted to v3)
v2 main — v2.x alias (explicit pin)
CI: new "Publish to GitHub Pages APT repo" step in the publish job runs on
tagged stable releases. Downloads the built .deb, places it in pool/v{major},
regenerates Packages.gz and a GPG-signed InRelease for the relevant dist(s),
and pushes to the gh-pages branch.
Setup: scripts/setup-apt-signing-key.sh generates the GPG key pair and prints
the exact `gh secret set` commands to run. Requires APT_SIGNING_KEY and
APT_SIGNING_KEY_PASSPHRASE secrets to be added to the repo before the first
tagged release.
README updated to point new installs at GitHub Pages; Cloudsmith testing
channel retained for beta builds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Motivated by #271 (silent v2→v3 upgrade). Establishes main (v2 compat),
v2 (alias), and v3 dist tracks so apt upgrade never crosses a major
version boundary. Gated on #230 (GitHub Pages apt repo).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Closes the [Unreleased] heading now that v3.0.0-1 is published to the
stable channel and the GitHub Release is live.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ADR-009 (PVE version support v3.x + v4.0 strategy) accepted.
ADR-008 marked Superseded by ADR-009.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When Proxmox fixes PVE::Storage::Plugin to use int($3) for the iscsi lun
field, this override becomes redundant. The comment now includes the exact
grep command to verify the upstream fix and confirms when removal is safe.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PVE::Storage::Plugin::qemu_blockdev_options builds the iscsi blockdev JSON
with lun => "$3" — a string capture from the URL regex. QEMU 9's strict
blockdev schema rejects a string where an integer is required, causing every
VM with TrueNAS storage to fail to start on PVE 9.
Override the method in TrueNAS.pm to return the blockdev hash directly from
_find_extent, with lun => int($ext->{lun_id}). This bypasses the base class
entirely and is correct on both PVE 8 and PVE 9.
Verified on pve01-hq (PVE 9.2.3 / QEMU 9 / kernel 7.0.6-2-pve):
VM 103 (TrueNAS-Scale01-Tank01, SCALE 24.10) starts successfully.
The upstream bug in PVE::Storage::Plugin is tracked separately — when
Proxmox fixes it, the override can be removed. See issue #266 for details.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- v3.x beta: tested on PVE 8.4.x, CORE 13.0-U6, SCALE 24.10 + 25.04 + 25.10
- v3.x PVE 9.x: split to planned row with known issue ref (#266)
- Version notices: PVE 8 now points to beta, PVE 9 warns of known issue
- Drop untested SCALE Cobia/Dragonfish from tested column
- "upcoming" → "beta", "In development" → "Beta — active development"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SCALE 25.10 (Goldeye) enforces strict integer validation. The log call
preceding the API call stringifies $size_b (sets Perl's POK flag), causing
JSON::XS to encode it as "8589934592" (quoted) instead of 8589934592.
int() at the encode site produces a fresh IV without the POK flag.
Also adds error parser support for SCALE 25.10's field-keyed validation
error format so the full Pydantic detail surfaces in PVE task logs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SCALE 24.10 rejects DELETE /iscsi/targetextent/id/<id> with 422 if
the associated target has an active iSCSI session — which happens when
removing an unused disk from a running VM (the VM still holds the target
session for its other disks).
Pass bare boolean true as the request body to force the targetextent
delete. Confirmed working on SCALE 24.10 and SCALE 25.04.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_find_extent returned only the first targetextent row for an extent.
If duplicate rows exist (from a prior failed alloc_image leaving an
orphan), free_image deleted one association and then hit a 422 on the
extent DELETE because TrueNAS still saw the second association as
keeping the target in use.
Return all rows in a new targetextents field and loop over them in
free_image so every association is removed before the extent DELETE.
Fixes: 422 Unprocessable Entity on free_image (SCALE 24.10, vm-103-disk-3)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Steps 2-4 of alloc_image (extent create, target find/create, targetextent
create) are now wrapped in an eval. On any failure the rollback runs in
reverse order, best-effort: target removed if it has no remaining extents
(i.e. it was just created for this disk), extent deleted with force=true,
zvol deleted recursively. This prevents the orphaned-resources-on-failure
state that previously required manual cleanup.
Also restore alias ("Proxmox VM <vmid>") on target creation. The alias was
removed during SCALE 25.04 debugging as a red herring — the actual bug was
int() type coercion (#264). Alias is unique per-VM and _resolve_vm_target
always checks for an existing target by name before attempting a POST, so
there is no alias-collision risk in normal operation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TrueNAS SCALE 25.04 introduced two breaking changes in its iSCSI API:
1. Unique alias constraint: POST /iscsi/target with alias collides on retry.
Remove alias field entirely — names are already unique per-VM.
2. Pydantic v2 strict integer validation: rejects JSON strings ("7") where
integers are expected. Perl scalars become dual-vars (string+int flags)
after log interpolation or hash-key lookups, causing encode_json to emit
quoted values. Fix: int() at all integer-ID pass sites (portal, target,
extent, lun) and omit null initiator/auth instead of sending explicit null.
Confirmed: alloc_image works on SCALE 25.04, SCALE 24.10, and CORE 13.0-U6.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SCALE 25.04 returns validation errors as a JSON array rather than a
{message: "..."} hash. The error detail was silently lost, making 4xx
failures appear as bare status lines in logs. Now handles both formats.
Prompted by a transient 422 on /iscsi/target during first-run testing
on SCALE 25.04 — root cause was iSCSI service mid-reload, not a format
incompatibility. Plugin is compatible with SCALE 25.04 as-is.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Captures v3.0.0 remaining work, completed items, the GitHub repo rename
deferral rationale (business timing, no FUNDING.yml impact), and
upcoming v3.1.0/v3.2.0 scope. Process rule added: business decisions
go here and in GitHub issue comments — not only in AI memory.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces truenas-proxmox as the real package and freenas-proxmox as a
transitional meta-package that Depends on truenas-proxmox. Existing
users running apt upgrade automatically pull in the new package; the
migration one-liner (apt install truenas-proxmox && apt purge
freenas-proxmox) completes the transition. The freenas-proxmox package
name will not exist in v4.0.
Changes:
- packaging/DEBIAN/: Package renamed to truenas-proxmox, INSTALL_DIR
and LOG_FILE updated to /usr/share/truenas-proxmox and
/var/log/truenas-proxmox-install.log; Replaces/Breaks fields added
- packaging/DEBIAN-transitional/: new empty meta-package that pulls in
truenas-proxmox and prints a deprecation notice on configure
- build.yml: PACKAGE_NAME=truenas-proxmox; both debs built and
published to Cloudsmith; transitional postinst added to shellcheck
- All GitHub repo URLs updated to TheGrandWazoo/truenas-proxmox
- apt install/remove/purge commands in docs updated to truenas-proxmox
- GitHub repo rename to truenas-proxmox pending (gh repo rename step)
Closes#262
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The API token was previously stored only in storage.cfg, which is
replicated in plaintext across all cluster nodes via pmxcfs. This adds
a keyfile mechanism: at startup each public entry point calls
_resolve_token(), which checks /etc/pve/priv/truenas-<storeid>.key
before falling back to truenas_api_key in storage.cfg. The resolved
token is cached in the per-host $state so the file is read at most once
per daemon session. truenas_api_key is now optional in storage.cfg;
either the keyfile or the config key must be present.
Docs updated in getting-started.md (section 4.1) and the in-app help
HTML with setup commands and the cluster-copy reminder.
Closes#261
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Kevin Adams