From 8286143fbe2d304ebe85e91f467ba65981af8469 Mon Sep 17 00:00:00 2001 From: Kevin Adams Date: Mon, 12 Oct 2026 11:57:43 -0400 Subject: [PATCH 1/8] fix: let path() return a placeholder for extent-less zvols (#292) PVE's content delete (pvesm free, GUI Remove) and qm destroy call path() only for the owner/vtype before vdisk_free. Dying when the zvol had no iSCSI extent made orphans from failed allocs or v2-era leftovers undeletable through PVE, even though free_image already handles the no-extent case. activate_volume and qemu_blockdev_options keep dying on the same condition, so the placeholder can never be opened by a VM. Adds the first Perl unit-test harness (t/, prove in CI lint job) with a mock TrueNAS API so plugin entry points can be tested off a PVE node. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 1bbcba136b394572891b709ee1cc095e79309794) --- .github/workflows/build.yml | 5 ++ README.md | 12 +++ docs/migration-troubleshooting.md | 2 + perl5/PVE/Storage/Custom/TrueNAS.pm | 26 ++++-- perl5/PVE/Storage/Custom/TrueNASMultipath.pm | 16 ++-- t/lib/MockTrueNAS.pm | 74 +++++++++++++++++ t/lib/PVE/Storage/Plugin.pm | 18 +++++ t/path.t | 84 ++++++++++++++++++++ 8 files changed, 226 insertions(+), 11 deletions(-) create mode 100644 t/lib/MockTrueNAS.pm create mode 100644 t/lib/PVE/Storage/Plugin.pm create mode 100644 t/path.t diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 15a5315..c58ba2c 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -73,6 +73,11 @@ jobs: packaging/DEBIAN-multipath/postrm echo "Shell scripts OK" + - name: Perl unit tests + run: | + echo "==> Running prove t/..." + prove -v t/ + - name: Test postinst/postrm service restarts run: | echo "==> Running tests/test-restart-services.sh..." diff --git a/README.md b/README.md index 6064ac8..1f48809 100644 --- a/README.md +++ b/README.md @@ -595,6 +595,18 @@ A quick reference: If you see iSCSI extents in TrueNAS that are not associated with any target, they can be safely deleted from the TrueNAS UI. v2.3.0 and later automatically roll back and clean up after a failed LUN creation. +### `pvesm free` / GUI Remove fails with "has no iSCSI extent" + +**Symptom:** removing a disk fails with `Volume 'vm-101-disk-0' has no iSCSI extent on . Was it created via this plugin?`. The zvol still exists on TrueNAS, but its iSCSI extent is gone. Typical causes are a failed `alloc_image`, an extent deleted by hand, or a leftover from v2.x. + +**Fixed in v4.0.2** ([#292](https://github.com/TheGrandWazoo/freenas-proxmox/issues/292)). `path()` now returns a placeholder for these orphans, so Proxmox can delete them normally. The task log shows `path: has no mapped iSCSI extent — returning placeholder path`. Starting a VM on such a disk still fails at `activate_volume`, as it should. + +**On v4.0.1 and earlier:** first confirm that no VM config references the disk (`grep -r '' /etc/pve/nodes/*/qemu-server/`). Then call `vdisk_free` directly, which skips the `path()` lookup: + +```bash +perl -MPVE::Storage -e 'PVE::Storage::vdisk_free(PVE::Storage::config(), "STORAGE:vm-101-disk-0")' +``` + ### Filing a Bug Report Please use the [GitHub issue tracker](https://github.com/TheGrandWazoo/truenas-proxmox/issues) and include the [bug report template](.github/ISSUE_TEMPLATE/bug_report.md). Include relevant log lines from `syslog` (search for `FreeNAS::`). diff --git a/docs/migration-troubleshooting.md b/docs/migration-troubleshooting.md index 94a019e..3136f58 100644 --- a/docs/migration-troubleshooting.md +++ b/docs/migration-troubleshooting.md @@ -110,6 +110,8 @@ The v2.x plugin may not have removed these automatically. They are harmless but Always verify a zvol is truly unused before deleting it — check `qm config` for all VMs. +**Zvols on a v3/v4 `truenas` storage** that have no extent show up in `pvesm list `. From v4.0.2 you can delete them with `pvesm free :` or the GUI's Remove button. On older versions that fails with "has no iSCSI extent"; see the README troubleshooting entry for the `vdisk_free` workaround ([#292](https://github.com/TheGrandWazoo/freenas-proxmox/issues/292)). + --- ## Old v2.x storage can't be removed from Proxmox diff --git a/perl5/PVE/Storage/Custom/TrueNAS.pm b/perl5/PVE/Storage/Custom/TrueNAS.pm index 6d9e6c3..871413e 100644 --- a/perl5/PVE/Storage/Custom/TrueNAS.pm +++ b/perl5/PVE/Storage/Custom/TrueNAS.pm @@ -397,6 +397,14 @@ sub _find_extent { }; } +# Placeholder path() result for a zvol with no mapped iSCSI extent (#292). +# Never a usable device: the IQN segment is not a valid IQN, so anything that +# tries to open it fails at connect time rather than hitting the wrong LUN. +sub _unmapped_path { + my ($scfg, $volname) = @_; + return 'iscsi://' . _portal($scfg) . "/unmapped/$volname"; +} + # Signal TrueNAS to reload the iSCSI service so new extents are visible sub _reload_iscsi { my ($scfg) = @_; @@ -614,12 +622,19 @@ sub path { my ($class, $scfg, $volname, $storeid, $snapname) = @_; _resolve_token($storeid, $scfg); - my $ext = _find_extent($scfg, $volname); - die "Volume '$volname' has no iSCSI extent on $scfg->{truenas_host}. " - . "Was it created via this plugin?\n" unless $ext; + my ($vtype, undef, $ds_vmid) = $class->parse_volname($volname); - die "Volume '$volname' is not mapped to any iSCSI target.\n" - unless defined $ext->{target_id}; + # A zvol with no extent (or an extent with no target mapping) is an orphan + # from a failed alloc_image, manual cleanup or the v2 era. PVE's content + # delete (pvesm free, GUI Remove) and qm destroy call path() only for the + # owner/vtype, so dying here made such orphans undeletable (#292). Return + # a placeholder instead — activate_volume and qemu_blockdev_options still + # die on the same condition, so nothing can actually run against it. + my $ext = _find_extent($scfg, $volname); + unless ($ext && defined $ext->{target_id}) { + _log('warning', "path: $volname has no mapped iSCSI extent — returning placeholder path"); + return (_unmapped_path($scfg, $volname), $ds_vmid, $vtype); + } # Look up the target that owns this extent (per-VM or legacy shared) my $t = _api($scfg, 'GET', "/iscsi/target/id/$ext->{target_id}") // {}; @@ -627,7 +642,6 @@ sub path { my $portal = _portal($scfg); my $dev_path = "iscsi://$portal/$iqn/$ext->{lun_id}"; - my ($vtype, undef, $ds_vmid) = $class->parse_volname($volname); return ($dev_path, $ds_vmid, $vtype); } diff --git a/perl5/PVE/Storage/Custom/TrueNASMultipath.pm b/perl5/PVE/Storage/Custom/TrueNASMultipath.pm index d7da0b9..9dbfe4b 100644 --- a/perl5/PVE/Storage/Custom/TrueNASMultipath.pm +++ b/perl5/PVE/Storage/Custom/TrueNASMultipath.pm @@ -95,11 +95,17 @@ sub path { PVE::Storage::Custom::TrueNAS::_resolve_token($storeid, $scfg); - my $ext = PVE::Storage::Custom::TrueNAS::_find_extent($scfg, $volname); - die "Volume '$volname' has no iSCSI extent on $scfg->{truenas_host}.\n" unless $ext; - - my $wwid = _naa_to_wwid($ext->{naa}) - or die "Volume '$volname' has no NAA/WWID on $scfg->{truenas_host}.\n"; + # Orphaned zvol (no extent / no NAA): return a placeholder with the real + # owner and vtype so pvesm free / GUI Remove can still delete it (#292). + # activate_volume dies on the same condition, so it is never opened. + my $ext = PVE::Storage::Custom::TrueNAS::_find_extent($scfg, $volname); + my $wwid = $ext ? _naa_to_wwid($ext->{naa}) : undef; + unless ($wwid) { + PVE::Storage::Custom::TrueNAS::_log('warning', + "multipath path: $volname has no iSCSI extent/WWID — returning placeholder path"); + my ($vtype, undef, $ds_vmid) = $class->parse_volname($volname); + return ("/dev/mapper/truenas-unmapped-$volname", $ds_vmid, $vtype); + } return ("/dev/mapper/$wwid", $ext->{lun_id}, $storeid); } diff --git a/t/lib/MockTrueNAS.pm b/t/lib/MockTrueNAS.pm new file mode 100644 index 0000000..21de15a --- /dev/null +++ b/t/lib/MockTrueNAS.pm @@ -0,0 +1,74 @@ +package MockTrueNAS; + +# In-memory fake of the TrueNAS API for unit tests. Replaces +# PVE::Storage::Custom::TrueNAS::_api (and silences _log) so plugin entry +# points can be exercised without a TrueNAS host or a PVE node. +# +# Usage: +# my $api = MockTrueNAS->install( +# extents => [ { id => 1, name => 'vm-100-disk-0', naa => '0x6589cfc...' } ], +# targetextents => [ { id => 1, extent => 1, target => 7, lunid => 0 } ], +# targets => [ { id => 7, name => 'proxmox-vm-100' } ], +# datasets => { 'tank/vm/vm-100-disk-0' => { volsize => { parsed => 1024 } } }, +# ); +# ... call plugin ... +# $api->calls # arrayref of [method, path, data] + +use strict; +use warnings; + +use URI::Escape (); +use PVE::Storage::Custom::TrueNAS; + +sub scfg { + return { + truenas_host => 'truenas.test', + truenas_api_key => 'test-key', + truenas_pool => 'tank', + truenas_dataset => 'vm', + }; +} + +sub install { + my ($class, %db) = @_; + my $self = bless { db => \%db, calls => [] }, $class; + + no warnings 'redefine'; ## no critic (TestingAndDebugging::ProhibitNoWarnings) + *PVE::Storage::Custom::TrueNAS::_log = sub { push @{ $self->{log} }, [@_]; return; }; + *PVE::Storage::Custom::TrueNAS::_api = sub { return $self->_route(@_); }; + + return $self; +} + +sub calls { my ($self) = @_; return $self->{calls}; } +sub log { my ($self) = @_; return $self->{log} // []; } ## no critic (Subroutines::ProhibitBuiltinHomonyms) + +sub _route { + my ($self, $scfg, $method, $path, $data) = @_; + push @{ $self->{calls} }, [ $method, $path, $data ]; + my $db = $self->{db}; + + if ($method eq 'GET') { + return { basename => 'iqn.2005-10.org.freenas.ctl' } if $path eq '/iscsi/global'; + return $db->{extents} // [] if $path eq '/iscsi/extent'; + + if ($path =~ m{^/iscsi/targetextent\?(extent|target)=(\d+)$}) { + my ($field, $id) = ($1, $2); + return [ grep { $_->{$field} == $id } @{ $db->{targetextents} // [] } ]; + } + if ($path =~ m{^/iscsi/target/id/(\d+)$}) { + my ($t) = grep { $_->{id} == $1 } @{ $db->{targets} // [] }; + return $t; + } + if ($path =~ m{^/pool/dataset/id/(.+)$}) { + my $name = URI::Escape::uri_unescape($1); + return $db->{datasets}{$name}; + } + } + + return if $method eq 'DELETE' || $method eq 'POST'; + + die "MockTrueNAS: unhandled $method $path\n"; +} + +1; diff --git a/t/lib/PVE/Storage/Plugin.pm b/t/lib/PVE/Storage/Plugin.pm new file mode 100644 index 0000000..90c062a --- /dev/null +++ b/t/lib/PVE/Storage/Plugin.pm @@ -0,0 +1,18 @@ +package PVE::Storage::Plugin; + +# Minimal stand-in for PVE::Storage::Plugin so the TrueNAS plugins can be +# loaded and unit-tested off a Proxmox host. Only what the plugins call on +# the base class at load time (or in tested code paths) is provided. + +use strict; +use warnings; + +sub new { return bless {}, shift; } +sub register { return; } +sub lookup_types { return []; } +sub properties { return {}; } +sub options { return {}; } + +sub qemu_blockdev_options { return {}; } + +1; diff --git a/t/path.t b/t/path.t new file mode 100644 index 0000000..cbd6905 --- /dev/null +++ b/t/path.t @@ -0,0 +1,84 @@ +#!/usr/bin/perl +# path() contract tests for both plugins. PVE expects ($path, $owner_vmid, $vtype); +# callers like pvesm free, GUI Remove and qm destroy rely on the owner/vtype +# even for volumes that cannot be opened. + +use strict; +use warnings; + +use FindBin; +use lib "$FindBin::Bin/lib", "$FindBin::Bin/../perl5"; + +use Test::More; + +use MockTrueNAS; +use PVE::Storage::Custom::TrueNAS; +use PVE::Storage::Custom::TrueNASMultipath; + +my $TN = 'PVE::Storage::Custom::TrueNAS'; +my $MP = 'PVE::Storage::Custom::TrueNASMultipath'; + +my %mapped = ( + extents => [ { id => 3, name => 'vm-100-disk-0', naa => '0x6589cfc000000abc' } ], + targetextents => [ { id => 5, extent => 3, target => 7, lunid => 2 } ], + targets => [ { id => 7, name => 'proxmox-vm-100' } ], +); + +subtest 'TrueNAS: mapped volume returns iscsi:// path' => sub { + MockTrueNAS->install(%mapped); + my ($path, $owner, $vtype) = $TN->path(MockTrueNAS::scfg(), 'vm-100-disk-0', 'store'); + is($path, 'iscsi://truenas.test/iqn.2005-10.org.freenas.ctl:proxmox-vm-100/2', 'path'); + is($owner, 100, 'owner is the VMID'); + is($vtype, 'images', 'vtype'); +}; + +# #292: an orphaned zvol (no extent) must still report owner/vtype so PVE can delete it. +subtest 'TrueNAS: extent-less zvol returns placeholder, not die (#292)' => sub { + my $api = MockTrueNAS->install(extents => []); + my ($path, $owner, $vtype) = eval { $TN->path(MockTrueNAS::scfg(), 'vm-101-disk-1', 'store') }; + is($@, '', 'does not die'); + is($path, 'iscsi://truenas.test/unmapped/vm-101-disk-1', 'placeholder path'); + is($owner, 101, 'owner is the VMID'); + is($vtype, 'images', 'vtype'); + ok((grep { $_->[0] eq 'warning' } @{ $api->log }), 'logs a warning'); +}; + +subtest 'TrueNAS: extent with no target mapping returns placeholder (#292)' => sub { + MockTrueNAS->install( + extents => [ { id => 3, name => 'vm-102-disk-0' } ], + targetextents => [], + ); + my ($path, $owner) = eval { $TN->path(MockTrueNAS::scfg(), 'vm-102-disk-0', 'store') }; + is($@, '', 'does not die'); + like($path, qr{/unmapped/vm-102-disk-0$}, 'placeholder path'); + is($owner, 102, 'owner is the VMID'); +}; + +subtest 'TrueNAS: activate_volume still dies for extent-less zvol' => sub { + MockTrueNAS->install(extents => []); + eval { $TN->activate_volume('store', MockTrueNAS::scfg(), 'vm-101-disk-1') }; + like($@, qr/has no iSCSI extent/, 'activate_volume dies'); +}; + +subtest 'TrueNAS: qemu_blockdev_options still dies for extent-less zvol' => sub { + MockTrueNAS->install(extents => []); + eval { $TN->qemu_blockdev_options(MockTrueNAS::scfg(), 'store', 'vm-101-disk-1') }; + like($@, qr/has no iSCSI extent/, 'qemu_blockdev_options dies'); +}; + +subtest 'Multipath: extent-less zvol returns placeholder, not die (#292)' => sub { + MockTrueNAS->install(extents => []); + my ($path, $owner, $vtype) = eval { $MP->path(MockTrueNAS::scfg(), 'vm-990-disk-0', 'store') }; + is($@, '', 'does not die'); + is($path, '/dev/mapper/truenas-unmapped-vm-990-disk-0', 'placeholder path'); + is($owner, 990, 'owner is the VMID'); + is($vtype, 'images', 'vtype'); +}; + +subtest 'Multipath: activate_volume still dies for extent-less zvol' => sub { + MockTrueNAS->install(extents => []); + eval { $MP->activate_volume('store', MockTrueNAS::scfg(), 'vm-990-disk-0') }; + like($@, qr/has no iSCSI extent/, 'activate_volume dies'); +}; + +done_testing(); From e0c5f46dbfd83413f719c816e9cbfb36c391693b Mon Sep 17 00:00:00 2001 From: Kevin Adams Date: Mon, 12 Oct 2026 12:58:42 -0400 Subject: [PATCH 2/8] fix: return owner VMID and vtype from multipath path() (#291) qm destroy calls path() and skips any disk whose owner != vmid. The multipath plugin returned the LUN id (usually 0) as the owner and the storeid as the vtype, so every truenas-multipath disk was silently left behind on TrueNAS while the task reported OK. Also make _naa_to_wwid return undef for a missing NAA: it returned a truthy "3", which pointed path()/activate_volume at /dev/mapper/3 and bypassed the #292 placeholder. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 147b461ea86466f161b814a40c911a589f1b9a8e) --- README.md | 15 ++++++++++ perl5/PVE/Storage/Custom/TrueNASMultipath.pm | 12 ++++++-- t/path.t | 31 ++++++++++++++++++++ 3 files changed, 55 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 1f48809..61a940d 100644 --- a/README.md +++ b/README.md @@ -607,6 +607,21 @@ If you see iSCSI extents in TrueNAS that are not associated with any target, the perl -MPVE::Storage -e 'PVE::Storage::vdisk_free(PVE::Storage::config(), "STORAGE:vm-101-disk-0")' ``` +### `qm destroy` left multipath disks behind on TrueNAS + +**Symptom:** you destroy a VM whose disks are on a `truenas-multipath` storage. The task reports `OK`, but the zvols, extents and the `proxmox-vm-` target are still on TrueNAS. The journal has no `free_image` line for those disks. + +**Cause:** in multipath versions up to and including 4.0.1, `path()` returned the LUN id where Proxmox expects the owning VMID. `qm destroy` skips any disk whose owner doesn't match, so it skipped all of them without a warning. **Fixed in v4.0.2** ([#291](https://github.com/TheGrandWazoo/freenas-proxmox/issues/291)). The base `truenas` storage type was never affected. + +**Finding orphans:** list the storage's volumes and look for VMIDs that have no config: + +```bash +pvesm list # every vm--* volume on the storage +ls /etc/pve/nodes/*/qemu-server/.conf 2>/dev/null # no output → VM no longer exists +``` + +To free an orphan, use `pvesm free :`. On 4.0.1 and earlier, use the `vdisk_free` one-liner from the entry above. Freeing a VM's last disk also removes its now-empty per-VM target. + ### Filing a Bug Report Please use the [GitHub issue tracker](https://github.com/TheGrandWazoo/truenas-proxmox/issues) and include the [bug report template](.github/ISSUE_TEMPLATE/bug_report.md). Include relevant log lines from `syslog` (search for `FreeNAS::`). diff --git a/perl5/PVE/Storage/Custom/TrueNASMultipath.pm b/perl5/PVE/Storage/Custom/TrueNASMultipath.pm index 9dbfe4b..54c4184 100644 --- a/perl5/PVE/Storage/Custom/TrueNASMultipath.pm +++ b/perl5/PVE/Storage/Custom/TrueNASMultipath.pm @@ -57,9 +57,11 @@ sub _portals { } # Convert TrueNAS NAA string (e.g. "0x6589cfc...") to the dm-multipath WWID -# used in /dev/mapper/ (e.g. "36589cfc..."). +# used in /dev/mapper/ (e.g. "36589cfc..."). Returns undef for a missing/empty +# NAA — a bare "3" would be truthy and send callers to /dev/mapper/3. sub _naa_to_wwid { my ($naa) = @_; + return unless defined $naa && length $naa; $naa =~ s/^0x//i; return "3$naa"; } @@ -95,6 +97,11 @@ sub path { PVE::Storage::Custom::TrueNAS::_resolve_token($storeid, $scfg); + # PVE's contract is ($path, $owner_vmid, $vtype). qm destroy skips any + # disk whose owner != vmid, so returning the LUN id here silently + # orphaned every multipath disk on destroy (#291). + my ($vtype, undef, $ds_vmid) = $class->parse_volname($volname); + # Orphaned zvol (no extent / no NAA): return a placeholder with the real # owner and vtype so pvesm free / GUI Remove can still delete it (#292). # activate_volume dies on the same condition, so it is never opened. @@ -103,11 +110,10 @@ sub path { unless ($wwid) { PVE::Storage::Custom::TrueNAS::_log('warning', "multipath path: $volname has no iSCSI extent/WWID — returning placeholder path"); - my ($vtype, undef, $ds_vmid) = $class->parse_volname($volname); return ("/dev/mapper/truenas-unmapped-$volname", $ds_vmid, $vtype); } - return ("/dev/mapper/$wwid", $ext->{lun_id}, $storeid); + return ("/dev/mapper/$wwid", $ds_vmid, $vtype); } # ── qemu_blockdev_options — delegate to Plugin.pm (host_device for /dev/mapper) ─ diff --git a/t/path.t b/t/path.t index cbd6905..746aaa7 100644 --- a/t/path.t +++ b/t/path.t @@ -66,6 +66,26 @@ subtest 'TrueNAS: qemu_blockdev_options still dies for extent-less zvol' => sub like($@, qr/has no iSCSI extent/, 'qemu_blockdev_options dies'); }; +# #291: qm destroy compares the owner to the VMID; the LUN id here made it skip every disk. +subtest 'Multipath: mapped volume returns owner VMID and vtype (#291)' => sub { + MockTrueNAS->install(%mapped); + my ($path, $owner, $vtype) = $MP->path(MockTrueNAS::scfg(), 'vm-100-disk-0', 'store'); + is($path, '/dev/mapper/36589cfc000000abc', 'path is the dm-multipath device'); + is($owner, 100, 'owner is the VMID, not the LUN id'); + is($vtype, 'images', 'vtype, not the storeid'); +}; + +subtest 'Multipath: base image owner is the template VMID (#291)' => sub { + MockTrueNAS->install( + extents => [ { id => 4, name => 'base-9000-disk-0', naa => '0x6589cfc000000def' } ], + targetextents => [ { id => 6, extent => 4, target => 8, lunid => 0 } ], + targets => [ { id => 8, name => 'proxmox-vm-9000' } ], + ); + my (undef, $owner, $vtype) = $MP->path(MockTrueNAS::scfg(), 'base-9000-disk-0', 'store'); + is($owner, 9000, 'owner'); + is($vtype, 'images', 'vtype'); +}; + subtest 'Multipath: extent-less zvol returns placeholder, not die (#292)' => sub { MockTrueNAS->install(extents => []); my ($path, $owner, $vtype) = eval { $MP->path(MockTrueNAS::scfg(), 'vm-990-disk-0', 'store') }; @@ -75,6 +95,17 @@ subtest 'Multipath: extent-less zvol returns placeholder, not die (#292)' => sub is($vtype, 'images', 'vtype'); }; +subtest 'Multipath: extent with no NAA returns placeholder, not /dev/mapper/3' => sub { + MockTrueNAS->install( + extents => [ { id => 3, name => 'vm-100-disk-0', naa => undef } ], + targetextents => [ { id => 5, extent => 3, target => 7, lunid => 0 } ], + targets => [ { id => 7, name => 'proxmox-vm-100' } ], + ); + my ($path, $owner) = $MP->path(MockTrueNAS::scfg(), 'vm-100-disk-0', 'store'); + is($path, '/dev/mapper/truenas-unmapped-vm-100-disk-0', 'placeholder path'); + is($owner, 100, 'owner'); +}; + subtest 'Multipath: activate_volume still dies for extent-less zvol' => sub { MockTrueNAS->install(extents => []); eval { $MP->activate_volume('store', MockTrueNAS::scfg(), 'vm-990-disk-0') }; From 3d092029dbab00e073353fb0ea18339800734ed7 Mon Sep 17 00:00:00 2001 From: Kevin Adams Date: Mon, 12 Oct 2026 16:14:01 -0400 Subject: [PATCH 3/8] fix: accept vm-N-cloudinit / base-N-cloudinit in parse_volname (#294) alloc_image already creates cloud-init drives under PVE's standard name, but parse_volname only knew disk-N and state-* volumes, so every later operation on the drive - including qm destroy - died with "unable to parse TrueNAS volume name". base- variant covers templates. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit c45c3b0abf51d582c521eb26d2463a81de97f835) --- perl5/PVE/Storage/Custom/TrueNAS.pm | 6 +++++ t/volname.t | 42 +++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+) create mode 100644 t/volname.t diff --git a/perl5/PVE/Storage/Custom/TrueNAS.pm b/perl5/PVE/Storage/Custom/TrueNAS.pm index 871413e..3f42242 100644 --- a/perl5/PVE/Storage/Custom/TrueNAS.pm +++ b/perl5/PVE/Storage/Custom/TrueNAS.pm @@ -424,6 +424,12 @@ sub parse_volname { return ('images', $volname, $vmid, undef, undef, $isBase, 'raw'); } + # Cloud-init drive: vm--cloudinit, or base--cloudinit once the + # VM is converted to a template. alloc_image already creates these (#294). + if ($volname =~ /^(vm|base)-(\d+)-cloudinit$/) { + return ('images', $volname, $2, undef, undef, ($1 eq 'base' ? 1 : 0), 'raw'); + } + # RAM snapshot state volume: vm--state- # PVE allocates one of these per snapshot when "Include RAM" is selected. if ($volname =~ /^vm-(\d+)-state-.+$/) { diff --git a/t/volname.t b/t/volname.t new file mode 100644 index 0000000..63c7f5c --- /dev/null +++ b/t/volname.t @@ -0,0 +1,42 @@ +#!/usr/bin/perl +# parse_volname: every volume name the plugin (or PVE on its behalf) creates +# must parse, or later operations on it (destroy, resize, regenerate) fail. + +use strict; +use warnings; + +use FindBin; +use lib "$FindBin::Bin/lib", "$FindBin::Bin/../perl5"; + +use Test::More; + +use PVE::Storage::Custom::TrueNAS; + +my $TN = 'PVE::Storage::Custom::TrueNAS'; + +my @ok = ( + # volname vmid isBase + [ 'vm-100-disk-0', 100, 0 ], + [ 'base-9000-disk-1', 9000, 1 ], + [ 'vm-100-state-pre_upg', 100, 0 ], + [ 'vm-9100-cloudinit', 9100, 0 ], # #294 + [ 'base-9000-cloudinit', 9000, 1 ], # #294 +); + +for my $case (@ok) { + my ($volname, $vmid, $isBase) = @$case; + my @r = eval { $TN->parse_volname($volname) }; + is($@, '', "$volname parses"); + is($r[0], 'images', "$volname vtype"); + is($r[1], $volname, "$volname name"); + is($r[2], $vmid, "$volname owner"); + is($r[5], $isBase, "$volname isBase"); + is($r[6], 'raw', "$volname format"); +} + +for my $bad (qw(vm-100 vm-abc-disk-0 vm-100-cloudinit.qcow2 subvol-100-disk-0 ../vm-100-disk-0)) { + eval { $TN->parse_volname($bad) }; + like($@, qr/unable to parse/, "$bad rejected"); +} + +done_testing(); From 80468804995db89a789532c92cf88ada79acb6ad Mon Sep 17 00:00:00 2001 From: Kevin Adams Date: Mon, 12 Oct 2026 16:14:42 -0400 Subject: [PATCH 4/8] fix: return (size, format, used, parent) from volume_size_info in list context (#294) PVE's content API (and the bpg/proxmox Terraform provider through it) calls volume_size_info in list context and dies "no format" when the second value is missing, so every disk on a truenas storage failed to refresh. Mirrors ZFSPoolPlugin: used comes from usedbydataset, and a missing zvol now dies instead of reporting a size of 0. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 664dfe6a3e2327f94153f57a3bca2ef529ec4f4c) --- perl5/PVE/Storage/Custom/TrueNAS.pm | 14 ++++++-- t/size.t | 55 +++++++++++++++++++++++++++++ 2 files changed, 67 insertions(+), 2 deletions(-) create mode 100644 t/size.t diff --git a/perl5/PVE/Storage/Custom/TrueNAS.pm b/perl5/PVE/Storage/Custom/TrueNAS.pm index 3f42242..868f9cc 100644 --- a/perl5/PVE/Storage/Custom/TrueNAS.pm +++ b/perl5/PVE/Storage/Custom/TrueNAS.pm @@ -618,10 +618,20 @@ sub volume_size_info { my ($class, $scfg, $storeid, $volname, $timeout) = @_; _resolve_token($storeid, $scfg); + my (undef, undef, undef, $parent, undef, undef, $format) = $class->parse_volname($volname); + my $zvol = _zvol_prefix($scfg) . "/$volname"; my $enc = uri_escape($zvol, "^A-Za-z0-9\\-_.~"); - my $ds = _api($scfg, 'GET', "/pool/dataset/id/$enc") // {}; - return $ds->{volsize}{parsed} // 0; + my $ds = _api($scfg, 'GET', "/pool/dataset/id/$enc"); + die "volume_size_info: zvol '$zvol' not found on $scfg->{truenas_host}\n" + unless ref $ds eq 'HASH' && defined $ds->{volsize}{parsed}; + + my $size = int($ds->{volsize}{parsed}); + my $used = int($ds->{usedbydataset}{parsed} // $ds->{used}{parsed} // 0); + + # List context is (size, format, used, parent), as in ZFSPoolPlugin. A bare + # scalar here made the content API die with "no format" (#294). + return wantarray ? ($size, $format, $used, $parent) : $size; } sub path { diff --git a/t/size.t b/t/size.t new file mode 100644 index 0000000..2e947a8 --- /dev/null +++ b/t/size.t @@ -0,0 +1,55 @@ +#!/usr/bin/perl +# volume_size_info contract: scalar context → size; list context → +# (size, format, used, parent). PVE's content API dies "no format" otherwise. + +use strict; +use warnings; + +use FindBin; +use lib "$FindBin::Bin/lib", "$FindBin::Bin/../perl5"; + +use Test::More; + +use MockTrueNAS; +use PVE::Storage::Custom::TrueNAS; + +my $TN = 'PVE::Storage::Custom::TrueNAS'; + +MockTrueNAS->install(datasets => { + 'tank/vm/vm-100-disk-0' => { + volsize => { parsed => 34359738368 }, + usedbydataset => { parsed => 1073741824 }, + used => { parsed => 2147483648 }, + }, + 'tank/vm/vm-100-cloudinit' => { + volsize => { parsed => 4194304 }, + used => { parsed => 65536 }, + }, +}); + +subtest 'scalar context returns size' => sub { + my $size = $TN->volume_size_info(MockTrueNAS::scfg(), 'store', 'vm-100-disk-0'); + is($size, 34359738368, 'size'); +}; + +subtest 'list context returns (size, format, used, parent) (#294)' => sub { + my @r = $TN->volume_size_info(MockTrueNAS::scfg(), 'store', 'vm-100-disk-0'); + is(scalar @r, 4, 'four values'); + is($r[0], 34359738368, 'size'); + is($r[1], 'raw', 'format'); + is($r[2], 1073741824, 'used prefers usedbydataset'); + ok(!defined $r[3], 'no parent'); +}; + +subtest 'used falls back to used when usedbydataset is absent' => sub { + my (undef, $fmt, $used) = $TN->volume_size_info(MockTrueNAS::scfg(), 'store', 'vm-100-cloudinit'); + is($fmt, 'raw', 'cloud-init drive has a format'); + is($used, 65536, 'used'); +}; + +subtest 'missing zvol dies instead of reporting size 0' => sub { + eval { $TN->volume_size_info(MockTrueNAS::scfg(), 'store', 'vm-100-disk-9') }; + like($@, qr/not found/, 'dies'); +}; + +done_testing(); From d2c8dd21af8c9a251ce256e038e459536d57173a Mon Sep 17 00:00:00 2001 From: Kevin Adams Date: Mon, 12 Oct 2026 16:17:26 -0400 Subject: [PATCH 5/8] feat: implement volume_resize for truenas and truenas-multipath (#294) Without an override the base Plugin.pm resize dies "can't resize this image format", which blocked qm disk resize, the GUI and Terraform's cloud-image provisioning. Grows the zvol via pool/dataset update (1 MiB-aligned, integer volsize for SCALE's Pydantic). The WebSocket transport gains the PUT -> pool.dataset.update mapping it needs. Multipath additionally rescans the target's kernel sessions and runs multipathd resize map when the device is active on this node, since QMP block_resize on a host_device can't grow past the stale dm size. Backport note: release/3.x is REST-only, so the WebSocket PUT -> pool.dataset.update mapping from the original commit is dropped. (cherry picked from commit 312cfa8) Co-Authored-By: Claude Opus 5.5 --- README.md | 13 ++++ perl5/PVE/Storage/Custom/TrueNAS.pm | 27 +++++++ perl5/PVE/Storage/Custom/TrueNASMultipath.pm | 30 ++++++++ t/lib/MockTrueNAS.pm | 2 +- t/resize.t | 77 ++++++++++++++++++++ 5 files changed, 148 insertions(+), 1 deletion(-) create mode 100644 t/resize.t diff --git a/README.md b/README.md index 61a940d..5fcc68e 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,7 @@ This plugin is a native `PVE::Storage::Custom` type — Proxmox discovers it aut - Bearer Token (API key) authentication — no username/password, no SSH keys - Per-VM iSCSI targets (`proxmox-vm-`) — removing one disk never affects other VMs - Automatic rollback when operations fail (no dangling iSCSI extents) +- Disk resize, including on a running VM. Cloud-init drives are supported (v4.0.2+). - Supports TrueNAS CORE 13.x and TrueNAS SCALE 24.10–25.10 --- @@ -607,6 +608,18 @@ If you see iSCSI extents in TrueNAS that are not associated with any target, the perl -MPVE::Storage -e 'PVE::Storage::vdisk_free(PVE::Storage::config(), "STORAGE:vm-101-disk-0")' ``` +### Resize, disk info or cloud-init operations fail on 4.0.1 and earlier + +These errors come from plugin gaps that are **fixed in v4.0.2** ([#294](https://github.com/TheGrandWazoo/freenas-proxmox/issues/294)): + +| Error | Cause | Workaround before upgrading | +|---|---|---| +| `can't resize this image format` (`qm disk resize`, GUI Resize, Terraform) | No `volume_resize` | Grow the zvol in TrueNAS (Datasets → zvol → Edit → Size). Then run `qm rescan --vmid ` so Proxmox picks up the new size. | +| `volume_size_info on '' failed - no format` (content API, `bpg/proxmox` Terraform refresh) | `volume_size_info` returned only a size | None. Upgrade. | +| `unable to parse TrueNAS volume name 'vm--cloudinit'` (`qm destroy`, cloud-init regenerate) | `parse_volname` rejected cloud-init names | Remove the cloud-init drive from the VM. Then delete `vm--cloudinit` in TrueNAS along with its iSCSI extent (and the `proxmox-vm-` target if it's now empty). | + +Resizing only grows the disk. You still need to grow the partition and filesystem inside the guest yourself (cloud-init's `growpart` does this on next boot). + ### `qm destroy` left multipath disks behind on TrueNAS **Symptom:** you destroy a VM whose disks are on a `truenas-multipath` storage. The task reports `OK`, but the zvols, extents and the `proxmox-vm-` target are still on TrueNAS. The journal has no `free_image` line for those disks. diff --git a/perl5/PVE/Storage/Custom/TrueNAS.pm b/perl5/PVE/Storage/Custom/TrueNAS.pm index 868f9cc..24d9cf3 100644 --- a/perl5/PVE/Storage/Custom/TrueNAS.pm +++ b/perl5/PVE/Storage/Custom/TrueNAS.pm @@ -634,6 +634,33 @@ sub volume_size_info { return wantarray ? ($size, $format, $used, $parent) : $size; } +# Grows the zvol. Without this override the base Plugin.pm version dies with +# "can't resize this image format" (#294). For a running VM, qemu-server then +# issues QMP block_resize, which makes libiscsi re-read the LUN capacity — +# TrueNAS resizes the LUN with the zvol, so no iSCSI-side call is needed here. +sub volume_resize { + my ($class, $scfg, $storeid, $volname, $size, $running, $snapname) = @_; + _resolve_token($storeid, $scfg); + + die "resizing a snapshot is not supported\n" if $snapname; + $class->parse_volname($volname); # dies on names we don't own + + # Align to 1 MiB like ZFSPoolPlugin, so volsize is always a multiple of + # the zvol's volblocksize (TrueNAS rejects anything else). + my $new_kb = int($size / 1024); + $new_kb += (1024 - $new_kb % 1024) % 1024; + + my $zvol = _zvol_prefix($scfg) . "/$volname"; + my $enc = uri_escape($zvol, "^A-Za-z0-9\\-_.~"); + + _log('info', "volume_resize: $zvol to ${new_kb}K"); + + # int() for a fresh IV: SCALE 25.10 Pydantic rejects string-typed integers. + _api($scfg, 'PUT', "/pool/dataset/id/$enc", { volsize => int($new_kb * 1024) }); + + return $new_kb; +} + sub path { my ($class, $scfg, $volname, $storeid, $snapname) = @_; _resolve_token($storeid, $scfg); diff --git a/perl5/PVE/Storage/Custom/TrueNASMultipath.pm b/perl5/PVE/Storage/Custom/TrueNASMultipath.pm index 54c4184..e38d3dc 100644 --- a/perl5/PVE/Storage/Custom/TrueNASMultipath.pm +++ b/perl5/PVE/Storage/Custom/TrueNASMultipath.pm @@ -116,6 +116,36 @@ sub path { return ("/dev/mapper/$wwid", $ds_vmid, $vtype); } +# ── volume_resize — grow zvol, then refresh kernel view of the LUN ──────────── + +# The base class grows the zvol. If the volume is active on this node, the +# kernel iSCSI sessions and the dm map still report the old size, and +# qemu-server's QMP block_resize on the host_device would fail. Rescan every +# session to the target, then have multipathd resize the map. +sub volume_resize { + my ($class, $scfg, $storeid, $volname, $size, $running, $snapname) = @_; + + my $new_kb = $class->SUPER::volume_resize($scfg, $storeid, $volname, $size, $running, $snapname); + + my $ext = PVE::Storage::Custom::TrueNAS::_find_extent($scfg, $volname); + my $wwid = $ext ? _naa_to_wwid($ext->{naa}) : undef; + return $new_kb unless $wwid && -b "/dev/mapper/$wwid" && defined $ext->{target_id}; + + my $t = PVE::Storage::Custom::TrueNAS::_api($scfg, 'GET', + "/iscsi/target/id/$ext->{target_id}") // {}; + my $iqn = PVE::Storage::Custom::TrueNAS::_basename($scfg) . ":$t->{name}"; + + my ($rc, $out) = _iscsiadm('-m', 'node', '-T', $iqn, '-R'); + die "iscsiadm rescan of $iqn failed (rc=$rc): $out\n" if $rc; + + my $multipathd = -x '/usr/sbin/multipathd' ? '/usr/sbin/multipathd' : '/sbin/multipathd'; + system($multipathd, 'resize', 'map', $wwid) == 0 + or die "multipathd resize map $wwid failed (rc=" . ($? >> 8) . ")\n"; + + PVE::Storage::Custom::TrueNAS::_log('info', "multipath resize: $wwid rescanned and resized"); + return $new_kb; +} + # ── qemu_blockdev_options — delegate to Plugin.pm (host_device for /dev/mapper) ─ # TrueNAS.pm overrides qemu_blockdev_options to return an iscsi:// blockdev — diff --git a/t/lib/MockTrueNAS.pm b/t/lib/MockTrueNAS.pm index 21de15a..0e489a2 100644 --- a/t/lib/MockTrueNAS.pm +++ b/t/lib/MockTrueNAS.pm @@ -66,7 +66,7 @@ sub _route { } } - return if $method eq 'DELETE' || $method eq 'POST'; + return if $method eq 'DELETE' || $method eq 'POST' || $method eq 'PUT'; die "MockTrueNAS: unhandled $method $path\n"; } diff --git a/t/resize.t b/t/resize.t new file mode 100644 index 0000000..a411e17 --- /dev/null +++ b/t/resize.t @@ -0,0 +1,77 @@ +#!/usr/bin/perl +# volume_resize (#294): grows the zvol via PUT /pool/dataset/id/ with a +# 1 MiB-aligned integer volsize, returns the new size in KiB (ZFSPoolPlugin +# convention), and refuses snapshots / foreign volume names. + +use strict; +use warnings; + +use FindBin; +use lib "$FindBin::Bin/lib", "$FindBin::Bin/../perl5"; + +use Test::More; +use JSON (); + +use MockTrueNAS; +use PVE::Storage::Custom::TrueNAS; +use PVE::Storage::Custom::TrueNASMultipath; + +my $TN = 'PVE::Storage::Custom::TrueNAS'; +my $MP = 'PVE::Storage::Custom::TrueNASMultipath'; +my $GiB = 1024 ** 3; + +sub puts { my ($api) = @_; return grep { $_->[0] eq 'PUT' } @{ $api->calls }; } + +subtest 'grows zvol to the requested size' => sub { + my $api = MockTrueNAS->install(); + my $kb = $TN->volume_resize(MockTrueNAS::scfg(), 'store', 'vm-100-disk-0', 40 * $GiB, 0); + is($kb, 40 * 1024 * 1024, 'returns new size in KiB'); + + my @p = puts($api); + is(scalar @p, 1, 'one PUT'); + is($p[0][1], '/pool/dataset/id/tank%2Fvm%2Fvm-100-disk-0', 'targets the zvol'); + is($p[0][2]{volsize}, 40 * $GiB, 'volsize in bytes'); + is(JSON->new->canonical->encode($p[0][2]), '{"volsize":' . (40 * $GiB) . '}', + 'volsize encodes as a JSON integer, not a string'); +}; + +subtest 'rounds up to a 1 MiB boundary' => sub { + my $api = MockTrueNAS->install(); + my $kb = $TN->volume_resize(MockTrueNAS::scfg(), 'store', 'vm-100-disk-0', 10 * $GiB + 1536, 0); + is($kb, 10 * 1024 * 1024 + 1024, 'next whole MiB'); + is((puts($api))[0][2]{volsize} % (1024 * 1024), 0, 'volsize is MiB-aligned'); +}; + +subtest 'cloud-init drive can be resized' => sub { + my $api = MockTrueNAS->install(); + $TN->volume_resize(MockTrueNAS::scfg(), 'store', 'vm-100-cloudinit', 8 * 1024 * 1024, 0); + is(scalar(puts($api)), 1, 'PUT issued'); +}; + +subtest 'snapshot resize refused' => sub { + my $api = MockTrueNAS->install(); + eval { $TN->volume_resize(MockTrueNAS::scfg(), 'store', 'vm-100-disk-0', $GiB, 0, 'snap1') }; + like($@, qr/snapshot/, 'dies'); + is(scalar(puts($api)), 0, 'no API write'); +}; + +subtest 'foreign volume name refused before any API write' => sub { + my $api = MockTrueNAS->install(); + eval { $TN->volume_resize(MockTrueNAS::scfg(), 'store', '../other-zvol', $GiB, 0) }; + like($@, qr/unable to parse/, 'dies'); + is(scalar(puts($api)), 0, 'no API write'); +}; + +subtest 'multipath: inactive volume only grows the zvol' => sub { + my $api = MockTrueNAS->install( + extents => [ { id => 3, name => 'vm-100-disk-0', naa => '0x6589cfc000000abc' } ], + targetextents => [ { id => 5, extent => 3, target => 7, lunid => 0 } ], + targets => [ { id => 7, name => 'proxmox-vm-100' } ], + ); + # /dev/mapper/36589cfc000000abc does not exist here, so no rescan is attempted. + my $kb = $MP->volume_resize(MockTrueNAS::scfg(), 'store', 'vm-100-disk-0', 20 * $GiB, 0); + is($kb, 20 * 1024 * 1024, 'returns new size in KiB'); + is(scalar(puts($api)), 1, 'zvol PUT issued'); +}; + +done_testing(); From e31b9ad10f240ce7f2612c9d2df0bdafd7629937 Mon Sep 17 00:00:00 2001 From: Kevin Adams Date: Tue, 13 Oct 2026 11:53:27 -0400 Subject: [PATCH 6/8] fix: pin truenas-proxmox-multipath to the exact base package version (#299) TrueNASMultipath.pm calls the base plugin's private helpers, and from 4.0.2 it also calls SUPER::volume_resize. So it only works with the truenas-proxmox built from the same commit. `>= 3.1.0` let apt pair 4.0.2 multipath with a 3.2.5 or 4.0.1 base, which fails at runtime. `(= ${VERSION})` makes apt upgrade or hold the two together. A CI check fails the build if the template ever drifts. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit ccc4ae27e1980ca73aa3701e9971991ac4458d30) --- .github/workflows/build.yml | 8 ++++++++ README.md | 2 +- packaging/DEBIAN-multipath/control.j2 | 2 +- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index c58ba2c..95a731a 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -271,6 +271,14 @@ jobs: echo "==> Package info (truenas-proxmox-multipath):" dpkg-deb --info "${{ steps.vars.outputs.multipath_deb_file }}" + # Multipath calls the base plugin's private helpers and SUPER:: methods, + # so it must be pinned to the exact base version built alongside it (#299). + VERSION="${{ steps.vars.outputs.version }}" + DEPENDS="$(dpkg-deb -f "${{ steps.vars.outputs.multipath_deb_file }}" Depends)" + echo "==> multipath Depends: ${DEPENDS}" + echo "${DEPENDS}" | grep -qF "truenas-proxmox (= ${VERSION})" \ + || { echo "::error::multipath must depend on truenas-proxmox (= ${VERSION})"; exit 1; } + - name: Upload package artifacts uses: actions/upload-artifact@v4 with: diff --git a/README.md b/README.md index 5fcc68e..dfe0cee 100644 --- a/README.md +++ b/README.md @@ -358,7 +358,7 @@ sudo apt update && sudo apt install truenas-proxmox-multipath -This installs `open-iscsi` and `multipath-tools` as dependencies and registers the `truenas-multipath` storage type. See [Configuration → Multipath Storage](#multipath-storage) for the extra field it needs, and [docs/architecture.md](docs/architecture.md) for how it differs from the default single-path `iscsi://` model. +This installs `open-iscsi` and `multipath-tools` as dependencies and registers the `truenas-multipath` storage type. From v4.0.2 on, the package requires the **exact same version** of `truenas-proxmox`, so apt always upgrades (or holds) the two together ([#299](https://github.com/TheGrandWazoo/freenas-proxmox/issues/299)). See [Configuration → Multipath Storage](#multipath-storage) for the extra field it needs, and [docs/architecture.md](docs/architecture.md) for how it differs from the default single-path `iscsi://` model. --- diff --git a/packaging/DEBIAN-multipath/control.j2 b/packaging/DEBIAN-multipath/control.j2 index 8d65a77..81ce793 100644 --- a/packaging/DEBIAN-multipath/control.j2 +++ b/packaging/DEBIAN-multipath/control.j2 @@ -2,7 +2,7 @@ Package: truenas-proxmox-multipath Version: ${VERSION} Architecture: all Maintainer: KSA Technologies, LLC -Depends: truenas-proxmox (>= 3.1.0), open-iscsi, multipath-tools +Depends: truenas-proxmox (= ${VERSION}), open-iscsi, multipath-tools Section: perl Priority: optional Homepage: https://github.com/TheGrandWazoo/truenas-proxmox From 54709a34927bd30a67aa4257c168d558b9283a32 Mon Sep 17 00:00:00 2001 From: Kevin Adams Date: Tue, 13 Oct 2026 12:36:54 -0400 Subject: [PATCH 7/8] chore: release housekeeping for v3.2.6 The 4.0.2 fixes above are cherry-picked for v3 users who can't move to v4 yet. This adds the version bump, CHANGELOG and Debian changelog entries, and README version numbers retargeted to the 3.x line (3.2.5 and earlier / fixed in 3.2.6), so v3 readers aren't told to upgrade to a version on a track they don't use. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 33 +++++++++++++++++++++ README.md | 16 +++++----- packaging/DEBIAN-multipath/changelog.Debian | 11 +++++++ packaging/changelog.Debian | 12 ++++++++ perl5/PVE/Storage/Custom/TrueNAS.pm | 2 +- 5 files changed, 65 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9762688..ea5711e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,39 @@ See each [GitHub Release](https://github.com/TheGrandWazoo/truenas-proxmox/relea --- +## [3.2.6] — 2026-10-13 + +Backport of the v4.0.2 volume lifecycle fixes to the v3.x line (REST +transport only). `storage.cfg` is unchanged. It requires PVE 9, like every v3.1+ release. + +### Added +- **Disk resize** (`volume_resize`) for `truenas` and `truenas-multipath`, + including on running VMs (#294). The zvol is grown through + `PUT /pool/dataset/id/`, aligned to 1 MiB with an integer `volsize`. + Multipath also rescans the kernel sessions and runs `multipathd resize map`. +- Perl unit-test harness (`t/`, mock TrueNAS API), run by `prove` in the CI + lint job (#292). + +### Fixed +- Cloud-init drives: `parse_volname` accepts `vm-N-cloudinit` / + `base-N-cloudinit` (#294). +- `volume_size_info` in list context returns `(size, format, used, parent)`, + which fixes the content API and Terraform `no format` error (#294). +- `path()` returns a placeholder for zvols with no iSCSI extent, so orphans + can be freed through PVE again (#292). +- Multipath `path()` returns the owner VMID and vtype, so `qm destroy` no + longer silently leaves multipath disks on TrueNAS. `_naa_to_wwid(undef)` + now returns undef (#291). + +### Changed +- `truenas-proxmox-multipath` depends on the exact same `truenas-proxmox` + version (#299). + +### Not backported +- The WebSocket `PUT` → `pool.dataset.update` mapping, because v3.x has no + WebSocket transport. SCALE 25.04+ still accepts REST through 25.10. Move to + v4 before SCALE 26.x removes it. + ## [Unreleased] — v4.0.0 (Rivendell), in development on the `next` branch Not yet released — tracked here as work lands, per this project's rule to keep diff --git a/README.md b/README.md index dfe0cee..fec476b 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ This plugin is a native `PVE::Storage::Custom` type — Proxmox discovers it aut - Bearer Token (API key) authentication — no username/password, no SSH keys - Per-VM iSCSI targets (`proxmox-vm-`) — removing one disk never affects other VMs - Automatic rollback when operations fail (no dangling iSCSI extents) -- Disk resize, including on a running VM. Cloud-init drives are supported (v4.0.2+). +- Disk resize, including on a running VM. Cloud-init drives are supported (v3.2.6+ / v4.0.2+). - Supports TrueNAS CORE 13.x and TrueNAS SCALE 24.10–25.10 --- @@ -358,7 +358,7 @@ sudo apt update && sudo apt install truenas-proxmox-multipath -This installs `open-iscsi` and `multipath-tools` as dependencies and registers the `truenas-multipath` storage type. From v4.0.2 on, the package requires the **exact same version** of `truenas-proxmox`, so apt always upgrades (or holds) the two together ([#299](https://github.com/TheGrandWazoo/freenas-proxmox/issues/299)). See [Configuration → Multipath Storage](#multipath-storage) for the extra field it needs, and [docs/architecture.md](docs/architecture.md) for how it differs from the default single-path `iscsi://` model. +This installs `open-iscsi` and `multipath-tools` as dependencies and registers the `truenas-multipath` storage type. From v3.2.6 (and v4.0.2) on, the package requires the **exact same version** of `truenas-proxmox`, so apt always upgrades (or holds) the two together ([#299](https://github.com/TheGrandWazoo/freenas-proxmox/issues/299)). See [Configuration → Multipath Storage](#multipath-storage) for the extra field it needs, and [docs/architecture.md](docs/architecture.md) for how it differs from the default single-path `iscsi://` model. --- @@ -600,17 +600,17 @@ If you see iSCSI extents in TrueNAS that are not associated with any target, the **Symptom:** removing a disk fails with `Volume 'vm-101-disk-0' has no iSCSI extent on . Was it created via this plugin?`. The zvol still exists on TrueNAS, but its iSCSI extent is gone. Typical causes are a failed `alloc_image`, an extent deleted by hand, or a leftover from v2.x. -**Fixed in v4.0.2** ([#292](https://github.com/TheGrandWazoo/freenas-proxmox/issues/292)). `path()` now returns a placeholder for these orphans, so Proxmox can delete them normally. The task log shows `path: has no mapped iSCSI extent — returning placeholder path`. Starting a VM on such a disk still fails at `activate_volume`, as it should. +**Fixed in v3.2.6** (and v4.0.2) ([#292](https://github.com/TheGrandWazoo/freenas-proxmox/issues/292)). `path()` now returns a placeholder for these orphans, so Proxmox can delete them normally. The task log shows `path: has no mapped iSCSI extent — returning placeholder path`. Starting a VM on such a disk still fails at `activate_volume`, as it should. -**On v4.0.1 and earlier:** first confirm that no VM config references the disk (`grep -r '' /etc/pve/nodes/*/qemu-server/`). Then call `vdisk_free` directly, which skips the `path()` lookup: +**On v3.2.5 and earlier:** first confirm that no VM config references the disk (`grep -r '' /etc/pve/nodes/*/qemu-server/`). Then call `vdisk_free` directly, which skips the `path()` lookup: ```bash perl -MPVE::Storage -e 'PVE::Storage::vdisk_free(PVE::Storage::config(), "STORAGE:vm-101-disk-0")' ``` -### Resize, disk info or cloud-init operations fail on 4.0.1 and earlier +### Resize, disk info or cloud-init operations fail on 3.2.5 and earlier -These errors come from plugin gaps that are **fixed in v4.0.2** ([#294](https://github.com/TheGrandWazoo/freenas-proxmox/issues/294)): +These errors come from plugin gaps that are **fixed in v3.2.6** (and v4.0.2) ([#294](https://github.com/TheGrandWazoo/freenas-proxmox/issues/294)): | Error | Cause | Workaround before upgrading | |---|---|---| @@ -624,7 +624,7 @@ Resizing only grows the disk. You still need to grow the partition and filesyste **Symptom:** you destroy a VM whose disks are on a `truenas-multipath` storage. The task reports `OK`, but the zvols, extents and the `proxmox-vm-` target are still on TrueNAS. The journal has no `free_image` line for those disks. -**Cause:** in multipath versions up to and including 4.0.1, `path()` returned the LUN id where Proxmox expects the owning VMID. `qm destroy` skips any disk whose owner doesn't match, so it skipped all of them without a warning. **Fixed in v4.0.2** ([#291](https://github.com/TheGrandWazoo/freenas-proxmox/issues/291)). The base `truenas` storage type was never affected. +**Cause:** in multipath versions up to and including 3.2.5 (and 4.0.1), `path()` returned the LUN id where Proxmox expects the owning VMID. `qm destroy` skips any disk whose owner doesn't match, so it skipped all of them without a warning. **Fixed in v3.2.6** (and v4.0.2) ([#291](https://github.com/TheGrandWazoo/freenas-proxmox/issues/291)). The base `truenas` storage type was never affected. **Finding orphans:** list the storage's volumes and look for VMIDs that have no config: @@ -633,7 +633,7 @@ pvesm list # every vm--* volume ls /etc/pve/nodes/*/qemu-server/.conf 2>/dev/null # no output → VM no longer exists ``` -To free an orphan, use `pvesm free :`. On 4.0.1 and earlier, use the `vdisk_free` one-liner from the entry above. Freeing a VM's last disk also removes its now-empty per-VM target. +To free an orphan, use `pvesm free :`. On 3.2.5 and earlier, use the `vdisk_free` one-liner from the entry above. Freeing a VM's last disk also removes its now-empty per-VM target. ### Filing a Bug Report diff --git a/packaging/DEBIAN-multipath/changelog.Debian b/packaging/DEBIAN-multipath/changelog.Debian index 7b202ec..ffc997a 100644 --- a/packaging/DEBIAN-multipath/changelog.Debian +++ b/packaging/DEBIAN-multipath/changelog.Debian @@ -1,3 +1,14 @@ +truenas-proxmox-multipath (3.2.6-1) trixie; urgency=medium + + * volume_resize: rescan kernel iSCSI sessions and multipathd resize map + so running VMs see the grown disk (#294) + * path() returns the owner VMID and vtype; qm destroy no longer silently + orphans multipath disks (#291) + * _naa_to_wwid(undef) returns undef instead of "3" (#291) + * Depend on the exact same truenas-proxmox version (#299) + + -- KSA Technologies, LLC Tue, 13 Oct 2026 00:00:00 +0000 + truenas-proxmox-multipath (3.2.2-1) bookworm; urgency=low * Version bump to match truenas-proxmox 3.2.2; api() fix inherited from TrueNAS.pm diff --git a/packaging/changelog.Debian b/packaging/changelog.Debian index 98ad8cb..dd326b7 100644 --- a/packaging/changelog.Debian +++ b/packaging/changelog.Debian @@ -1,3 +1,15 @@ +truenas-proxmox (3.2.6-1) trixie; urgency=medium + + * Backport of v4.0.2 volume lifecycle fixes (REST transport only) + * Add volume_resize so qm disk resize works, including on running VMs (#294) + * parse_volname accepts vm-N-cloudinit / base-N-cloudinit (#294) + * volume_size_info returns (size, format, used, parent) in list context; + fixes content API "no format" (#294) + * path() returns a placeholder for extent-less zvols so orphans can be + freed through PVE (#292) + + -- KSA Technologies, LLC Tue, 13 Oct 2026 00:00:00 +0000 + truenas-proxmox (3.2.4-1) bookworm; urgency=medium * Fix postinst/postrm to also restart pvestatd, pve-ha-lrm, pve-ha-crm on diff --git a/perl5/PVE/Storage/Custom/TrueNAS.pm b/perl5/PVE/Storage/Custom/TrueNAS.pm index 24d9cf3..b416568 100644 --- a/perl5/PVE/Storage/Custom/TrueNAS.pm +++ b/perl5/PVE/Storage/Custom/TrueNAS.pm @@ -29,7 +29,7 @@ use PVE::Storage::Plugin; use base qw(PVE::Storage::Plugin); -our $VERSION = '3.2.5'; +our $VERSION = '3.2.6'; # Per-host runtime state cache my $state = {}; From e3add6802b853de59fcd46a678d872e21989b827 Mon Sep 17 00:00:00 2001 From: Kevin Adams Date: Tue, 13 Oct 2026 16:27:15 -0400 Subject: [PATCH 8/8] docs: document v3 + SCALE online cloud-init update race (3.2.6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit During the 3.2.6 live test, `qm cloudinit update` failed on a running VM on all three SCALE storages, with "Tray of the device is not open". It passed on CORE and on v4.0.2 (even 1 s after VM start), and succeeded on retry. Timing the REST call showed why: SCALE answers in 0.8–3.1 s, compared with 0.09 s on CORE. That stretches PVE's open-tray → attach → insert sequence long enough for the guest to close the tray. It isn't a backport defect (3.2.5 couldn't do cloud-init at all). It's the slow deprecated REST API that v4's WebSocket transport replaces. This adds a README troubleshooting entry with workarounds, and the CHANGELOG known-limitation and verification notes. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 19 +++++++++++++++++++ README.md | 15 +++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index ea5711e..c3a07c0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -41,6 +41,25 @@ transport only). `storage.cfg` is unchanged. It requires PVE 9, like every v3.1+ WebSocket transport. SCALE 25.04+ still accepts REST through 25.10. Move to v4 before SCALE 26.x removes it. +### Known limitation +- On TrueNAS SCALE, `qm cloudinit update` on a **running** VM can fail with + `blockdev-insert-medium ... Tray of the device is not open`. SCALE's REST + API is slow (about 0.8 s per call on 25.04 and 3.1 s on 25.10, compared with + 0.09 s on CORE). The guest closes the virtual tray before PVE can insert the + regenerated disc. Retrying, or regenerating while the VM is stopped, works. + v4.x over WebSocket does not hit this. See the README troubleshooting entry. + +### Verified (live, PVE 9.2.21, pve01, 3.2.6 dev build) +- Full cycle on CORE 13.0-U6.8, SCALE 25.10.6, SCALE 25.04.2.6 and multipath + on 25.04.2.6, all over REST: + - create with a cloud-init drive; + - resize +1G while stopped; + - start; + - resize +1G while running (QEMU sees 3 GiB); + - `qm destroy --purge`, leaving no orphans and no iSCSI sessions. +- Online `qm cloudinit update` passed on CORE. On the three SCALE storages it + hit the known limitation above, and succeeded on retry. + ## [Unreleased] — v4.0.0 (Rivendell), in development on the `next` branch Not yet released — tracked here as work lands, per this project's rule to keep diff --git a/README.md b/README.md index fec476b..7559ddb 100644 --- a/README.md +++ b/README.md @@ -620,6 +620,21 @@ These errors come from plugin gaps that are **fixed in v3.2.6** (and v4.0.2) ([# Resizing only grows the disk. You still need to grow the partition and filesystem inside the guest yourself (cloud-init's `growpart` does this on next boot). +### `qm cloudinit update` on a running VM fails with "Tray of the device is not open" (v3.x + TrueNAS SCALE) + +**Symptom:** `qm cloudinit update ` (or **Cloud-Init → Regenerate Image** in the GUI) on a **running** VM fails with: + +``` +VM qmp command 'blockdev-insert-medium' failed - Tray of the device is not open +``` + +**Cause:** to swap the cloud-init CD in a running VM, PVE opens the virtual tray, removes the old disc, asks the plugin for the new disc's iSCSI path, and inserts it. v3.x reaches TrueNAS SCALE over the REST API, which SCALE 25.x has deprecated and serves slowly. Measured from a PVE node, `GET /iscsi/extent` took about 0.8 s on SCALE 25.04 and about 3.1 s on 25.10, compared with 0.09 s on CORE 13. The plugin makes about six such calls while PVE is attaching the new disc. That's long enough for the guest to close the tray before the insert. TrueNAS CORE is fast enough not to hit this, and so is v4.x's WebSocket transport for SCALE 25.04+. + +**Workarounds:** +- Run the command again. The second attempt usually succeeds. +- Or regenerate while the VM is stopped. PVE also regenerates the cloud-init drive automatically at every VM start. +- Or move to v4.x on PVE 9 ([upgrade path](docs/upgrade-paths.md)), which uses the much faster WebSocket API for SCALE 25.04+. + ### `qm destroy` left multipath disks behind on TrueNAS **Symptom:** you destroy a VM whose disks are on a `truenas-multipath` storage. The task reports `OK`, but the zvols, extents and the `proxmox-vm-` target are still on TrueNAS. The journal has no `free_image` line for those disks.