Merge pull request #173 from TheGrandWazoo/develop

Develop

README.md

@@ -1,31 +1,59 @@
-# TrueNAS ZFS over iSCSI interface  [![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TCLNEMBUYQUXN&source=url)
+# TrueNAS ZFS over iSCSI Plugin for Proxmox VE
 
-## :rotating_light: ATTENTION 2023-08-09 :rotating_light: :construction: New repo coming soon :construction: Thanks for your patience.
+## 📢: ATTENTION 2023-08-16 📢: New repos are now online at [Cloudsmith](#new-installs).
 
-### Updates 2023-02-12<br/>  - Added `systemctl restart pvescheduler.service` command to the package.
-#### Roadmap
-* Fix automated builds.
-  * Production - 'main' repo component.
-* Package the patches with the deb package.
-  * Remove the need for the git dependency.
-* Change to LWP::UserAgent
-  * Remove depenancy of the REST::Client because LWP::UserAgent is already installed and used by Proxmox VE.
-* Change from FreeNAS to TrueNAS.
-  * Cleanup the FreeNAS repo and name everything to TrueNAS to be inline with the product.
-* Add API key for direct TrueNAS services.
-  * Will be a new enable field and API key and will only be used by the plugin.
-  * You will still need the SSH keys, username, and password because of Proxmox VE using `iscsiadm` to get the list of disks.
-    * This is tricky because the format needs to be that of the output of 'zfs list' which is not part of the LunCmd but that of the backend Proxmox VE system and the API's do a bunch of JSON stuff.
+## Activity
 
-## Thank you for all that have recently donated to the project - Updated 2022-06-04
+<details>
+ <summary>Expand to see the activity tree</summary>
+
+ <blockquote>
+  
+ <details>
+  <summary>2023-08-18</summary>
+
+  - Update and cleanup the README.md
+
+  </details>
+
+  <details><summary>2023-08-16</summary>
+   
+  - Fixed repos. https://github.com/TheGrandWazoo/freenas-proxmox/issues/151, https://github.com/TheGrandWazoo/freenas-proxmox/issues/152, https://github.com/TheGrandWazoo/freenas-proxmox/issues/153 See [New Installs](#new-installs).
+  - Fixed PayPal issues. https://github.com/TheGrandWazoo/freenas-proxmox/issues/154
+  - Updated README.md
+
+  </details>
+
+  <details><summary>2023-08-12</summary>
+   
+  - Fixed postinst issue with Windows-based EOL. https://github.com/TheGrandWazoo/freenas-proxmox/issues/149
+
+  </details>
+
+  <details><summary>2023-02-12</summary>
+   
+  - Added `systemctl restart pvescheduler.service` command to the package based on https://github.com/TheGrandWazoo/freenas-proxmox/issues/109#issuecomment-1367527917
+
+  </details>
+
+  </blockquote>
+</details>
+
+## Donations [![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TCLNEMBUYQUXN&source=url)
+
+<details>Donators<summary>Thank you for all that have donated to the project - Updated 2023-08-18</summary>
+ 
     Alexander Finkhäuser - Recurring
     Bjarte Kvamme - Recurring
     Jonathan Schober - Recurring
-    Frederic Silvi
-    Security Camera
-    Vincent Cui
-    Jakub Jochec
+    Carlos Galvez from Security Camera
+    Sebastian Fischer
+    Eugene van der Merwe
+    Martin Gonzalez
     
+    Jakub Jochec
+    Frederic Silvi
+    Vincent Cui
     Mark Komarinski
     Jesse Bryan
     Maksym Vasylenko
@@ -37,44 +65,156 @@
     Marc Hodler
     Martin Gonzalez
 
-### 'main' repo (Follows a release branch - Current 2.x) Currently unavailable.
-Will be production ready code that has been tested (as best as possible) from the 'testing' repo.
+</details>
 
-### 'testing' repo (Follows the master branch)
-Will be 'beta' code for features, bugs, and updates.
+Their donations have allowed for:
+- A 4 Node Proxmox VE Cluster for testing and development.
+  - Spin up old and new revisions of FreeNAS and TrueNAS.
+- 10Gb Ethernet Testing.
+- Multihomed configuration testing.
+  - In progress and as best I can in a flat network.
 
-## New Installs.
-Issue the following from a command line:
-```bash
-curl https://ksatechnologies.jfrog.io/artifactory/ksa-repo-gpg/ksatechnologies-release.gpg -o /etc/apt/trusted.gpg.d/ksatechnologies-release.gpg
-curl https://ksatechnologies.jfrog.io/artifactory/ksa-repo-gpg/ksatechnologies-repo.list -o /etc/apt/sources.list.d/ksatechnologies-repo.list
-```
+## Roadmap
+<details><summary>Roadmap details</summary>
 
-Then issue the following to install the package
-```bash
-apt update
-apt install freenas-proxmox
-```
+* Update the documentation - <i>In Progress</i>.
+  * Restructure the main README.md for better readability. 
+  * Add some screenshots.
+* Fix Max Lun Limit issue.
+  * https://github.com/TheGrandWazoo/freenas-proxmox/issues/150
+* Fix automated builds - <i>In Progress</i>.
+  * Production - 'main' repo component.
+* Autoinstall the SSH keys.
+  * Tech spike to see if it is even doable.
+* Hashicorp Vault integration.
+  * Pull in secrets from a Hashicorp Vault service.  
+  * Tech spike to see if it is even doable.
+* Package the patches with the deb package.
+  * Remove the need for git dependency.
+* Change to LWP::UserAgent
+  * Remove dependency of the REST::Client because LWP::UserAgent is already installed and used by Proxmox VE.
+* Change from FreeNAS to TrueNAS - <i>In Progress</i>.
+  * Cleanup the FreeNAS repo and name everything to TrueNAS to be inline with the product.
+* Add API key for direct TrueNAS services - <i>In Progress</i>.
+  * Will be a new enable field and API key and will only be used by the plugin.
+  * You will still need the SSH keys, username, and password because of Proxmox VE using `iscsiadm` to get the list of disks.
+    * This is tricky because the format needs to be that of the output of 'zfs list' which is not part of the LunCmd but that of the backend Proxmox VE system and the API's do a bunch of JSON stuff.
 
-Before freenas-proxmox-2.2.0-0-beta8 please issue the following:
-```
-systemctl restart pvescheduler.service
-```
-due to this post https://github.com/TheGrandWazoo/freenas-proxmox/issues/109#issuecomment-1367527917
+</details>
 
-Then just do your regular upgrade via apt at the command line or the Proxmox Update subsystem; the package will automatically issue all commands to patch the files.
-```bash
-apt update
-apt [full|dist]-upgrade
-```
+## New Install Instructions
 
-If you wish not to use the package you may remove it at anytime with
-```
-apt [remove|purge] freenas-proxmox
-```
-This will place you back to a normal and non-patched Proxmox VE install.
+### Select at least one `Step 1.x` based on your preference. Can be combined.
 
-#### NOTE: Please be aware that this plugin uses the TrueNAS APIs but still uses SSH keys due to the underlying Proxmox VE perl modules that use the ```iscsiadm``` command.
+<details><summary>Step 1.0: For stable releases. <b>Enabled</b> by default.</summary>
+
+ ### truenas-proxmox repo - Currently follows the 2.0 branch.
+
+ Select one of the following GPG Key locations based on your preference.
+
+ ```bash
+ # Preferred - based on documentation. Copy and paste to bash command line:
+ keyring_location=/usr/share/keyrings/ksatechnologies-truenas-proxmox-keyring.gpg
+ ```
+
+ ```bash
+ # Alternative - If you wish to continue with the old ways.  Copy and paste to bash command line:
+ keyring_location=/etc/apt/trusted.gpg.d/ksatechnologies-truenas-proxmox.gpg
+ ```
+
+ Copy and paste to bash command line to load the GPG key to the location selected above:
+ ```bash
+ curl -1sLf 'https://dl.cloudsmith.io/public/ksatechnologies/truenas-proxmox/gpg.284C106104A8CE6D.key' |  gpg --dearmor >> ${keyring_location}
+ ```
+
+ Copy and paste the following code to bash command line to create '/etc/apt/sources.list.d/ksatechnologies-repo.list'
+ ```bash
+ cat << EOF > /etc/apt/sources.list.d/ksatechnologies-repo.list
+ # Source: KSATechnologies
+ # Site: https://cloudsmith.io
+ # Repository: KSATechnologies / truenas-proxmox
+ # Description: TrueNAS plugin for Proxmox VE - Production
+ deb [signed-by=${keyring_location}] https://dl.cloudsmith.io/public/ksatechnologies/truenas-proxmox/deb/debian any-version main
+
+ EOF
+ ```
+
+</details>
+
+<details><summary>Step 1.1: For development releases. <i>Disabled</i> by default.</summary>
+
+ ### truenas-proxmox-testing repo - Follows the master branch and you wish to test before a stable release (beta).
+ 
+ Select one of the following GPG Key locations based on your preference.
+
+ ```bash
+ # Preferred - based on documentation. Copy and paste to bash command line:
+ keyring_location=/usr/share/keyrings/ksatechnologies-truenas-proxmox-testing-keyring.gpg
+ ```
+
+ ```bash
+ # Alternative - If you wish to continue with the old ways.  Copy and paste to bash command line:
+ keyring_location=/etc/apt/trusted.gpg.d/ksatechnologies-truenas-proxmox-testing.gpg
+ ```
+
+ Copy and paste to bash command line to load the GPG key to the location selected above:
+ ```bash
+ curl -1sLf 'https://dl.cloudsmith.io/public/ksatechnologies/truenas-proxmox-testing/gpg.CACC9EE03F2DFFCC.key' |  gpg --dearmor >> ${keyring_location}
+ ```
+
+ Copy and paste the following code to bash command line to create '/etc/apt/sources.list.d/ksatechnologies-testing-repo.list'
+ ```bash
+ cat << EOF > /etc/apt/sources.list.d/ksatechnologies-testing-repo.list
+ # Source: KSATechnologies
+ # Site: https://cloudsmith.io
+ # Repository: KSATechnologies / truenas-proxmox-testing
+ # Description: TrueNAS plugin for Proxmox VE - Testing
+ deb [signed-by=${keyring_location}] https://dl.cloudsmith.io/public/ksatechnologies/truenas-proxmox-testing/deb/debian any-version main
+
+ EOF
+ ```
+
+</details>
+
+<details><summary>Step 2.0: Next step after completing any combination of the 1.x steps</summary>
+
+ ### Update apt
+
+ Then issue the following to install the package
+ ```bash
+ apt update
+ apt install freenas-proxmox
+ ```
+
+ </details>
+
+ <details><summary>Step 3.0: Maintenance.</summary>
+
+  Then just do your regular upgrade via apt at the command line or the Proxmox Update subsystem; the package will automatically issue all commands to patch the files.
+  ```bash
+  apt update
+  apt [full|dist]-upgrade
+  ```
+
+ </details>
+
+</details>
+
+## Uninstall truenas-proxmox
+
+<details><summary>If you wish not to use the package you may remove it at anytime with the following:</summary>
+
+ ```
+  apt [remove|purge] freenas-proxmox
+ ```
+
+ This will place you back to a normal and non-patched Proxmox VE install.
+ 
+</details>
+
+## Notes:
+
+### Please be aware that this plugin uses the TrueNAS APIs but still uses SSH keys due to the underlying Proxmox VE perl modules that use the ```iscsiadm``` command.
 
 You will still need to configure the SSH connector for listing the ZFS Pools because this is currently being done in a Proxmox module (ZFSPoolPlugin.pm). To configure this please follow the steps at https://pve.proxmox.com/wiki/Storage:_ZFS_over_iSCSI that have to do with SSH between Proxmox VE and TrueNAS. The code segment should start out `mkdir /etc/pve/priv/zfs`.
 

perl5/PVE/Storage/LunCmd/FreeNAS.pm

@@ -128,10 +128,13 @@ sub run_lun_command {
 
     syslog("info",(caller(0))[3] . " : $method(@params)");
 
-    if(!defined($scfg->{'freenas_user'}) || !defined($scfg->{'freenas_password'})) {
-        die "Undefined freenas_user and/or freenas_password.";
+    if (defined($scfg->{'truenas_token_auth'}) && $scfg->{'truenas_token_auth'}) {
+        if (!defined($scfg->{'truenas_secret'})) {
+            die "Undefined `truenas_secret` variable.";
+        }
+    } elsif (!defined($scfg->{'freenas_user'}) || !defined($scfg->{'freenas_password'})) {
+        die "Undefined `freenas_user` and/or `freenas_password` variables.";
     }
-
     if (!defined $freenas_server_list->{defined($scfg->{freenas_apiv4_host}) ? $scfg->{freenas_apiv4_host} : $scfg->{portal}}) {
         freenas_api_check($scfg);
     }
@@ -341,7 +344,13 @@ sub freenas_api_connect {
     }
     $freenas_server_list->{$apihost}->setHost($scheme . '://' . $apihost);
     $freenas_server_list->{$apihost}->addHeader('Content-Type', 'application/json');
-    $freenas_server_list->{$apihost}->addHeader('Authorization', 'Basic ' . encode_base64($scfg->{freenas_user} . ':' . $scfg->{freenas_password}));
+    if (defined($scfg->{'truenas_token_auth'})) {
+        syslog("info", (caller(0))[3] . " : Authentication using Bearer Token Auth");
+        $freenas_server_list->{$apihost}->addHeader('Authorization', 'Bearer ' . $scfg->{truenas_secret});
+    } else {
+        syslog("info", (caller(0))[3] . " : Authentication using Basic Auth");
+        $freenas_server_list->{$apihost}->addHeader('Authorization', 'Basic ' . encode_base64($scfg->{freenas_user} . ':' . $scfg->{freenas_password}));
+    }
     # If using SSL, don't verify SSL certs
     if ($scfg->{freenas_use_ssl}) {
         $freenas_server_list->{$apihost}->getUseragent()->ssl_opts(verify_hostname => 0);

perl5/PVE/Storage/ZFSPlugin-8.1.3_1.pm.patch

@@ -0,0 +1,157 @@
+--- ZFSPlugin.pm.orig	2023-12-31 09:56:18.895228853 -0500
++++ ZFSPlugin.pm	2023-12-31 09:57:08.830488875 -0500
+@@ -10,6 +10,7 @@
+ 
+ use base qw(PVE::Storage::ZFSPoolPlugin);
+ use PVE::Storage::LunCmd::Comstar;
++use PVE::Storage::LunCmd::FreeNAS;
+ use PVE::Storage::LunCmd::Istgt;
+ use PVE::Storage::LunCmd::Iet;
+ use PVE::Storage::LunCmd::LIO;
+@@ -26,13 +27,14 @@
+     modify_lu   => 1,
+     add_view    => 1,
+     list_view   => 1,
++    list_extent => 1,
+     list_lu     => 1,
+ };
+ 
+ my $zfs_unknown_scsi_provider = sub {
+     my ($provider) = @_;
+ 
+-    die "$provider: unknown iscsi provider. Available [comstar, istgt, iet, LIO]";
++    die "$provider: unknown iscsi provider. Available [comstar, freenas, istgt, iet, LIO]";
+ };
+ 
+ my $zfs_get_base = sub {
+@@ -40,6 +42,8 @@
+ 
+     if ($scfg->{iscsiprovider} eq 'comstar') {
+         return PVE::Storage::LunCmd::Comstar::get_base;
++    } elsif ($scfg->{iscsiprovider} eq 'freenas') {
++        return PVE::Storage::LunCmd::FreeNAS::get_base;
+     } elsif ($scfg->{iscsiprovider} eq 'istgt') {
+         return PVE::Storage::LunCmd::Istgt::get_base;
+     } elsif ($scfg->{iscsiprovider} eq 'iet') {
+@@ -62,6 +66,8 @@
+     if ($lun_cmds->{$method}) {
+         if ($scfg->{iscsiprovider} eq 'comstar') {
+             $msg = PVE::Storage::LunCmd::Comstar::run_lun_command($scfg, $timeout, $method, @params);
++        } elsif ($scfg->{iscsiprovider} eq 'freenas') {
++            $msg = PVE::Storage::LunCmd::FreeNAS::run_lun_command($scfg, $timeout, $method, @params);
+         } elsif ($scfg->{iscsiprovider} eq 'istgt') {
+             $msg = PVE::Storage::LunCmd::Istgt::run_lun_command($scfg, $timeout, $method, @params);
+         } elsif ($scfg->{iscsiprovider} eq 'iet') {
+@@ -166,6 +172,15 @@
+     die "lun_number for guid $guid is not a number";
+ }
+ 
++# Part of the multipath enhancement
++sub zfs_get_wwid_number {
++    my ($class, $scfg, $guid) = @_;
++
++    die "could not find lun_number for guid $guid" if !$guid;
++
++    return $class->zfs_request($scfg, undef, 'list_extent', $guid);
++}
++
+ # Configuration
+ 
+ sub type {
+@@ -184,6 +199,32 @@
+ 	    description => "iscsi provider",
+ 	    type => 'string',
+ 	},
++	# This is for FreeNAS iscsi and API intergration
++	# And some enhancements asked by the community
++	freenas_user => {
++	    description => "FreeNAS API Username",
++	    type => 'string',
++	},
++	freenas_password => {
++	    description => "FreeNAS API Password (Deprecated)",
++	    type => 'string',
++	},
++	truenas_secret => {
++	    description => "TrueNAS API Secret",
++	    type => 'string',
++	},
++	truenas_token_auth => {
++	    description => "TrueNAS API Authentication with Token",
++	    type => 'boolean',
++	},
++	freenas_use_ssl => {
++	    description => "FreeNAS API access via SSL",
++	    type => 'boolean',
++	},
++	freenas_apiv4_host => {
++	    description => "FreeNAS API Host",
++	    type => 'string',
++	},
+ 	# this will disable write caching on comstar and istgt.
+ 	# it is not implemented for iet. iet blockio always operates with
+ 	# writethrough caching when not in readonly mode
+@@ -211,14 +252,20 @@
+ 	nodes => { optional => 1 },
+ 	disable => { optional => 1 },
+ 	portal => { fixed => 1 },
+-	target => { fixed => 1 },
+-	pool => { fixed => 1 },
++	target => { fixed => 0 },
++	pool => { fixed => 0 },
+ 	blocksize => { fixed => 1 },
+ 	iscsiprovider => { fixed => 1 },
+ 	nowritecache => { optional => 1 },
+ 	sparse => { optional => 1 },
+ 	comstar_hg => { optional => 1 },
+ 	comstar_tg => { optional => 1 },
++	freenas_user => { optional => 1 },
++	freenas_password => { optional => 1 },
++	truenas_secret => { optional => 1 },
++	truenas_token_auth => { optional => 1 },
++	freenas_use_ssl => { optional => 1 },
++	freenas_apiv4_host => { optional => 1 },
+ 	lio_tpg => { optional => 1 },
+ 	content => { optional => 1 },
+ 	bwlimit => { optional => 1 },
+@@ -243,6 +290,40 @@
+ 
+     my $path = "iscsi://$portal/$target/$lun";
+ 
++    # Multipath enhancement
++    eval {
++	my $wwid = $class->zfs_get_wwid_number($scfg, $guid);
++#	syslog(info,"JD: path get_lun_number guid $guid");
++
++	if ($wwid =~ /^([-\@\w.]+)$/) {
++	    $wwid = $1;                     # $data now untainted
++	} else {
++	    die "Bad data in '$wwid'";      # log this somewhere
++	}
++	my $wwid_end = substr $wwid, 16;
++
++	my $mapper = '';
++	sleep 3;
++	run_command("iscsiadm -m session --rescan");
++	sleep 3;
++	my $line = `/usr/sbin/multipath -ll | grep \"$wwid_end\"`;
++	my ($mapper_device) = split(' ', $line);
++	$mapper_device = "" unless $mapper_device;
++	$mapper .= $mapper_device;
++
++	if ($mapper =~ /^([-\@\w.]+)$/) {
++	    $mapper = $1;                   # $data now untainted
++	} else {
++	    $mapper = '';
++	}
++
++#	syslog(info,"Multipath mapper found: $mapper\n");
++	if ($mapper ne "") {
++	    $path = "/dev/mapper/$mapper";
++	    sleep 5;
++	}
++    };
++
+     return ($path, $vmid, $vtype);
+ }
+ 

perl5/PVE/Storage/ZFSPlugin.pm.patch

@@ -1,5 +1,5 @@
---- ZFSPlugin.pm.orig	2022-02-04 12:08:01.000000000 -0500
-+++ ZFSPlugin.pm	2022-03-26 13:51:40.660068908 -0400
+--- ZFSPlugin.pm.orig	2023-12-31 09:56:18.895228853 -0500
++++ ZFSPlugin.pm	2023-12-31 09:57:08.830488875 -0500
 @@ -10,6 +10,7 @@
  
  use base qw(PVE::Storage::ZFSPoolPlugin);
@@ -58,7 +58,7 @@
  # Configuration
  
  sub type {
-@@ -184,6 +199,24 @@
+@@ -184,6 +199,32 @@
  	    description => "iscsi provider",
  	    type => 'string',
  	},
@@ -69,9 +69,17 @@
 +	    type => 'string',
 +	},
 +	freenas_password => {
-+	    description => "FreeNAS API Password",
++	    description => "FreeNAS API Password (Deprecated)",
 +	    type => 'string',
 +	},
++	truenas_secret => {
++	    description => "TrueNAS API Secret",
++	    type => 'string',
++	},
++	truenas_token_auth => {
++	    description => "TrueNAS API Authentication with Token",
++	    type => 'boolean',
++	},
 +	freenas_use_ssl => {
 +	    description => "FreeNAS API access via SSL",
 +	    type => 'boolean',
@@ -83,7 +91,7 @@
  	# this will disable write caching on comstar and istgt.
  	# it is not implemented for iet. iet blockio always operates with
  	# writethrough caching when not in readonly mode
-@@ -211,14 +244,18 @@
+@@ -211,14 +252,20 @@
  	nodes => { optional => 1 },
  	disable => { optional => 1 },
  	portal => { fixed => 1 },
@@ -99,12 +107,14 @@
  	comstar_tg => { optional => 1 },
 +	freenas_user => { optional => 1 },
 +	freenas_password => { optional => 1 },
++	truenas_secret => { optional => 1 },
++	truenas_token_auth => { optional => 1 },
 +	freenas_use_ssl => { optional => 1 },
 +	freenas_apiv4_host => { optional => 1 },
  	lio_tpg => { optional => 1 },
  	content => { optional => 1 },
  	bwlimit => { optional => 1 },
-@@ -243,6 +280,40 @@
+@@ -243,6 +290,40 @@
  
      my $path = "iscsi://$portal/$target/$lun";
  

pve-docs/api-viewer/apidoc-8.0.5_1.js.patch

@@ -0,0 +1,91 @@
+--- apidoc.js.orig	2024-01-06 13:02:06.730512378 -0500
++++ apidoc.js	2024-01-06 13:02:55.349787105 -0500
+@@ -50336,6 +50336,37 @@
+                            "type" : "string",
+                            "typetext" : "<string>"
+                         },
++                        "freenas_user" : {
++                           "description" : "FreeNAS user for API access",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
++                        "freenas_password" : {
++                           "description" : "FreeNAS password for API access (Deprecated)",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
++                        "truenas_secret" : {
++                           "description" : "TrueNAS Secret for API access",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
++                        "freenas_use_ssl" : {
++                           "description" : "FreeNAS API access via SSL",
++                           "optional" : 1,
++                           "type" : "boolean",
++                           "typetext" : "<boolean>"
++                        },
++                        "freenas_apiv4_host" : {
++                           "description" : "FreeNAS API Host via IPv4",
++                           "format" : "address",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
+                         "fuse" : {
+                            "description" : "Mount CephFS through FUSE.",
+                            "optional" : 1,
+@@ -50555,6 +50586,12 @@
+                            "type" : "boolean",
+                            "typetext" : "<boolean>"
+                         },
++                        "target" : {
++                           "description" : "iSCSI target.",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
+                         "transport" : {
+                            "description" : "Gluster transport: tcp or rdma",
+                            "enum" : [
+@@ -50854,6 +50891,37 @@
+                      "optional" : 1,
+                      "type" : "string",
+                      "typetext" : "<string>"
++                  },
++                  "freenas_user" : {
++                     "description" : "FreeNAS user for API access",
++                     "optional" : 1,
++                     "type" : "string",
++                     "typetext" : "<string>"
++                  },
++                  "freenas_password" : {
++                     "description" : "FreeNAS password for API access (Deprecated)",
++                     "optional" : 1,
++                     "type" : "string",
++                     "typetext" : "<string>"
++                  },
++                  "truenas_secret" : {
++                     "description" : "TrueNAS secret for API access",
++                     "optional" : 1,
++                     "type" : "string",
++                     "typetext" : "<string>"
++                  },
++                  "freenas_use_ssl" : {
++                     "description" : "FreeNAS API access via SSL",
++                     "optional" : 1,
++                     "type" : "boolean",
++                     "typetext" : "<boolean>"
++                  },
++                  "freenas_apiv4_host" : {
++                     "description" : "FreeNAS API Host via IPv4",
++                     "format" : "address",
++                     "optional" : 1,
++                     "type" : "string",
++                     "typetext" : "<string>"
+                   },
+                   "fuse" : {
+                      "description" : "Mount CephFS through FUSE.",

pve-docs/api-viewer/apidoc.js.patch

@@ -1,6 +1,6 @@
---- apidoc.js.orig	2021-11-15 10:07:34.000000000 -0500
-+++ apidoc.js	2021-12-06 08:04:01.648822707 -0500
-@@ -44064,6 +44064,31 @@
+--- apidoc.js.orig      2023-08-16 06:03:55.000000000 -0400
++++ apidoc.js   2023-12-26 14:45:47.202566775 -0500
+@@ -47579,6 +47579,37 @@
                             "type" : "string",
                             "typetext" : "<string>"
                          },
@@ -11,7 +11,13 @@
 +                           "typetext" : "<string>"
 +                        },
 +                        "freenas_password" : {
-+                           "description" : "FreeNAS password for API access",
++                           "description" : "FreeNAS password for API access (Deprecated)",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
++                        "truenas_secret" : {
++                           "description" : "TrueNAS Secret for API access",
 +                           "optional" : 1,
 +                           "type" : "string",
 +                           "typetext" : "<string>"
@@ -32,7 +38,7 @@
                          "fuse" : {
                             "description" : "Mount CephFS through FUSE.",
                             "optional" : 1,
-@@ -44275,6 +44300,12 @@
+@@ -47798,6 +47829,12 @@
                             "type" : "boolean",
                             "typetext" : "<boolean>"
                          },
@@ -45,7 +51,7 @@
                          "transport" : {
                             "description" : "Gluster transport: tcp or rdma",
                             "enum" : [
-@@ -44547,6 +44578,31 @@
+@@ -48097,6 +48134,37 @@
                       "optional" : 1,
                       "type" : "string",
                       "typetext" : "<string>"
@@ -57,7 +63,13 @@
 +                     "typetext" : "<string>"
 +                  },
 +                  "freenas_password" : {
-+                     "description" : "FreeNAS password for API access",
++                     "description" : "FreeNAS password for API access (Deprecated)",
++                     "optional" : 1,
++                     "type" : "string",
++                     "typetext" : "<string>"
++                  },
++                  "truenas_secret" : {
++                     "description" : "TrueNAS secret for API access",
 +                     "optional" : 1,
 +                     "type" : "string",
 +                     "typetext" : "<string>"

pve-manager/js/pvemanagerlib.js.patch

@@ -1,58 +1,162 @@
---- pvemanagerlib.js.orig	2022-03-17 09:08:40.000000000 -0400
-+++ pvemanagerlib.js	2022-04-03 08:54:10.229689187 -0400
-@@ -8068,6 +8068,7 @@
+--- pvemanagerlib.js.orig	2023-12-30 15:36:27.913505863 -0500
++++ pvemanagerlib.js	2024-01-02 09:30:56.000000000 -0500
+@@ -9228,6 +9228,7 @@
      alias: ['widget.pveiScsiProviderSelector'],
      comboItems: [
  	['comstar', 'Comstar'],
-+	['freenas', 'FreeNAS-API'],
++    ['freenas', 'FreeNAS/TrueNAS API'],
  	['istgt', 'istgt'],
  	['iet', 'IET'],
  	['LIO', 'LIO'],
-@@ -49636,6 +49637,7 @@
+@@ -58017,16 +58018,24 @@
+ 	me.callParent();
+     },
+ });
++
+ Ext.define('PVE.storage.ZFSInputPanel', {
+     extend: 'PVE.panel.StorageBase',
+ 
+     viewModel: {
+ 	parent: null,
  	data: {
++isComstar: true,
++	    isFreeNAS: false,
  	    isLIO: false,
- 	    isComstar: true,
-+        isFreeNAS: false,
+-	    isComstar: true,
++	    isToken: false,
  	    hasWriteCacheOption: true,
  	},
++formulas: {
++            hideUsername: function(get) {
++                return (!get('isFreeNAS') || !(get('isFreeNAS') && !get('isToken')));
++            },
++	},
      },
-@@ -49648,10 +49650,26 @@
+ 
+     controller: {
+@@ -58034,13 +58043,42 @@
+ 	control: {
+ 	    'field[name=iscsiprovider]': {
+ 		change: 'changeISCSIProvider',
++},
++	    'field[name=truenas_token_auth]': {
++		change: 'changeUsername',
  	    },
  	},
  	changeISCSIProvider: function(f, newVal, oldVal) {
-+	    var me = this;
++var me = this;
  	    var vm = this.getViewModel();
  	    vm.set('isLIO', newVal === 'LIO');
  	    vm.set('isComstar', newVal === 'comstar');
 -	    vm.set('hasWriteCacheOption', newVal === 'comstar' || newVal === 'istgt');
 +	    vm.set('isFreeNAS', newVal === 'freenas');
-+	    vm.set('hasWriteCacheOption', newVal === 'comstar' || newVal === 'freenas' || newVal === 'istgt');
-+	    if (newVal !== 'freenas') {
-+		me.lookupReference('freenas_use_ssl_field').setValue(false);
-+		me.lookupReference('freenas_apiv4_host_field').setValue('');
-+		me.lookupReference('freenas_user_field').setValue('');
-+		me.lookupReference('freenas_user_field').allowBlank = true;
-+		me.lookupReference('freenas_password_field').setValue('');
-+		me.lookupReference('freenas_password_field').allowBlank = true;
-+		me.lookupReference('freenas_confirmpw_field').setValue('');
-+		me.lookupReference('freenas_confirmpw_field').allowBlank = true;
-+	    } else {
-+		me.lookupReference('freenas_user_field').allowBlank = false;
-+		me.lookupReference('freenas_password_field').allowBlank = false;
-+		me.lookupReference('freenas_confirmpw_field').allowBlank = false;
-+	    }
++        vm.set('hasWriteCacheOption', newVal === 'comstar' || newVal === 'freenas' || newVal === 'istgt');
++        if (newVal !== 'freenas') {
++            me.lookupReference('freenas_use_ssl_field').setValue(false);
++            me.lookupReference('truenas_token_auth_field').setValue(false);
++            me.lookupReference('freenas_apiv4_host_field').setValue('');
++            me.lookupReference('freenas_user_field').setValue('');
++            me.lookupReference('freenas_user_field').allowBlank = true;
++            me.lookupReference('truenas_secret_field').setValue('');
++            me.lookupReference('truenas_secret_field').allowBlank = true;
++            me.lookupReference('truenas_confirm_secret_field').setValue('');
++            me.lookupReference('truenas_confirm_secret_field').allowBlank = true;
++        } else {
++            me.lookupReference('freenas_user_field').allowBlank = false;
++            me.lookupReference('truenas_secret_field').allowBlank = false;
++            me.lookupReference('truenas_confirm_secret_field').allowBlank = false;
++        }
++    },
++    changeUsername: function(f, newVal, oldVal) {
++        var me = this;
++        var vm = me.getViewModel();
++        vm.set('isToken', newVal);
++        me.lookupReference('freenas_user_field').allowBlank = newVal;
++        if (newVal) {
++            me.lookupReference('freenas_user_field').setValue('');
++        }
  	},
      },
  
-@@ -49669,6 +49687,7 @@
+@@ -58053,28 +58091,78 @@
+ 
+ 	values.nowritecache = values.writecache ? 0 : 1;
+ 	delete values.writecache;
++    console.warn(values.freenas_password);
++	if (values.freenas_password) {
++	    values.truenas_secret = values.freenas_password;
++	}
++	console.warn(values.truenas_secret);
+ 
+ 	return me.callParent([values]);
      },
  
      setValues: function(values) {
-+        values.freenas_confirmpw = values.freenas_password;
- 	values.writecache = values.nowritecache ? 0 : 1;
- 	this.callParent([values]);
+-	values.writecache = values.nowritecache ? 0 : 1;
+-	this.callParent([values]);
++        if (values.freenas_password) {
++            values.truenas_secret = values.freenas_password;
++        }
++        values.truenas_confirm_secret = values.truenas_secret;
++        values.writecache = values.nowritecache ? 0 : 1;
++        this.callParent([values]);
      },
-@@ -49685,7 +49704,7 @@
+ 
+     initComponent: function() {
+-	var me = this;
++    var me = this;
++
++    var tnsecret = Ext.create('Ext.form.TextField', { 
++        xtype: 'proxmoxtextfield',
++        name: 'truenas_secret',
++       reference: 'truenas_secret_field',
++        inputType: me.isCreate ? '' : 'password',
++        value: '',
++        editable: true,
++        emptyText: Proxmox.Utils.noneText,
++        bind: {
++            hidden: '{!isFreeNAS}'
++        },
++        fieldLabel: gettext('API Password'),
++        change: function(f, value) {
++            if (f.rendered) {
++                f.up().down('field[name=truenas_confirm_secret]').validate();
++            }
++        },
++    });
+ 
+-	me.column1 = [
+-	    {
+-		xtype: me.isCreate ? 'textfield' : 'displayfield',
+-		name: 'portal',
++    var tnconfirmsecret = Ext.create('Ext.form.TextField', {
++        xtype: 'proxmoxtextfield',
++        name: 'truenas_confirm_secret',
++        reference: 'truenas_confirm_secret_field',
++        inputType: me.isCreate ? '' : 'password',
++        value: '',
++        editable: true,
++        submitValue: false,
++        emptyText: Proxmox.Utils.noneText,
++        bind: {
++            hidden: '{!isFreeNAS}'
++        },
++        fieldLabel: gettext('Confirm API Password'),
++        validator: function(value) {
++            var pw = me.up().down('field[name=truenas_secret]').getValue();
++            if (pw !== value) {
++                return "Secrets do not match!";
++            }
++            return true;
++        },
++    });
++
++    me.column1 = [
++        {
++        xtype: me.isCreate ? 'textfield' : 'displayfield',
++        name: 'portal',
+ 		value: '',
+ 		fieldLabel: gettext('Portal'),
  		allowBlank: false,
  	    },
  	    {
@@ -61,7 +165,7 @@
  		name: 'pool',
  		value: '',
  		fieldLabel: gettext('Pool'),
-@@ -49695,11 +49714,11 @@
+@@ -58084,11 +58172,11 @@
  		xtype: me.isCreate ? 'textfield' : 'displayfield',
  		name: 'blocksize',
  		value: '4k',
@@ -75,7 +179,7 @@
  		name: 'target',
  		value: '',
  		fieldLabel: gettext('Target'),
-@@ -49710,9 +49729,34 @@
+@@ -58099,8 +58187,59 @@
  		name: 'comstar_tg',
  		value: '',
  		fieldLabel: gettext('Target group'),
@@ -84,7 +188,7 @@
 +		    hidden: '{!isComstar}'
 +		},
  		allowBlank: true,
- 	    },
++},
 +	    {
 +		xtype: 'proxmoxcheckbox',
 +		name: 'freenas_use_ssl',
@@ -98,6 +202,32 @@
 +		fieldLabel: gettext('API use SSL'),
 +	    },
 +	    {
++		xtype: 'proxmoxcheckbox',
++		name: 'truenas_token_auth',
++		reference: 'truenas_token_auth_field',
++		inputId: 'truenas_use_token_auth_field',
++		checked: false,
++		listeners: {
++		    change: function(field, newValue) {
++			if (newValue === true) {
++			    tnsecret.labelEl.update('API Token');
++			    tnconfirmsecret.labelEl.update('Confirm API Token');
++			    me.lookupReference('freenas_user_field').setValue('');
++			    me.lookupReference('freenas_user_field').allowBlank = true;
++			} else {
++			    tnsecret.labelEl.update('API Password');
++			    tnconfirmsecret.labelEl.update('Confirm API Password');
++			    me.lookupReference('freenas_user_field').allowBlank = false;
++			}
++		    },
++		},
++		bind: {
++		    hidden: '{!isFreeNAS}'
++		},
++		uncheckedValue: 0,
++		fieldLabel: gettext('API Token Auth'),
++	    },
++	    {
 +		xtype: 'textfield',
 +		name: 'freenas_user',
 +		reference: 'freenas_user_field',
@@ -105,13 +235,12 @@
 +		value: '',
 +		fieldLabel: gettext('API Username'),
 +		bind: {
-+		    hidden: '{!isFreeNAS}'
++		    hidden: '{hideUsername}'
 +		},
-+	    },
+ 	    },
  	];
  
- 	me.column2 = [
-@@ -49742,7 +49786,9 @@
+@@ -58131,7 +58270,9 @@
  		xtype: me.isCreate ? 'textfield' : 'displayfield',
  		name: 'comstar_hg',
  		value: '',
@@ -122,18 +251,19 @@
  		fieldLabel: gettext('Host group'),
  		allowBlank: true,
  	    },
-@@ -49750,9 +49796,62 @@
+@@ -58139,15 +58280,32 @@
  		xtype: me.isCreate ? 'textfield' : 'displayfield',
  		name: 'lio_tpg',
  		value: '',
 -		bind: me.isCreate ? { disabled: '{!isLIO}' } : { hidden: '{!isLIO}' },
 -		allowBlank: false,
+-		fieldLabel: gettext('Target portal group'),
 +		bind: {
 +		    hidden: '{!isLIO}'
 +		},
- 		fieldLabel: gettext('Target portal group'),
-+		allowBlank: true
-+	    },
++				fieldLabel: gettext('Target portal group'),
++	    allowBlank: true
+ 	    },
 +	    {
 +		xtype: 'proxmoxtextfield',
 +		name: 'freenas_apiv4_host',
@@ -146,44 +276,14 @@
 +		},
 +		fieldLabel: gettext('API IPv4 Host'),
 +	    },
-+	    {
-+		xtype: 'proxmoxtextfield',
-+		name: 'freenas_password',
-+		reference: 'freenas_password_field',
-+		inputType: me.isCreate ? '' : 'password',
-+		value: '',
-+		editable: true,
-+		emptyText: Proxmox.Utils.noneText,
-+		bind: {
-+		    hidden: '{!isFreeNAS}'
-+		},
-+		fieldLabel: gettext('API Password'),
-+		change: function(f, value) {
-+		    if (f.rendered) {
-+			f.up().down('field[name=freenas_confirmpw]').validate();
-+		    }
-+		},
-+	    },
-+	    {
-+		xtype: 'proxmoxtextfield',
-+		name: 'freenas_confirmpw',
-+		reference: 'freenas_confirmpw_field',
-+		inputType: me.isCreate ? '' : 'password',
-+		value: '',
-+		editable: true,
-+		submitValue: false,
-+		emptyText: Proxmox.Utils.noneText,
-+		bind: {
-+		    hidden: '{!isFreeNAS}'
-+		},
-+		fieldLabel: gettext('Confirm Password'),
-+		validator: function(value) {
-+		    var pw = this.up().down('field[name=freenas_password]').getValue();
-+		    if (pw !== value) {
-+			return "Passwords do not match!";
-+		    }
-+		    return true;
-+		},
- 	    },
++	    tnsecret,
++	    tnconfirmsecret,
  	];
  
+ 	me.callParent();
+     },
+ });
++
+ Ext.define('PVE.storage.ZFSPoolSelector', {
+     extend: 'PVE.form.ComboBoxSetStoreNode',
+     alias: 'widget.pveZFSPoolSelector',

stable-7/pve-manager/js/pvemanagerlib.js.patch

@@ -1,189 +0,0 @@
---- pvemanagerlib.js.orig	2022-03-17 09:08:40.000000000 -0400
-+++ pvemanagerlib.js	2022-04-03 08:54:10.229689187 -0400
-@@ -8068,6 +8068,7 @@
-     alias: ['widget.pveiScsiProviderSelector'],
-     comboItems: [
- 	['comstar', 'Comstar'],
-+	['freenas', 'FreeNAS-API'],
- 	['istgt', 'istgt'],
- 	['iet', 'IET'],
- 	['LIO', 'LIO'],
-@@ -49636,6 +49637,7 @@
- 	data: {
- 	    isLIO: false,
- 	    isComstar: true,
-+        isFreeNAS: false,
- 	    hasWriteCacheOption: true,
- 	},
-     },
-@@ -49648,10 +49650,26 @@
- 	    },
- 	},
- 	changeISCSIProvider: function(f, newVal, oldVal) {
-+	    var me = this;
- 	    var vm = this.getViewModel();
- 	    vm.set('isLIO', newVal === 'LIO');
- 	    vm.set('isComstar', newVal === 'comstar');
--	    vm.set('hasWriteCacheOption', newVal === 'comstar' || newVal === 'istgt');
-+	    vm.set('isFreeNAS', newVal === 'freenas');
-+	    vm.set('hasWriteCacheOption', newVal === 'comstar' || newVal === 'freenas' || newVal === 'istgt');
-+	    if (newVal !== 'freenas') {
-+		me.lookupReference('freenas_use_ssl_field').setValue(false);
-+		me.lookupReference('freenas_apiv4_host_field').setValue('');
-+		me.lookupReference('freenas_user_field').setValue('');
-+		me.lookupReference('freenas_user_field').allowBlank = true;
-+		me.lookupReference('freenas_password_field').setValue('');
-+		me.lookupReference('freenas_password_field').allowBlank = true;
-+		me.lookupReference('freenas_confirmpw_field').setValue('');
-+		me.lookupReference('freenas_confirmpw_field').allowBlank = true;
-+	    } else {
-+		me.lookupReference('freenas_user_field').allowBlank = false;
-+		me.lookupReference('freenas_password_field').allowBlank = false;
-+		me.lookupReference('freenas_confirmpw_field').allowBlank = false;
-+	    }
- 	},
-     },
- 
-@@ -49669,6 +49687,7 @@
-     },
- 
-     setValues: function(values) {
-+        values.freenas_confirmpw = values.freenas_password;
- 	values.writecache = values.nowritecache ? 0 : 1;
- 	this.callParent([values]);
-     },
-@@ -49685,7 +49704,7 @@
- 		allowBlank: false,
- 	    },
- 	    {
--		xtype: me.isCreate ? 'textfield' : 'displayfield',
-+		xtype: 'textfield',
- 		name: 'pool',
- 		value: '',
- 		fieldLabel: gettext('Pool'),
-@@ -49695,11 +49714,11 @@
- 		xtype: me.isCreate ? 'textfield' : 'displayfield',
- 		name: 'blocksize',
- 		value: '4k',
--		fieldLabel: gettext('Block Size'),
-+		fieldLabel: gettext('ZFS Block Size'),
- 		allowBlank: false,
- 	    },
- 	    {
--		xtype: me.isCreate ? 'textfield' : 'displayfield',
-+		xtype: 'textfield',
- 		name: 'target',
- 		value: '',
- 		fieldLabel: gettext('Target'),
-@@ -49710,9 +49729,34 @@
- 		name: 'comstar_tg',
- 		value: '',
- 		fieldLabel: gettext('Target group'),
--		bind: me.isCreate ? { disabled: '{!isComstar}' } : { hidden: '{!isComstar}' },
-+		bind: {
-+		    hidden: '{!isComstar}'
-+		},
- 		allowBlank: true,
- 	    },
-+	    {
-+		xtype: 'proxmoxcheckbox',
-+		name: 'freenas_use_ssl',
-+		reference: 'freenas_use_ssl_field',
-+		inputId: 'freenas_use_ssl_field',
-+		checked: false,
-+		bind: {
-+		    hidden: '{!isFreeNAS}'
-+		},
-+		uncheckedValue: 0,
-+		fieldLabel: gettext('API use SSL'),
-+	    },
-+	    {
-+		xtype: 'textfield',
-+		name: 'freenas_user',
-+		reference: 'freenas_user_field',
-+		inputId: 'freenas_user_field',
-+		value: '',
-+		fieldLabel: gettext('API Username'),
-+		bind: {
-+		    hidden: '{!isFreeNAS}'
-+		},
-+	    },
- 	];
- 
- 	me.column2 = [
-@@ -49742,7 +49786,9 @@
- 		xtype: me.isCreate ? 'textfield' : 'displayfield',
- 		name: 'comstar_hg',
- 		value: '',
--		bind: me.isCreate ? { disabled: '{!isComstar}' } : { hidden: '{!isComstar}' },
-+		bind: {
-+		    hidden: '{!isComstar}'
-+		},
- 		fieldLabel: gettext('Host group'),
- 		allowBlank: true,
- 	    },
-@@ -49750,9 +49796,62 @@
- 		xtype: me.isCreate ? 'textfield' : 'displayfield',
- 		name: 'lio_tpg',
- 		value: '',
--		bind: me.isCreate ? { disabled: '{!isLIO}' } : { hidden: '{!isLIO}' },
--		allowBlank: false,
-+		bind: {
-+		    hidden: '{!isLIO}'
-+		},
- 		fieldLabel: gettext('Target portal group'),
-+		allowBlank: true
-+	    },
-+	    {
-+		xtype: 'proxmoxtextfield',
-+		name: 'freenas_apiv4_host',
-+		reference: 'freenas_apiv4_host_field',
-+		value: '',
-+		editable: true,
-+		emptyText: Proxmox.Utils.noneText,
-+		bind: {
-+		    hidden: '{!isFreeNAS}'
-+		},
-+		fieldLabel: gettext('API IPv4 Host'),
-+	    },
-+	    {
-+		xtype: 'proxmoxtextfield',
-+		name: 'freenas_password',
-+		reference: 'freenas_password_field',
-+		inputType: me.isCreate ? '' : 'password',
-+		value: '',
-+		editable: true,
-+		emptyText: Proxmox.Utils.noneText,
-+		bind: {
-+		    hidden: '{!isFreeNAS}'
-+		},
-+		fieldLabel: gettext('API Password'),
-+		change: function(f, value) {
-+		    if (f.rendered) {
-+			f.up().down('field[name=freenas_confirmpw]').validate();
-+		    }
-+		},
-+	    },
-+	    {
-+		xtype: 'proxmoxtextfield',
-+		name: 'freenas_confirmpw',
-+		reference: 'freenas_confirmpw_field',
-+		inputType: me.isCreate ? '' : 'password',
-+		value: '',
-+		editable: true,
-+		submitValue: false,
-+		emptyText: Proxmox.Utils.noneText,
-+		bind: {
-+		    hidden: '{!isFreeNAS}'
-+		},
-+		fieldLabel: gettext('Confirm Password'),
-+		validator: function(value) {
-+		    var pw = this.up().down('field[name=freenas_password]').getValue();
-+		    if (pw !== value) {
-+			return "Passwords do not match!";
-+		    }
-+		    return true;
-+		},
- 	    },
- 	];
- 

stable-8/perl5/PVE/Storage/ZFSPlugin-8.0.5_1.pm.patch

@@ -0,0 +1,157 @@
+--- ZFSPlugin.pm.orig	2023-12-31 09:56:18.895228853 -0500
++++ ZFSPlugin.pm	2023-12-31 09:57:08.830488875 -0500
+@@ -10,6 +10,7 @@
+ 
+ use base qw(PVE::Storage::ZFSPoolPlugin);
+ use PVE::Storage::LunCmd::Comstar;
++use PVE::Storage::LunCmd::FreeNAS;
+ use PVE::Storage::LunCmd::Istgt;
+ use PVE::Storage::LunCmd::Iet;
+ use PVE::Storage::LunCmd::LIO;
+@@ -26,13 +27,14 @@
+     modify_lu   => 1,
+     add_view    => 1,
+     list_view   => 1,
++    list_extent => 1,
+     list_lu     => 1,
+ };
+ 
+ my $zfs_unknown_scsi_provider = sub {
+     my ($provider) = @_;
+ 
+-    die "$provider: unknown iscsi provider. Available [comstar, istgt, iet, LIO]";
++    die "$provider: unknown iscsi provider. Available [comstar, freenas, istgt, iet, LIO]";
+ };
+ 
+ my $zfs_get_base = sub {
+@@ -40,6 +42,8 @@
+ 
+     if ($scfg->{iscsiprovider} eq 'comstar') {
+         return PVE::Storage::LunCmd::Comstar::get_base;
++    } elsif ($scfg->{iscsiprovider} eq 'freenas') {
++        return PVE::Storage::LunCmd::FreeNAS::get_base;
+     } elsif ($scfg->{iscsiprovider} eq 'istgt') {
+         return PVE::Storage::LunCmd::Istgt::get_base;
+     } elsif ($scfg->{iscsiprovider} eq 'iet') {
+@@ -62,6 +66,8 @@
+     if ($lun_cmds->{$method}) {
+         if ($scfg->{iscsiprovider} eq 'comstar') {
+             $msg = PVE::Storage::LunCmd::Comstar::run_lun_command($scfg, $timeout, $method, @params);
++        } elsif ($scfg->{iscsiprovider} eq 'freenas') {
++            $msg = PVE::Storage::LunCmd::FreeNAS::run_lun_command($scfg, $timeout, $method, @params);
+         } elsif ($scfg->{iscsiprovider} eq 'istgt') {
+             $msg = PVE::Storage::LunCmd::Istgt::run_lun_command($scfg, $timeout, $method, @params);
+         } elsif ($scfg->{iscsiprovider} eq 'iet') {
+@@ -166,6 +172,15 @@
+     die "lun_number for guid $guid is not a number";
+ }
+ 
++# Part of the multipath enhancement
++sub zfs_get_wwid_number {
++    my ($class, $scfg, $guid) = @_;
++
++    die "could not find lun_number for guid $guid" if !$guid;
++
++    return $class->zfs_request($scfg, undef, 'list_extent', $guid);
++}
++
+ # Configuration
+ 
+ sub type {
+@@ -184,6 +199,32 @@
+ 	    description => "iscsi provider",
+ 	    type => 'string',
+ 	},
++	# This is for FreeNAS iscsi and API intergration
++	# And some enhancements asked by the community
++	freenas_user => {
++	    description => "FreeNAS API Username",
++	    type => 'string',
++	},
++	freenas_password => {
++	    description => "FreeNAS API Password (Deprecated)",
++	    type => 'string',
++	},
++	truenas_secret => {
++	    description => "TrueNAS API Secret",
++	    type => 'string',
++	},
++	truenas_token_auth => {
++	    description => "TrueNAS API Authentication with Token",
++	    type => 'boolean',
++	},
++	freenas_use_ssl => {
++	    description => "FreeNAS API access via SSL",
++	    type => 'boolean',
++	},
++	freenas_apiv4_host => {
++	    description => "FreeNAS API Host",
++	    type => 'string',
++	},
+ 	# this will disable write caching on comstar and istgt.
+ 	# it is not implemented for iet. iet blockio always operates with
+ 	# writethrough caching when not in readonly mode
+@@ -211,14 +252,20 @@
+ 	nodes => { optional => 1 },
+ 	disable => { optional => 1 },
+ 	portal => { fixed => 1 },
+-	target => { fixed => 1 },
+-	pool => { fixed => 1 },
++	target => { fixed => 0 },
++	pool => { fixed => 0 },
+ 	blocksize => { fixed => 1 },
+ 	iscsiprovider => { fixed => 1 },
+ 	nowritecache => { optional => 1 },
+ 	sparse => { optional => 1 },
+ 	comstar_hg => { optional => 1 },
+ 	comstar_tg => { optional => 1 },
++	freenas_user => { optional => 1 },
++	freenas_password => { optional => 1 },
++	truenas_secret => { optional => 1 },
++	truenas_token_auth => { optional => 1 },
++	freenas_use_ssl => { optional => 1 },
++	freenas_apiv4_host => { optional => 1 },
+ 	lio_tpg => { optional => 1 },
+ 	content => { optional => 1 },
+ 	bwlimit => { optional => 1 },
+@@ -243,6 +290,40 @@
+ 
+     my $path = "iscsi://$portal/$target/$lun";
+ 
++    # Multipath enhancement
++    eval {
++	my $wwid = $class->zfs_get_wwid_number($scfg, $guid);
++#	syslog(info,"JD: path get_lun_number guid $guid");
++
++	if ($wwid =~ /^([-\@\w.]+)$/) {
++	    $wwid = $1;                     # $data now untainted
++	} else {
++	    die "Bad data in '$wwid'";      # log this somewhere
++	}
++	my $wwid_end = substr $wwid, 16;
++
++	my $mapper = '';
++	sleep 3;
++	run_command("iscsiadm -m session --rescan");
++	sleep 3;
++	my $line = `/usr/sbin/multipath -ll | grep \"$wwid_end\"`;
++	my ($mapper_device) = split(' ', $line);
++	$mapper_device = "" unless $mapper_device;
++	$mapper .= $mapper_device;
++
++	if ($mapper =~ /^([-\@\w.]+)$/) {
++	    $mapper = $1;                   # $data now untainted
++	} else {
++	    $mapper = '';
++	}
++
++#	syslog(info,"Multipath mapper found: $mapper\n");
++	if ($mapper ne "") {
++	    $path = "/dev/mapper/$mapper";
++	    sleep 5;
++	}
++    };
++
+     return ($path, $vmid, $vtype);
+ }
+ 

stable-8/perl5/PVE/Storage/ZFSPlugin.pm.orig

@@ -0,0 +1,422 @@
+package PVE::Storage::ZFSPlugin;
+
+use strict;
+use warnings;
+use IO::File;
+use POSIX;
+use PVE::Tools qw(run_command);
+use PVE::Storage::ZFSPoolPlugin;
+use PVE::RPCEnvironment;
+
+use base qw(PVE::Storage::ZFSPoolPlugin);
+use PVE::Storage::LunCmd::Comstar;
+use PVE::Storage::LunCmd::Istgt;
+use PVE::Storage::LunCmd::Iet;
+use PVE::Storage::LunCmd::LIO;
+
+
+my @ssh_opts = ('-o', 'BatchMode=yes');
+my @ssh_cmd = ('/usr/bin/ssh', @ssh_opts);
+my $id_rsa_path = '/etc/pve/priv/zfs';
+
+my $lun_cmds = {
+    create_lu   => 1,
+    delete_lu   => 1,
+    import_lu   => 1,
+    modify_lu   => 1,
+    add_view    => 1,
+    list_view   => 1,
+    list_lu     => 1,
+};
+
+my $zfs_unknown_scsi_provider = sub {
+    my ($provider) = @_;
+
+    die "$provider: unknown iscsi provider. Available [comstar, istgt, iet, LIO]";
+};
+
+my $zfs_get_base = sub {
+    my ($scfg) = @_;
+
+    if ($scfg->{iscsiprovider} eq 'comstar') {
+        return PVE::Storage::LunCmd::Comstar::get_base;
+    } elsif ($scfg->{iscsiprovider} eq 'istgt') {
+        return PVE::Storage::LunCmd::Istgt::get_base;
+    } elsif ($scfg->{iscsiprovider} eq 'iet') {
+        return PVE::Storage::LunCmd::Iet::get_base;
+    } elsif ($scfg->{iscsiprovider} eq 'LIO') {
+        return PVE::Storage::LunCmd::LIO::get_base;
+    } else {
+        $zfs_unknown_scsi_provider->($scfg->{iscsiprovider});
+    }
+};
+
+sub zfs_request {
+    my ($class, $scfg, $timeout, $method, @params) = @_;
+
+    $timeout = PVE::RPCEnvironment->is_worker() ? 60*60 : 10
+	if !$timeout;
+
+    my $msg = '';
+
+    if ($lun_cmds->{$method}) {
+        if ($scfg->{iscsiprovider} eq 'comstar') {
+            $msg = PVE::Storage::LunCmd::Comstar::run_lun_command($scfg, $timeout, $method, @params);
+        } elsif ($scfg->{iscsiprovider} eq 'istgt') {
+            $msg = PVE::Storage::LunCmd::Istgt::run_lun_command($scfg, $timeout, $method, @params);
+        } elsif ($scfg->{iscsiprovider} eq 'iet') {
+            $msg = PVE::Storage::LunCmd::Iet::run_lun_command($scfg, $timeout, $method, @params);
+        } elsif ($scfg->{iscsiprovider} eq 'LIO') {
+            $msg = PVE::Storage::LunCmd::LIO::run_lun_command($scfg, $timeout, $method, @params);
+        } else {
+            $zfs_unknown_scsi_provider->($scfg->{iscsiprovider});
+        }
+    } else {
+
+	my $target = 'root@' . $scfg->{portal};
+
+	my $cmd = [@ssh_cmd, '-i', "$id_rsa_path/$scfg->{portal}_id_rsa", $target];
+
+        if ($method eq 'zpool_list') {
+	    push @$cmd, 'zpool', 'list';
+	} else {
+	    push @$cmd, 'zfs', $method;
+        }
+
+	push @$cmd, @params;
+
+	my $output = sub {
+	    my $line = shift;
+	    $msg .= "$line\n";
+        };
+
+        run_command($cmd, outfunc => $output, timeout => $timeout);
+    }
+
+    return $msg;
+}
+
+sub zfs_get_lu_name {
+    my ($class, $scfg, $zvol) = @_;
+
+    my $base = $zfs_get_base->($scfg);
+
+    $zvol = ($class->parse_volname($zvol))[1];
+
+    my $object = ($zvol =~ /^.+\/.+/) ? "$base/$zvol" : "$base/$scfg->{pool}/$zvol";
+
+    my $lu_name = $class->zfs_request($scfg, undef, 'list_lu', $object);
+
+    return $lu_name if $lu_name;
+
+    die "Could not find lu_name for zvol $zvol";
+}
+
+sub zfs_add_lun_mapping_entry {
+    my ($class, $scfg, $zvol, $guid) = @_;
+
+    if (!defined($guid)) {
+	$guid = $class->zfs_get_lu_name($scfg, $zvol);
+    }
+
+    $class->zfs_request($scfg, undef, 'add_view', $guid);
+}
+
+sub zfs_delete_lu {
+    my ($class, $scfg, $zvol) = @_;
+
+    my $guid = $class->zfs_get_lu_name($scfg, $zvol);
+
+    $class->zfs_request($scfg, undef, 'delete_lu', $guid);
+}
+
+sub zfs_create_lu {
+    my ($class, $scfg, $zvol) = @_;
+
+    my $base = $zfs_get_base->($scfg);
+    my $guid = $class->zfs_request($scfg, undef, 'create_lu', "$base/$scfg->{pool}/$zvol");
+
+    return $guid;
+}
+
+sub zfs_import_lu {
+    my ($class, $scfg, $zvol) = @_;
+
+    my $base = $zfs_get_base->($scfg);
+    $class->zfs_request($scfg, undef, 'import_lu', "$base/$scfg->{pool}/$zvol");
+}
+
+sub zfs_resize_lu {
+    my ($class, $scfg, $zvol, $size) = @_;
+
+    my $guid = $class->zfs_get_lu_name($scfg, $zvol);
+
+    $class->zfs_request($scfg, undef, 'modify_lu', "${size}K", $guid);
+}
+
+sub zfs_get_lun_number {
+    my ($class, $scfg, $guid) = @_;
+
+    die "could not find lun_number for guid $guid" if !$guid;
+
+    if ($class->zfs_request($scfg, undef, 'list_view', $guid) =~ /^(\d+)$/) {
+	return $1;
+    }
+
+    die "lun_number for guid $guid is not a number";
+}
+
+# Configuration
+
+sub type {
+    return 'zfs';
+}
+
+sub plugindata {
+    return {
+	content => [ {images => 1}, { images => 1 }],
+    };
+}
+
+sub properties {
+    return {
+	iscsiprovider => {
+	    description => "iscsi provider",
+	    type => 'string',
+	},
+	# this will disable write caching on comstar and istgt.
+	# it is not implemented for iet. iet blockio always operates with
+	# writethrough caching when not in readonly mode
+	nowritecache => {
+	    description => "disable write caching on the target",
+	    type => 'boolean',
+	},
+	comstar_tg => {
+	    description => "target group for comstar views",
+	    type => 'string',
+	},
+	comstar_hg => {
+	    description => "host group for comstar views",
+	    type => 'string',
+	},
+	lio_tpg => {
+	    description => "target portal group for Linux LIO targets",
+	    type => 'string',
+	},
+    };
+}
+
+sub options {
+    return {
+	nodes => { optional => 1 },
+	disable => { optional => 1 },
+	portal => { fixed => 1 },
+	target => { fixed => 1 },
+	pool => { fixed => 1 },
+	blocksize => { fixed => 1 },
+	iscsiprovider => { fixed => 1 },
+	nowritecache => { optional => 1 },
+	sparse => { optional => 1 },
+	comstar_hg => { optional => 1 },
+	comstar_tg => { optional => 1 },
+	lio_tpg => { optional => 1 },
+	content => { optional => 1 },
+	bwlimit => { optional => 1 },
+    };
+}
+
+# Storage implementation
+
+sub path {
+    my ($class, $scfg, $volname, $storeid, $snapname) = @_;
+
+    die "direct access to snapshots not implemented"
+	if defined($snapname);
+
+    my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+
+    my $target = $scfg->{target};
+    my $portal = $scfg->{portal};
+
+    my $guid = $class->zfs_get_lu_name($scfg, $name);
+    my $lun = $class->zfs_get_lun_number($scfg, $guid);
+
+    my $path = "iscsi://$portal/$target/$lun";
+
+    return ($path, $vmid, $vtype);
+}
+
+sub create_base {
+    my ($class, $storeid, $scfg, $volname) = @_;
+
+    my $snap = '__base__';
+
+    my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) =
+        $class->parse_volname($volname);
+
+    die "create_base not possible with base image\n" if $isBase;
+
+    my $newname = $name;
+    $newname =~ s/^vm-/base-/;
+
+    my $newvolname = $basename ? "$basename/$newname" : "$newname";
+
+    $class->zfs_delete_lu($scfg, $name);
+    $class->zfs_request($scfg, undef, 'rename', "$scfg->{pool}/$name", "$scfg->{pool}/$newname");
+
+    my $guid = $class->zfs_create_lu($scfg, $newname);
+    $class->zfs_add_lun_mapping_entry($scfg, $newname, $guid);
+
+    my $running  = undef; #fixme : is create_base always offline ?
+
+    $class->volume_snapshot($scfg, $storeid, $newname, $snap, $running);
+
+    return $newvolname;
+}
+
+sub clone_image {
+    my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
+
+    my $name = $class->SUPER::clone_image($scfg, $storeid, $volname, $vmid, $snap);
+
+    # get ZFS dataset name from PVE volname
+    my (undef, $clonedname) = $class->parse_volname($name);
+
+    my $guid = $class->zfs_create_lu($scfg, $clonedname);
+    $class->zfs_add_lun_mapping_entry($scfg, $clonedname, $guid);
+
+    return $name;
+}
+
+sub alloc_image {
+    my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
+    
+    die "unsupported format '$fmt'" if $fmt ne 'raw';
+
+    die "illegal name '$name' - should be 'vm-$vmid-*'\n"
+    if $name && $name !~ m/^vm-$vmid-/;
+
+    my $volname = $name;
+
+    $volname = $class->find_free_diskname($storeid, $scfg, $vmid, $fmt) if !$volname;
+    
+    $class->zfs_create_zvol($scfg, $volname, $size);
+ 
+    my $guid = $class->zfs_create_lu($scfg, $volname);
+    $class->zfs_add_lun_mapping_entry($scfg, $volname, $guid);
+
+    return $volname;
+}
+
+sub free_image {
+    my ($class, $storeid, $scfg, $volname, $isBase) = @_;
+
+    my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+
+    $class->zfs_delete_lu($scfg, $name);
+
+    eval { $class->zfs_delete_zvol($scfg, $name); };
+    if (my $err = $@) {
+        my $guid = $class->zfs_create_lu($scfg, $name);
+        $class->zfs_add_lun_mapping_entry($scfg, $name, $guid);
+        die $err;
+    }
+
+    return undef;
+}
+
+sub volume_resize {
+    my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+
+    $volname = ($class->parse_volname($volname))[1];
+
+    my $new_size = $class->SUPER::volume_resize($scfg, $storeid, $volname, $size, $running);
+
+    $class->zfs_resize_lu($scfg, $volname, $new_size);
+
+    return $new_size;
+}
+
+sub volume_snapshot_delete {
+    my ($class, $scfg, $storeid, $volname, $snap, $running) = @_;
+
+    $volname = ($class->parse_volname($volname))[1];
+
+    $class->zfs_request($scfg, undef, 'destroy', "$scfg->{pool}/$volname\@$snap");
+}
+
+sub volume_snapshot_rollback {
+    my ($class, $scfg, $storeid, $volname, $snap) = @_;
+
+    $volname = ($class->parse_volname($volname))[1];
+
+    $class->zfs_delete_lu($scfg, $volname);
+
+    $class->zfs_request($scfg, undef, 'rollback', "$scfg->{pool}/$volname\@$snap");
+
+    $class->zfs_import_lu($scfg, $volname);
+
+    $class->zfs_add_lun_mapping_entry($scfg, $volname);
+}
+
+sub storage_can_replicate {
+    my ($class, $scfg, $storeid, $format) = @_;
+
+    return 0;
+}
+
+sub volume_has_feature {
+    my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running) = @_;
+
+    my $features = {
+	snapshot => { current => 1, snap => 1},
+	clone => { base => 1},
+	template => { current => 1},
+	copy => { base => 1, current => 1},
+    };
+
+    my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) =
+	$class->parse_volname($volname);
+
+    my $key = undef;
+
+    if ($snapname) {
+	$key = 'snap';
+    } else {
+	$key = $isBase ? 'base' : 'current';
+    }
+
+    return 1 if $features->{$feature}->{$key};
+
+    return undef;
+}
+
+sub activate_storage {
+    my ($class, $storeid, $scfg, $cache) = @_;
+
+    return 1;
+}
+
+sub deactivate_storage {
+    my ($class, $storeid, $scfg, $cache) = @_;
+
+    return 1;
+}
+
+sub activate_volume {
+    my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+
+    die "unable to activate snapshot from remote zfs storage" if $snapname;
+
+    return 1;
+}
+
+sub deactivate_volume {
+    my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+
+    die "unable to deactivate snapshot from remote zfs storage" if $snapname;
+
+    return 1;
+}
+
+1;

stable-8/perl5/PVE/Storage/ZFSPlugin.pm.patch

@@ -1,5 +1,5 @@
---- ZFSPlugin.pm.orig	2022-02-04 12:08:01.000000000 -0500
-+++ ZFSPlugin.pm	2022-03-26 13:51:40.660068908 -0400
+--- ZFSPlugin.pm.orig	2023-12-31 09:56:18.895228853 -0500
++++ ZFSPlugin.pm	2023-12-31 09:57:08.830488875 -0500
 @@ -10,6 +10,7 @@
  
  use base qw(PVE::Storage::ZFSPoolPlugin);
@@ -58,7 +58,7 @@
  # Configuration
  
  sub type {
-@@ -184,6 +199,24 @@
+@@ -184,6 +199,32 @@
  	    description => "iscsi provider",
  	    type => 'string',
  	},
@@ -69,9 +69,17 @@
 +	    type => 'string',
 +	},
 +	freenas_password => {
-+	    description => "FreeNAS API Password",
++	    description => "FreeNAS API Password (Deprecated)",
 +	    type => 'string',
 +	},
++	truenas_secret => {
++	    description => "TrueNAS API Secret",
++	    type => 'string',
++	},
++	truenas_token_auth => {
++	    description => "TrueNAS API Authentication with Token",
++	    type => 'boolean',
++	},
 +	freenas_use_ssl => {
 +	    description => "FreeNAS API access via SSL",
 +	    type => 'boolean',
@@ -83,7 +91,7 @@
  	# this will disable write caching on comstar and istgt.
  	# it is not implemented for iet. iet blockio always operates with
  	# writethrough caching when not in readonly mode
-@@ -211,14 +244,18 @@
+@@ -211,14 +252,20 @@
  	nodes => { optional => 1 },
  	disable => { optional => 1 },
  	portal => { fixed => 1 },
@@ -99,12 +107,14 @@
  	comstar_tg => { optional => 1 },
 +	freenas_user => { optional => 1 },
 +	freenas_password => { optional => 1 },
++	truenas_secret => { optional => 1 },
++	truenas_token_auth => { optional => 1 },
 +	freenas_use_ssl => { optional => 1 },
 +	freenas_apiv4_host => { optional => 1 },
  	lio_tpg => { optional => 1 },
  	content => { optional => 1 },
  	bwlimit => { optional => 1 },
-@@ -243,6 +280,40 @@
+@@ -243,6 +290,40 @@
  
      my $path = "iscsi://$portal/$target/$lun";
  

stable-8/pve-docs/api-viewer/apidoc-8.0.5_1.js.patch

@@ -0,0 +1,91 @@
+--- apidoc.js.orig	2024-01-06 13:02:06.730512378 -0500
++++ apidoc.js	2024-01-06 13:02:55.349787105 -0500
+@@ -50336,6 +50336,37 @@
+                            "type" : "string",
+                            "typetext" : "<string>"
+                         },
++                        "freenas_user" : {
++                           "description" : "FreeNAS user for API access",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
++                        "freenas_password" : {
++                           "description" : "FreeNAS password for API access (Deprecated)",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
++                        "truenas_secret" : {
++                           "description" : "TrueNAS Secret for API access",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
++                        "freenas_use_ssl" : {
++                           "description" : "FreeNAS API access via SSL",
++                           "optional" : 1,
++                           "type" : "boolean",
++                           "typetext" : "<boolean>"
++                        },
++                        "freenas_apiv4_host" : {
++                           "description" : "FreeNAS API Host via IPv4",
++                           "format" : "address",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
+                         "fuse" : {
+                            "description" : "Mount CephFS through FUSE.",
+                            "optional" : 1,
+@@ -50555,6 +50586,12 @@
+                            "type" : "boolean",
+                            "typetext" : "<boolean>"
+                         },
++                        "target" : {
++                           "description" : "iSCSI target.",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
+                         "transport" : {
+                            "description" : "Gluster transport: tcp or rdma",
+                            "enum" : [
+@@ -50854,6 +50891,37 @@
+                      "optional" : 1,
+                      "type" : "string",
+                      "typetext" : "<string>"
++                  },
++                  "freenas_user" : {
++                     "description" : "FreeNAS user for API access",
++                     "optional" : 1,
++                     "type" : "string",
++                     "typetext" : "<string>"
++                  },
++                  "freenas_password" : {
++                     "description" : "FreeNAS password for API access (Deprecated)",
++                     "optional" : 1,
++                     "type" : "string",
++                     "typetext" : "<string>"
++                  },
++                  "truenas_secret" : {
++                     "description" : "TrueNAS secret for API access",
++                     "optional" : 1,
++                     "type" : "string",
++                     "typetext" : "<string>"
++                  },
++                  "freenas_use_ssl" : {
++                     "description" : "FreeNAS API access via SSL",
++                     "optional" : 1,
++                     "type" : "boolean",
++                     "typetext" : "<boolean>"
++                  },
++                  "freenas_apiv4_host" : {
++                     "description" : "FreeNAS API Host via IPv4",
++                     "format" : "address",
++                     "optional" : 1,
++                     "type" : "string",
++                     "typetext" : "<string>"
+                   },
+                   "fuse" : {
+                      "description" : "Mount CephFS through FUSE.",

stable-8/pve-docs/api-viewer/apidoc.js.patch

@@ -1,6 +1,6 @@
---- apidoc.js.orig	2021-11-15 10:07:34.000000000 -0500
-+++ apidoc.js	2021-12-06 08:04:01.648822707 -0500
-@@ -44064,6 +44064,31 @@
+--- apidoc.js.orig	2024-01-06 13:02:06.730512378 -0500
++++ apidoc.js	2024-01-06 13:02:55.349787105 -0500
+@@ -50336,6 +50336,37 @@
                             "type" : "string",
                             "typetext" : "<string>"
                          },
@@ -11,7 +11,13 @@
 +                           "typetext" : "<string>"
 +                        },
 +                        "freenas_password" : {
-+                           "description" : "FreeNAS password for API access",
++                           "description" : "FreeNAS password for API access (Deprecated)",
++                           "optional" : 1,
++                           "type" : "string",
++                           "typetext" : "<string>"
++                        },
++                        "truenas_secret" : {
++                           "description" : "TrueNAS Secret for API access",
 +                           "optional" : 1,
 +                           "type" : "string",
 +                           "typetext" : "<string>"
@@ -32,7 +38,7 @@
                          "fuse" : {
                             "description" : "Mount CephFS through FUSE.",
                             "optional" : 1,
-@@ -44275,6 +44300,12 @@
+@@ -50555,6 +50586,12 @@
                             "type" : "boolean",
                             "typetext" : "<boolean>"
                          },
@@ -45,7 +51,7 @@
                          "transport" : {
                             "description" : "Gluster transport: tcp or rdma",
                             "enum" : [
-@@ -44547,6 +44578,31 @@
+@@ -50854,6 +50891,37 @@
                       "optional" : 1,
                       "type" : "string",
                       "typetext" : "<string>"
@@ -57,7 +63,13 @@
 +                     "typetext" : "<string>"
 +                  },
 +                  "freenas_password" : {
-+                     "description" : "FreeNAS password for API access",
++                     "description" : "FreeNAS password for API access (Deprecated)",
++                     "optional" : 1,
++                     "type" : "string",
++                     "typetext" : "<string>"
++                  },
++                  "truenas_secret" : {
++                     "description" : "TrueNAS secret for API access",
 +                     "optional" : 1,
 +                     "type" : "string",
 +                     "typetext" : "<string>"

stable-8/pve-manager/js/pvemanagerlib-8.0.5_1.js.patch

@@ -0,0 +1,289 @@
+--- pvemanagerlib.js.orig	2023-12-30 15:36:27.913505863 -0500
++++ pvemanagerlib.js	2024-01-02 09:30:56.000000000 -0500
+@@ -9228,6 +9228,7 @@
+     alias: ['widget.pveiScsiProviderSelector'],
+     comboItems: [
+ 	['comstar', 'Comstar'],
++    ['freenas', 'FreeNAS/TrueNAS API'],
+ 	['istgt', 'istgt'],
+ 	['iet', 'IET'],
+ 	['LIO', 'LIO'],
+@@ -58017,16 +58018,24 @@
+ 	me.callParent();
+     },
+ });
++
+ Ext.define('PVE.storage.ZFSInputPanel', {
+     extend: 'PVE.panel.StorageBase',
+ 
+     viewModel: {
+ 	parent: null,
+ 	data: {
++isComstar: true,
++	    isFreeNAS: false,
+ 	    isLIO: false,
+-	    isComstar: true,
++	    isToken: false,
+ 	    hasWriteCacheOption: true,
+ 	},
++formulas: {
++            hideUsername: function(get) {
++                return (!get('isFreeNAS') || !(get('isFreeNAS') && !get('isToken')));
++            },
++	},
+     },
+ 
+     controller: {
+@@ -58034,13 +58043,42 @@
+ 	control: {
+ 	    'field[name=iscsiprovider]': {
+ 		change: 'changeISCSIProvider',
++},
++	    'field[name=truenas_token_auth]': {
++		change: 'changeUsername',
+ 	    },
+ 	},
+ 	changeISCSIProvider: function(f, newVal, oldVal) {
++var me = this;
+ 	    var vm = this.getViewModel();
+ 	    vm.set('isLIO', newVal === 'LIO');
+ 	    vm.set('isComstar', newVal === 'comstar');
+-	    vm.set('hasWriteCacheOption', newVal === 'comstar' || newVal === 'istgt');
++	    vm.set('isFreeNAS', newVal === 'freenas');
++        vm.set('hasWriteCacheOption', newVal === 'comstar' || newVal === 'freenas' || newVal === 'istgt');
++        if (newVal !== 'freenas') {
++            me.lookupReference('freenas_use_ssl_field').setValue(false);
++            me.lookupReference('truenas_token_auth_field').setValue(false);
++            me.lookupReference('freenas_apiv4_host_field').setValue('');
++            me.lookupReference('freenas_user_field').setValue('');
++            me.lookupReference('freenas_user_field').allowBlank = true;
++            me.lookupReference('truenas_secret_field').setValue('');
++            me.lookupReference('truenas_secret_field').allowBlank = true;
++            me.lookupReference('truenas_confirm_secret_field').setValue('');
++            me.lookupReference('truenas_confirm_secret_field').allowBlank = true;
++        } else {
++            me.lookupReference('freenas_user_field').allowBlank = false;
++            me.lookupReference('truenas_secret_field').allowBlank = false;
++            me.lookupReference('truenas_confirm_secret_field').allowBlank = false;
++        }
++    },
++    changeUsername: function(f, newVal, oldVal) {
++        var me = this;
++        var vm = me.getViewModel();
++        vm.set('isToken', newVal);
++        me.lookupReference('freenas_user_field').allowBlank = newVal;
++        if (newVal) {
++            me.lookupReference('freenas_user_field').setValue('');
++        }
+ 	},
+     },
+ 
+@@ -58053,28 +58091,78 @@
+ 
+ 	values.nowritecache = values.writecache ? 0 : 1;
+ 	delete values.writecache;
++    console.warn(values.freenas_password);
++	if (values.freenas_password) {
++	    values.truenas_secret = values.freenas_password;
++	}
++	console.warn(values.truenas_secret);
+ 
+ 	return me.callParent([values]);
+     },
+ 
+     setValues: function(values) {
+-	values.writecache = values.nowritecache ? 0 : 1;
+-	this.callParent([values]);
++        if (values.freenas_password) {
++            values.truenas_secret = values.freenas_password;
++        }
++        values.truenas_confirm_secret = values.truenas_secret;
++        values.writecache = values.nowritecache ? 0 : 1;
++        this.callParent([values]);
+     },
+ 
+     initComponent: function() {
+-	var me = this;
++    var me = this;
++
++    var tnsecret = Ext.create('Ext.form.TextField', { 
++        xtype: 'proxmoxtextfield',
++        name: 'truenas_secret',
++       reference: 'truenas_secret_field',
++        inputType: me.isCreate ? '' : 'password',
++        value: '',
++        editable: true,
++        emptyText: Proxmox.Utils.noneText,
++        bind: {
++            hidden: '{!isFreeNAS}'
++        },
++        fieldLabel: gettext('API Password'),
++        change: function(f, value) {
++            if (f.rendered) {
++                f.up().down('field[name=truenas_confirm_secret]').validate();
++            }
++        },
++    });
+ 
+-	me.column1 = [
+-	    {
+-		xtype: me.isCreate ? 'textfield' : 'displayfield',
+-		name: 'portal',
++    var tnconfirmsecret = Ext.create('Ext.form.TextField', {
++        xtype: 'proxmoxtextfield',
++        name: 'truenas_confirm_secret',
++        reference: 'truenas_confirm_secret_field',
++        inputType: me.isCreate ? '' : 'password',
++        value: '',
++        editable: true,
++        submitValue: false,
++        emptyText: Proxmox.Utils.noneText,
++        bind: {
++            hidden: '{!isFreeNAS}'
++        },
++        fieldLabel: gettext('Confirm API Password'),
++        validator: function(value) {
++            var pw = me.up().down('field[name=truenas_secret]').getValue();
++            if (pw !== value) {
++                return "Secrets do not match!";
++            }
++            return true;
++        },
++    });
++
++    me.column1 = [
++        {
++        xtype: me.isCreate ? 'textfield' : 'displayfield',
++        name: 'portal',
+ 		value: '',
+ 		fieldLabel: gettext('Portal'),
+ 		allowBlank: false,
+ 	    },
+ 	    {
+-		xtype: me.isCreate ? 'textfield' : 'displayfield',
++		xtype: 'textfield',
+ 		name: 'pool',
+ 		value: '',
+ 		fieldLabel: gettext('Pool'),
+@@ -58084,11 +58172,11 @@
+ 		xtype: me.isCreate ? 'textfield' : 'displayfield',
+ 		name: 'blocksize',
+ 		value: '4k',
+-		fieldLabel: gettext('Block Size'),
++		fieldLabel: gettext('ZFS Block Size'),
+ 		allowBlank: false,
+ 	    },
+ 	    {
+-		xtype: me.isCreate ? 'textfield' : 'displayfield',
++		xtype: 'textfield',
+ 		name: 'target',
+ 		value: '',
+ 		fieldLabel: gettext('Target'),
+@@ -58099,8 +58187,59 @@
+ 		name: 'comstar_tg',
+ 		value: '',
+ 		fieldLabel: gettext('Target group'),
+-		bind: me.isCreate ? { disabled: '{!isComstar}' } : { hidden: '{!isComstar}' },
++		bind: {
++		    hidden: '{!isComstar}'
++		},
+ 		allowBlank: true,
++},
++	    {
++		xtype: 'proxmoxcheckbox',
++		name: 'freenas_use_ssl',
++		reference: 'freenas_use_ssl_field',
++		inputId: 'freenas_use_ssl_field',
++		checked: false,
++		bind: {
++		    hidden: '{!isFreeNAS}'
++		},
++		uncheckedValue: 0,
++		fieldLabel: gettext('API use SSL'),
++	    },
++	    {
++		xtype: 'proxmoxcheckbox',
++		name: 'truenas_token_auth',
++		reference: 'truenas_token_auth_field',
++		inputId: 'truenas_use_token_auth_field',
++		checked: false,
++		listeners: {
++		    change: function(field, newValue) {
++			if (newValue === true) {
++			    tnsecret.labelEl.update('API Token');
++			    tnconfirmsecret.labelEl.update('Confirm API Token');
++			    me.lookupReference('freenas_user_field').setValue('');
++			    me.lookupReference('freenas_user_field').allowBlank = true;
++			} else {
++			    tnsecret.labelEl.update('API Password');
++			    tnconfirmsecret.labelEl.update('Confirm API Password');
++			    me.lookupReference('freenas_user_field').allowBlank = false;
++			}
++		    },
++		},
++		bind: {
++		    hidden: '{!isFreeNAS}'
++		},
++		uncheckedValue: 0,
++		fieldLabel: gettext('API Token Auth'),
++	    },
++	    {
++		xtype: 'textfield',
++		name: 'freenas_user',
++		reference: 'freenas_user_field',
++		inputId: 'freenas_user_field',
++		value: '',
++		fieldLabel: gettext('API Username'),
++		bind: {
++		    hidden: '{hideUsername}'
++		},
+ 	    },
+ 	];
+ 
+@@ -58131,7 +58270,9 @@
+ 		xtype: me.isCreate ? 'textfield' : 'displayfield',
+ 		name: 'comstar_hg',
+ 		value: '',
+-		bind: me.isCreate ? { disabled: '{!isComstar}' } : { hidden: '{!isComstar}' },
++		bind: {
++		    hidden: '{!isComstar}'
++		},
+ 		fieldLabel: gettext('Host group'),
+ 		allowBlank: true,
+ 	    },
+@@ -58139,15 +58280,32 @@
+ 		xtype: me.isCreate ? 'textfield' : 'displayfield',
+ 		name: 'lio_tpg',
+ 		value: '',
+-		bind: me.isCreate ? { disabled: '{!isLIO}' } : { hidden: '{!isLIO}' },
+-		allowBlank: false,
+-		fieldLabel: gettext('Target portal group'),
++		bind: {
++		    hidden: '{!isLIO}'
++		},
++				fieldLabel: gettext('Target portal group'),
++	    allowBlank: true
+ 	    },
++	    {
++		xtype: 'proxmoxtextfield',
++		name: 'freenas_apiv4_host',
++		reference: 'freenas_apiv4_host_field',
++		value: '',
++		editable: true,
++		emptyText: Proxmox.Utils.noneText,
++		bind: {
++		    hidden: '{!isFreeNAS}'
++		},
++		fieldLabel: gettext('API IPv4 Host'),
++	    },
++	    tnsecret,
++	    tnconfirmsecret,
+ 	];
+ 
+ 	me.callParent();
+     },
+ });
++
+ Ext.define('PVE.storage.ZFSPoolSelector', {
+     extend: 'PVE.form.ComboBoxSetStoreNode',
+     alias: 'widget.pveZFSPoolSelector',

stable-8/pve-manager/js/pvemanagerlib.js.patch

@@ -0,0 +1,289 @@
+--- pvemanagerlib.js.orig	2023-12-30 15:36:27.913505863 -0500
++++ pvemanagerlib.js	2024-01-02 09:30:56.000000000 -0500
+@@ -9228,6 +9228,7 @@
+     alias: ['widget.pveiScsiProviderSelector'],
+     comboItems: [
+ 	['comstar', 'Comstar'],
++    ['freenas', 'FreeNAS/TrueNAS API'],
+ 	['istgt', 'istgt'],
+ 	['iet', 'IET'],
+ 	['LIO', 'LIO'],
+@@ -58017,16 +58018,24 @@
+ 	me.callParent();
+     },
+ });
++
+ Ext.define('PVE.storage.ZFSInputPanel', {
+     extend: 'PVE.panel.StorageBase',
+ 
+     viewModel: {
+ 	parent: null,
+ 	data: {
++isComstar: true,
++	    isFreeNAS: false,
+ 	    isLIO: false,
+-	    isComstar: true,
++	    isToken: false,
+ 	    hasWriteCacheOption: true,
+ 	},
++formulas: {
++            hideUsername: function(get) {
++                return (!get('isFreeNAS') || !(get('isFreeNAS') && !get('isToken')));
++            },
++	},
+     },
+ 
+     controller: {
+@@ -58034,13 +58043,42 @@
+ 	control: {
+ 	    'field[name=iscsiprovider]': {
+ 		change: 'changeISCSIProvider',
++},
++	    'field[name=truenas_token_auth]': {
++		change: 'changeUsername',
+ 	    },
+ 	},
+ 	changeISCSIProvider: function(f, newVal, oldVal) {
++var me = this;
+ 	    var vm = this.getViewModel();
+ 	    vm.set('isLIO', newVal === 'LIO');
+ 	    vm.set('isComstar', newVal === 'comstar');
+-	    vm.set('hasWriteCacheOption', newVal === 'comstar' || newVal === 'istgt');
++	    vm.set('isFreeNAS', newVal === 'freenas');
++        vm.set('hasWriteCacheOption', newVal === 'comstar' || newVal === 'freenas' || newVal === 'istgt');
++        if (newVal !== 'freenas') {
++            me.lookupReference('freenas_use_ssl_field').setValue(false);
++            me.lookupReference('truenas_token_auth_field').setValue(false);
++            me.lookupReference('freenas_apiv4_host_field').setValue('');
++            me.lookupReference('freenas_user_field').setValue('');
++            me.lookupReference('freenas_user_field').allowBlank = true;
++            me.lookupReference('truenas_secret_field').setValue('');
++            me.lookupReference('truenas_secret_field').allowBlank = true;
++            me.lookupReference('truenas_confirm_secret_field').setValue('');
++            me.lookupReference('truenas_confirm_secret_field').allowBlank = true;
++        } else {
++            me.lookupReference('freenas_user_field').allowBlank = false;
++            me.lookupReference('truenas_secret_field').allowBlank = false;
++            me.lookupReference('truenas_confirm_secret_field').allowBlank = false;
++        }
++    },
++    changeUsername: function(f, newVal, oldVal) {
++        var me = this;
++        var vm = me.getViewModel();
++        vm.set('isToken', newVal);
++        me.lookupReference('freenas_user_field').allowBlank = newVal;
++        if (newVal) {
++            me.lookupReference('freenas_user_field').setValue('');
++        }
+ 	},
+     },
+ 
+@@ -58053,28 +58091,78 @@
+ 
+ 	values.nowritecache = values.writecache ? 0 : 1;
+ 	delete values.writecache;
++    console.warn(values.freenas_password);
++	if (values.freenas_password) {
++	    values.truenas_secret = values.freenas_password;
++	}
++	console.warn(values.truenas_secret);
+ 
+ 	return me.callParent([values]);
+     },
+ 
+     setValues: function(values) {
+-	values.writecache = values.nowritecache ? 0 : 1;
+-	this.callParent([values]);
++        if (values.freenas_password) {
++            values.truenas_secret = values.freenas_password;
++        }
++        values.truenas_confirm_secret = values.truenas_secret;
++        values.writecache = values.nowritecache ? 0 : 1;
++        this.callParent([values]);
+     },
+ 
+     initComponent: function() {
+-	var me = this;
++    var me = this;
++
++    var tnsecret = Ext.create('Ext.form.TextField', { 
++        xtype: 'proxmoxtextfield',
++        name: 'truenas_secret',
++       reference: 'truenas_secret_field',
++        inputType: me.isCreate ? '' : 'password',
++        value: '',
++        editable: true,
++        emptyText: Proxmox.Utils.noneText,
++        bind: {
++            hidden: '{!isFreeNAS}'
++        },
++        fieldLabel: gettext('API Password'),
++        change: function(f, value) {
++            if (f.rendered) {
++                f.up().down('field[name=truenas_confirm_secret]').validate();
++            }
++        },
++    });
+ 
+-	me.column1 = [
+-	    {
+-		xtype: me.isCreate ? 'textfield' : 'displayfield',
+-		name: 'portal',
++    var tnconfirmsecret = Ext.create('Ext.form.TextField', {
++        xtype: 'proxmoxtextfield',
++        name: 'truenas_confirm_secret',
++        reference: 'truenas_confirm_secret_field',
++        inputType: me.isCreate ? '' : 'password',
++        value: '',
++        editable: true,
++        submitValue: false,
++        emptyText: Proxmox.Utils.noneText,
++        bind: {
++            hidden: '{!isFreeNAS}'
++        },
++        fieldLabel: gettext('Confirm API Password'),
++        validator: function(value) {
++            var pw = me.up().down('field[name=truenas_secret]').getValue();
++            if (pw !== value) {
++                return "Secrets do not match!";
++            }
++            return true;
++        },
++    });
++
++    me.column1 = [
++        {
++        xtype: me.isCreate ? 'textfield' : 'displayfield',
++        name: 'portal',
+ 		value: '',
+ 		fieldLabel: gettext('Portal'),
+ 		allowBlank: false,
+ 	    },
+ 	    {
+-		xtype: me.isCreate ? 'textfield' : 'displayfield',
++		xtype: 'textfield',
+ 		name: 'pool',
+ 		value: '',
+ 		fieldLabel: gettext('Pool'),
+@@ -58084,11 +58172,11 @@
+ 		xtype: me.isCreate ? 'textfield' : 'displayfield',
+ 		name: 'blocksize',
+ 		value: '4k',
+-		fieldLabel: gettext('Block Size'),
++		fieldLabel: gettext('ZFS Block Size'),
+ 		allowBlank: false,
+ 	    },
+ 	    {
+-		xtype: me.isCreate ? 'textfield' : 'displayfield',
++		xtype: 'textfield',
+ 		name: 'target',
+ 		value: '',
+ 		fieldLabel: gettext('Target'),
+@@ -58099,8 +58187,59 @@
+ 		name: 'comstar_tg',
+ 		value: '',
+ 		fieldLabel: gettext('Target group'),
+-		bind: me.isCreate ? { disabled: '{!isComstar}' } : { hidden: '{!isComstar}' },
++		bind: {
++		    hidden: '{!isComstar}'
++		},
+ 		allowBlank: true,
++},
++	    {
++		xtype: 'proxmoxcheckbox',
++		name: 'freenas_use_ssl',
++		reference: 'freenas_use_ssl_field',
++		inputId: 'freenas_use_ssl_field',
++		checked: false,
++		bind: {
++		    hidden: '{!isFreeNAS}'
++		},
++		uncheckedValue: 0,
++		fieldLabel: gettext('API use SSL'),
++	    },
++	    {
++		xtype: 'proxmoxcheckbox',
++		name: 'truenas_token_auth',
++		reference: 'truenas_token_auth_field',
++		inputId: 'truenas_use_token_auth_field',
++		checked: false,
++		listeners: {
++		    change: function(field, newValue) {
++			if (newValue === true) {
++			    tnsecret.labelEl.update('API Token');
++			    tnconfirmsecret.labelEl.update('Confirm API Token');
++			    me.lookupReference('freenas_user_field').setValue('');
++			    me.lookupReference('freenas_user_field').allowBlank = true;
++			} else {
++			    tnsecret.labelEl.update('API Password');
++			    tnconfirmsecret.labelEl.update('Confirm API Password');
++			    me.lookupReference('freenas_user_field').allowBlank = false;
++			}
++		    },
++		},
++		bind: {
++		    hidden: '{!isFreeNAS}'
++		},
++		uncheckedValue: 0,
++		fieldLabel: gettext('API Token Auth'),
++	    },
++	    {
++		xtype: 'textfield',
++		name: 'freenas_user',
++		reference: 'freenas_user_field',
++		inputId: 'freenas_user_field',
++		value: '',
++		fieldLabel: gettext('API Username'),
++		bind: {
++		    hidden: '{hideUsername}'
++		},
+ 	    },
+ 	];
+ 
+@@ -58131,7 +58270,9 @@
+ 		xtype: me.isCreate ? 'textfield' : 'displayfield',
+ 		name: 'comstar_hg',
+ 		value: '',
+-		bind: me.isCreate ? { disabled: '{!isComstar}' } : { hidden: '{!isComstar}' },
++		bind: {
++		    hidden: '{!isComstar}'
++		},
+ 		fieldLabel: gettext('Host group'),
+ 		allowBlank: true,
+ 	    },
+@@ -58139,15 +58280,32 @@
+ 		xtype: me.isCreate ? 'textfield' : 'displayfield',
+ 		name: 'lio_tpg',
+ 		value: '',
+-		bind: me.isCreate ? { disabled: '{!isLIO}' } : { hidden: '{!isLIO}' },
+-		allowBlank: false,
+-		fieldLabel: gettext('Target portal group'),
++		bind: {
++		    hidden: '{!isLIO}'
++		},
++				fieldLabel: gettext('Target portal group'),
++	    allowBlank: true
+ 	    },
++	    {
++		xtype: 'proxmoxtextfield',
++		name: 'freenas_apiv4_host',
++		reference: 'freenas_apiv4_host_field',
++		value: '',
++		editable: true,
++		emptyText: Proxmox.Utils.noneText,
++		bind: {
++		    hidden: '{!isFreeNAS}'
++		},
++		fieldLabel: gettext('API IPv4 Host'),
++	    },
++	    tnsecret,
++	    tnconfirmsecret,
+ 	];
+ 
+ 	me.callParent();
+     },
+ });
++
+ Ext.define('PVE.storage.ZFSPoolSelector', {
+     extend: 'PVE.form.ComboBoxSetStoreNode',
+     alias: 'widget.pveZFSPoolSelector',