feat: add Help button to TrueNAS storage panel and ship docs
- Inject pveOnlineHelpInfo entry in truenas-storage.js so PVE wires the Help button automatically via the StorageBase framework - Add onlineHelp: 'storage_truenas' to the input panel definition - Add ui/truenas-storage-help.html — local help page installed to /usr/share/pve-docs/truenas-storage.html (no internet required) - Add docs/getting-started.md and docs/architecture.md - Update postinst/postrm to install and remove the help HTML file - Update build.yml to include the help HTML in the package staging dir Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
feb1776aa1
commit
c48a992385
|
|
@ -159,8 +159,9 @@ jobs:
|
|||
chmod 0755 "${STAGING}/DEBIAN/postinst" "${STAGING}/DEBIAN/postrm"
|
||||
|
||||
# Plugin files (postinst copies these to their final destinations)
|
||||
cp perl5/PVE/Storage/Custom/TrueNAS.pm "${STAGING}/usr/share/freenas-proxmox/TrueNAS.pm"
|
||||
cp ui/truenas-storage.js "${STAGING}/usr/share/freenas-proxmox/truenas-storage.js"
|
||||
cp perl5/PVE/Storage/Custom/TrueNAS.pm "${STAGING}/usr/share/freenas-proxmox/TrueNAS.pm"
|
||||
cp ui/truenas-storage.js "${STAGING}/usr/share/freenas-proxmox/truenas-storage.js"
|
||||
cp ui/truenas-storage-help.html "${STAGING}/usr/share/freenas-proxmox/truenas-storage-help.html"
|
||||
|
||||
echo "==> Package contents:"
|
||||
find "${STAGING}" | sort
|
||||
|
|
|
|||
|
|
@ -0,0 +1,620 @@
|
|||
# freenas-proxmox v3.0 — Architecture & Developer Guide
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Architecture Overview](#1-architecture-overview)
|
||||
2. [Per-VM iSCSI Target Model](#2-per-vm-iscsi-target-model)
|
||||
3. [The `iscsi://` Path Model](#3-the-iscsi-path-model)
|
||||
4. [API Flow — TrueNAS REST v2.0](#4-api-flow--truenas-rest-v20)
|
||||
5. [alloc_image Deep Dive](#5-alloc_image-deep-dive)
|
||||
6. [free_image Deep Dive](#6-free_image-deep-dive)
|
||||
7. [Build Pipeline and Version/Channel Routing](#7-build-pipeline-and-versionchannel-routing)
|
||||
8. [Debugging Guide](#8-debugging-guide)
|
||||
9. [Contributing](#9-contributing)
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture Overview
|
||||
|
||||
### Where This Plugin Sits
|
||||
|
||||
Proxmox VE organizes storage backends through a plugin registry. Every storage type — directory, LVM, Ceph, ZFS-over-iSCSI — is a Perl module that subclasses `PVE::Storage::Plugin`. The `pvedaemon` process loads all registered storage plugins at startup and delegates storage operations (create volume, delete volume, list volumes, etc.) to the correct plugin based on the `type` field in `/etc/pve/storage.cfg`.
|
||||
|
||||
`PVE::Storage::Custom::TrueNAS` is loaded by PVE's auto-discovery mechanism. Any module installed under `/usr/share/perl5/PVE/Storage/Custom/` is automatically registered as a custom storage type. No patching of PVE's core `ZFSPlugin.pm`, `pvemanagerlib.js`, or `apidoc.js` is required — this is the fundamental architectural change from v2.x.
|
||||
|
||||
```
|
||||
Proxmox VE (pvedaemon)
|
||||
└─ PVE::Storage # core storage dispatch
|
||||
└─ PVE::Storage::Custom::TrueNAS # this plugin
|
||||
└─ TrueNAS REST API v2.0 # all state lives here
|
||||
├─ /pool/dataset # zvol create/delete/resize
|
||||
└─ /iscsi/* # extent, target, targetextent CRUD
|
||||
```
|
||||
|
||||
### What the Plugin Manages
|
||||
|
||||
The plugin manages the entire lifecycle of iSCSI-backed block devices on behalf of Proxmox VE:
|
||||
|
||||
- **ZFS volumes (zvols)** — the actual block storage, created and destroyed via `POST /pool/dataset` and `DELETE /pool/dataset/id/{id}`
|
||||
- **iSCSI extents** — TrueNAS's representation of a device to export over iSCSI, one per zvol
|
||||
- **iSCSI targets** — the access point a client connects to, one per VM (`proxmox-vm-<vmid>`)
|
||||
- **targetextent mappings** — the join record linking an extent to a target at a specific LUN ID
|
||||
|
||||
The plugin does NOT manage the iSCSI initiator side. QEMU's built-in libiscsi opens the `iscsi://` URI returned by `path()` directly, without any host-side session management via `iscsiadm`.
|
||||
|
||||
### What the Plugin Does Not Do
|
||||
|
||||
- **Pool listing**: Proxmox's `status()` call returns pool capacity via the TrueNAS API, but ZFS pool enumeration for the "Add Storage" wizard still requires SSH access through PVE's upstream `ZFSPoolPlugin.pm`. This is a Proxmox limitation outside the plugin's scope.
|
||||
- **Snapshots on PVE 8.x**: The `volume_snapshot` family of methods is defined but will return an unsupported error on PVE 8. PVE 9.0's Snapshot-as-Volume-Chains feature is the target integration point (v3.1.0, ADR-008).
|
||||
- **TPM state disks**: swtpm (the virtual TPM backend) cannot use `iscsi://` URIs; it requires a local filesystem path. TPM disks must be placed on a different storage type.
|
||||
|
||||
### Installed File Locations
|
||||
|
||||
| File | Destination on Proxmox host |
|
||||
|---|---|
|
||||
| `perl5/PVE/Storage/Custom/TrueNAS.pm` | `/usr/share/perl5/PVE/Storage/Custom/TrueNAS.pm` |
|
||||
| `ui/truenas-storage.js` | `/usr/share/pve-manager/js/truenas-storage.js` |
|
||||
| One `<script>` tag | Injected into `/usr/share/pve-manager/index.html.tpl` |
|
||||
|
||||
`postinst` copies both files, injects the script tag into `index.html.tpl` (idempotent — skipped if already present), and restarts `pvedaemon` and `pveproxy`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Per-VM iSCSI Target Model
|
||||
|
||||
### Why Per-VM Targets
|
||||
|
||||
The v2.x approach used a single shared iSCSI target for all VMs. Every disk from every VM was a different LUN on the same target IQN. This created several problems:
|
||||
|
||||
- Deleting one disk required stopping the iSCSI service (`force=true` on extent delete does not work cleanly when multiple active sessions are on the same target)
|
||||
- Live migration of a single disk while others on the same target are active caused session disruption
|
||||
- LUN exhaustion — TrueNAS has a practical limit of 256 LUNs per target
|
||||
|
||||
In v3.0, each VM gets its own iSCSI target: `proxmox-vm-<vmid>`. Disks belonging to that VM become LUN 0, LUN 1, LUN 2, etc. on that target.
|
||||
|
||||
### The TrueNAS Object Graph
|
||||
|
||||
For a VM with `vmid=100` and two disks, TrueNAS holds:
|
||||
|
||||
```
|
||||
ZFS pool "tank"
|
||||
└── tank/proxmox/vdisks/
|
||||
├── vm-100-disk-0 (zvol, 32 GiB)
|
||||
└── vm-100-disk-1 (zvol, 64 GiB)
|
||||
|
||||
iSCSI extents
|
||||
├── vm-100-disk-0 (type=DISK, disk=zvol/tank/proxmox/vdisks/vm-100-disk-0)
|
||||
└── vm-100-disk-1 (type=DISK, disk=zvol/tank/proxmox/vdisks/vm-100-disk-1)
|
||||
|
||||
iSCSI target
|
||||
└── proxmox-vm-100 (IQN: iqn.2005-10.org.freenas.ctl:proxmox-vm-100)
|
||||
|
||||
targetextent mappings
|
||||
├── target=proxmox-vm-100, extent=vm-100-disk-0, lunid=0
|
||||
└── target=proxmox-vm-100, extent=vm-100-disk-1, lunid=1
|
||||
```
|
||||
|
||||
### Target Lifecycle
|
||||
|
||||
**Creation**: `_resolve_vm_target($scfg, $vmid)` is called on the first `alloc_image` for a VM. It queries `GET /iscsi/target`, looks for an entry with `name eq "proxmox-vm-$vmid"`, and creates one via `POST /iscsi/target` if it does not exist. The new target is assigned the same portal group and initiator group settings as an existing reference target (the `truenas_target` config value, or any existing target on the configured portal IP, as a fallback).
|
||||
|
||||
**Portal group discovery** (`_portal_groups_for_new_target`): Queries `GET /iscsi/portal` and finds all portals listening on the configured `truenas_portal_ip` (or `0.0.0.0`/`::`). Then queries `GET /iscsi/target` and finds the groups from the base target that reference those portal IDs. These groups are cloned for the new per-VM target, preserving `portal`, `initiator`, `authmethod`, and `auth` settings.
|
||||
|
||||
**Deletion**: `_maybe_cleanup_vm_target($scfg, $vmid, $target_id)` is called after each disk is removed. It queries `GET /iscsi/targetextent?target=<id>` to check for remaining extents. If the target is now empty, it is deleted via `DELETE /iscsi/target/id/<id>`. This keeps TrueNAS clean — no accumulation of empty targets.
|
||||
|
||||
**Runtime caching**: Both the target lookup and the global iSCSI config (`GET /iscsi/global`) are cached in the module-level `$state` hashref, keyed by `truenas_host`. `deactivate_storage` clears the entire cache for a host.
|
||||
|
||||
---
|
||||
|
||||
## 3. The `iscsi://` Path Model
|
||||
|
||||
### How QEMU Connects
|
||||
|
||||
The `path()` method returns a URI of the form:
|
||||
|
||||
```
|
||||
iscsi://<portal_ip>/<iqn>/<lun_id>
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```
|
||||
iscsi://192.168.1.50/iqn.2005-10.org.freenas.ctl:proxmox-vm-100/0
|
||||
```
|
||||
|
||||
QEMU implements libiscsi internally. When it receives an `iscsi://` path, it opens an iSCSI session directly from the QEMU process to the TrueNAS portal. No `iscsiadm` command is run, no kernel iSCSI session is created, and no `/dev/sdX` or `/dev/disk/by-path/` device node appears on the Proxmox host.
|
||||
|
||||
### How `path()` Works Internally
|
||||
|
||||
```perl
|
||||
sub path {
|
||||
my ($class, $scfg, $volname, $storeid, $snapname) = @_;
|
||||
|
||||
my $ext = _find_extent($scfg, $volname);
|
||||
my $t = _api($scfg, 'GET', "/iscsi/target/id/$ext->{target_id}");
|
||||
my $iqn = _basename($scfg) . ":$t->{name}";
|
||||
my $portal = _portal($scfg);
|
||||
return ("iscsi://$portal/$iqn/$ext->{lun_id}", $vmid, $vtype);
|
||||
}
|
||||
```
|
||||
|
||||
`_find_extent` calls `GET /iscsi/extent` to locate the extent by name, then `GET /iscsi/targetextent?extent=<id>` to find the LUN ID and target association. The IQN is assembled from the cached global basename (`GET /iscsi/global` → `basename` field) and the target name.
|
||||
|
||||
### activate_volume and deactivate_volume
|
||||
|
||||
Because QEMU manages the connection itself, `activate_volume` only verifies that the extent exists (catching configuration errors early) and logs readiness. It does not call `iscsiadm login` or any iSCSI session management command. `deactivate_volume` is a no-op.
|
||||
|
||||
### Constraints of the iscsi:// Model
|
||||
|
||||
- **No multipath**: libiscsi in QEMU does not do multipath. If the portal is unreachable, QEMU retries but does not fail over to a second portal. Multi-portal TrueNAS setups are not effective with this model (tracked for v3.1.0, #256).
|
||||
- **TPM disks excluded**: `swtpm` cannot use `iscsi://` URIs. Any disk assigned to a VM's TPM must be on a different storage type (e.g., local-lvm, directory).
|
||||
- **No host-visible device**: Because there is no kernel iSCSI session, tools like `lsscsi`, `iscsiadm --mode session`, or `ls /dev/disk/by-path` will not show the disk on the Proxmox host. This is expected and correct.
|
||||
|
||||
---
|
||||
|
||||
## 4. API Flow — TrueNAS REST v2.0
|
||||
|
||||
### Authentication
|
||||
|
||||
All requests use `Authorization: Bearer <token>` with the API key stored in `truenas_api_key` in storage.cfg. Basic auth was removed in v3.0 (ADR-005). The API key is generated in TrueNAS under:
|
||||
|
||||
- SCALE: System Settings → API Keys → Add
|
||||
- CORE 13: gear icon (top right) → API Keys → Add
|
||||
|
||||
### Transport
|
||||
|
||||
The plugin uses `LWP::UserAgent` with a 30-second timeout. A single `LWP::UserAgent` instance is created per `truenas_host` and cached in `$state->{$host}{ua}`. SSL verification is off by default (`truenas_ssl_verify => 0`) to accommodate self-signed certificates. When `truenas_ssl_verify` is false, `ssl_opts(verify_hostname => 0, SSL_verify_mode => 0)` is applied to the UA.
|
||||
|
||||
### The `_api` Function
|
||||
|
||||
All TrueNAS calls go through `_api($scfg, $method, $path, $data)`:
|
||||
|
||||
```perl
|
||||
sub _api {
|
||||
my ($scfg, $method, $path, $data) = @_;
|
||||
|
||||
my $url = "$scheme://$scfg->{truenas_host}/api/v2.0$path";
|
||||
my $req = HTTP::Request->new(uc($method) => $url);
|
||||
$req->header('Authorization' => "Bearer $scfg->{truenas_api_key}");
|
||||
$req->content(encode_json($data)) if defined $data;
|
||||
|
||||
my $res = _ua($scfg)->request($req);
|
||||
die "TrueNAS API $method $path: " . $res->status_line unless $res->is_success;
|
||||
return decode_json($res->content);
|
||||
}
|
||||
```
|
||||
|
||||
On HTTP error, the function extracts the `message` field from the JSON response body, logs it via `syslog('err', ...)`, and dies with a descriptive string. HTTP 204 (No Content) responses — returned by most DELETE calls — return `undef` cleanly.
|
||||
|
||||
### Endpoints Used
|
||||
|
||||
| Operation | Method | Endpoint |
|
||||
|---|---|---|
|
||||
| Global iSCSI config | GET | `/iscsi/global` |
|
||||
| List portals | GET | `/iscsi/portal` |
|
||||
| List targets | GET | `/iscsi/target` |
|
||||
| Get target by ID | GET | `/iscsi/target/id/{id}` |
|
||||
| Create target | POST | `/iscsi/target` |
|
||||
| Delete target | DELETE | `/iscsi/target/id/{id}` |
|
||||
| List extents | GET | `/iscsi/extent` |
|
||||
| Create extent | POST | `/iscsi/extent` |
|
||||
| Delete extent | DELETE | `/iscsi/extent/id/{id}` |
|
||||
| List targetextents (by target) | GET | `/iscsi/targetextent?target={id}` |
|
||||
| List targetextents (by extent) | GET | `/iscsi/targetextent?extent={id}` |
|
||||
| Create targetextent | POST | `/iscsi/targetextent` |
|
||||
| Delete targetextent | DELETE | `/iscsi/targetextent/id/{id}` |
|
||||
| Reload iSCSI service | POST | `/service/reload` |
|
||||
| Get pool dataset | GET | `/pool/dataset?id={pool}` |
|
||||
| Get zvol by ID | GET | `/pool/dataset/id/{encoded_path}` |
|
||||
| List all zvols | GET | `/pool/dataset?type=VOLUME` |
|
||||
| Create zvol | POST | `/pool/dataset` |
|
||||
| Delete zvol | DELETE | `/pool/dataset/id/{encoded_path}` |
|
||||
|
||||
### URL Encoding for Dataset Paths
|
||||
|
||||
TrueNAS uses the full ZFS dataset path as the resource identifier in DELETE and GET by-ID calls. Forward slashes must be URL-encoded. The plugin uses `URI::Escape::uri_escape($path, "^A-Za-z0-9\\-_.~")` which encodes `/` as `%2F`:
|
||||
|
||||
```perl
|
||||
my $enc = uri_escape("tank/proxmox/vdisks/vm-100-disk-0", "^A-Za-z0-9\\-_.~");
|
||||
# Result: tank%2Fproxmox%2Fvdisks%2Fvm-100-disk-0
|
||||
_api($scfg, 'DELETE', "/pool/dataset/id/$enc", { recursive => JSON::true });
|
||||
```
|
||||
|
||||
### iSCSI Service Reload
|
||||
|
||||
After creating a new extent and targetextent, the plugin calls:
|
||||
|
||||
```perl
|
||||
_api($scfg, 'POST', '/service/reload', { service => 'iscsitarget' });
|
||||
```
|
||||
|
||||
This signals TrueNAS to reload its iSCSI configuration so the new LUN is visible to connecting initiators. The reload is non-fatal — if it fails, a warning is logged but `alloc_image` still succeeds.
|
||||
|
||||
---
|
||||
|
||||
## 5. alloc_image Deep Dive
|
||||
|
||||
`alloc_image($class, $storeid, $scfg, $vmid, $fmt, $name, $size_kb)` is called by PVE when a new virtual disk is created.
|
||||
|
||||
### Step 1: Name Resolution
|
||||
|
||||
If `$name` is not supplied, `find_free_diskname` (base-class method) generates `vm-<vmid>-disk-<N>` where `N` is the lowest unused integer. Only `raw` format is accepted.
|
||||
|
||||
### Step 2: Zvol Creation
|
||||
|
||||
```perl
|
||||
_api($scfg, 'POST', '/pool/dataset', {
|
||||
name => "$prefix/$name", # e.g. tank/proxmox/vdisks/vm-100-disk-0
|
||||
type => 'VOLUME',
|
||||
volsize => $size_kb * 1024, # API takes bytes
|
||||
sparse => JSON::true,
|
||||
});
|
||||
```
|
||||
|
||||
`$prefix` is assembled by `_zvol_prefix`: `truenas_pool` + optionally `/ truenas_dataset`. Sparse provisioning is always on.
|
||||
|
||||
### Step 3: iSCSI Extent Creation
|
||||
|
||||
```perl
|
||||
_api($scfg, 'POST', '/iscsi/extent', {
|
||||
name => $name, # e.g. vm-100-disk-0
|
||||
type => 'DISK',
|
||||
disk => "zvol/$prefix/$name", # zvol/ prefix required by TrueNAS
|
||||
ro => JSON::false,
|
||||
});
|
||||
```
|
||||
|
||||
The extent name matches the volume name exactly. This 1:1 naming is load-bearing — `_find_extent` searches extents by name.
|
||||
|
||||
### Step 4: Per-VM Target Resolution
|
||||
|
||||
`_resolve_vm_target($scfg, $vmid)` returns `{ id, iqn }`. If the target does not yet exist, it is created here. LUN ID assignment (`_next_lun_id`): queries `GET /iscsi/targetextent?target=<id>` and returns the lowest unused non-negative integer.
|
||||
|
||||
### Step 5: targetextent Creation
|
||||
|
||||
```perl
|
||||
_api($scfg, 'POST', '/iscsi/targetextent', {
|
||||
target => $target_id,
|
||||
extent => $extent_id,
|
||||
lunid => $lun_id,
|
||||
});
|
||||
```
|
||||
|
||||
### Step 6: Service Reload and Return
|
||||
|
||||
`_reload_iscsi` triggers the TrueNAS iSCSI service reload. The function returns `$name`, which PVE stores in the VM config.
|
||||
|
||||
### Rollback on Failure
|
||||
|
||||
The current implementation does not explicitly roll back a partially-created extent if targetextent creation fails — the extent and zvol remain on TrueNAS. ADR-004 documents the intended `eval {}` rollback pattern. This is tracked as a known gap in issue #250.
|
||||
|
||||
---
|
||||
|
||||
## 6. free_image Deep Dive
|
||||
|
||||
`free_image($class, $storeid, $scfg, $volname, $isBase)` is called when a disk is deleted. The deletion order is deliberately sequenced to avoid dangling references.
|
||||
|
||||
### Step 1: Locate the Extent
|
||||
|
||||
`_find_extent($scfg, $volname)` calls `GET /iscsi/extent` (by name) then `GET /iscsi/targetextent?extent=<id>`. Returns `{ extent_id, targetextent_id, lun_id, target_id }`. If no extent exists, a warning is logged and extent removal is skipped.
|
||||
|
||||
### Step 2: Unmap the targetextent
|
||||
|
||||
```perl
|
||||
_api($scfg, 'DELETE', "/iscsi/targetextent/id/$ext->{targetextent_id}");
|
||||
```
|
||||
|
||||
This must happen before the extent DELETE. Removing the targetextent severs this LUN's association without affecting the target session or other LUNs on the same target — critical for the live-migration case.
|
||||
|
||||
### Step 3: Delete the Extent
|
||||
|
||||
```perl
|
||||
_api($scfg, 'DELETE', "/iscsi/extent/id/$ext->{extent_id}", { force => JSON::true });
|
||||
```
|
||||
|
||||
`force=true` is required to delete an extent that may still have a residual QEMU libiscsi session open (e.g., detached disk on a running VM).
|
||||
|
||||
### Step 4: Clean Up the Per-VM Target if Empty
|
||||
|
||||
`_maybe_cleanup_vm_target($scfg, $vmid, $target_id)` queries `GET /iscsi/targetextent?target=<id>` after the targetextent was removed. If no mappings remain, `DELETE /iscsi/target/id/<id>` removes the empty target.
|
||||
|
||||
### Step 5: Delete the Zvol
|
||||
|
||||
```perl
|
||||
my $enc = uri_escape($zvol, "^A-Za-z0-9\\-_.~");
|
||||
_api($scfg, 'DELETE', "/pool/dataset/id/$enc", { recursive => JSON::true });
|
||||
```
|
||||
|
||||
`recursive=true` handles any ZFS snapshots under the zvol. Zvol deletion is last — if it fails, the error propagates after extent and targetextent are already cleaned up.
|
||||
|
||||
---
|
||||
|
||||
## 7. Build Pipeline and Version/Channel Routing
|
||||
|
||||
### Single-Repo Architecture (ADR-001)
|
||||
|
||||
All build and packaging logic lives in `.github/workflows/build.yml`. The old two-repo dispatch approach (this repo → `freenas-proxmox-packer`) was eliminated. Files are embedded in the `.deb` at build time; no `git clone` or `patch` runs at install time.
|
||||
|
||||
### Four-Job Pipeline
|
||||
|
||||
```
|
||||
lint → build → security → publish
|
||||
```
|
||||
|
||||
**Job 1: Lint**
|
||||
- `perl -c -I/tmp/pve-stub perl5/PVE/Storage/Custom/TrueNAS.pm` — syntax check with PVE module stubs
|
||||
- `perlcritic --profile .perlcriticrc` — static analysis at severity 4
|
||||
- `shellcheck --severity=warning packaging/DEBIAN/postinst packaging/DEBIAN/postrm`
|
||||
|
||||
**Job 2: Build**
|
||||
- Reads `our $VERSION` from `TrueNAS.pm` as the base version (ADR-006)
|
||||
- Resolves version string and target channel based on `$GITHUB_REF`
|
||||
- Generates `dist/DEBIAN/control` from `packaging/DEBIAN/control.j2` via `sed`
|
||||
- Runs `dpkg-deb -Zgzip --build dist <deb_file>`
|
||||
- Uploads the `.deb` as a workflow artifact (30-day retention)
|
||||
|
||||
**Job 3: Security**
|
||||
- Trivy filesystem scan (secrets + misconfig) on the repository
|
||||
- Trivy scan of extracted `.deb` contents (vuln + secret)
|
||||
- Both scans use `exit-code: 1` — HIGH or CRITICAL findings fail the pipeline
|
||||
|
||||
**Job 4: Publish**
|
||||
- Skipped for PRs and branches with `channel=none`
|
||||
- Cloudsmith `deb` push to the resolved repo
|
||||
- On tagged releases: creates a draft GitHub Release with the `.deb` attached
|
||||
|
||||
### Version String Logic
|
||||
|
||||
| Git ref | Debian version string | Channel | Cloudsmith repo |
|
||||
|---|---|---|---|
|
||||
| `refs/tags/v3.0.0` | `3.0.0-1` | stable | `truenas-proxmox` |
|
||||
| `refs/heads/master` or `refs/heads/release/3.x` | `3.0.0~beta+abc1234` | testing | `truenas-proxmox-testing` |
|
||||
| `refs/heads/release/*` (other) | `3.0.0~alpha+abc1234` | development | `truenas-proxmox-snapshots` |
|
||||
| feature branches, PRs | `3.0.0~dev+abc1234` | none | not published |
|
||||
|
||||
The tilde (`~`) in the Debian version sorts below the base version in `dpkg`, guaranteeing pre-release builds never auto-upgrade over a stable release.
|
||||
|
||||
### Running the Build Locally
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
sudo apt-get install -y libperl-critic-perl shellcheck libwww-perl libjson-perl liburi-perl
|
||||
|
||||
# Create PVE stubs
|
||||
mkdir -p /tmp/pve-stub/PVE/Storage
|
||||
printf 'package PVE::SafeSyslog;\nuse Exporter "import";\nour @EXPORT = qw(syslog);\nsub syslog {}\n1;\n' \
|
||||
> /tmp/pve-stub/PVE/SafeSyslog.pm
|
||||
printf 'package PVE::Tools;\nuse Exporter "import";\nour @EXPORT_OK = qw(run_command);\nsub run_command {}\n1;\n' \
|
||||
> /tmp/pve-stub/PVE/Tools.pm
|
||||
printf 'package PVE::Storage::Plugin;\nsub new {}\nsub register {}\nsub lookup_types { [] }\nsub properties { {} }\nsub options { {} }\n1;\n' \
|
||||
> /tmp/pve-stub/PVE/Storage/Plugin.pm
|
||||
|
||||
# Syntax check + lint
|
||||
perl -c -I/tmp/pve-stub perl5/PVE/Storage/Custom/TrueNAS.pm
|
||||
perlcritic --profile .perlcriticrc perl5/PVE/Storage/Custom/TrueNAS.pm
|
||||
shellcheck --severity=warning packaging/DEBIAN/postinst packaging/DEBIAN/postrm
|
||||
|
||||
# Build the package
|
||||
VERSION="3.0.0-dev"
|
||||
mkdir -p dist/DEBIAN dist/usr/share/freenas-proxmox
|
||||
sed "s/\${VERSION}/$VERSION/" packaging/DEBIAN/control.j2 > dist/DEBIAN/control
|
||||
cp packaging/DEBIAN/postinst packaging/DEBIAN/postrm dist/DEBIAN/
|
||||
chmod 0755 dist/DEBIAN/postinst dist/DEBIAN/postrm
|
||||
cp perl5/PVE/Storage/Custom/TrueNAS.pm dist/usr/share/freenas-proxmox/TrueNAS.pm
|
||||
cp ui/truenas-storage.js dist/usr/share/freenas-proxmox/truenas-storage.js
|
||||
sudo dpkg-deb -Zgzip --build dist freenas-proxmox_${VERSION}_all.deb
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Debugging Guide
|
||||
|
||||
### Layer Model
|
||||
|
||||
Errors fall into one of four distinct layers:
|
||||
|
||||
```
|
||||
PVE task / VM operation
|
||||
└─ Plugin (TrueNAS.pm) — syslog "TrueNASPlugin: ..."
|
||||
└─ TrueNAS REST API — HTTP errors, JSON responses
|
||||
└─ iSCSI / QEMU libiscsi — QEMU logs, dmesg
|
||||
└─ ZFS — TrueNAS ZFS pool logs
|
||||
```
|
||||
|
||||
### Plugin Logs (Layer 1)
|
||||
|
||||
```bash
|
||||
# All plugin messages
|
||||
grep -i 'TrueNASPlugin' /var/log/syslog
|
||||
|
||||
# Follow live
|
||||
tail -f /var/log/syslog | grep -i TrueNAS
|
||||
```
|
||||
|
||||
Log levels: `info` (normal operation), `warning` (non-fatal issues), `err` (fatal — operation dies).
|
||||
|
||||
Example sequence for a successful `alloc_image`:
|
||||
|
||||
```
|
||||
TrueNASPlugin: alloc_image: creating zvol tank/proxmox/vdisks/vm-100-disk-0 (34359738368 bytes) for VM 100
|
||||
TrueNASPlugin: Created per-VM target: iqn.2005-10.org.freenas.ctl:proxmox-vm-100 (id=42)
|
||||
TrueNASPlugin: alloc_image: vm-100-disk-0 ready at lun 0 on iqn.2005-10.org.freenas.ctl:proxmox-vm-100
|
||||
```
|
||||
|
||||
### PVE Task Logs (Layer 1)
|
||||
|
||||
```bash
|
||||
# Recent storage tasks
|
||||
pvesh get /nodes/<nodename>/tasks --limit 50 | grep -i storage
|
||||
|
||||
# Specific task log
|
||||
pvesh get /nodes/<nodename>/tasks/<upid>/log
|
||||
|
||||
# VM start failures
|
||||
tail -f /var/log/pve/qemu-server/<vmid>.log
|
||||
```
|
||||
|
||||
### TrueNAS API Errors (Layer 2)
|
||||
|
||||
Common errors and causes:
|
||||
|
||||
| Error | Likely cause |
|
||||
|---|---|
|
||||
| `401 Unauthorized` | API key missing, revoked, or wrong |
|
||||
| `404 Not Found` on extent DELETE | Extent already deleted or never created |
|
||||
| `422 Unprocessable Entity` | Payload validation error — check TrueNAS system log for detail |
|
||||
| `500 Internal Server Error` | TrueNAS-side error — check TrueNAS system log |
|
||||
| Connection refused / timeout | TrueNAS host unreachable; check `truenas_host` or `truenas_portal_ip` |
|
||||
|
||||
**Probe the API manually:**
|
||||
|
||||
```bash
|
||||
# Test connectivity and auth
|
||||
curl -sk -H "Authorization: Bearer <token>" \
|
||||
https://<truenas_host>/api/v2.0/iscsi/global | python3 -m json.tool
|
||||
|
||||
# List all extents
|
||||
curl -sk -H "Authorization: Bearer <token>" \
|
||||
https://<truenas_host>/api/v2.0/iscsi/extent | python3 -m json.tool
|
||||
|
||||
# List all targetextents
|
||||
curl -sk -H "Authorization: Bearer <token>" \
|
||||
https://<truenas_host>/api/v2.0/iscsi/targetextent | python3 -m json.tool
|
||||
```
|
||||
|
||||
### Dangling Resources (Layer 2)
|
||||
|
||||
If `alloc_image` fails partway through, orphaned extents may remain on TrueNAS. Identify unmapped extents:
|
||||
|
||||
```bash
|
||||
# Get all extent IDs
|
||||
curl -sk -H "Authorization: Bearer $TOKEN" \
|
||||
https://<truenas>/api/v2.0/iscsi/extent | python3 -c \
|
||||
"import json,sys; [print(e['id'],e['name']) for e in json.load(sys.stdin)]"
|
||||
|
||||
# Get all mapped extent IDs
|
||||
curl -sk -H "Authorization: Bearer $TOKEN" \
|
||||
https://<truenas>/api/v2.0/iscsi/targetextent | python3 -c \
|
||||
"import json,sys; [print(te['extent']) for te in json.load(sys.stdin)]"
|
||||
```
|
||||
|
||||
Manual cleanup of a dangling extent:
|
||||
|
||||
```bash
|
||||
curl -sk -X DELETE -H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"force": true}' \
|
||||
https://<truenas>/api/v2.0/iscsi/extent/id/<id>
|
||||
```
|
||||
|
||||
### iSCSI / QEMU Connection Issues (Layer 3)
|
||||
|
||||
```bash
|
||||
tail -f /var/log/pve/qemu-server/<vmid>.log
|
||||
```
|
||||
|
||||
Common libiscsi errors:
|
||||
|
||||
| Error | Meaning |
|
||||
|---|---|
|
||||
| `Failed to connect to iSCSI portal` | Portal IP wrong, TrueNAS iSCSI service not running, or firewall |
|
||||
| `Initiator has no access to this target` | Initiator group does not include the Proxmox node's IQN |
|
||||
| `LUN not found` | Extent not mapped, or iSCSI service not reloaded after extent creation |
|
||||
| `Target not found` | IQN mismatch — check `_basename()` vs TrueNAS's actual global basename |
|
||||
|
||||
Check the Proxmox node's initiator IQN:
|
||||
|
||||
```bash
|
||||
cat /etc/iscsi/initiatorname.iscsi
|
||||
```
|
||||
|
||||
### Plugin Not Loading
|
||||
|
||||
```bash
|
||||
grep -i 'TrueNAS\|Storage::Custom\|Can.t locate' /var/log/syslog
|
||||
|
||||
# Verify syntax directly
|
||||
perl -c /usr/share/perl5/PVE/Storage/Custom/TrueNAS.pm
|
||||
|
||||
# Check required Perl modules
|
||||
dpkg -l libwww-perl libjson-perl liburi-perl libio-socket-ssl-perl
|
||||
```
|
||||
|
||||
### Install / Removal Log
|
||||
|
||||
```bash
|
||||
cat /var/log/freenas-proxmox-install.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Contributing
|
||||
|
||||
### Quick Iteration on a Live Proxmox Node
|
||||
|
||||
For changes to `TrueNAS.pm` only:
|
||||
|
||||
```bash
|
||||
scp perl5/PVE/Storage/Custom/TrueNAS.pm \
|
||||
root@<proxmox-node>:/usr/share/perl5/PVE/Storage/Custom/TrueNAS.pm
|
||||
ssh root@<proxmox-node> "pvedaemon restart && pveproxy restart"
|
||||
```
|
||||
|
||||
For UI changes (`truenas-storage.js`):
|
||||
|
||||
```bash
|
||||
scp ui/truenas-storage.js \
|
||||
root@<proxmox-node>:/usr/share/pve-manager/js/truenas-storage.js
|
||||
# Hard-refresh browser (Ctrl+Shift+R) — no service restart needed
|
||||
```
|
||||
|
||||
### Coding Standards
|
||||
|
||||
**Perl:**
|
||||
- `use strict` and `use warnings` — no exceptions
|
||||
- All TrueNAS API calls through `_api()` — never raw `LWP::UserAgent` outside that function
|
||||
- Multi-step operations must use `eval {}` rollback: if step N fails, undo steps 1 through N-1 in reverse before dying (ADR-004)
|
||||
- `JSON::true` / `JSON::false` for boolean JSON fields
|
||||
|
||||
**Shell (`postinst`/`postrm`):**
|
||||
- `set -e` at top
|
||||
- All output via the `log()` function (writes to `$LOG_FILE`)
|
||||
- Operations must be idempotent — check before acting
|
||||
- `shellcheck --severity=warning` must pass
|
||||
|
||||
### Branch and PR Workflow
|
||||
|
||||
1. Open a GitHub issue before writing code
|
||||
2. Branch from `master`: `git checkout -b feature/short-description`
|
||||
3. Run lint locally before pushing
|
||||
4. Open a PR against `master` — CI runs lint → build → security automatically
|
||||
5. Close the issue with commit SHA and version when the change ships
|
||||
|
||||
### Architecture Decision Records
|
||||
|
||||
Major design decisions live in `.claude/cos/adrs/`. Check before making changes that affect the plugin's architecture, auth model, build pipeline, or supported platform matrix.
|
||||
|
||||
| ADR | Decision |
|
||||
|---|---|
|
||||
| ADR-001 | Consolidate build pipeline into this repo; no install-time git clone |
|
||||
| ADR-002 | Full `PVE::Storage::Custom` plugin; no ZFSPlugin.pm or pvemanagerlib.js patches |
|
||||
| ADR-003 | Transition from Cloudsmith to GitHub Pages apt repo (accepted, not yet implemented) |
|
||||
| ADR-004 | `eval {}` rollback pattern for all multi-step TrueNAS operations |
|
||||
| ADR-005 | Bearer token (API key) as primary auth; basic auth removed in v3.0 |
|
||||
| ADR-006 | `$VERSION` in `TrueNAS.pm` is the single source of truth for package versioning |
|
||||
| ADR-007 | v2.x keeps `FreeNAS` naming; v3.x uses `TrueNAS` everywhere |
|
||||
| ADR-008 | v3.0 supports PVE 8+ (core); PVE 9+ snapshot interface targeted for v3.1.0 |
|
||||
|
||||
---
|
||||
|
||||
## Key File Reference
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| [perl5/PVE/Storage/Custom/TrueNAS.pm](../perl5/PVE/Storage/Custom/TrueNAS.pm) | Main plugin — all storage operations |
|
||||
| [ui/truenas-storage.js](../ui/truenas-storage.js) | Ext.js UI panel for the `truenas` storage type |
|
||||
| [packaging/DEBIAN/postinst](../packaging/DEBIAN/postinst) | Install script — copies files, injects script tag, restarts PVE |
|
||||
| [packaging/DEBIAN/postrm](../packaging/DEBIAN/postrm) | Removal script — reverses postinst changes |
|
||||
| [packaging/DEBIAN/control.j2](../packaging/DEBIAN/control.j2) | Debian package control file template |
|
||||
| [.github/workflows/build.yml](../.github/workflows/build.yml) | Full CI/CD pipeline |
|
||||
| [.claude/cos/adrs/](../.claude/cos/adrs/) | Architecture Decision Records |
|
||||
|
|
@ -0,0 +1,307 @@
|
|||
# Getting Started with freenas-proxmox v3.0
|
||||
|
||||
## 1. What This Plugin Does
|
||||
|
||||
Proxmox VE is a hypervisor — it runs your virtual machines. Those VMs need somewhere to store their disk images. If you have a TrueNAS storage server on your network, you might want to store those disks on TrueNAS so they live on dedicated, reliable, ZFS-managed storage instead of inside the Proxmox host itself. This plugin makes that possible. It registers a new storage type called **TrueNAS (ZFS/iSCSI)** inside Proxmox. When you add a VM disk to that storage, Proxmox talks to your TrueNAS server through its REST API, creates a ZFS volume on TrueNAS, exposes it as an iSCSI block device, and gives your VM a direct high-speed path to it — all automatically. You do not need to touch the TrueNAS UI every time you create or delete a VM disk.
|
||||
|
||||
---
|
||||
|
||||
## 2. Prerequisites
|
||||
|
||||
Complete all of these before installing the plugin.
|
||||
|
||||
### 2.1 Proxmox VE Version
|
||||
|
||||
Your Proxmox VE node (or cluster) must be running **PVE 8.x or PVE 9.x**. Check with:
|
||||
|
||||
```bash
|
||||
pveversion
|
||||
```
|
||||
|
||||
If the output shows `pve-manager/7.x`, stay on plugin v2.x — v3.0 does not support PVE 7.
|
||||
|
||||
### 2.2 TrueNAS Version
|
||||
|
||||
| TrueNAS product | Minimum version |
|
||||
|:----------------|:----------------|
|
||||
| TrueNAS SCALE | Electric Eel (24.10) or later |
|
||||
| TrueNAS CORE | 13.0-U6 or later |
|
||||
|
||||
### 2.3 iSCSI Service on TrueNAS
|
||||
|
||||
The plugin creates per-VM iSCSI targets for you automatically. It does **not** create the iSCSI service infrastructure — you must set that up once.
|
||||
|
||||
**On TrueNAS SCALE:**
|
||||
1. Go to **Shares → iSCSI → Configure**
|
||||
2. Under **Portals**, add a portal that listens on the IP address your Proxmox nodes use to reach TrueNAS (or `0.0.0.0` to listen on all interfaces). Note the portal's IP address — you will need it.
|
||||
3. Under **Initiators**, add an initiator group. You can leave it open (allow all) for initial setup; lock it down to specific Proxmox node IPs later.
|
||||
4. Enable the iSCSI service and confirm it is running.
|
||||
|
||||
**On TrueNAS CORE:**
|
||||
1. Go to **Sharing → Block Shares (iSCSI)**
|
||||
2. Under **Portals**, add a portal (same as above)
|
||||
3. Under **Initiators**, add an initiator group
|
||||
4. Go to **Services** and enable **iSCSI**
|
||||
|
||||
You do **not** need to create any targets or extents by hand — the plugin creates those automatically when you allocate VM disks.
|
||||
|
||||
### 2.4 TrueNAS API Key
|
||||
|
||||
The plugin authenticates with TrueNAS using an API key (Bearer token). No username or password is used.
|
||||
|
||||
**On TrueNAS SCALE:**
|
||||
Go to **System Settings → API Keys → Add**. Give it a descriptive name such as `proxmox-plugin`. Copy the key — it is shown only once.
|
||||
|
||||
**On TrueNAS CORE:**
|
||||
Click the gear icon in the top-right corner → **API Keys → Add**. Copy the key.
|
||||
|
||||
Store the key somewhere safe temporarily. You will paste it into the Proxmox UI during storage configuration.
|
||||
|
||||
### 2.5 Network Reachability
|
||||
|
||||
- Every Proxmox node must be able to reach the TrueNAS host on **TCP 443** (HTTPS API) or **TCP 80** (HTTP, not recommended)
|
||||
- Every Proxmox node must be able to reach the iSCSI portal on **TCP 3260**
|
||||
|
||||
No SSH keys are required. No pre-created iSCSI targets are required.
|
||||
|
||||
### 2.6 What You Cannot Store on This Storage
|
||||
|
||||
TPM state disks (`tpmstate0`) cannot live on this storage type. If you plan to create VMs with Secure Boot / TPM enabled, configure a separate local-lvm or NFS storage for those disks. All other disk types (virtio, scsi, IDE, EFI) work normally.
|
||||
|
||||
---
|
||||
|
||||
## 3. Installation
|
||||
|
||||
Run all of these commands on **each Proxmox node** in your cluster, or on your single Proxmox host. Run them as root.
|
||||
|
||||
### Step 1 — Import the GPG Signing Key
|
||||
|
||||
```bash
|
||||
curl -fsSL https://dl.cloudsmith.io/public/ksatechnologies/truenas-proxmox/gpg.284C106104A8CE6D.key \
|
||||
| gpg --dearmor \
|
||||
| tee /usr/share/keyrings/ksatechnologies-truenas-proxmox-keyring.gpg > /dev/null
|
||||
```
|
||||
|
||||
### Step 2 — Add the Package Repository
|
||||
|
||||
```bash
|
||||
cat > /etc/apt/sources.list.d/ksatechnologies-repo.list << 'EOF'
|
||||
deb [signed-by=/usr/share/keyrings/ksatechnologies-truenas-proxmox-keyring.gpg] \
|
||||
https://dl.cloudsmith.io/public/ksatechnologies/truenas-proxmox/deb/debian any-version main
|
||||
EOF
|
||||
```
|
||||
|
||||
### Step 3 — Install the Package
|
||||
|
||||
```bash
|
||||
apt update && apt install freenas-proxmox
|
||||
```
|
||||
|
||||
The installer:
|
||||
- Copies `TrueNAS.pm` to `/usr/share/perl5/PVE/Storage/Custom/`
|
||||
- Copies `truenas-storage.js` to `/usr/share/pve-manager/js/`
|
||||
- Adds one `<script>` tag to `/usr/share/pve-manager/index.html.tpl` so the UI loads the new panel
|
||||
- Restarts `pvedaemon` and `pveproxy`
|
||||
|
||||
It does **not** patch any PVE system files.
|
||||
|
||||
### Step 4 — Refresh Your Browser
|
||||
|
||||
Open (or reload) the Proxmox web UI. Do a hard refresh: **Ctrl+Shift+R** (Windows/Linux) or **Cmd+Shift+R** (Mac). The TrueNAS storage type will not appear until the browser picks up the new JavaScript.
|
||||
|
||||
---
|
||||
|
||||
## 4. First Storage Configuration in the Proxmox UI
|
||||
|
||||
In the Proxmox web UI:
|
||||
|
||||
1. Click **Datacenter** in the left tree
|
||||
2. Click **Storage** in the top tabs
|
||||
3. Click **Add**
|
||||
4. Choose **TrueNAS (ZFS/iSCSI)** from the dropdown
|
||||
|
||||
If **TrueNAS (ZFS/iSCSI)** does not appear in the list, see [section 6.1](#61-truenas-zfsiscsi-does-not-appear-in-the-add-storage-dropdown) below.
|
||||
|
||||
### Field Reference
|
||||
|
||||
Fill in the form using the table below. Fields not listed can be left at their defaults.
|
||||
|
||||
| Field | What to enter | Notes |
|
||||
|-------|---------------|-------|
|
||||
| **ID** | A short name, e.g. `truenas-vms` | How Proxmox refers to the storage internally. Letters, numbers, and hyphens only. Cannot be changed later. |
|
||||
| **TrueNAS Host** | IP address or hostname of your TrueNAS server | Example: `192.168.10.50` or `truenas.local`. Also used as the iSCSI portal address unless you set Portal IP separately. |
|
||||
| **API Key** | Paste the API key you generated in step 2.4 | Treated as a password — hidden by default. Click the eye icon to show it while pasting. |
|
||||
| **Pool / Dataset Path** | The ZFS pool or dataset path where VM disks should be created | Use `tank` for the pool root, or `tank/proxmox/vdisks` for a specific dataset. |
|
||||
| **Use SSL** | Leave checked (on) | Recommended. Turn off only if TrueNAS has no HTTPS configured. |
|
||||
| **Verify SSL Certificate** | Leave unchecked | Turn on only if TrueNAS has a valid CA-signed certificate. Most homelab setups use self-signed certs — leave this off. |
|
||||
| **Shared** | Leave checked (on) | Required for clusters. Tells Proxmox all nodes can access this storage. |
|
||||
| **Portal IP** | Optional — leave blank | If your TrueNAS management interface and iSCSI data interface are on different IPs, put the iSCSI data IP here. Otherwise the TrueNAS Host IP is used for both. |
|
||||
| **Target IQN** | Optional — leave blank | The plugin auto-discovers the portal and initiator group from existing iSCSI targets. Leave blank unless you want to force it to copy settings from a specific target. |
|
||||
|
||||
Click **Add**. Proxmox contacts TrueNAS and confirms the pool is reachable. If it fails, check the error message — see [section 6](#6-common-first-run-problems) for common causes.
|
||||
|
||||
---
|
||||
|
||||
## 5. Creating Your First VM Disk on TrueNAS Storage
|
||||
|
||||
### During VM Creation
|
||||
|
||||
1. In the Proxmox UI, click **Create VM**
|
||||
2. On the **Disks** tab, change **Storage** from `local-lvm` to the storage ID you just created (e.g., `truenas-vms`)
|
||||
3. Set the disk size
|
||||
4. Complete the rest of the VM setup and click **Finish**
|
||||
|
||||
Proxmox will:
|
||||
- Create a ZFS volume (`zvol`) on TrueNAS sized to your request
|
||||
- Create an iSCSI extent pointing to that zvol
|
||||
- Create a dedicated iSCSI target named `proxmox-vm-<vmid>` on TrueNAS (e.g., `proxmox-vm-100`)
|
||||
- Map the extent to the target
|
||||
- Give QEMU an `iscsi://` path to connect directly
|
||||
|
||||
When the VM starts, QEMU opens an iSCSI connection directly to TrueNAS — no kernel session management (`iscsiadm`) is involved.
|
||||
|
||||
### Adding a Disk to an Existing VM
|
||||
|
||||
1. Select the VM in the left tree
|
||||
2. Click **Hardware**
|
||||
3. Click **Add → Hard Disk**
|
||||
4. Set **Storage** to your TrueNAS storage
|
||||
5. Set the disk size
|
||||
6. Click **Add**
|
||||
|
||||
### What You Will See on TrueNAS
|
||||
|
||||
After creating a VM disk, log into TrueNAS and check:
|
||||
- **Datasets** (SCALE) or **Storage → Pools** (CORE): a new zvol named `vm-100-disk-0` (or similar) will appear under your configured pool/dataset
|
||||
- **Shares → iSCSI → Targets**: a target named `proxmox-vm-100` will appear
|
||||
- **Shares → iSCSI → Extents**: an extent named `vm-100-disk-0` will appear
|
||||
|
||||
When you delete the disk from Proxmox, all three are removed automatically. If the deleted disk was the last disk on that VM's target, the target is removed too.
|
||||
|
||||
---
|
||||
|
||||
## 6. Common First-Run Problems
|
||||
|
||||
### 6.1 "TrueNAS (ZFS/iSCSI)" Does Not Appear in the Add Storage Dropdown
|
||||
|
||||
**Cause:** Browser has cached the old Proxmox JavaScript.
|
||||
|
||||
**Fix:** Hard-refresh the browser: **Ctrl+Shift+R** (Windows/Linux) or **Cmd+Shift+R** (Mac). If it still does not appear, check that the package installed correctly:
|
||||
|
||||
```bash
|
||||
ls /usr/share/perl5/PVE/Storage/Custom/TrueNAS.pm
|
||||
ls /usr/share/pve-manager/js/truenas-storage.js
|
||||
```
|
||||
|
||||
Both files should exist. If they are missing, re-run `apt install freenas-proxmox`. Also check whether `pvedaemon` and `pveproxy` restarted cleanly:
|
||||
|
||||
```bash
|
||||
systemctl status pvedaemon pveproxy
|
||||
```
|
||||
|
||||
### 6.2 Storage Shows as Unavailable or Add Dialog Returns an Error
|
||||
|
||||
**Cause:** The plugin cannot reach the TrueNAS API.
|
||||
|
||||
**Check the logs first:**
|
||||
|
||||
```bash
|
||||
journalctl -u pvedaemon --since "10 minutes ago" | grep -i truenas
|
||||
```
|
||||
|
||||
Common specific errors:
|
||||
|
||||
| Log message | Fix |
|
||||
|-------------|-----|
|
||||
| `TrueNAS API key is not configured` | API key field was left blank; edit the storage and paste the key |
|
||||
| `401 Unauthorized` | API key is wrong, expired, or revoked; generate a new one in TrueNAS |
|
||||
| `500 Internal Server Error` or connection refused | SSL mismatch — if TrueNAS uses a self-signed certificate and Use SSL is on, try toggling Verify SSL Certificate off. Or check that TrueNAS is reachable on HTTPS from the Proxmox node |
|
||||
| `Pool dataset 'tank' not found` | The pool name you typed does not match what TrueNAS has; check under **Storage → Pools** in TrueNAS |
|
||||
|
||||
**Quick connectivity test** from the Proxmox node shell:
|
||||
|
||||
```bash
|
||||
# Replace 192.168.10.50 and your-api-key with real values
|
||||
curl -sk -H "Authorization: Bearer your-api-key" \
|
||||
https://192.168.10.50/api/v2.0/iscsi/global | python3 -m json.tool
|
||||
```
|
||||
|
||||
If this returns JSON with a `basename` field, the API is working. If it returns an HTML login page or a TLS error, you have a connectivity or SSL problem.
|
||||
|
||||
### 6.3 Disk Creation Fails with "No iSCSI Portals Found"
|
||||
|
||||
**Cause:** The plugin could not match the TrueNAS Host IP (or Portal IP) to any portal configured in TrueNAS iSCSI settings.
|
||||
|
||||
**Fix:** In TrueNAS, go to **Shares → iSCSI → Portals** and check what IP the portal is listening on. Either:
|
||||
- Set the portal to listen on `0.0.0.0` (all interfaces), or
|
||||
- In Proxmox, edit the storage and set **Portal IP** to the exact IP the TrueNAS portal is listening on
|
||||
|
||||
### 6.4 VM Starts but Disk Is Not Accessible / VM Won't Boot
|
||||
|
||||
**Cause:** The iSCSI data path is blocked. Even if the plugin successfully created the disk (API path), the VM's QEMU process must open a separate iSCSI TCP connection to TrueNAS.
|
||||
|
||||
**Checks:**
|
||||
1. Confirm TrueNAS iSCSI service is running
|
||||
2. From the Proxmox node, test TCP connectivity to the iSCSI port:
|
||||
```bash
|
||||
nc -zv 192.168.10.50 3260
|
||||
```
|
||||
3. Check that the initiator group in TrueNAS does not block the Proxmox node's IP
|
||||
4. Check that the per-VM target in TrueNAS (`proxmox-vm-<vmid>`) has the extent associated with it under **iSCSI → Target/Extent**
|
||||
|
||||
### 6.5 API Key Stopped Working After Upgrading TrueNAS SCALE to 25.04
|
||||
|
||||
TrueNAS SCALE 25.04 may revoke API keys created under certain conditions during upgrade, and enforces HTTPS for API key authentication.
|
||||
|
||||
**Fix:**
|
||||
1. Log into TrueNAS SCALE → **Credentials → API Keys** → generate a new key
|
||||
2. In Proxmox: **Datacenter → Storage** → select your TrueNAS storage → **Edit**
|
||||
3. Paste the new API key into the **API Key** field
|
||||
4. Ensure **Use SSL** is checked
|
||||
|
||||
### 6.6 Disk Size Shown in TrueNAS Is Larger Than What I Entered in Proxmox
|
||||
|
||||
This is expected and not an error. Proxmox creates disks in GiB (base-2) but TrueNAS reports sizes in GB (base-10). An 80 GiB disk shows as approximately 85.90 GB in TrueNAS. The disk inside your VM is exactly what you asked for.
|
||||
|
||||
### 6.7 TPM State Disk Fails to Create on TrueNAS Storage
|
||||
|
||||
TPM state (`tpmstate0`) is a known limitation — this storage type cannot hold TPM state disks. When creating a VM with TPM/Secure Boot, configure a different storage (such as `local-lvm` or NFS) for the `tpmstate0` disk. All other disk types work normally.
|
||||
|
||||
---
|
||||
|
||||
## 7. Uninstalling
|
||||
|
||||
Before uninstalling, remove any storage configurations that use this plugin from Proxmox, and migrate any VM disks off TrueNAS storage. Uninstalling while VMs are still using TrueNAS disks will not delete your data, but Proxmox will lose track of those disks.
|
||||
|
||||
### Step 1 — Remove Storage Configurations
|
||||
|
||||
In the Proxmox UI: **Datacenter → Storage** → select each TrueNAS storage entry → **Remove**.
|
||||
|
||||
### Step 2 — Uninstall the Package
|
||||
|
||||
```bash
|
||||
apt remove freenas-proxmox
|
||||
```
|
||||
|
||||
The removal script:
|
||||
- Deletes `/usr/share/perl5/PVE/Storage/Custom/TrueNAS.pm`
|
||||
- Deletes `/usr/share/pve-manager/js/truenas-storage.js`
|
||||
- Removes the `<script>` tag from `/usr/share/pve-manager/index.html.tpl`
|
||||
- Restarts `pvedaemon` and `pveproxy`
|
||||
|
||||
To also remove install logs:
|
||||
|
||||
```bash
|
||||
apt purge freenas-proxmox
|
||||
```
|
||||
|
||||
### Step 3 — Refresh Your Browser
|
||||
|
||||
Hard-refresh the browser after uninstalling. The TrueNAS option will disappear from the Add Storage dropdown.
|
||||
|
||||
---
|
||||
|
||||
## Getting Help
|
||||
|
||||
- GitHub Issues: https://github.com/TheGrandWazoo/freenas-proxmox/issues
|
||||
- When reporting a bug, include log lines from `journalctl -u pvedaemon | grep -i truenas` and the output of `pveversion`
|
||||
|
|
@ -7,6 +7,7 @@ set -e
|
|||
INSTALL_DIR="/usr/share/freenas-proxmox"
|
||||
PLUGIN_DST="/usr/share/perl5/PVE/Storage/Custom/TrueNAS.pm"
|
||||
JS_DST="/usr/share/pve-manager/js/truenas-storage.js"
|
||||
HELP_DST="/usr/share/pve-docs/truenas-storage.html"
|
||||
TPL="/usr/share/pve-manager/index.html.tpl"
|
||||
SCRIPT_TAG=' <script type="text/javascript" src="/pve2/js/truenas-storage.js?ver=[% version %]"></script>'
|
||||
LOG_FILE="/var/log/freenas-proxmox-install.log"
|
||||
|
|
@ -25,6 +26,9 @@ install_ui() {
|
|||
log "Installing ${JS_DST}"
|
||||
cp "${INSTALL_DIR}/truenas-storage.js" "$JS_DST"
|
||||
|
||||
log "Installing ${HELP_DST}"
|
||||
cp "${INSTALL_DIR}/truenas-storage-help.html" "$HELP_DST"
|
||||
|
||||
if grep -qF 'truenas-storage.js' "$TPL" 2>/dev/null; then
|
||||
log "index.html.tpl already has truenas-storage.js — skipping"
|
||||
return 0
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ set -e
|
|||
|
||||
PLUGIN_DST="/usr/share/perl5/PVE/Storage/Custom/TrueNAS.pm"
|
||||
JS_DST="/usr/share/pve-manager/js/truenas-storage.js"
|
||||
HELP_DST="/usr/share/pve-docs/truenas-storage.html"
|
||||
TPL="/usr/share/pve-manager/index.html.tpl"
|
||||
LOG_FILE="/var/log/freenas-proxmox-install.log"
|
||||
|
||||
|
|
@ -12,7 +13,8 @@ log() {
|
|||
}
|
||||
|
||||
remove_ui() {
|
||||
rm -f "$JS_DST" && log "Removed ${JS_DST}"
|
||||
rm -f "$JS_DST" && log "Removed ${JS_DST}"
|
||||
rm -f "$HELP_DST" && log "Removed ${HELP_DST}"
|
||||
|
||||
if [ -f "$TPL" ] && grep -qF 'truenas-storage.js' "$TPL"; then
|
||||
log "Removing <script> tag from ${TPL}"
|
||||
|
|
@ -36,7 +38,7 @@ case "$1" in
|
|||
;;
|
||||
purge)
|
||||
log "Purging freenas-proxmox ..."
|
||||
rm -f "$PLUGIN_DST" "$JS_DST" "$LOG_FILE"
|
||||
rm -f "$PLUGIN_DST" "$JS_DST" "$HELP_DST" "$LOG_FILE"
|
||||
[ -f "$TPL" ] && sed -i '/truenas-storage\.js/d' "$TPL"
|
||||
;;
|
||||
upgrade|failed-upgrade|disappear|abort-upgrade|abort-remove|abort-deconfigure)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,231 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>TrueNAS Storage — Help</title>
|
||||
<style>
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
|
||||
font-size: 14px;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
margin: 0;
|
||||
padding: 20px 28px;
|
||||
max-width: 860px;
|
||||
}
|
||||
h1 { font-size: 1.4em; margin: 0 0 4px 0; color: #1a1a1a; }
|
||||
h2 { font-size: 1.05em; margin: 22px 0 8px 0; color: #1a1a1a;
|
||||
border-bottom: 1px solid #ddd; padding-bottom: 4px; }
|
||||
p { margin: 0 0 10px 0; }
|
||||
ul { margin: 0 0 10px 0; padding-left: 20px; }
|
||||
li { margin-bottom: 4px; }
|
||||
code {
|
||||
font-family: "SFMono-Regular", Consolas, monospace;
|
||||
font-size: 0.88em;
|
||||
background: #f4f4f4;
|
||||
border: 1px solid #ddd;
|
||||
border-radius: 3px;
|
||||
padding: 1px 4px;
|
||||
}
|
||||
pre {
|
||||
background: #f4f4f4;
|
||||
border: 1px solid #ddd;
|
||||
border-radius: 4px;
|
||||
padding: 10px 14px;
|
||||
overflow-x: auto;
|
||||
font-size: 0.88em;
|
||||
margin: 0 0 10px 0;
|
||||
}
|
||||
table {
|
||||
border-collapse: collapse;
|
||||
width: 100%;
|
||||
margin-bottom: 12px;
|
||||
font-size: 0.93em;
|
||||
}
|
||||
th {
|
||||
background: #f0f0f0;
|
||||
text-align: left;
|
||||
padding: 6px 10px;
|
||||
border: 1px solid #ccc;
|
||||
font-weight: 600;
|
||||
}
|
||||
td {
|
||||
padding: 5px 10px;
|
||||
border: 1px solid #ddd;
|
||||
vertical-align: top;
|
||||
}
|
||||
tr:nth-child(even) td { background: #fafafa; }
|
||||
.warn {
|
||||
background: #fff8e1;
|
||||
border-left: 4px solid #f9a825;
|
||||
padding: 8px 12px;
|
||||
margin-bottom: 12px;
|
||||
border-radius: 0 4px 4px 0;
|
||||
}
|
||||
.note {
|
||||
background: #e8f5e9;
|
||||
border-left: 4px solid #43a047;
|
||||
padding: 8px 12px;
|
||||
margin-bottom: 12px;
|
||||
border-radius: 0 4px 4px 0;
|
||||
}
|
||||
.footer {
|
||||
margin-top: 28px;
|
||||
padding-top: 12px;
|
||||
border-top: 1px solid #ddd;
|
||||
font-size: 0.88em;
|
||||
color: #666;
|
||||
}
|
||||
.footer a { color: #1565c0; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<h1>TrueNAS (ZFS/iSCSI) Storage</h1>
|
||||
<p>Manages TrueNAS ZFS volumes over iSCSI via the TrueNAS REST API.
|
||||
Each VM gets its own dedicated iSCSI target — no SSH keys or pre-created targets required.</p>
|
||||
|
||||
<h2>Prerequisites</h2>
|
||||
<ul>
|
||||
<li>Proxmox VE 8.x or 9.x</li>
|
||||
<li>TrueNAS CORE 13.0-U6+ or TrueNAS SCALE 24.10 (Electric Eel)+</li>
|
||||
<li>iSCSI service enabled on TrueNAS with at least one portal and one initiator group configured</li>
|
||||
<li>TCP 443 (HTTPS API) and TCP 3260 (iSCSI) reachable from every Proxmox node to TrueNAS</li>
|
||||
<li>An API key generated in TrueNAS (System Settings → API Keys)</li>
|
||||
</ul>
|
||||
|
||||
<div class="warn">
|
||||
<strong>TPM state disks</strong> (<code>tpmstate0</code>) cannot be stored on this storage type.
|
||||
If your VM uses Secure Boot / TPM, store the TPM state disk on <code>local-lvm</code> or NFS.
|
||||
All other disk types (virtio, scsi, IDE, EFI) work normally.
|
||||
</div>
|
||||
|
||||
<h2>Field Reference</h2>
|
||||
<table>
|
||||
<tr><th>Field</th><th>Required</th><th>Description</th></tr>
|
||||
<tr>
|
||||
<td><code>ID</code></td><td>Yes</td>
|
||||
<td>Storage name used internally by Proxmox. Letters, numbers, and hyphens only.
|
||||
Cannot be changed after creation.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>TrueNAS Host</code></td><td>Yes</td>
|
||||
<td>IP address or hostname of the TrueNAS server. Also used as the iSCSI portal address
|
||||
unless <code>Portal IP</code> is set separately. Cannot be changed after creation.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>API Key</code></td><td>Yes</td>
|
||||
<td>Bearer token generated in TrueNAS. Stored in <code>/etc/pve/storage.cfg</code>.
|
||||
Click the eye icon to show/hide while pasting.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>Pool / Dataset Path</code></td><td>Yes</td>
|
||||
<td>ZFS pool or dataset where volumes are created. Examples: <code>tank</code> or
|
||||
<code>tank/proxmox/vdisks</code>. Cannot be changed after creation.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>Sub-dataset</code></td><td>No</td>
|
||||
<td>Extra sub-path appended below Pool / Dataset Path. Leave blank in most cases.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>Shared</code></td><td>—</td>
|
||||
<td>Should be checked for all cluster deployments. Marks storage accessible from all nodes.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>Use SSL</code></td><td>—</td>
|
||||
<td>Use HTTPS for API calls. Recommended. Disable only if TrueNAS has no HTTPS configured.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>Verify SSL Certificate</code></td><td>—</td>
|
||||
<td>Validate the TrueNAS HTTPS certificate against a CA. Leave unchecked for self-signed
|
||||
certificates (common in homelab setups).</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>Portal IP</code></td><td>No</td>
|
||||
<td>Override the iSCSI portal address. Use when the TrueNAS management IP differs from
|
||||
the iSCSI data IP. Defaults to TrueNAS Host if blank.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>Target IQN</code></td><td>No</td>
|
||||
<td>Reference target used to discover portal and initiator group settings for new per-VM
|
||||
targets. Auto-discovered if blank.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<h2>How It Works</h2>
|
||||
<p>When you create a VM disk, the plugin:</p>
|
||||
<ol>
|
||||
<li>Creates a ZFS volume (<code>zvol</code>) on TrueNAS under your configured pool/dataset</li>
|
||||
<li>Creates an iSCSI extent pointing to that zvol</li>
|
||||
<li>Creates a dedicated iSCSI target <code>proxmox-vm-<vmid></code> if one does not exist</li>
|
||||
<li>Maps the extent to the target at the next available LUN ID</li>
|
||||
</ol>
|
||||
<p>QEMU connects directly to TrueNAS via an <code>iscsi://</code> URI — no <code>iscsiadm</code>
|
||||
session management is needed on the Proxmox host. When the last disk for a VM is deleted,
|
||||
the per-VM target is automatically removed.</p>
|
||||
|
||||
<h2>Common Problems</h2>
|
||||
|
||||
<table>
|
||||
<tr><th>Symptom</th><th>Fix</th></tr>
|
||||
<tr>
|
||||
<td>"TrueNAS (ZFS/iSCSI)" missing from Add Storage dropdown</td>
|
||||
<td>Hard-refresh the browser: <strong>Ctrl+Shift+R</strong> (Windows/Linux) or
|
||||
<strong>Cmd+Shift+R</strong> (Mac)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>401 Unauthorized in logs</td>
|
||||
<td>API key is wrong, expired, or revoked — generate a new one in TrueNAS</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>No iSCSI portals found</td>
|
||||
<td>Set the TrueNAS portal to listen on <code>0.0.0.0</code>, or set Portal IP to match
|
||||
the exact IP the portal is listening on</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>VM won't boot / disk not accessible</td>
|
||||
<td>Check that TCP 3260 is reachable from the Proxmox node:
|
||||
<code>nc -zv <truenas-ip> 3260</code>. Check the initiator group allows the
|
||||
Proxmox node's IP.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>API key stopped working after SCALE upgrade</td>
|
||||
<td>SCALE 25.04+ may revoke keys on upgrade. Generate a new API key and update the storage
|
||||
config in Proxmox.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<h2>Checking Logs</h2>
|
||||
<pre>
|
||||
# Plugin log messages on the Proxmox node
|
||||
grep -i TrueNASPlugin /var/log/syslog
|
||||
|
||||
# PVE daemon journal
|
||||
journalctl -u pvedaemon --since "30 minutes ago" | grep -i truenas
|
||||
|
||||
# Install / removal log
|
||||
cat /var/log/freenas-proxmox-install.log
|
||||
</pre>
|
||||
|
||||
<div class="note">
|
||||
<strong>Quick API test</strong> from the Proxmox node shell:<br>
|
||||
<code>curl -sk -H "Authorization: Bearer <your-api-key>"
|
||||
https://<truenas-host>/api/v2.0/iscsi/global | python3 -m json.tool</code><br>
|
||||
A JSON response with a <code>basename</code> field confirms the API key and connectivity are working.
|
||||
</div>
|
||||
|
||||
<div class="footer">
|
||||
<a href="https://github.com/TheGrandWazoo/freenas-proxmox" target="_blank">
|
||||
freenas-proxmox on GitHub
|
||||
</a>
|
||||
·
|
||||
<a href="https://github.com/TheGrandWazoo/freenas-proxmox/issues" target="_blank">
|
||||
Report an issue
|
||||
</a>
|
||||
·
|
||||
Full documentation: <code>docs/getting-started.md</code> in the repository
|
||||
</div>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -18,6 +18,13 @@
|
|||
document.head.appendChild(style);
|
||||
}());
|
||||
|
||||
// Register our help page in PVE's online help system so the Help button works
|
||||
if (typeof pveOnlineHelpInfo !== 'undefined') {
|
||||
pveOnlineHelpInfo['storage_truenas'] = {
|
||||
link: '/pve-docs/truenas-storage.html',
|
||||
};
|
||||
}
|
||||
|
||||
// Register TrueNAS in the storage type dropdown
|
||||
PVE.Utils.storageSchema.truenas = {
|
||||
name: 'TrueNAS (ZFS/iSCSI)',
|
||||
|
|
@ -28,6 +35,7 @@ PVE.Utils.storageSchema.truenas = {
|
|||
|
||||
Ext.define('PVE.storage.TrueNASInputPanel', {
|
||||
extend: 'PVE.panel.StorageBase',
|
||||
onlineHelp: 'storage_truenas',
|
||||
|
||||
initComponent: function () {
|
||||
let me = this;
|
||||
|
|
|
|||
Loading…
Reference in New Issue