From c87033dbb6baa98d6814e4c1c3d199b6d9e5699a Mon Sep 17 00:00:00 2001 From: Kevin Adams Date: Tue, 1 Sep 2026 23:59:51 -0400 Subject: [PATCH] docs: document v4.0.1 multipath re-verification through the real daemon MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The post-ADR-014 multipath re-check was only run as a standalone script, the exact testing gap that caused #290 in the first place. Closed it by re-verifying through a real pveproxy/pvedaemon-driven VM disk allocate + start + teardown cycle against .92, and documented the result in ADR-014 and architecture.md §10 alongside the original AnyEvent-era verification. Co-Authored-By: Claude Sonnet 5 --- ...-014-websocket-transport-library-change.md | 42 ++++++++++++++++++- docs/architecture.md | 2 + 2 files changed, 43 insertions(+), 1 deletion(-) diff --git a/.claude/cos/adrs/ADR-014-websocket-transport-library-change.md b/.claude/cos/adrs/ADR-014-websocket-transport-library-change.md index 0bce7d8..781c5a2 100644 --- a/.claude/cos/adrs/ADR-014-websocket-transport-library-change.md +++ b/.claude/cos/adrs/ADR-014-websocket-transport-library-change.md @@ -91,7 +91,10 @@ implementation, against the same real hosts: - Full snapshot cycle (`volume_snapshot`/`_info`/`_rollback`/`_delete`) — pass - Multipath (`activate_volume`/`deactivate_volume`, real 2-path - `dm-multipath` device) — pass + `dm-multipath` device) — pass, but via a standalone test script, same as + every other bullet above. See the dedicated daemon-context re-check below — + #290's whole lesson was that standalone-script results don't prove + anything about behavior inside `pveproxy`/`pvedaemon`. - **The actual bug reproduction, re-run against the fix**: same `POST /api2/json/storage` request via the real `pveproxy`/`pvedaemon` on `pve03-hq` that previously reproduced the error — now returns HTTP 200, @@ -100,6 +103,43 @@ implementation, against the same real hosts: actually matters for this ADR — everything else was already known-good from ADR-012 and was re-run to confirm the transport swap didn't regress it. +### Multipath, specifically, through the real daemon (2026-09-01, follow-up) + +The bug reproduction above proved the fix for the base `truenas` storage type +through the real daemon path. It did not prove anything about +`truenas-multipath`, which shares `_ws_connect`/`_ws_call` but is a separate +plugin (`TrueNASMultipath.pm`) with its own `activate_volume`/`iscsiadm`/ +`dm-multipath` assembly code. Leaving that on standalone-script evidence alone +would repeat the exact gap that caused #290, so it was re-checked the same +way: + +- Deployed v4.0.1's `TrueNAS.pm`/`TrueNASMultipath.pm` to `pve01-hq` (the lab + node with the `TrueNAS-Scale2504-Multipath` storage config against `.92`), + confirmed `libprotocol-websocket-perl`/`libio-socket-ssl-perl` already + present, restarted `pvedaemon`/`pveproxy`/`pvestatd`. +- `GET .../storage/TrueNAS-Scale2504-Multipath/status` via real `pveproxy` → + HTTP 200, `active:1` — confirms `activate_storage` connects and authenticates + over the new transport in-daemon. +- Allocated a real disk via `POST .../storage/.../content`, created a minimal + test VM (990) referencing it, started it via + `POST .../qemu/990/status/start` — all through real `pveproxy`/`pvedaemon`, + no standalone scripts anywhere in this path. +- `multipath -ll` on `pve01-hq` showed a genuine 2-path device + (`36589cfc...`, paths `sde`/`sdf`, one per portal `192.168.69.92` and + `172.31.69.92`), and `iscsiadm -m session` showed both portal sessions + logged in for `vm-990`. +- Stopped and destroyed the VM via the real API — `multipath -ll` and + `iscsiadm -m session` afterward show a clean teardown, no orphaned sessions + or devices. +- `pve01-hq` was restored to its apt-tracked `3.2.4` build afterward (its + `sources.list` is still pinned to the `error`/v3 dist track — switching it + to `rivendell`/v4 is a separate decision, not a side effect of a + verification pass). + +Conclusion: `truenas-multipath` needed zero code or packaging changes for +v4.0.1, and that conclusion now rests on the same daemon-context evidence +standard as the base plugin fix, not just a standalone script. + ## Consequences - `packaging/DEBIAN/control.j2` Depends: `libanyevent-perl` and diff --git a/docs/architecture.md b/docs/architecture.md index 4254ca9..12b0043 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -690,6 +690,8 @@ Ships as a separate package, `truenas-proxmox-multipath` (v3.2.0+, #256), becaus Everything in §6–§7 (`alloc_image`, `free_image`, zvol/extent/target lifecycle) is **inherited unchanged** — `TrueNASMultipath.pm` calls the shared `PVE::Storage::Custom::TrueNAS::_api(...)` helper directly (a fully-qualified sub call, not a virtual method), so any future transport change to that helper (e.g. the WebSocket work in [ADR-012](../.claude/cos/adrs/ADR-012-websocket-transport-v4.md)) applies to multipath automatically with no separate implementation work. **Confirmed live 2026-08-31**, not just by code inspection: a full `alloc_image`/`activate_volume`/`path`/`deactivate_volume`/`free_image` cycle against a WebSocket-transport SCALE host (`.92`, 25.04.2.6) worked with zero code changes to this file — see ADR-012's Consequences section. +When the transport's internals were rewritten for v4.0.1 ([ADR-014](../.claude/cos/adrs/ADR-014-websocket-transport-library-change.md), fixing #290 — `AnyEvent::WebSocket::Client` deadlocked inside `pveproxy`/`pvedaemon`'s own `AnyEvent` reactor), this inheritance claim was re-checked **through the real daemon, not a standalone script** — the #290 root cause was specifically that standalone-script tests can't see daemon-context deadlocks. A real disk was allocated and a real VM started against `.92` via `pveproxy`'s actual HTTP API, producing a genuine 2-path `dm-multipath` device, then torn down cleanly via the same API — all with zero changes to `TrueNASMultipath.pm`. See ADR-014's "Multipath, specifically, through the real daemon" section for the full trace. + Only four methods are overridden, replacing the `iscsi://` model from §3 with a kernel iSCSI + `dm-multipath` model: - **`path`** — returns `/dev/mapper/` instead of an `iscsi://` URI. The WWID is derived from the extent's TrueNAS NAA identifier.