docs: troubleshooting and changelog for the WebSocket idle-close fix (#303)

Operators on 4.0.0-4.0.2 see random 'closed unexpectedly' failures and
a SCALE storage flapping to inactive, with nothing pointing at the cause.
The README entry names the symptom, the fixed version, and the stopgap
(restart pvedaemon) plus the lock: create cleanup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kevin Adams
2026-10-13 22:10:03 -04:00
co-authored by Claude Opus 5.5
parent 946e6313d4
commit 8d27a0a6aa
2 changed files with 45 additions and 0 deletions
+24
View File
@@ -8,6 +8,30 @@ See each [GitHub Release](https://github.com/TheGrandWazoo/truenas-proxmox/relea
---
## [Unreleased] — 4.0.3
### Fixed
- **WebSocket calls failed after the connection sat idle in a PVE daemon**
(#303). TrueNAS closes an idle WebSocket after 30–60 s (measured on SCALE
25.04 and 25.10). Long-lived `pvedaemon` workers reused their cached
connection after that and died with `closed unexpectedly during call
'pool.dataset.query'`. That failed VM creates (once leaving a VM in
`lock: create`), returned HTTP 500 on content listings, and made `status()`
report SCALE storages as inactive at random. The CLI was unaffected
because each `qm`/`pvesm` is a fresh process.
- Before every reuse, a cached connection must have nothing pending on
the socket, and if it has been idle for more than 10 s it must answer
`core.ping` within 5 s. Otherwise the plugin reconnects. A ping takes
2–4 ms, compared with 1–4 s for a reconnect and login.
- Read-only calls (`*.query`, `*.config`) that still hit a closed socket
reconnect and retry once. Writes are never retried.
### Verified
- Read-only idle probe on pve01: 4.0.2 fails after 60 s and 90 s of idle
time on SCALE 25.04 and 25.10. A long-lived process with the fix ran
`status` plus `list_images` every 45 s for 5 minutes against both hosts,
14/14 OK.
## [4.0.2] — 2026-10-13
Volume lifecycle fixes. Every change is backward compatible and `storage.cfg` is unchanged.
+21
View File
@@ -648,6 +648,27 @@ ls /etc/pve/nodes/*/qemu-server/<id>.conf 2>/dev/null # no output → VM no lon
To free an orphan, use `pvesm free <storage>:<vol>`. On 4.0.1 and earlier, use the `vdisk_free` one-liner from the entry above. Freeing a VM's last disk also removes its now-empty per-VM target.
### "WebSocket connection ... closed unexpectedly", or a SCALE storage flapping to inactive (4.0.0–4.0.2)
**Symptom:** on TrueNAS SCALE 25.04+, requests through the Proxmox API, GUI or Terraform fail at random. Retrying sometimes helps. The error looks like this:
```
TrueNAS WebSocket connection to <host> closed unexpectedly during call 'pool.dataset.query'
```
Other symptoms:
- VM creation fails, sometimes leaving the VM in `lock: create`.
- Content listings return HTTP 500.
- The storage shows `active: 0` for some requests and `active: 1` for others.
- `qm`/`pvesm` on the command line work fine.
**Cause:** TrueNAS closes an idle WebSocket after 30–60 s. Each long-lived `pvedaemon` worker kept its cached connection and reused it after TrueNAS had closed it. **Fixed in v4.0.3** ([#303](https://github.com/TheGrandWazoo/freenas-proxmox/issues/303)). Now a cached connection is checked before every reuse and pinged if it has been idle, and read-only calls reconnect and retry once. TrueNAS CORE and SCALE < 25.04 (REST) were never affected.
**On 4.0.0–4.0.2:**
- Upgrade.
- If you can't upgrade yet, `systemctl restart pvedaemon` clears the stale connections, but only until they go idle again.
- A VM left in `lock: create` with no disks can be removed with `qm unlock <vmid> && qm destroy <vmid>`.
### Filing a Bug Report
Please use the [GitHub issue tracker](https://github.com/TheGrandWazoo/truenas-proxmox/issues) and include the [bug report template](.github/ISSUE_TEMPLATE/bug_report.md). Include relevant log lines from `syslog` (search for `FreeNAS::`).