mirror of
https://github.com/TheGrandWazoo/freenas-proxmox.git
synced 2026-10-20 09:38:29 +02:00
docs: troubleshooting and changelog for the WebSocket idle-close fix (#303)
Operators on 4.0.0-4.0.2 see random 'closed unexpectedly' failures and a SCALE storage flapping to inactive, with nothing pointing at the cause. The README entry names the symptom, the fixed version, and the stopgap (restart pvedaemon) plus the lock: create cleanup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
946e6313d4
commit
8d27a0a6aa
@@ -8,6 +8,30 @@ See each [GitHub Release](https://github.com/TheGrandWazoo/truenas-proxmox/relea
|
||||
|
||||
---
|
||||
|
||||
## [Unreleased] — 4.0.3
|
||||
|
||||
### Fixed
|
||||
- **WebSocket calls failed after the connection sat idle in a PVE daemon**
|
||||
(#303). TrueNAS closes an idle WebSocket after 30–60 s (measured on SCALE
|
||||
25.04 and 25.10). Long-lived `pvedaemon` workers reused their cached
|
||||
connection after that and died with `closed unexpectedly during call
|
||||
'pool.dataset.query'`. That failed VM creates (once leaving a VM in
|
||||
`lock: create`), returned HTTP 500 on content listings, and made `status()`
|
||||
report SCALE storages as inactive at random. The CLI was unaffected
|
||||
because each `qm`/`pvesm` is a fresh process.
|
||||
- Before every reuse, a cached connection must have nothing pending on
|
||||
the socket, and if it has been idle for more than 10 s it must answer
|
||||
`core.ping` within 5 s. Otherwise the plugin reconnects. A ping takes
|
||||
2–4 ms, compared with 1–4 s for a reconnect and login.
|
||||
- Read-only calls (`*.query`, `*.config`) that still hit a closed socket
|
||||
reconnect and retry once. Writes are never retried.
|
||||
|
||||
### Verified
|
||||
- Read-only idle probe on pve01: 4.0.2 fails after 60 s and 90 s of idle
|
||||
time on SCALE 25.04 and 25.10. A long-lived process with the fix ran
|
||||
`status` plus `list_images` every 45 s for 5 minutes against both hosts,
|
||||
14/14 OK.
|
||||
|
||||
## [4.0.2] — 2026-10-13
|
||||
|
||||
Volume lifecycle fixes. Every change is backward compatible and `storage.cfg` is unchanged.
|
||||
|
||||
@@ -648,6 +648,27 @@ ls /etc/pve/nodes/*/qemu-server/<id>.conf 2>/dev/null # no output → VM no lon
|
||||
|
||||
To free an orphan, use `pvesm free <storage>:<vol>`. On 4.0.1 and earlier, use the `vdisk_free` one-liner from the entry above. Freeing a VM's last disk also removes its now-empty per-VM target.
|
||||
|
||||
### "WebSocket connection ... closed unexpectedly", or a SCALE storage flapping to inactive (4.0.0–4.0.2)
|
||||
|
||||
**Symptom:** on TrueNAS SCALE 25.04+, requests through the Proxmox API, GUI or Terraform fail at random. Retrying sometimes helps. The error looks like this:
|
||||
|
||||
```
|
||||
TrueNAS WebSocket connection to <host> closed unexpectedly during call 'pool.dataset.query'
|
||||
```
|
||||
|
||||
Other symptoms:
|
||||
- VM creation fails, sometimes leaving the VM in `lock: create`.
|
||||
- Content listings return HTTP 500.
|
||||
- The storage shows `active: 0` for some requests and `active: 1` for others.
|
||||
- `qm`/`pvesm` on the command line work fine.
|
||||
|
||||
**Cause:** TrueNAS closes an idle WebSocket after 30–60 s. Each long-lived `pvedaemon` worker kept its cached connection and reused it after TrueNAS had closed it. **Fixed in v4.0.3** ([#303](https://github.com/TheGrandWazoo/freenas-proxmox/issues/303)). Now a cached connection is checked before every reuse and pinged if it has been idle, and read-only calls reconnect and retry once. TrueNAS CORE and SCALE < 25.04 (REST) were never affected.
|
||||
|
||||
**On 4.0.0–4.0.2:**
|
||||
- Upgrade.
|
||||
- If you can't upgrade yet, `systemctl restart pvedaemon` clears the stale connections, but only until they go idle again.
|
||||
- A VM left in `lock: create` with no disks can be removed with `qm unlock <vmid> && qm destroy <vmid>`.
|
||||
|
||||
### Filing a Bug Report
|
||||
|
||||
Please use the [GitHub issue tracker](https://github.com/TheGrandWazoo/truenas-proxmox/issues) and include the [bug report template](.github/ISSUE_TEMPLATE/bug_report.md). Include relevant log lines from `syslog` (search for `FreeNAS::`).
|
||||
|
||||
Reference in New Issue
Block a user