fix(shutdown): exec uvicorn as PID 1 in Docker, and bound the graceful-shutdown wait

Two defects, both invisible until you ask the app to stop.

Docker never shut down gracefully at all. CMD ["sh","-c","uvicorn ..."] left
the shell as PID 1 with uvicorn as its child, and dash does not forward
signals, so docker stop SIGTERMed the shell and uvicorn never heard about it.
Measured on the shipped image: the full 10s grace period, exit 137, and no
"Shutting down" line in the log. Every stop, restart and image update was a
hard kill -- no WAL checkpoint, no MQTT disconnect, no virtual-printer
teardown. `exec` makes uvicorn PID 1; the rebuilt image now stops in 1s with
exit 0 and checkpoints the WAL.

Separately, uvicorn's timeout_graceful_shutdown defaults to None -- wait
forever for in-flight requests. An MJPEG camera stream is a response that
never completes (httptools' connection shutdown() only flips keep_alive on an
in-flight cycle, it never closes the transport), so one open camera tile
pinned the process until systemd SIGKILLed at 90s. The ordering makes it
unfixable from inside the app: uvicorn fires the lifespan shutdown -- the code
that tears the streams down -- only after connections drain.

All six launchers now pass --timeout-graceful-shutdown 5: Dockerfile,
deploy/bambuddy.service, the systemd unit and launchd plist from
install/install.sh, the SpoolBuddy installer's unit, and the Windows NSSM
registration. On timeout uvicorn cancels the request tasks; the camera
generators already unwind cleanly on CancelledError.

TimeoutStopSec raised to 30s on the units and stop_grace_period: 30s added to
compose, as backstops rather than the mechanism. On Windows NSSM's default
1500ms AppStopMethodConsole was force-killing uvicorn mid-teardown; raised to
15s, with the WM_CLOSE and thread-message stages skipped (uvicorn is a console
app with neither a window nor a message loop).
This commit is contained in:
maziggy
2026-07-11 14:44:45 +02:00
parent aba00598bb
commit ba1394db3e
8 changed files with 202 additions and 7 deletions
+13 -3
View File
@@ -34,14 +34,24 @@ Environment="PATH=INSTALL_PATH/venv/bin:/usr/local/bin:/usr/bin:/bin"
# Server configuration
# --loop asyncio is required: uvloop's SSL layer can silently truncate VP FTP
# uploads on a ragged client close over slow storage (#1896). Do not remove.
ExecStart=INSTALL_PATH/venv/bin/uvicorn backend.app.main:app --host 0.0.0.0 --port ${PORT:-8000} --loop asyncio
#
# --timeout-graceful-shutdown is also required. Uvicorn's default is to wait
# forever for in-flight requests, and an MJPEG camera stream is a response that
# never completes — one open camera tile would hang the stop until systemd gave
# up and SIGKILLed, skipping the WAL checkpoint, the MQTT disconnect and the
# virtual-printer teardown entirely. On timeout uvicorn cancels the request
# tasks; the camera generators unwind cleanly on CancelledError.
ExecStart=INSTALL_PATH/venv/bin/uvicorn backend.app.main:app --host 0.0.0.0 --port ${PORT:-8000} --loop asyncio --timeout-graceful-shutdown 5
# Restart policy
Restart=on-failure
RestartSec=5
# Graceful shutdown
TimeoutStopSec=10
# Graceful shutdown. Uvicorn now bounds its own wait at 5s and the app's own
# teardown takes ~1-2s, so this only has to be comfortably longer than that —
# it is the backstop, not the mechanism. The old 10s could clip a slow teardown
# on a Pi with several virtual printers.
TimeoutStopSec=30
# Kill zombie ffmpeg processes (timelapse processing)
ExecStartPre=-/usr/bin/pkill -9 -f "ffmpeg.*bambuddy"