After an RTSP session dropped, ffmpeg could keep writing the same JPEG
indefinitely with no connection to the printer left (two hours at ~29 fps
on a P2S), and every check that counts frames saw a healthy stream. The
stream now restarts ffmpeg after 20 s without a changed frame, on the
existing reconnect path, while viewers stay attached. If the first frame
after the restart is the same picture again, the camera really shows a
still scene (a dark, idle chamber): it is then only re-checked every
5 minutes until the picture changes.
ffmpeg opens every run with ~20 lines of version and build banner and prints
its diagnosis last, so the stderr[:200] eight of the nine call sites used kept
the banner and threw the error away. The reporter's twelve capture failures all
read "ffmpeg version 7.1.4 ... configuration: --prefix=/usr --extra-version=",
identical on every install; the exit code was the only usable byte.
The banner-stripping summariser written for #925 lived private to the camera
route. It now lives in backend/app/utils/ffmpeg_output.py and every ffmpeg and
ffprobe stderr goes through it. Two things the scattered copies also got wrong:
four logged the input URL unmasked, publishing a printer access code or camera
password, and four called a bare .decode() on bytes ffmpeg copies stream
fragments into.
-----
Delete the files a no-3MF archive owns, without taking a printer folder
Both delete paths derived the directory from file_path, which such an archive
does not have, so they removed nothing and logged it at ERROR under a SECURITY
banner. That was true when the archive was an empty row and stopped being true
once one could hold a timelapse and finish photos in <archive_dir>/<id>/ and an
uploaded source in archive/no_source/<id>/.
The two are cleaned up by different means, because <archive_dir>/<id> shares a
namespace with the per-printer folders: a normal archive lives at
<archive_dir>/<printer_id>/<timestamp>_<name>/, so archive/1 is printer 1's
folder and also the directory the shared helper hands archive id 1. Ids come
from unrelated sequences, so the first few archives collide with the printers
on every install, and an rmtree there takes every print that printer made --
measured on a scratch tree. no_source/<id> is a level deeper under a name no
printer id can take and is removed whole; the id-named directory gives up only
its photos subdirectory and the video the row records, then goes only if that
left it empty. The depth guard moves from one to two for the same reason: a
file_path that lost a path component could point the delete at a printer
folder, and no archive directory has been one level deep since the first
commit.
Hard delete had its own copy of these rules, which the helper's docstring says
it exists to prevent, and it had diverged -- it skipped the print-log thumbnail
cleanup whenever a guard tripped.
-----
Stop the RTSPS proxy leaving a handler behind at shutdown
asyncio.start_server keeps only a weak reference to the connection callback's
task, so a handler still awaiting its forwarders could be collected while
pending -- "Task was destroyed but it is pending!", at ERROR with a traceback
into camera.py, once every few hundred snapshots. Teardown had the matching
gap: server.close() leaves established connections running, so the close waited
on a handler that only finishes when the peer drops, and ffmpeg has already
been reaped by then.
Handlers are held for as long as they run and cancelled at shutdown, which is
Server.close_clients() by hand -- that landed in 3.13 and Bambuddy supports
3.10. Both the snapshot path and the streaming endpoint share the shutdown.
On a printer with an external camera, watching the live view while a print
ran meant the layer timelapse recorded almost nothing and the finish photo
went out with no image. The reporter measured 0 of 87 layer captures on one
print and 0 of 105 on another, both watched throughout. A USB camera allows
one V4L2 handle, so a capture during a live view fails outright.
The built-in camera has had this rule since #1348 and #1271: reuse the
viewer's buffered frame rather than opening a second connection. It was
never extended to the external paths, and it could not have been -- the
buffer it depends on was only ever populated by the built-in paths.
generate_mjpeg_stream yields multipart-wrapped chunks, so the route layer
could not recover the JPEG, and a guarded caller would have found an empty
buffer and skipped every time.
So the stream now publishes each raw frame through a new on_frame callback
(parallel to on_process from #2675), and the six one-shot consumers reuse
it: layer timelapse, the finish-photo moment and its background fallback,
the notification snapshot, Obico polling, and the plate check. A viewer
attached with nothing buffered yet skips that one attempt rather than
competing -- kicking the viewer off is worse than missing a frame.
on_frame exceptions are logged and swallowed, like iter_subscriber's
on_unsubscribe: buffering is a side effect and must never be able to take
the live stream down with it. The external stream's teardown now releases
the buffered frame too, ownership-checked so a concurrent viewer of the
same printer keeps its own.
Two side effects on paths not touched here, both improvements: the snapshot
endpoint and the finish-photo fallback chain consult get_buffered_frame and
can now serve an external camera's live frame. plate_detection's docstring
already claimed this behaviour while implementing it only for the built-in
fallback; that drift is resolved.
ffmpeg is spawned with stderr=PIPE and it was only read on the error
paths, so for the life of a working stream nobody read that pipe. ffmpeg
writes its banner, the input analysis, then a progress line at a steady
rate; a 64 KiB pipe fills eventually, ffmpeg blocks writing to it, frames
stop, and the stream's own 30s timeout fires -- logged as "RTSP read
timeout" with no hint that we starved it ourselves.
How long that takes is unmeasured and evidently long: one H2D upstream
ran 21m36s without stalling, and an earlier 512 B/s extrapolation of mine
was mostly the one-off startup banner. So this is a bounded resource being
treated as unbounded, not a fault anyone has reported.
_FfmpegStderrTail drains the pipe continuously and keeps a 16 KiB rolling
tail. That tail is what the error paths now report, which is better
material than before: it holds what ffmpeg said as things went wrong,
where the on-demand read returned whatever was printed first -- usually
the banner, which the summariser strips anyway.
Three readers wanted this one pipe, and asyncio rejects concurrent reads
on a StreamReader, so the collector is authoritative: it registers by pid,
_read_ffmpeg_stderr returns its tail when present and otherwise reads the
pipe unchanged, and _terminate_ffmpeg skips its own stderr drain when the
collector owns it (the collector keeps draining through teardown, which is
all wait() needs). The generator starts it after the immediate-failure
check, which reads the pipe directly because the process is already dead,
and releases it after _terminate_ffmpeg.
text() goes through _summarize_ffmpeg_stderr like every other stderr log
here, so the access code ffmpeg echoes in its input URL stays masked.
aclose() awaits the cancelled pump rather than firing and forgetting, so
no pending task survives into loop teardown.
Closing a camera view and reopening it immediately could leave the new
stream unregistered while it was running and delivering frames. The
damage was all indirect: is_stream_active() reported no viewer, so Obico
polling and snapshots opened a second camera connection against the live
view (the thing #1348 and #1271 exist to prevent); the janitor's /proc
scan found an ffmpeg missing from _active_streams and killed the live
stream as an orphan; and /camera/stop reported "Stopped 0" with a stream
running.
The fan-out stream id was f"{printer_id}-fanout" -- constant per printer,
so every successive stream shared one registry key, and the departing
generator's finally popped whatever was under it, including its
successor's entry. The same finally also cleared the per-printer frame
buffer unconditionally, discarding the new stream's frame. It needed the
two streams to overlap, which the 4s teardown made easy.
Each stream now gets its own key via _new_fanout_stream_id(), so a
generator can only clean up after itself -- the external-camera path
already does this (#2675) and this brings the fan-out path in line. The
per-printer dicts are released through _release_printer_frame_state(),
which checks that no other stream for the printer is still running; both
the RTSP and chamber-image cleanups had the same unconditional pop.
Also hoisted time and uuid to module level and dropped four
function-local `import time` statements. A local import shadows the name
for the whole function, so any use on a branch that doesn't reach the
import raises UnboundLocalError -- a real hazard in camera_stream, whose
external-camera branch imported both while the RTSP path needs them too.
A test pins camera_stream as free of function-local imports.
Closing a camera view logged "ffmpeg didn't terminate gracefully,
killing" followed by "ffmpeg did not exit within 2.0s of SIGKILL;
abandoning wait", on every single close. Both waits expired every time,
so teardown took a fixed 4.00s -- and since the firmware allows one
camera connection, that was 4s in which nothing else could use it.
ffmpeg is spawned with stdout and stderr as pipes and the teardown paths
have stopped reading them, so it sits blocked in write() on a full 64 KiB
pipe. SIGTERM cannot be acted on there: the handler only sets a flag that
the main loop polls, and the loop never gets back to the check. SIGKILL
does kill it, but asyncio resolves Process.wait()'s waiter through
_try_finish(), which requires every pipe transport to report
disconnected; paused, unread pipes never reach EOF, so wait() blocks with
returncode already set. A negative-control test shows returncode=-9 at
the instant the abandon fires.
Draining both pipes while stopping the process fixes both halves: 4.00s
becomes ~0.15s. The signal ladder and its bounds stay as backstops, so a
genuinely wedged process still cannot hang a stream, a Stop request or
the janitor.
This corrects _FFMPEG_KILL_TIMEOUT's premise and #2580's conclusion. That
12-hour hang was the unbounded form of this same self-inflicted stall, not
an ffmpeg stuck in uninterruptible I/O -- the process observed doing it
was in state S, which cannot survive a delivered SIGKILL. Bounding the
wait capped the symptom without removing the cause.
Subprocess output and user-supplied URLs are scrubbed of credentials
before they reach the application log. Adds a shared redaction helper in
core/logging_filters and routes the existing support-bundle sanitizer
through the same pattern.
Closing an external USB (V4L2) camera view abruptly could leave its ffmpeg
running and holding /dev/videoN open -- LED stuck on, and reopening the view
failed or took 10-30s fighting for exclusive device access. Same class of leak
as #776 (built-in RTSP path), but the external path was never wired into that
fix: external streams registered into none of the _active_streams /
_disconnect_events / spawned-PID registries, so /camera/stop returned
{"stopped": 0} for a live USB stream and the orphan janitor's /proc net matched
only rtsp(s)://bblp: cmdlines. Cleanup ran only via the stream generator's own
finally, which an abrupt disconnect can skip.
- Thread an on_process callback + stop_event through generate_mjpeg_stream into
_stream_usb / _stream_rtsp; register the process before the startup probe so a
process that hangs on a locked device (not just one that exits) is reapable.
- Register external streams into the shared registries under a unique
{printer_id}-ext-{token} id so /camera/stop and cleanup_orphaned_streams find
and kill them; stop_event prevents the reconnect loops from respawning.
- Extend the /proc safety-net scan to also match USB (-f v4l2) ffmpeg, excluding
still-active streams and unrelated ffmpeg.
The remaining routes of the idle-in-transaction class: the file-manager,
storage, camera-snapshot and timelapse routes each took their printer row
via Depends(get_db) and then talked FTP/camera on the same held session, so
a farm dashboard polling cover/snapshot tiles (offline printers included)
crept the pool to exhaustion over ~23h. They now read in a short session and
release before the I/O; timelapse re-opens a fresh session only for the write.
Also caps the four bare-executor FTP helpers with asyncio.wait_for so a
saturated 48-worker pool can't pin a caller (and its DB connection)
indefinitely, and runs the synchronous smtplib send off the event loop with
an explicit timeout so a wedged relay can't freeze the loop.
After an RTSP read timeout the stream cleanup killed the stalled ffmpeg
and then awaited process.wait() unbounded. A SIGKILLed ffmpeg stuck in
uninterruptible I/O on a dead RTSP socket can take arbitrarily long to
be reaped, so the fan-out stream coroutine sat parked in that wait (12
hours in the reported case) while every new viewer attached to the
stalled broadcaster and received no frames.
Bound the post-kill wait to 2s in all three places it existed: the
stream generator's _terminate_ffmpeg (the reported hang), the camera
stop endpoint (which would hang the recovery request itself; now uses
the shared helper instead of an inline copy), and the orphan-cleanup
janitor (whose hang would disable the safety net). On timeout the
zombie is abandoned; the janitor's /proc scan reaps it next pass and
the stream proceeds to its normal reconnect.
/camera/stream took its printer row via Depends(get_db). get_db is a
yield dependency, so its session stayed open until the response body
finished streaming — for a live MJPEG stream, as long as the browser
tab is open (hours). Every open camera tile pinned one pooled DB
connection idle-in-transaction, draining the pool on large farms.
Fetch the printer in a short-lived async_session() and release the
connection before returning the StreamingResponse. expire_on_commit=
False keeps the already-loaded columns readable during the stream.
streams when one viewer closes
1) Offline tiles now show OFF (not LIVE)
CameraWall.modeByPrinter assigned 'live' to any visible printer
without considering status.connected, so a disconnected X1C wasted
a live-budget slot AND rendered the red LIVE chip on top of the
WifiOff placeholder. Disconnected printers now map to 'paused' and
don't decrement liveBudget — the existing WifiOff + Off chip
rendering takes over.
2) /camera/stop no longer kills other viewers' streams
The cam-wall tile, EmbeddedCameraViewer, and the /camera/:id popup
all subscribe to the same fan-out broadcaster for a printer.
/camera/stop used to unconditionally shutdown_broadcaster() + kill
every ffmpeg process for the printer, so closing the embedded viewer
while the cam-wall tile of the same printer was live force-killed
the source the tile was pulling from — the tile's <img> errored.
New get_subscriber_count(key) accessor in camera_fanout.py exposes
the broadcaster's subscriber list length. /camera/stop now reads
that first; when >= 1 subscriber is still attached, return
{stopped: 0, skipped: true} and leave the broadcaster + ffmpeg
processes alone. The leaving viewer's HTTP teardown still runs the
natural iter_subscriber.finally -> unsubscribe path, so its slot is
released; the broadcaster keeps serving the other viewers. Single-
viewer close still hits the immediate force-teardown (count is 0).
A previous attempt swapped `-timeout` → `-stimeout` unconditionally to
fix EADDRINUSE on the reporter's transitional ffmpeg. That broke every
install on a modern ffmpeg (5+/6+/7+) — current Debian/Ubuntu/Homebrew
— where `-stimeout` was removed and `-timeout` is back to meaning
socket I/O. Verified locally: `ffmpeg -stimeout ...` errors
"Unrecognized option 'stimeout'" on ffmpeg 7.1.
install on a modern ffmpeg (5+/6+/7+) — current Debian/Ubuntu/Homebrew
— where `-stimeout` was removed and `-timeout` is back to meaning
socket I/O. Verified locally: `ffmpeg -stimeout ...` errors
"Unrecognized option 'stimeout'" on ffmpeg 7.1.
ffmpeg has shipped THREE arrangements of this option over time and
Bambuddy supports the full range:
- Pre-deprecation (early 4.x and earlier): `-timeout` is socket I/O.
- Transitional (~late-4.x, Jammy-era): `-timeout` is deprecated and
repurposed to RTSP listen-mode timeout; any non-zero value implies
`-listen`, which makes ffmpeg bind the TLS-proxy port and fail with
EADDRINUSE. `-stimeout` is the replacement socket I/O option.
- Modern (5.x / 6.x / 7.x): `-stimeout` REMOVED. `-timeout` is back to
socket I/O — the original meaning.
So no single literal is correct on all installs.
Fix: `rtsp_socket_timeout_flag()` in services/camera.py probes
`ffmpeg -h demuxer=rtsp` once and picks `-stimeout` when ffmpeg
advertises it (covers transitional + older builds that kept it as an
alias), else `-timeout` (modern + pre-deprecation). Cached at module
level for the process lifetime — ffmpeg doesn't swap mid-run.
The function returns the option name without a leading dash; callers
prepend it themselves so a formatting bug can't pass an empty flag.
Wired into both RTSP ffmpeg call sites in lockstep: routes/camera.py
(printer camera) and services/external_camera.py (external RTSP),
which use the same TLS-proxy + ffmpeg pattern and would hit the same
regression on either ffmpeg cohort.
Tests: 8 in test_ffmpeg_rtsp_timeout_flag.py — 6 probe unit tests
(prefers stimeout when advertised, falls back to timeout on modern,
defaults to timeout when ffmpeg missing or probe raises, caches across
calls, trailing-space substring guard against `-listen_timeout`
false-positives), 2 parametrised guards against either RTSP ffmpeg
argv re-hard-coding a literal instead of consuming the probe. 37
probe + existing external-camera tests green.
A P2S support bundle on 0.2.5b1 — per-model probesize fix already
applied — showed the camera still failing: ffmpeg connects, stays alive
30+ seconds, emits zero JPEG bytes, the 30s stdout.read times out,
reconnect loop repeats. No ffmpeg stderr appeared anywhere in the log to
explain why.
The cause was a diagnostic bug, not the camera path. _read_ffmpeg_stderr
called process.stderr.read() — read-to-EOF. A stalled-but-still-alive
ffmpeg (the P2S RTSP failure mode) never closes stderr, so the read
blocked until the 2s wait_for timeout and returned None, discarding the
banner + stream-analysis lines ffmpeg had already printed. ffmpeg stderr
was captured only when it fully exited; once the probesize bump turned
the earlier crash into a hang, the diagnostic went dark.
Drain stderr incrementally in bounded 8KB chunks (64KB cap), returning
whatever ffmpeg printed so far whether or not it has exited. Also log
the resolved per-model probesize/analyzeduration on the info-level
"Starting RTSP camera stream" line, and log the full ffmpeg argv at
debug level with only the credential-bearing camera URL redacted
instead of hiding the entire command.
No behaviour change to streaming — this makes the unresolved P2S RTSP
stall diagnosable in the next support bundle.
Step 2 of the camera architecture overhaul agreed after #1395. When
the camera viewer hits its error state OR before a print at any
time, a Diagnose button runs a staged check against the printer and
renders the result inline: which stage failed, how long it took,
and a translated remediation hint. Cuts off the "user opens a
'camera broken' ticket → ask for support bundle → triage" loop at
the user's screen.
Backend
- New `backend/app/services/camera_diagnose.py` orchestrator with
CameraDiagnoseResult / CameraDiagnoseStage dataclasses.
- New POST /printers/{id}/camera/diagnose route in camera.py.
- Stages:
tcp_reachable — TCP socket open to 322 (RTSP) / 6000 (chamber)
with 3 s timeout. Distinguishes timeout, refused, and host-
unreachable into distinct summary codes so the frontend can
show a precise remediation (firewall vs LAN-only off vs
wrong IP).
first_frame — captures one JPEG end-to-end via the existing
capture_camera_frame_bytes pipeline. Auth + RTSP handshake +
first keyframe collapse into one stage; the user-facing
answer is the same regardless of which sub-layer failed.
- Live-stream shortcut: when a viewer is currently watching the
camera with a buffered frame < 10 s old, the diagnostic skips
the real test and returns live_stream_active_healthy. Opening a
fresh socket would kick the live viewer off on single-camera-
connection firmwares (the #1348 reconnect-storm trigger), so we
trust the real-world evidence instead.
- Response surfaces protocol, port, and profile name for support
triage — lets us ask "what does your modal say?" instead of
"send the support bundle".
Frontend
- New CameraDiagnoseModal renders one row per stage with green-
check / red-X / grey-skipped icons, the per-stage duration in
ms, a remediation banner styled by overall status, and a Run
again button.
- Two entry points:
1. The viewer's error overlay grows a Diagnose button next to
Retry. Retry stays the primary action; Diagnose is the
escape hatch for users who can't see what's wrong.
2. A stethoscope icon in the viewer's always-visible control
bar, between Refresh and Fullscreen. Pre-flight testing
("did my firmware update break the camera?", "is the
camera up before I send a print?") doesn't require waiting
for the stream to fail first.
- Also lifted the previously-hard-coded "Camera unavailable" /
"Retry" strings into camera.unavailable / camera.retry so the
error UI is fully translated alongside the new keys.
Reporter on a P2S running firmware 01.02.00.00 saw the camera connect
for a few seconds then time out, repeating. P1S on the same install
worked fine — different protocol (chamber-image port 6000 vs RTSP via
ffmpeg).
The P2S RTSP path was running ffmpeg with `-probesize 32
-analyzeduration 0`, tuned for X1/H2 fast startup. The P2S's slower
keyframe pacing means ffmpeg can't lock onto the stream within 32
bytes — its own stderr says "consider increasing probesize" before
giving up after ~2s. Bambuddy reconnects, cycle repeats.
Instead of bumping the globals (which would regress every other RTSP
model's startup latency), this lifts the per-model tuning into a new
`camera_profiles` registry. CameraProfile dataclass holds the
previously-global knobs (probesize, analyzeduration, rtsp_reconnect_max,
rtsp_reconnect_delay, plus an extra_ffmpeg_input_args hook for future
per-model flags). get_camera_profile(model) returns the model's profile
or DEFAULT_PROFILE.
Default profile preserves the historical X1/H2 fast-startup values
verbatim — X1, X1C, X1E, X2D, H2C, H2D, H2D Pro, H2S all see no
behaviour change. P2S is the only override:
P2S: probesize=1_000_000, analyzeduration=500_000
SSDP internal codes (N7→P2S) resolve via an alias map so the camera
path works during the early-connect window before the display name
is settled.
This is the first step of the camera-architecture overhaul agreed
after #1395. Adding the next quirky model is a config entry, not
another module-level constant.
Obico polling could freeze the live camera stream within seconds of opening
the viewer. Cause: when the buffer-reuse path in obico_detection._capture_frame
saw an empty _last_frames[printer_id] entry (stream startup before the first
JPEG lands, or upstream mid-reconnect after a 30s read timeout), it fell
through to capture_camera_frame_bytes() and opened a second RTSP socket. On
firmwares that allow only one camera connection, that second socket forced
the printer to drop the live fan-out connection - the viewer's ffmpeg then
hit its own 30s timeout, looped through 30 reconnects at 0.2s, all racing
the next Obico poll, and the broadcaster pump exited.
Widen the gate from "do we have a buffered frame?" to "is any fan-out stream
registered for this printer?". New is_stream_active() helper checks
_active_streams / _active_chamber_streams independently of buffer state.
_capture_frame consults it first: if a viewer is attached, it returns the
buffered frame when available or None (skip this poll cycle) when not. Never
opens a competing socket while a viewer is connected.
Cost: at most one missed Obico detection cycle per viewer-attach (~10s lag).
Benefit: zero competing-socket events while any viewer is connected.
try_get_active_buffered_frame() refactored to delegate to is_stream_active()
so the two helpers stay in lockstep. The /camera/snapshot caller is unchanged
behaviorally (snapshot is a user-initiated single-shot; falling through to
fresh capture on an empty buffer is the desired behavior there).
Reporter @Andlar94 hit a permanent "Build plate not empty" on every print
start on an A1 with an external RTSP camera. The runtime auto-check at
main.py:1819 called check_plate_empty with use_external=external_camera_enabled,
but the manual UI routes (camera.py) declared use_external: bool = False
and the frontend client always sent use_external=false. So calibration
captured a built-in frame and stamped it as the reference; the runtime
check captured an external frame and diffed it against that reference --
a permanent mismatch.
Centralise the default on the backend: both routes now take bool | None,
deriving the default from the printer's external_camera_enabled +
external_camera_url + external_camera_type. The frontend client stops
sending the flag unless the caller explicitly sets it, so the existing
UI call sites immediately benefit and any future caller gets the right
camera automatically. Explicit overrides still win.
Adds 4 regression tests pinning the new default for both the
external-enabled and external-disabled cases, plus the explicit override
path so a future "always built-in" caller stays supported.
The MJPEG fan-out broadcaster from #1089 only solved viewer-side
concurrency. Obico polling (every 5s) and the manual /camera/snapshot
endpoint kept opening their own fresh RTSP sockets, which X1/H2/P2
firmwares tolerated but X2D firmware 01.01.00.00 enforces strict
single-connection on — every poll kicked the live stream.
Add try_get_active_buffered_frame(printer_id): returns the broadcaster's
last buffered frame when a viewer is connected, None otherwise. Obico
and /camera/snapshot consult it before opening a fresh socket. When no
viewer is active they fall through to the existing fresh-capture path.
plate_detection and layer_timelapse intentionally not converted.
go2rtc and several IP cameras still emit a warm-up / black frame on every
fresh MJPEG connection — even with the v0.2.4b2 warm-up-skip fix it
slipped through intermittently for @nkm8's setup. His own bisect named
the clean solution: go2rtc exposes /api/frame.jpeg as a dedicated
single-frame endpoint that never returns the encoder's stale keyframe.
Adds an optional external_camera_snapshot_url column on printers. When
set, every single-frame capture path (snapshot endpoint, [SNAPSHOT]
notification thumbnails, [PHOTO-BG] finish photo, layer timelapse,
Obico ML, plate-detect / calibrate-plate) routes through _capture_snapshot
on the override URL via plain HTTP GET, bypassing the warm-up dance.
Live view stays on the configured stream URL — only single-frame
captures use the override. Override is camera-type-agnostic. SSRF guard
applies (existing _sanitize_camera_url allowlist). Empty string treated
as unset.
Settings UI: new "Snapshot URL (optional)" input + Test button under
External Cameras, hidden for camera_type=snapshot since the live URL is
already a single-frame source. en + de fully translated; 6 other locales
seeded with English copy.
5 backend tests pin the routing contract; 3 frontend tests pin the
input + debounced PATCH. Documented in
bambuddy-wiki/docs/features/camera.md with the go2rtc example.
Most Bambu Lab printers only allow one concurrent camera connection, but
GET /printers/{id}/camera/stream opened a fresh upstream per viewer.
Two browser tabs → second viewer fails or kicks the first off.
New MjpegBroadcaster (services/camera_fanout.py) owns one upstream per
printer and fans MJPEG chunks out to N subscribers. 5 s grace window
absorbs tab refreshes without reconnecting. Bounded subscriber queues
drop frames for slow viewers rather than blocking the broadcaster.
Audit-pass fixes:
- _stream_start_times set with setdefault() so stream_uptime reflects
the shared upstream's age, not the most-recent viewer's
- subscribe() retried once on RuntimeError to close a tiny grace race
- unsubscribe() returns post-removal count atomically so the detach log
no longer races with concurrent leavers
Permission gates unchanged; broadcaster has no FastAPI surface.
Tests: 13 broadcaster unit tests + 2 integration tests on /camera/stop.
External-camera path untouched.
The periodic camera cleanup task scans /proc for ffmpeg processes and
kills any not in the active-streams registry. The Obico detection
service's capture_camera_frame_bytes() spawns short-lived ffmpeg for
snapshots but never registered the PID — so cleanup killed it as
"orphaned" mid-capture (SIGKILL, exit -9), producing false errors and
missed detection frames.
Track capture PIDs in _active_capture_pids and exclude them from the
cleanup kill list.
Two bugs surfaced while investigating camera reconnect behaviour in #925.
The camera page briefly displayed "Reconnecting attempt 6 of 5" before
giving up, because the attempt counter could be incremented to the
maximum while the reconnect banner was still rendering. The displayed
value is now clamped to the configured maximum.
Every failed ffmpeg spawn logged the full ~20-line ffmpeg version,
configuration, and lib* banner, producing hundreds of lines of noise
per failed camera click (one reported click produced 555 log lines
across 30 retries). A new _summarize_ffmpeg_stderr helper strips the
banner and caps output at the last 10 meaningful lines, applied at
all three stderr log sites (immediate-failure, stream-ended,
read-timeout). Covered by unit tests for empty input, banner
stripping, line cap, blank-line filtering, and banner-only input.
The underlying "camera service stops accepting connections after
prolonged uptime" behaviour in the X1C firmware is still under
investigation — these two fixes are independent of that root cause.
Camera snapshot, test, and plate detection endpoints created temporary
JPEG files with default 0644 permissions. Switch from NamedTemporaryFile
to mkstemp with explicit 0600 permissions.
Camera streams, snapshots, thumbnails, timelapse videos, photos, QR
codes, and cover images served via <img>/<video> tags were previously
unauthenticated because browser media elements cannot send Authorization
headers. When auth is enabled, these endpoints are now protected by a
reusable stream token (?token=xxx) obtained from POST
/printers/camera/stream-token (requires CAMERA_VIEW permission).
When a user closed the camera viewer, the stop endpoint killed the
ffmpeg process but never signaled the stream generator's disconnect
event. The generator saw "process died" as a dropped RTSP session
and respawned ffmpeg — up to 30 times per stream. The orphan cleanup
couldn't catch these because they were still tracked as active.
- Add per-stream disconnect events dict so stop endpoint can signal
generators to stop reconnecting before killing the process
- Check if stream_id was removed from _active_streams before
reconnecting (belt-and-suspenders with the event)
- Track frame timestamps per stream_id instead of per printer_id
so stale detection isn't fooled by newer streams for the same
printer
- Reduce stale thresholds from 120s+60s to 60s+30s
- Signal disconnect events from cleanup when killing stale streams
The Debian ffmpeg package uses GnuTLS, whose hardened defaults reject
TLS renegotiation and legacy ciphers that some Bambu printer firmwares
(notably P2S) rely on — causing RTSP sessions to drop after a few
seconds.
Add a local TLS termination proxy (Python ssl/OpenSSL) that handles
the TLS connection to the printer and exposes a plain RTSP port to
ffmpeg. The proxy rewrites RTSP request-line URLs (rtsp://proxy →
rtsps://printer) while preserving Authorization headers so Digest
auth hashes remain valid.
Also:
- Reduce RTSP reconnect delay from 1.0s to 0.2s
- Add ffmpeg fast-start flags (-probesize 32, -analyzeduration 0,
-fflags nobuffer, -flags low_delay)
- Fix external camera double rate-limiting causing choppy streams
- Apply TLS proxy to external camera rtsps:// URLs and snapshot capture
- Update orphan ffmpeg cleanup to match rtsp:// (proxied) URLs
- Add unit tests for RTSP URL rewriting and proxy lifecycle
Fix P2S camera stream dropping and snapshot capture race (#661)
P2S firmware's TLS renegotiation is rejected by Debian's hardened GnuTLS
defaults, causing ffmpeg RTSP sessions to drop after ~3 seconds. Add
GnuTLS config allowing unsafe renegotiation and legacy ciphers. Also add
ffmpeg fast-start flags, reduce reconnect delay from 1.0s to 0.2s,
remove double rate-limiting on external camera streams, and fix orphan
cleanup killing snapshot capture ffmpeg processes (exit code -9).
Or as a single combined commit:
Fix P2S camera streaming, snapshot race, and energy stats (#661, #695)
Camera: P2S firmware's TLS renegotiation rejected by Debian's hardened
GnuTLS defaults, dropping RTSP sessions after ~3s. Add GnuTLS compat
config, ffmpeg fast-start flags, reduce reconnect delay to 0.2s, remove
external camera double rate-limiting, and register snapshot ffmpeg PIDs
with the orphan tracker to prevent SIGKILL during capture.
The P2S firmware drops RTSP sessions after a few seconds with an I/O
error. The backend treated this as fatal, ending the MJPEG stream and
forcing the frontend through a full reconnection cycle. Added transparent
auto-reconnection: when ffmpeg's RTSP connection dies, it respawns
immediately and continues streaming MJPEG frames to the browser with
only a brief freeze (~1s). Up to 30 reconnections before giving up.
On Windows, process.terminate() on ffmpeg broadcasts CTRL_C_EVENT to
the entire process group, causing uvicorn to shut down. Spawn ffmpeg
with CREATE_NEW_PROCESS_GROUP so cleanup doesn't affect the server.
Fix camera button permissions & ffmpeg process leak (#550)
Camera button on printer card was clickable without camera:view
permission. ffmpeg processes (~240MB each) accumulated after closing
camera streams because: (1) stop endpoint called terminate() without
wait()/kill(), (2) HTTP disconnect detection only ran between frames
so was blocked when the generator was stuck on stdout read, and
(3) no mechanism caught processes orphaned by generator abandonment
or app restarts.
- Add camera:view permission check + tooltip to camera button
- Fix stop endpoint: terminate() → wait(2s) → kill() → wait()
- Add background disconnect monitor (polls every 2s, kills ffmpeg
directly on disconnect)
- Add periodic /proc scan (every 60s) that SIGKILLs any ffmpeg
with rtsps://bblp: not in an active stream
- Add noCamera i18n key to all 6 locales
- Fix camera API test mocks for async wait() and pid attribute
The snapshot endpoint always used the internal printer camera even when
an external camera was configured. Now checks for external camera first,
matching the stream endpoint pattern. Also added retry logic (3 attempts,
2s delay) to MJPEG and RTSP stream generators so they reconnect on
timeout instead of silently ending the stream.
- Add USB camera type to external camera service
- Auto-detect available V4L2 devices on Linux
- New API endpoint: GET /api/v1/printers/usb-cameras
- Use ffmpeg for USB camera capture and streaming
- Add "USB Camera (V4L2)" option in Settings UI
- Debounce camera URL input to avoid saving on every keystroke
Closes#143
Automatically detect if objects are on the build plate before printing
and pause the print immediately if detected.
Features:
- Per-printer toggle to enable/disable plate detection
- Multi-reference calibration: store up to 5 reference images per printer
for different plate types (textured, smooth, high-temp, etc.)
- Automatic print pause when objects detected at print start
- Push notification and WebSocket alert when print is paused
- ROI (Region of Interest) calibration UI with sliders to adjust
detection area
- Reference management: view thumbnails, add labels, delete references
- Works with both built-in and external cameras
- Uses buffered camera frames when stream is active (no blocking)
- Split button UI: main button opens modal, chevron toggles on/off
- Green visual indicator when plate detection is enabled
- Included in backup/restore
Add support for external network cameras (MJPEG, RTSP, HTTP snapshot)
that replace a printer's built-in camera when configured.
Features:
- Live streaming on printers page (replaces built-in camera)
- Finish photo capture from external camera on print complete
- Layer-based timelapse: captures frame on each layer change,
stitches to MP4 video on print completion
Backend changes:
- Add external_camera_url, external_camera_type, external_camera_enabled
fields to Printer model with database migration
- New external_camera.py service: MJPEG/RTSP/snapshot frame capture,
connection testing, MJPEG stream generation
- New layer_timelapse.py service: TimelapseSession management,
layer-by-layer frame capture, ffmpeg video stitching
- Add on_layer_change callback to MQTT client and printer manager
- Update camera routes with external camera streaming and tracking
- Update print lifecycle hooks for timelapse start/stitch/cancel
- Add external camera fields to backup/restore
- Rate limiting for external camera streams (prevents browser freeze)
Frontend changes:
- Add external camera configuration UI in Settings > Camera
- Per-printer enable toggle, URL input, type selector, test button
- Toast notification on save
Closes#143
Features:
- Resizable printer cards (S/M/L/XL) with toolbar controls on Printers page
- Queue Only mode for staging prints without automatic scheduling
- "Queue Only" option in add/edit queue modals
- Purple "Staged" badge and Play button to release to queue
- manual_start field in database with migration
Fixes:
- Improved camera stream stuck detection with automatic reconnection
Tests:
- Added 16 integration tests for print queue API endpoints
prints (not just completed), matching get_printer_total_hours behavior
- Add option to keep or delete archives when deleting a printer
- Custom maintenance types no longer auto-assign to all printers
- Add UI to manually assign/remove custom maintenance types per printer
- Add backend endpoints for assigning types to printers and removing items
- Exclude static/assets from large file pre-commit check
- Fix A1/P1 camera streaming with extended timeouts and lower FPS cap
maziggy