A Kubernetes Secret written as a block scalar carries a trailing newline, and
the schema bounds the four required variables by max_length only, so an
unstripped issuer_url was stored and enabled and then raised httpx.InvalidURL
on the first click of the SSO button -- the authorize-time failure the
all-or-nothing rule exists to prevent. Whitespace-only values got through the
same way, contradicting the reader's own "an empty required var counts as
unset". The optional variables have always treated blank as unset; the
required ones now do too.
Also registers BAMBUDDY_LOCAL_LOGIN (#1589) in the typo guard, which logged
"possible typo" for it on every boot while listing every BAMBUDDY_OIDC_*
variable as legitimate.
Promoting env_bool to strict rejection made BAMBUDDY_LOCAL_LOGIN=on raise
EnvOIDCConfigError uncaught on the login/forgot-password path -- a 500 on
the exact recovery endpoint the bypass exists to keep open. env_bool gains
a strict flag (default True for the startup OIDC reader); the local-login
caller opts out so an unrecognized value falls back to "off" instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016q8EAf9Rj7ZHL92sPnXYxy
_env_bool returned the default for anything outside {true,1,yes}, so
BAMBUDDY_OIDC_REQUIRE_EMAIL_VERIFIED=on silently read as OFF and
BAMBUDDY_OIDC_ENABLED=on silently disabled the provider -- the exact
opposite of what .env.example claimed. Unrecognized values now raise
EnvOIDCConfigError, caught in _apply_env_oidc_provider the same way a
bad DEFAULT_GROUP or a ValidationError already is: logged and left
running, never released on a typo.
Also promotes _env_bool to env_bool now that it has a call site in
auth.py, and corrects the boolean-parsing sentence in .env.example.
BAMBUDDY_OIDC_SCOPES, _EMAIL_CLAIM and _ICON_URL fell back to their
default only when the key was absent, so `BAMBUDDY_OIDC_ICON_URL=` in
a compose file (as .env.example ships it, commented) reached the
schema validator as an empty string and got the whole provider
refused. default_group already treated blank as unset; these three
now follow the same rule.
Saving a K-profile was fire-and-forget. set_kprofiles_batch published
and returned True, and the printer's extrusion_cali_set answer was
logged at DEBUG and dropped, so a write the printer refused was
reported to the user as saved (#2718, reporter @jmoore-skild).
The reason it could not simply be gated on: the answer itself was
wrong. Single-nozzle firmware returned result:"fail" with
reason:"invalid tray_id" on writes that demonstrably applied.
Measured against an X1C and an H2D over MQTT, the cause is the
tray_id:-1 Bambuddy itself put in the payload. Sending three
otherwise identical writes isolated it: tray_id:-1 fails, tray_id:0
succeeds, and cali_idx:-1 is accepted either way, so only that one
field is at fault. The H2D ignores the value entirely; the X1C
validates it, complains, and applies the write anyway. BambuStudio
always sends a real tray_id and defaults it to 0 for a manually
entered profile.
With tray_id:0 the acknowledgement is honest, and the printer echoes
back the sequence_id we sent -- confirmed for extrusion_cali_get,
_set and _del on both printer classes -- so it can be matched to the
write that caused it. Writes now return their sequence_id and the
routes await the verdict, turning a real failure into an error that
carries the printer's own reason. A printer that stays silent is
still treated as success: no answer is not evidence of refusal, and
firmware that never answers must not turn every save into an error.
Raises the ack to INFO. It sat at DEBUG, so the one line that
explains a failed save was absent from every support bundle -- the
same reasoning that put ams_filament_drying at INFO for #1447.
Also fixes extrusion_cali_set building its payload from
str(self._sequence_id) without incrementing first, reusing the
previous command's id. Harmless while nothing correlated on it,
fatal now that the write path does.
Adds supports_nozzle_flow_type() for the Standard / High Flow choice,
which the K-Profiles UI previously showed as "Not reported by
printer" -- not a value anyone can save. Most printers omit the
nozzle identity from their calibration table entirely, and the slicer
treats that as Standard rather than unknown; Bambuddy now does the
same and keeps the choice editable. The field is hidden only where
the model ships a single nozzle variant, using the slicer's own rule
(len(nozzle_volume) // len(nozzle_diameter) > 1 over the machine
preset) evaluated across every bundled Bambu profile. That puts only
A1, A1 Mini and A2L on the hidden side -- it is not the single-
versus-dual-nozzle split, since P1P, P1S, P2S, X1, X1C, X1E and H2S
are all single-nozzle and all carry two variants. Editing a profile
also no longer writes back an empty nozzle_id.
Wiki records that on printers which omit the field the chosen flow
type is discarded by the firmware and reads back as Standard, in
Bambu Studio as well, so it does not get filed as a bug again.
Every K-profile came back as 0.4mm on printers running any other
nozzle (#1748, reporters @Liquidmasl and @jmoore-skild). The printer
puts nozzle_diameter on the extrusion_cali_get envelope only; the
per-filament entries carry setting_id, filament_id, name, k_value,
n_coef and cali_idx, and nothing else. The parser read the field per
entry with a hardcoded "0.4" fallback, so the fallback fired on every
profile of every response. The envelope value was already in scope,
read into response_nozzle and used only to match the request.
This never reproduced on H2D because that firmware does include the
field per entry. Both construction sites are in the same handler, so
the code path is shared; what differs is the payload, and every
single-nozzle model omits it.
The display was the least of it. Editing is delete-and-re-add on
single-nozzle printers, and the dialog rebuilt nozzle_id and
nozzle_diameter from its own greyed-out selects, so saving an
untouched 0.6mm profile rewrote it on the printer as HH00-0.4.
Deleting aimed extrusion_cali_del at the wrong nozzle the same way.
Both now pass through what the printer reported. The cali_idx cascade
in inventory.py, spoolman_inventory.py and spoolman.py matches on
nozzle_diameter, so on a 0.6 or 0.8 nozzle it never found the
printer-side entry and the assignment silently failed to stick --
that is the "cannot auto-map a K-profile" half of the report, fixed
at the source without touching those three call sites.
nozzle_id has no source in the payload at all, and state.nozzles
carries material (hardened_steel), not flow, so it cannot honestly
produce HH/HS. Rather than keep inventing one, the UI now says the
printer did not report it: the card shows the diameter alone, the
dialog shows "Not reported by printer", and the High Flow / Standard
filter is hidden instead of being offered as a control that can only
ever empty the list. Import stops stamping HH00 on profiles whose
source reported none.
Also correlates K-profile requests by sequence_id. Responses were
matched by nozzle diameter through a single shared expectation slot,
so a second request overwrote the first's and the first's valid
answer was discarded as a mismatch -- the "Failed to get K-profiles
after 3 attempts" in the same logs, with the printer having answered
correctly both times. Pending state is now one entry per request,
keyed by the id we already send, with the nozzle match kept as a
fallback for firmware that does not echo it back.
Fixes the flow-type select naming a new profile with the opposite
label, which contradicted the identical expression 44 lines above it.
Enabling Cloud Profiles for a Git backup produced nothing, and said it had
worked. Two independent faults, either one sufficient.
The collector looked for a "setting" list. The Bambu Cloud listing endpoint
is keyed by preset type instead, each key holding private and public arrays,
so the loop body never executed once — and the entries carry no type of
their own either, which routes/cloud.py already knew: it takes the type from
the outer key and maps Bambu's "print" to process. Two bugs on one line.
It also asked build_authenticated_cloud for the credential store used when
authentication is disabled. With auth on, tokens live on User rows, so the
collector returned at "Cloud not authenticated" before ever reaching the bad
key. Every multi-user install was collecting from zero accounts.
Neither failure surfaced. backup_metadata.json recorded the configured flag
rather than the outcome, so it claimed cloud_profiles: true on runs that
wrote nothing, and the log read "Collected cloud profiles: 0 filament, 0
printer, 0 process" at INFO — which is exactly what a successful backup of
an empty account looks like.
Cloud profiles now come from every connected account across both clouds. The
toggle predates Orca Cloud entirely, and Orca has the same three preset
types, so both are collected and grouped the same way:
cloud_profiles/bambu/user-3/{filament,printer,process}.json
cloud_profiles/orca/user-3/{filament,printer,process}.json
Accounts are keyed by Bambuddy user id, "global" when auth is off. Never by
email: a backup repository can be public, and the Bambu listing's user_id is
dropped for the same reason. Both credential stores are read on every run,
because a Settings row survives someone enabling auth later and dropping it
would silently stop backing that account up.
Bambu costs one get_setting_detail per private preset. The listing is
metadata only, and without base_id and setting the backup is a list of names
that create_setting cannot rebuild from. Public presets are skipped — Bambu's
bundled catalogue is the same hundreds of entries for everyone, always
re-downloadable, not recreatable under your account, and would rewrite the
repository on every run. Orca needs no second call; its sync-pull carries
each profile's content inline. Where the Orca route drops a profile whose
content.type it cannot map, the backup writes it to other.json instead:
silently omitting a profile because Orca added a type is the same class of
bug as this one.
Failures are contained per account and per preset, and counted rather than
swallowed. A partial backup that looks complete is how this stayed invisible.
The metadata now reports what was collected, per cloud and per account, and a
run that collects nothing while the category is enabled warns with the reason
instead of an INFO line that reads like success.
The checkbox gated on the viewer's own Bambu sign-in, which is not the same
question as whether there is anything to back up — with auth enabled the
accounts belong to individual users, and an administrator who never signed
in personally saw the category disabled with plenty in scope. It now gates
on the total across both clouds and shows the counts. That comes from its
own endpoint rather than a field on /config, since /config answers null
until the first save and would disable the toggle during the very setup it
belongs to. Counts only, never identities.
One deliberate restraint. _build_authenticated_service clears stored
credentials when a refresh is rejected, which is right for a route — the
user is on the page and can pair again — and wrong for a scheduled job.
Orca reports every rejection with one composite reason ("unknown, expired,
revoked, or already used"), so a genuine revocation cannot be told apart
from a lost token-rotation race, and acting destructively on a signal that
cannot be disambiguated is the #2562 mistake in a different cloud. It also
gains nothing: the Profiles route hits the same failure and clears it then,
with the user present. Background callers now pass clear_on_auth_failure=
False and skip the account. A successful refresh is still persisted either
way — by that point the old token is consumed, so dropping the new pair
would break a working pairing for real.
Restore is not part of this. Nothing reads cloud_profiles/* yet; the format
carries base_id/setting for Bambu and content for Orca so that it can.
Round-3 review of the "Save AMS mapping" PR.
The queue item's ams_mapping was set unconditionally, on the reasoning that
honouring the slicer's own pick is a correctness fix rather than a feature.
It is both. Storing a resolved mapping makes _ensure_ams_mapping return
early, so _compute_ams_mapping_for_printer never runs — and that function is
where prefer_lowest_filament lives, along with the AMS-filament-backup gate
that qualifies it (#1766), the inventory-remain overrides, and the per-slot
force-colour overrides. Every existing queue-mode VP pointed at a printer
would have quietly lost all of it on upgrade, without a setting to turn it
back on.
So save_ams_mapping now gates the queue item too, not just the archive
persistence. Off is exactly the old behaviour. The correctness case the PR
was written for — two spools of the same red PLA, and the slot the user
picked in the slicer thrown away — is still fixed, for anyone who asks for
it.
Force color match wins over it when both are on. Its only effect on a
fixed-printer item is the filament_overrides written onto the queue item,
and those are read inside the function a stored mapping skips, so the two
toggles sitting next to each other on the same card silently cancelled. The
dispatch now matches strictly, as asked, while the slicer's pick is still
saved onto the archive — that is what the toggle's name promises, and a
later reprint is a separate decision from this print. The queue-add fallback
applies the same rule to a request that carries force-colour overrides.
A mapping shorter than a plate's highest slot id cannot address that plate's
own slots, and _ensure_ams_mapping would have kept it anyway, since it only
rejects an all-unresolved one. Each plate now checks the length it needs and
falls back to a computed mapping if the array does not reach. Bambu Studio
sends a file-global array, so this normally never fires; it also means a
multi-plate Send All degrades safely if that ever stops being true.
The badges claimed more than they delivered. Both rendered whenever a saved
mapping existed, ignoring which printer it belonged to, while the tooltips
promised the reprint would reuse those exact spools — true only on the
printer the trays were resolved against. The queue row's flag is now
computed against that row's own printer, which is precisely when dispatch
reuses the mapping, and the archive card names the printer instead of
implying any of them will do. It hides itself when that printer no longer
exists. Retranslated in all 13 locales.
Frontend tests, which the PR had none of. The printer-scoping rule is now a
pure function rather than an inline expression, covered for the mismatched
printer, the no-printer-selected case that would otherwise compare undefined
against undefined, and malformed extra_data. The toggle's undo bookkeeping
is covered for unresolved slots, short mappings, and hand-made picks —
preserved when the toggle never wrote that slot, replaced when it did, which
is behaviour worth pinning either way.
Also reverts all three queue-mode switches when a save fails, not just the
new one; without it the card shows a setting the server rejected.
A bundle described everything except the process it runs in. So a report of
memory climbing over days until the OOM killer fires arrives with no way to
act on it: the numbers that name the mechanism only exist while it is
happening, and by the time anyone asks, the container has been restarted.
The new `process` section carries what actually separates the candidates.
Resident against virtual memory: 650MB RSS with 12.9GB VMS is address
space — thread stacks or allocator arenas — not a heap full of live data,
and that reading is the opposite of the one the reporter drew from the same
figures. Thread count and child-process count then split those two apart,
and a census of live objects by type names what a growing heap is filling
up with. Open files, sockets and uptime round it out.
Three constraints worth keeping:
The heap census is skipped above 2GB. gc.get_objects() materialises every
tracked object, so it costs most on exactly the process that can least
afford it — a bundle generated to diagnose runaway memory must not be the
allocation that tips the host over. Everything else is still collected, and
the skip is recorded with its reason rather than silently omitted.
Children are recorded by executable name only. An ffmpeg command line
carries the camera URL, and with it the camera's password.
Collection runs off the event loop and every metric is independently
best-effort. psutil raises on hardened kernels and in restricted
containers, and the bundle is how someone reports a problem in the first
place — it has to be produced even when half the numbers are unavailable.
This does not fix#2734, and nothing here should be read as having found
its cause. The bundle's own evidence contradicts both proposed causes: the
orphan janitor ran 7 times in 26 days over 725 stream-ends and killed no
orphaned ffmpeg, which is not the #776 signature; and the 5 "database is
locked" errors all fall between two OOM kills, making them a symptom of the
memory pressure rather than a source of it.
The recovery rollback after a failed commit was itself unguarded, so a
rollback that raises on a wedged connection would still take the boot
down -- the exact failure the never-raise contract exists to prevent.
Suppressed; the caller's `async with` discards the session regardless.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016q8EAf9Rj7ZHL92sPnXYxy
A heavy MakerWorld model — one Bambu Studio also takes a long time over —
failed after five minutes with "Slicer sidecar unreachable". The sidecar
was reachable the whole time and still slicing when we hung up on it.
SlicerApiService carried a hardcoded 300s timeout, passed to httpx as a
bare float so it covered connect, read, write and pool alike. On a single
long request that is not a health check, it is a cap on how long a model is
allowed to take. And because httpx.ReadTimeout subclasses RequestError,
expiry landed in the same handler as a refused connection and was reported
as an unreachable sidecar — so the reporter went and updated their sidecar
container, which was never the problem.
The information to do better was already being collected. _poll_progress
polls /slice/progress/{id} once a second alongside the blocking POST to
drive the live progress toast, so at minute five Bambuddy had fresh
evidence the slicer was working. It killed the request anyway.
So the read timeout comes off the HTTP call and the poller supervises
instead: the deadline moves forward on every progress update, and only
genuine silence ends the wait. A model that keeps reporting runs to
completion however long it takes. Connect and pool keep short timeouts —
a sidecar that will not accept a connection is unreachable and should
still say so quickly.
Only a *changed* progress payload counts as alive. The sidecar re-serves
its last snapshot on every poll, so counting repeats would leave the
watchdog unable to detect a stall at all.
The window is floored at three poll intervals: liveness can only be
observed as fast as the poller ticks, so anything shorter would expire in
the gap between two polls and fail every slice instantly.
New setting slicer_stall_timeout_minutes (Settings > Workflow > Slicer),
default 15, range 1-240, alongside the sidecar URL and gated on
use_slicer_api like its neighbours. Sidecars too old to report progress
have no liveness signal, so for those the same number bounds total elapsed
time — the old behaviour, configurable and no longer 300s flat. The
message says which case applies and where to change it.
SlicerTimeoutError is its own type and maps to 504, not 502: the sidecar
answered throughout, we stopped waiting. Connection failures keep
SlicerApiUnavailableError. The preview slice path gets the same treatment.
_local_login_env_bypass() re-inlined the same {"true", "1", "yes"} set
oidc_env._env_bool already enforces, so the typo-guard existed in two
places. oidc_env has no module-scope import of models (its model
imports are lazy inside apply_env_oidc_provider), so importing
_env_bool at module scope here is not a cycle -- confirmed by
importing backend.app.main and this module directly.
A name collision with a provider that was NOT already env-managed
overwrites its issuer, client id and secret in place and locks it
behind the env-managed 409 -- but it logged the same routine "applied"
line as an ordinary re-apply, giving no signal a UI provider was just
taken over. Adoption is now a WARNING with its own wording; a routine
re-apply of an already env-managed provider keeps the INFO line.
The db.execute/db.commit calls in the upsert and release paths sat
outside the try/except that only wrapped OIDCProviderCreate, so a
commit failure at startup (connection blip, WAL lock) propagated out
of the lifespan and took the instance down -- the exact outcome this
module exists to avoid. The body now runs inside a private
_apply_env_oidc_provider(), with the public entry point catching,
logging and rolling back on any exception.
Deleting a print that belonged to a project left it on the project page as
a card with a missing thumbnail, and there was no way to remove it.
Deleting a print is a soft delete by default (#1343): the files go from
disk, the row stays so global Quick Stats keeps counting its filament,
time and cost. Every other consumer filters those rows out. The projects
module filtered none of them — the only deleted_at check in the whole file
was for LibraryFile — so a deleted print kept its project_id and kept
being listed, pointing at a thumbnail that no longer existed. The same
broken previews appeared on the overview cards, and in the timeline, where
the entry links to an archive that no longer opens. Unassigning was
impossible because the only UI that can change a print's project lives on
the Archives page, which correctly hides deleted prints: visible on the
project, unreachable from anywhere.
All eight project-scoped archive queries now filter, counts included. That
last part is a deliberate divergence from #1343, where the whole point of
the soft delete is that the contribution survives: a project is a piece of
work with a definite membership, not a lifetime total, so a project that
lists eleven prints must not claim twelve. The reasoning is recorded at
the constant so nobody later "fixes" it back.
remove_archives_from_project keeps working on hidden rows on purpose — it
is the repair path for links written before this. The BOM print_name
lookups are left alone; naming a since-deleted print is still correct.
Two more consumers had the same gap. The CSV/Excel export handed back rows
the interface says are gone — filtered at the base query, since the export
is the list you are looking at saved to a file. Per-project failure
analysis measured a failure rate against prints deleted from the project,
and disagreed with the project's own numbers; only the project-scoped
branch filters, global analysis still counts every run including orphans
as #1390 established.
Finally, the project page needed a manual reload to catch up. staleTime is
60s and the delete mutations invalidated only ['archives'], so a project
visited within the minute served its cached copy, print still there. The
project-assign mutations had the mirror-image bug: ['projects'] refreshed
the overview cards but never ['project', id]. Both now go through one
shared helper covering every project-derived key, as bare prefixes so all
cached project ids are matched.
The reporter's printer lost its session to a keep-alive timeout at 02:19
and did not come back until 11:24 — nine hours offline, with the web UI
open throughout.
check_staleness() was never going to catch it. Its first line is
`if self.state.connected and self.is_stale()`, so it only ever handles the
half-broken session that is still connected but has gone quiet. This
client had connected=False from 02:19:42 (the offline notification fired a
minute later), so every call returned immediately, and paho's own retry was
the only thing left watching. When that stopped making progress nothing
noticed.
Adds a sweep every 60s that rebuilds a client when all four hold: it is
disconnected, it had a working session before, it has been silent for five
minutes, and its MQTT port still answers. The port check is what keeps this
from becoming a nuisance — a switched-off printer is left to paho, so a
farm powering down overnight causes no client churn and no log spam. The
five-minute grace sits well past the 60s stale timeout and the 30s max
reconnect backoff, so a session recovering on its own is never interrupted.
The rebuild goes through force_reconnect_stale_session from async context,
which takes the hard-reset path: fresh client_id and paho's QoS 1 queue
dropped, so a project_file left unacked on the dead session cannot replay
into the new one and trip 0500_4003 (#1136). Rate-limited per printer,
cooldown cleared when the printer returns, and the sweep continues past a
client that throws rather than abandoning the rest of the farm. The log
line names how long the printer was gone and the last connect error, so a
session that dies repeatedly leaves a trail.
check_port gains a public alias in printer_diagnostic rather than having
the watchdog reach for the private name.
Also corrects the Developer Mode path added in the previous commit: the
wiki documents it under Settings > Network, not Settings > General. The
menu path is dropped from the translated string entirely, since it varies
by model and firmware and the wiki carries the detail.
A P1S on firmware 01.10.00.00 rejected every control command and said so:
HMS 0500-0500-0001-0007, "MQTT command verification failed". Bambuddy
received that, dropped it, and reported a healthy printer instead.
The frontend filtered it out. This code's meaning lives in attr's low half
(0500) and code's high half (0001), both of which the MMMM_EEEE short form
discards, so it collapsed to "0500_0007" — no catalog entry, no firmware
actions, and filterKnownHMSErrors drops uncatalogued action-less errors.
Catalog lookups now try full_code first, in both the description and the
filter, and errors matched that way display the four-group code the
printer's own screen shows. The remedy line is ours, not Bambu's: their
wiki says to update Studio or Handy, which does not apply to a print sent
from Bambuddy.
The developer-mode probe made it worse. It read anything that was not an
explicit refusal as confirmation, and this firmware answers the probe with
an empty result while refusing everything else — so an inference drawn
from a non-answer became "developer_mode: pass" in the support bundle of a
printer that had not accepted a command all day. The probe now has three
outcomes: explicit success enables, explicit verify-failure disables,
anything else stays unknown and the diagnostic reports skip.
The HMS is authoritative over that inference in both directions. It forces
developer_mode False when present, and clears back to unknown when the
printer stops reporting it, so enabling Developer Mode and restarting the
printer is picked up without restarting Bambuddy.
Dispatch no longer treats a refusal as a wedge. The watchdog latches the
HMS across both phases and fails the item on the first attempt naming the
code and the fix, rather than spending three uploads and 270s a lap to
arrive at a message about SD cards. The check runs after the active-state
exit in both phases, so a lingering HMS can never abort a print that is
visibly running.
Also: the "wrong or mis-cased serial number" hint no longer fires in the
moment after a reconnect. _report_messages_since_connect is reset by
_on_connect, so a reconnect landing microseconds before the staleness
check leaves it at 0 for reasons that have nothing to do with the serial —
this reporter's healthy printer was told to go check its serial 1 ms after
reconnecting.
Obico's ml_api container takes an optional ML_API_TOKEN environment variable.
With it set, ml_api/auth.py answers a bare 401 to any request whose
Authorization header isn't "Bearer <token>"; with it unset it ignores the
header entirely. Bambuddy never sent one, so pointing it at a protected server
meant deleting the token there — which the reporter had set for their Home
Assistant integration and did not want to undo.
Settings -> Failure Detection gains an ML API Token field. When it is empty no
header is sent, so an unconfigured install's request stays byte-identical to
what shipped before the setting existed.
This failed in the worst possible way, and that is the more important half of
the change. Obico decorates /p/ with token_required but leaves /hc/ open. Test
Connection pinged /hc/, so it reported success against a server that was
rejecting every real detection call, the settings looked right, and detection
silently never ran. The only symptom was a generic "ML API call failed" buried
in the status card.
So the test now proves what it claims. After health passes it probes GET /p/
with no img parameter: the auth decorator runs before the handler, so 401 means
the token was rejected and 422 ("Invalid request params") means it was
accepted. No inference work is done either way. A probe that itself errors
reports the token as unknown rather than as working — the UI says it could not
be checked instead of claiming success.
The detection loop checks for 401 before raise_for_status, so a rejected token
is reported as a rejected token, naming the setting and the environment
variable, instead of surfacing "401 Unauthorized" with no hint of what to do.
The message never contains the token; a test pins that.
The setting name carries "token", so the support bundle's keyword redactor
masks it with no new rule. Resolving "field omitted" to the saved token is the
route's job, keeping test_connection a pure outbound call with no database
access.
Second fix, same issue: support bundles misreported which printers Obico
watches. The bundle split obico_enabled_printers on commas and read an empty
value as "no printers". The settings UI writes a JSON array, and empty means
*all* printers — the default — so a working Obico setup showed obico_enabled
false against every printer in its own bundle. That is the reporter's bundle
exactly, and it points anyone reading it at the wrong subsystem. The bundle now
parses the setting the way ObicoDetectionService does, keeps a comma fallback
for any install that stored the legacy shape, and factors in the global switch.
The photo fired the moment layer_num reached total_layer_num. That edge is
where the printer *starts* its final layer, not where it finishes it: the
reporter's H2C capture shows it arriving at 92% with mc_remaining_time=2,
three minutes and seventeen seconds and one filament change before the print
actually ended, so the frame caught the toolhead mid-print over the model.
The trigger also latched _finish_photo_captured, which locked out both the
stage-22 and FINISH triggers for the rest of the print — so on firmware that
never reports an end-of-print filament unload (H2C and A1 Mini confirmed)
nothing could replace the bad frame.
Remove the last-layer trigger. The photo is now taken at the FINISH-state
trigger, which every model sends and which lands after the toolhead parks.
Since Bambu's end G-code drops the plate ~100mm just before that, restore the
framing before capturing: absolute G90/G1 Z to max_z_height + 10mm clearance,
settle, capture, then drop it back so the print is as reachable as the printer
left it. Absolute is the safety argument — that Z is a height the toolhead
occupied seconds earlier, so it is inside the travel limits by construction and
leaves the nozzle above the part, and it is unambiguous across model families
because Z is the nozzle-to-bed gap whether the bed moves or the toolhead does.
M211 is never touched (#2579). This is what #1145, #1397 and #1565 asked for.
The height is only trusted when two independent sources agree: the archive is
matched by the finished print's subtask_name by equality (not LIKE, so "Cube"
cannot resolve to "Cube v2"), and its layer count from the 3MF must match the
layer count the printer reported over MQTT. Matching on "most recent archive
for this printer" was not safe — on_print_complete pops the _active_prints
binding concurrently, and a print Bambuddy failed to archive would have
resolved to its predecessor. A wrong height is the one failure that could drive
the nozzle into the model.
The move is additionally skipped when the print height is unknown, when a queue
item is pending for the printer, when the printer has left FINISH, and when the
new finish_photo_restore_plate setting is off.
for every FINISH-state capture — which is what shipped the mid-print photo —
the bank is used only when the dispatcher recorded that it injected End G-code
into this print, since a SwapMod snippet may have ejected the plate. The flag is
handed over in two steps (mark_pending at dispatch, adopt at print start) so it
can never outlive its print: a job started from the slicer or SD card adopts
False rather than inheriting its predecessor's answer. Those prints also skip
the plate move outright, bank or no bank.
The bank now refreshes on mc_percent advances as well as layer changes, via a
new on_print_progress callback. Layer changes stop the instant the final layer
begins, which left the #1867 fallback frame stale by the whole length of that
layer; progress keeps ticking there and freezes before the End G-code runs, so
a swapped plate still cannot reach the bank. The last-layer throttle exemption
is dropped, since it would now fire a grab on every percent tick.
On the timelapse path the moment producer returns early, so the consumer does
the restore itself before its live-grab fallback — the documented usual outcome
on P1-series, where the video has not transferred by the time the notification
goes out and the shipped photo was of an already-dropped plate. The two waits
are now derived from the settle window and the video poll timeout rather than
hardcoded; at the old flat 75s that fallback was guaranteed to be cut off
mid-settle.
extract_max_z_height_from_3mf reads only a bounded prefix of the plate G-code,
since a sliced plate is routinely tens of megabytes and the header is ~40 lines.
It returns None for missing, unparseable, zero and negative values so callers
must treat "don't know" as such rather than defaulting.
Round-2 review fixes for #2700.
Blocking: the toggle didn't actually gate the archive write. archive.py's
promotion fired for any print_data carrying ams_mapping, but bambu_mqtt's
request-topic interception captures ams_mapping unconditionally for every
print source (slicer-direct LAN prints included). Since main.py's
real-printer auto-archive path forwards the full MQTT payload as
print_data, every archive on any install — VP or not — grew
extra_data.slicer_ams_mapping. Fixed by replacing the print_data-sniffing
with an explicit `slicer_ams_mapping` param on archive_print() that only
the VP-queue path (already gated on save_ams_mapping) ever passes.
Blocking: a saved mapping could get reused on a printer it was never
resolved against — tray IDs only mean something relative to one printer's
AMS layout. extra_data.slicer_ams_mapping is now stored as
{mapping, printer_id} instead of a bare array:
- add_to_queue's fallback only fires when the reprint's target printer_id
matches the mapping's origin printer.
- The frontend's archiveAmsMapping only surfaces (and the Mapping button
only appears) when the print modal's selected printer matches too.
- A model-based VP (target_printer_id=None, no MQTT bridge to any real
printer) never stamps a mapping in the first place — there's no live AMS
layout for the slicer to have resolved tray IDs against.
Also from review:
- Multi-plate archives now get the Mapping button too (the per-plate
FilamentMapping loop was missing archiveAmsMapping entirely).
- Added coverage for the previously-untested late-MQTT archive patch path
(_restamp_recent_queue_item), including the model-based-VP skip case.
- usingArchiveMapping now also resets on printer change, not just
plate/archive (it already worked via the printer-scoping above, but is
now an explicit dependency too).
- The Mapping button's revert (OFF) now undoes only the slots it itself
set, not every manual pick in scope — matches the comment above it.
- Added a comment on why negative-value slots (external spool) are
skipped rather than cleared when applying a saved mapping.
Lets a reprint reuse the AMS slot the slicer itself picked, instead of
re-deriving one from the file's static type/color.
When a Print Queue VP has "Save AMS mapping" on, the slicer's own
live-resolved ams_mapping (from the project_file MQTT command) is
persisted onto the archive as extra_data.slicer_ams_mapping. A later
reprint can reuse it via a new "Mapping" button in the filament-mapping
panel — one click snaps every slot to the saved pick, click again
reverts to auto-match. Archive cards and queue rows get an "AMS mapping
saved" badge so it's visible beforehand. add_to_queue also falls back
to the saved mapping automatically when the caller sends no explicit
ams_mapping (e.g. a plain reprint with no per-slot edits).
The queue item's own ams_mapping (used for that dispatch) is still
captured unconditionally whenever the slicer provides it — that part is
a correctness fix, not gated behind the toggle. Only the archive
persistence for future reprints is opt-in.
Split out from the original combined PR per review: this half is
genuinely opt-in and low-risk (#2684). The dispatch-time validation
gate that keeps a stored mapping honest (#1308) changes behaviour for
every existing user and will land as its own PR.
Review fixes applied:
- _extract_slicer_ams_mapping_json: dropped the unreachable `v is None`
arm and rejected bool explicitly (isinstance(v, int) accepts bool).
- Translated the Russian docstring text to English.
- save_ams_mapping's model comment moved to a trailing comment on the
column line, matching the file's convention.
- usingArchiveMapping now resets when the plate or archive changes, so
the Mapping button can't read ON against a mapping it never applied.
- Translated "Click to change slot assignment" and "Re-read".
- add_to_queue's fallback is now called out explicitly in code comments
and covered by three new integration tests (fallback fires, explicit
mapping wins, unrelated extra_data doesn't false-trigger).
Closes#2684
Review follow-ups on applying camera_rotation to finish photos and
layer-timelapse frames.
Rotating the frame popped from _stage22_finish_frames rotated one of its
sources twice. The cache has two kinds of feeder: live grabs, which are raw,
and the #1867 in-print bank, whose bytes come from
_capture_snapshot_for_notification and have already been rotated on the way
in. The consumer cannot tell them apart, so on the finish_state trigger - the
path the bank exists to serve, on firmware that never emits stg_cur=22 - a 180
degree rotation cancelled itself out and the photo was upside-down again,
which is the reported symptom exactly; 90 and 270 landed 180 out. Rotation now
happens where each frame is captured, so every entry in the cache carries one
rotation whatever produced it, and the invariant is stated both where the
cache is declared and where it is consumed.
Two finish-photo sources were still writing unrotated files: the built-in
camera's own capture_finish_photo, and the still extracted from a
printer-recorded timelapse - which is the *preferred* source for a built-in
camera print, so a user with a rotation set got a correctly oriented photo or
not depending on which source happened to win. Neither ever holds the frame as
bytes; ffmpeg writes the file and they return a filename. apply_camera_rotation_to_file
handles that case and is best-effort - a failed rotate leaves the unrotated
file rather than losing a delivered photo. The archived video itself is the
printer's own file and is not re-encoded, so it still plays at the camera's
native orientation; the CHANGELOG says so rather than leaving it to be
discovered.
apply_camera_rotation logs at debug, not info. It was on a path that runs once
per layer, where a tall print would have put hundreds of lines in the log for
something the surrounding capture already reports at debug.
The moved rotation logic had no test of its own - every existing test patches
it out and asserts the call, so a flipped sign or a dropped expand=True would
have shipped green. test_camera_rotation.py drives the real round trip: a
corner marker pins which way it turns, the dimensions pin that the frame is
not cropped, and an undecodable frame comes back by identity because a capture
path must not lose a frame to a failed rotate.
Tests for the fix itself sit on both sides of the cache. The producer half is
driven directly; the consumer half is a closure nested inside on_print_complete
with nothing able to reach it, so it is pinned by an AST guard - checked
against the source because the alternative is no check at all. Reverting
main.py to the pre-fix shape fails three of the five, the guard among them.
The three new tests used Path("/tmp/test") for a patched base_dir, which Bandit
flagged (B108); they take tmp_path now.
Review follow-ups on the orphaned timelapse session cleanup.
The sweep's own docstring said min_age_seconds made it safe to call mid-run.
It did not. on_print_complete drops the session from _active_sessions before
handing frames_dir to ffmpeg, so for the length of a stitch the directory
matches no active session, and its mtime is the last layer's frame write -
which on a tall print's final layer is easily older than the margin. The
default margin is 300s and the stitch timeout is also 300s, so the two were
tied with no headroom at all: a sweep landing in that window deleted ffmpeg's
input from under it. _finalizing_sessions now covers the stitch, set as the
session leaves _active_sessions and cleared in a finally so a failed stitch
cannot leak the marker and make that printer's leftovers permanently
un-sweepable. The docstring names all three guards and which gap each covers,
including that the margin does have real headroom for the two cases it suits -
a session mid-creation, and the freshly written .mp4 awaiting attach.
The file branch now requires the timelapse_<session_id>.mp4 shape its own
comment describes. It previously deleted any file under
timelapse_frames/<printer_id>/ past the margin; nothing else writes there
today, but age alone is not a reason to delete a file this feature did not
create.
Dropped ignore_errors=True from the rmtree. It made the surrounding
except OSError unreachable, so a read-only mount or a permissions problem was
counted and logged as a successful removal - and that log is the only evidence
an operator has of what was deleted.
Tests 5 -> 9: sparing a session mid-stitch, the finalizing marker cleared even
when the stitch raises, unrelated files left alone, and a failed removal not
counted. The failure test's rmtree stub honours the real contract and returns
silently when ignore_errors=True, because that silent no-op is exactly what the
old call could never observe; a stub that raised unconditionally would have
passed against both versions and proved nothing.
main.py is unchanged: it has no module-level logger, and the inline
logging.getLogger(__name__) the sweep uses is the idiom throughout lifespan.
Review follow-ups on the external-camera capture coalescing.
The coalescing was transplanted from camera.py, which is keyed by printer IP
and so has nothing to hide in a log line. These keys carry the camera URL, and
an RTSP camera URL routinely embeds user:pass@ - so the five new log lines
printed the password, one of them at warning level, where it reaches support
bundles. All five now go through _log_key(), which redacts before truncating:
slicing first can cut the URL short of the @ the pattern anchors on and leave
the password intact, which is why every other URL log in the module already
does it in that order.
_capture_frame_uncoalesced gained the blanket catch its camera.py counterpart
has. That is load-bearing once captures are shared: the wrapper hands one
task's outcome to every caller waiting on it and can only give a follower its
own turn for an outcome it recognises, so an escaping exception reached all of
them at once and none retried - one caller's failure becoming N. The per-type
helpers catch narrowly (aiohttp.ClientError / OSError / timeouts), so the
guarantee belongs here rather than resting on their coverage. CancelledError
is re-raised ahead of it, since the wrapper distinguishes a cancelled leader
from a failed one.
test_connection reports whether it shared a capture. It reaches capture_frame
like any other consumer, so a test landing while Obico is polling got that
frame back and answered "connected" for a connection it never made - the one
answer a connection test must not give silently. It still shares rather than
forcing its own capture, because forcing one would open the second handle to a
single-reader device that this whole mechanism exists to prevent. The response
carries `coalesced`, which also gives capture_in_flight() the consumer its
camera.py counterpart has in the Diagnose tool, and the Test button says
"shared with a capture already running" instead of a bare success.
Tests 12 -> 20: an unexpected error reported as a failed capture, a raising
leader whose follower still gets a frame, the three coalesced states, and
redaction on each log line that can carry a URL. The raising-leader test
patches _capture_rtsp_frame rather than _capture_frame_uncoalesced, since a
stand-in installed in the latter's place sits above the catch and would test
the wrapper against a shape it can no longer be handed.
Same gap as the finish-photo fix: camera_rotation was only ever wired
into the notification-snapshot path, so a layer-timelapse video came
out upside-down whenever a rotation was configured - every frame
(fresh or reused from the live view's buffer) was written to disk raw.
Extracts the rotation logic out of main.py into a shared
apply_camera_rotation(image_data, rotation, logger) in services/camera.py
(taking the rotation value directly rather than a printer object, so
both main.py's printer-shaped callers and layer_timelapse's plain int
field can use it). main.py's _apply_camera_rotation becomes a thin
compatibility wrapper so its existing call sites are unchanged.
Threads a `rotation` field through TimelapseSession/start_session,
set from printer.camera_rotation in _maybe_start_layer_timelapse, and
applies it (via asyncio.to_thread, since PIL rotation is CPU-bound) in
capture_layer before each frame is written - after #2707's
live_frame_for_capture() resolves the frame, regardless of whether it
came fresh or from the live view's buffer.
Rebased onto #2707's landed implementation: capture_layer now calls
live_frame_for_capture() instead of the older is_stream_active/
try_get_active_buffered_frame pair this was originally written
against, so the layer-timelapse tests are updated to match, and the
now-redundant TestCaptureLayerAvoidsCompetingWithLiveViewer class
(superseded by #2707's own test_external_camera_live_frame_reuse.py)
is dropped.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
_apply_camera_rotation was only ever wired into
_capture_snapshot_for_notification (print-start photo, in-print frame
bank) - the finish-photo pipeline (_background_finish_photo's three
branches: stage-22 pre-captured frame, external-camera fallback,
built-in buffered-frame fallback) saved frames straight to disk with
no rotation, so a finish photo came out upside-down whenever
camera_rotation was configured. Pre-existing gap, unrelated to
yesterday's collision fixes - just more visible now that a finish
photo reliably gets captured at all.
Built-in camera's final capture_finish_photo() fallback (writes its
own file, returns only a filename) is left unrotated - out of scope
here since Carl uses an external camera; would need its own change to
services/camera.py.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
_active_sessions is in-memory only, so a process restart mid-print
loses track of any active layer-timelapse session without ever calling
cancel_session()/cleanup() - the frames directory (and, if stitching
had already produced output before the restart, a stray
timelapse_<session_id>.mp4) are then orphaned on disk permanently, with
no equivalent to the ffmpeg orphan janitor to reap them.
Confirmed live: 38MB of exactly this leftover on the OrangePi after
several restarts during this week's testing, including two corrupt
48-byte .mp4s from stitches that got interrupted mid-write.
Adds cleanup_orphaned_timelapse_sessions(), run once at startup: for
each printer_id under timelapse_frames/, remove any frame directory or
stitched-output file that doesn't match that printer's current active
session (if any) and is older than a defensive margin (5 min default).
A restart-recovered print never gets a new timelapse session either
(#1353's _maybe_start_layer_timelapse only fires on fresh PRINT_START
events), so nothing orphaned here can ever be resumed - safe to always
remove once it's old enough not to be a startup race.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
#2705 fixed simultaneous captures colliding on the built-in camera path,
keyed by printer IP through capture_camera_frame_bytes(). External
cameras reach the same collision through a different function -
external_camera.capture_frame() - that #2705 didn't touch, and a V4L2
USB device allows exactly one open handle just like Bambu's own RTSP
limit.
Nothing coalesced two one-shot capturers here either: Obico polling,
the in-print frame bank, the finish-photo moment, plate detection and
the notification snapshot could each open their own connection to the
same USB camera and collide - is_stream_active() only stops a
capturer from competing with an attached viewer, not with another
capturer (that's what #2707 fixed).
capture_frame() is now a single-flight coalescing wrapper (actual
dispatch moved to _capture_frame_uncoalesced), keyed by (url,
camera_type, snapshot_url) - snapshot_url is part of the key since
#1177's override routes to a completely different endpoint. Mirrors
#2705's shape: coalesces, doesn't cache (a call after the previous one
finishes always captures fresh); each caller keeps its own timeout via
wait_for(shield(...)) rather than inheriting the leader's; a follower
whose leader fails takes its own turn instead of inheriting a failure
it never had a chance to avoid, bounded at two rounds; cancellation is
disambiguated via leader.cancelled() so a follower's own cancellation
still propagates while a cancelled leader is treated as a failed one.
12 tests mirroring test_camera_capture_coalescing.py's structure.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
device.airduct is pushed field by field - the modeCur handler reads it with
an "in" check for that reason - so a frame can carry parts without carrying
every fan. Absence in that list is what tells us a kit is not fitted, and
taken from a truncated frame it made both accessory badges vanish mid-print
and started rejecting fan=aux2 on a printer that has the fan.
A parts list now counts as a full inventory only when it carries ids 1 (part
cooling) and 2 (aux). Neither is optional on a machine that reports an
airduct at all, and both appear in every layout in the support-package
archive - P2S base 1,2 / P2S+kit 1,2,3 / X2D 1,2,3,10 / H2C,H2D,H2S 1,2,3,6.
Anything narrower is a diff frame: its speeds are applied, presence is left
alone. Presence can still be added from a partial frame; only retraction
needs the full list, so a kit that really is removed still disappears.
Also compose showChamberFan from both model lists rather than branching
between them, so the P2S/X2D entries in MODELS_WITH_CHAMBER_FAN stay
reachable instead of reading as dead, and note in the fan-speed docstring
that the aux2 gate also rejects between connect and the first airduct push.
The filament list is positional from the modal down to the CLI's
filament_N.json parts, but for a source that already carries slice_info the
requirements endpoint returns only the slots the plate consumes. A
MakerWorld model declaring four filaments and painting with slot 4 alone
therefore showed one dropdown, whose PETG pick the CLI bound to slot 1 —
slot 4 sliced with the profile baked into the source, and the print came out
PLA.
The endpoint now takes full_slots, which widens that answer to every
project slot with used_in_plate flags, and only the slice modal passes it.
Print-time AMS matching shares the endpoint and keeps the used-only list, so
it still asks for exactly the spools the job needs.
setInterval does not await an async callback. Slicing a large project
blocks the backend for seconds, so poll ticks piled up behind one stalled
request, each holding a snapshot taken while the job was still active.
They resolved together, and every one of them ran the completion path —
one toast and two query invalidations each. A 20s stall against the 1.5s
interval produced 13 "Sliced X" toasts from a single slice.
Only one poll round is now in flight at a time, which also stops queueing
requests against a backend that is already saturated. Completion is
recorded once per job id, and a round still awaiting a response when the
effect tears down now returns instead of acting.
The two-flagged-row state was never released, so no installation can carry it
in -- the reason to release every flagged row is not repair, it is that the
sweep's invariant is not enforced anywhere and scalar_one_or_none() turns a
broken one into a failed boot. Comment, test docstring and test name say that
instead.
Without it every account auto-created through the env provider fell back to
Viewers (routes/mfa.py), and because the provider is locked the UI could not
correct it either -- a real limitation for a declarative deployment running
BAMBUDDY_OIDC_AUTO_CREATE_USERS=true.
BAMBUDDY_OIDC_DEFAULT_GROUP names a group rather than an id: ids are handed out
per installation, so the same compose file would point at a different group on
the next deployment. The name is matched exactly, resolved against the database
before anything is written, and default_group_id joins _APPLIED_FIELDS so
dropping the variable clears the group again -- the environment is the whole
truth for this row.
A name that matches no group is refused rather than defaulted: silently landing
users in Viewers is the failure this variable exists to remove, and the API
already answers 422 for a default_group_id that does not exist. The refusal is
logged and survivable, and it says which of the two cases happened, because
they differ sharply -- an existing provider keeps running on its last good
config, while on a first boot nothing is created and no SSO button appears.
Raised by maziggy in review of #2625 as a scope decision; documented in
.env.example and in the companion wiki PR.
apply_env_oidc_provider matches the provider by BAMBUDDY_OIDC_NAME but never
cleared is_env_managed from the row it managed previously. Renaming the
variable therefore left two flagged rows, and both consequences are reachable
by ordinary config edits: the old row stayed enabled with a stale issuer and
secret on the login page while _refuse_if_env_managed answered 409 to every
attempt to edit, disable or delete it -- the dead end reachable only through
the database that the release path exists to prevent -- and unsetting the
variables later hit scalar_one_or_none() on two rows, so MultipleResultsFound
propagated out of the lifespan and the app stopped booting.
The upsert now sweeps the flag off every other row, the same shape the
autologin sweep one block down already uses: disable and release rather than
delete, for the same cascade reason as everywhere else in this branch. The
release path releases every flagged row it finds instead of exactly one -- the
sweep should keep that at one, but a release path that dies with
MultipleResultsFound the moment that invariant breaks is a second way to lose
the boot, and the query costs the same either way.
Releasing now clears is_autologin as well. Without it a released row keeps a
latent autologin claim: update_oidc_provider only re-runs the exclusivity sweep
when a request sets is_autologin=True, so merely re-enabling the row in the UI
would silently make it the autologin target again.
Reported by maziggy in review of #2625, with the rename reproduction.
feat(db): warn when the connection pool can outgrow the PostgreSQL server
fix(mqtt): an unusable layer_num must not drop the printer connection
test: patch settings.base_dir via monkeypatch so it unwinds on error
test: restore the config module after reloading it
On a printer with an external camera, watching the live view while a print
ran meant the layer timelapse recorded almost nothing and the finish photo
went out with no image. The reporter measured 0 of 87 layer captures on one
print and 0 of 105 on another, both watched throughout. A USB camera allows
one V4L2 handle, so a capture during a live view fails outright.
The built-in camera has had this rule since #1348 and #1271: reuse the
viewer's buffered frame rather than opening a second connection. It was
never extended to the external paths, and it could not have been -- the
buffer it depends on was only ever populated by the built-in paths.
generate_mjpeg_stream yields multipart-wrapped chunks, so the route layer
could not recover the JPEG, and a guarded caller would have found an empty
buffer and skipped every time.
So the stream now publishes each raw frame through a new on_frame callback
(parallel to on_process from #2675), and the six one-shot consumers reuse
it: layer timelapse, the finish-photo moment and its background fallback,
the notification snapshot, Obico polling, and the plate check. A viewer
attached with nothing buffered yet skips that one attempt rather than
competing -- kicking the viewer off is worse than missing a frame.
on_frame exceptions are logged and swallowed, like iter_subscriber's
on_unsubscribe: buffering is a side effect and must never be able to take
the live stream down with it. The external stream's teardown now releases
the buffered frame too, ownership-checked so a concurrent viewer of the
same printer keeps its own.
Two side effects on paths not touched here, both improvements: the snapshot
endpoint and the finish-photo fallback chain consult get_buffered_frame and
can now serve an external camera's live frame. plate_detection's docstring
already claimed this behaviour while implementing it only for the built-in
fallback; that drift is resolved.
ffmpeg is spawned with stderr=PIPE and it was only read on the error
paths, so for the life of a working stream nobody read that pipe. ffmpeg
writes its banner, the input analysis, then a progress line at a steady
rate; a 64 KiB pipe fills eventually, ffmpeg blocks writing to it, frames
stop, and the stream's own 30s timeout fires -- logged as "RTSP read
timeout" with no hint that we starved it ourselves.
How long that takes is unmeasured and evidently long: one H2D upstream
ran 21m36s without stalling, and an earlier 512 B/s extrapolation of mine
was mostly the one-off startup banner. So this is a bounded resource being
treated as unbounded, not a fault anyone has reported.
_FfmpegStderrTail drains the pipe continuously and keeps a 16 KiB rolling
tail. That tail is what the error paths now report, which is better
material than before: it holds what ffmpeg said as things went wrong,
where the on-demand read returned whatever was printed first -- usually
the banner, which the summariser strips anyway.
Three readers wanted this one pipe, and asyncio rejects concurrent reads
on a StreamReader, so the collector is authoritative: it registers by pid,
_read_ffmpeg_stderr returns its tail when present and otherwise reads the
pipe unchanged, and _terminate_ffmpeg skips its own stderr drain when the
collector owns it (the collector keeps draining through teardown, which is
all wait() needs). The generator starts it after the immediate-failure
check, which reads the pipe directly because the process is already dead,
and releases it after _terminate_ffmpeg.
text() goes through _summarize_ffmpeg_stderr like every other stderr log
here, so the access code ffmpeg echoes in its input URL stays masked.
aclose() awaits the cancelled pump rather than firing and forgetting, so
no pending task survives into loop teardown.
Closing a camera view and reopening it immediately could leave the new
stream unregistered while it was running and delivering frames. The
damage was all indirect: is_stream_active() reported no viewer, so Obico
polling and snapshots opened a second camera connection against the live
view (the thing #1348 and #1271 exist to prevent); the janitor's /proc
scan found an ffmpeg missing from _active_streams and killed the live
stream as an orphan; and /camera/stop reported "Stopped 0" with a stream
running.
The fan-out stream id was f"{printer_id}-fanout" -- constant per printer,
so every successive stream shared one registry key, and the departing
generator's finally popped whatever was under it, including its
successor's entry. The same finally also cleared the per-printer frame
buffer unconditionally, discarding the new stream's frame. It needed the
two streams to overlap, which the 4s teardown made easy.
Each stream now gets its own key via _new_fanout_stream_id(), so a
generator can only clean up after itself -- the external-camera path
already does this (#2675) and this brings the fan-out path in line. The
per-printer dicts are released through _release_printer_frame_state(),
which checks that no other stream for the printer is still running; both
the RTSP and chamber-image cleanups had the same unconditional pop.
Also hoisted time and uuid to module level and dropped four
function-local `import time` statements. A local import shadows the name
for the whole function, so any use on a branch that doesn't reach the
import raises UnboundLocalError -- a real hazard in camera_stream, whose
external-camera branch imported both while the RTSP path needs them too.
A test pins camera_stream as free of function-local imports.
Closing a camera view logged "ffmpeg didn't terminate gracefully,
killing" followed by "ffmpeg did not exit within 2.0s of SIGKILL;
abandoning wait", on every single close. Both waits expired every time,
so teardown took a fixed 4.00s -- and since the firmware allows one
camera connection, that was 4s in which nothing else could use it.
ffmpeg is spawned with stdout and stderr as pipes and the teardown paths
have stopped reading them, so it sits blocked in write() on a full 64 KiB
pipe. SIGTERM cannot be acted on there: the handler only sets a flag that
the main loop polls, and the loop never gets back to the check. SIGKILL
does kill it, but asyncio resolves Process.wait()'s waiter through
_try_finish(), which requires every pipe transport to report
disconnected; paused, unread pipes never reach EOF, so wait() blocks with
returncode already set. A negative-control test shows returncode=-9 at
the instant the abandon fires.
Draining both pipes while stopping the process fixes both halves: 4.00s
becomes ~0.15s. The signal ladder and its bounds stay as backstops, so a
genuinely wedged process still cannot hang a stream, a Stop request or
the janitor.
This corrects _FFMPEG_KILL_TIMEOUT's premise and #2580's conclusion. That
12-hour hang was the unbounded form of this same self-inflicted stall, not
an ffmpeg stuck in uninterruptible I/O -- the process observed doing it
was in state S, which cannot survive a delivered SIGKILL. Bounding the
wait capped the symptom without removing the cause.
maziggy