From 46c246c504ddff92e4174656ad60e0e2d450083a Mon Sep 17 00:00:00 2001 From: maziggy Date: Thu, 16 Apr 2026 09:36:44 +0200 Subject: [PATCH] fix(archive): resume on subtask_id, short-circuit 550, cache 3mf (#972) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Second wave of #972 — reproducer on a 37.5 MB BambuStudio print to an A1 showed three stacking root causes when Bambuddy restarts mid-print. 1. Archive start_time lost on container restart. The name-based dedup cancelled any "printing" archive older than 4h and recreated it with started_at=now(), so a 13h print that saw a restart 10h in ended up showing ~1.5h duration. Persist MQTT subtask_id on every archive and match on that first, regardless of age — same id means same print, resume in place. Also revives Stale-cancelled rows for users upgrading mid-print. 2. 3MF FTP search tried non-existent paths for ~48 min. Order was /cache → /model → /data → /data/Metadata → / with 11×30s retries each; BambuStudio actually pushes to / on A1, so the real path was tested last. Reorder to / first, and raise a new FileNotOnPrinterError sentinel from download_to_file on 550 so with_ftp_retry short-circuits via non_retry_exceptions. 425 / SSL EOF / connection resets still retry as before. 3. Cover endpoint and archive flow downloaded the same 36 MB twice and competed for the printer's single FTP socket, producing 425 errors that fed cause-2's retry storm. Add an in-memory _threemf_path_cache keyed on (printer_id, normalized filename); whichever flow fetches first populates it, the other reuses the file read-only. Eviction runs on on_print_complete and deletes the temp file. Backend: 14 new tests across test_bambu_ftp.py and a new test_subtask_archive_resume.py. Existing suite: 2737 pass. ruff clean, frontend build clean. --- CHANGELOG.md | 1 + backend/app/api/routes/printers.py | 92 +++++---- backend/app/core/database.py | 6 + backend/app/main.py | 171 +++++++++++++--- backend/app/models/archive.py | 6 + backend/app/services/archive.py | 5 + backend/app/services/bambu_ftp.py | 106 +++++++++- backend/tests/unit/services/test_bambu_ftp.py | 174 +++++++++++++++- .../tests/unit/test_subtask_archive_resume.py | 185 ++++++++++++++++++ 9 files changed, 677 insertions(+), 69 deletions(-) create mode 100644 backend/tests/unit/test_subtask_archive_resume.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 8ab014152..29b8f273c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,7 @@ All notable changes to Bambuddy will be documented in this file. - **Clear Plate Confirmation Bypassed on Power Cycle** ([#961](https://github.com/maziggy/bambuddy/issues/961)) — With Auto Off enabled and another job queued, the smart plug would cut power when a print finished and immediately re-power when the scheduler saw the queue, at which point the printer booted fresh into `IDLE` and the next job auto-dispatched without the "Clear Plate & Start Next" confirmation. Root cause: the plate-cleared gate lived only in the in-memory `PrinterManager._plate_cleared` set, and the scheduler's idle check treated `IDLE` as always-idle regardless of whether a previous finish had been acknowledged — so the gate was lost across both Bambuddy restarts and the IDLE-on-boot state transition. The gate is now an `awaiting_plate_clear` column on the `printers` table, set by `on_print_complete` when a print finishes or fails, cleared by the `/printers/{id}/clear-plate` endpoint and by the scheduler when it dispatches the next job, and rehydrated from the DB into `PrinterManager` on startup. `_is_printer_idle` now short-circuits to not-idle whenever `require_plate_clear` is on and the printer is awaiting ack, regardless of the currently reported state — so the prompt survives Auto Off cycles, Bambuddy restarts, and the printer booting back into `IDLE`. The clear-plate endpoint no longer requires the printer to currently report `FINISH`/`FAILED` (it accepts the ack whenever the awaiting flag is set), and the Printers page widget prompts based on the flag rather than the reported state. Thanks to @miaopas for reporting. - **Insecure Temp File Creation in Backup Export** — The manual backup download endpoint used `tempfile.mktemp()`, which is vulnerable to a symlink race condition (CWE-377). Replaced with `tempfile.mkstemp()` which atomically creates the file, eliminating the TOCTOU window. - **Spoolman Iframe Blocked After 0.2.3b4 Security Headers** — The Spoolman page (Inventory → Spoolman iframe) failed to load when Spoolman was served from the same host as Bambuddy via a reverse proxy. The security-headers middleware added in 0.2.3b4 set `X-Frame-Options: DENY` on every response, which blocked even same-origin iframing. Relaxed to `SAMEORIGIN` so Spoolman (and any other same-origin tool behind the same reverse proxy) can be embedded again, while still preventing cross-origin clickjacking. +- **Large 3MF Print Restart Mid-Job Kept Duplicate Archive With Wrong Duration** ([#972](https://github.com/maziggy/bambuddy/issues/972)) — Second wave of #972 reports — a reproducer on a 37.5 MB BambuStudio-pushed print to an A1 surfaced three distinct problems that compounded across a Bambuddy container restart mid-print. (1) *Archive start_time lost*: the print-start handler only deduped existing `printing` archives by filename and marked them cancelled once older than 4 h — so a 13 h print that had a restart 10 h in got its archive cancelled, a brand-new archive created with `started_at = now()`, and the final duration displayed as ~1.5 h for a job that actually ran 13 h. Fixed by persisting the MQTT-provided `subtask_id` on every archive row (new `subtask_id` column, auto-added via the existing inline migration runner) and matching on that id first, regardless of age. Same id means same print; the row is resumed in place with its original `started_at`. Also revives `Stale`-cancelled rows from the legacy path if an earlier Bambuddy version already ran the old cancel-then-recreate logic. (2) *3MF search retried non-existent paths for ~48 min*: the path order was `/cache/ → /model/ → /data/ → /data/Metadata/ → /`, and every missing path burned the full retry budget (user had `ftp_retry_count = 10` with 30 s delay ⇒ 11 × 30 s × 4 missing paths ≈ 22 min before the real `/` root path was even tried). BambuStudio/OrcaSlicer actually push to `/` on A1-family printers, so the "most likely" path was tested last. Fixed by reordering to try `/` first, and by raising a new `FileNotOnPrinterError` sentinel from `download_to_file` when the FTP response is a 550 (file not found) so `with_ftp_retry`'s `non_retry_exceptions` short-circuits instead of waiting out the full delay ×11 retries against a path that will never have the file. Transient errors (425 "can't open data connection", SSL EOF, connection resets) still retry as before. (3) *Same 36 MB downloaded twice* — the cover-thumbnail endpoint and the archive-metadata handler each opened their own FTP session for the same file during the print, and the second session often hit 425 because the first was still using the printer's single FTP socket. Added a small in-memory `_threemf_path_cache` keyed on (printer_id, normalized filename): whichever flow fetches the 3MF first populates the cache, the other flow reuses the file read-only, and `on_print_complete` evicts the entry + deletes the temp file. Normalization collapses `Broly_X`, `Broly_X.3mf`, `Broly_X.gcode.3mf`, `Broly X`, and case variants to the same slot so both flows agree on the key. Net effect for the reproducer: what took ~48 min with a lost start time now takes seconds and the archive keeps its original row + timestamps. Thanks to @mstko for the reproducer and support bundles. - **Large 3MF Files Silently Dropped After Print Finish** ([#972](https://github.com/maziggy/bambuddy/issues/972)) — After large prints, the Files tab rows arrived with no thumbnail, no filament breakdown and no cost — the archive row got created as a fallback with no 3MF even when the file was sittable on disk. Two root causes in the 3MF-fetch path. (1) The configured `ftp_timeout` setting (default 30 s, reporter had raised it to 300 s) was only plumbed through as the FTP *socket* timeout; the outer `asyncio.wait_for` wrapping `run_in_executor` was stuck on the hardcoded 60 s default, so the user's 300 s value never applied — every 3MF download was capped at 60 s regardless. (2) `asyncio.wait_for` cannot cancel `run_in_executor` threads: when the 60 s outer timeout fired, the executor thread kept running `ftplib.retrbinary` and frequently completed the download successfully ~30–60 s later — logging `"Successfully downloaded … N bytes"` and caching the working FTP mode — but by then the async wrapper had already returned `False`, so the retry loop kept re-attempting the same path, each attempt truncating the file the zombie thread had just written. After all 4 attempts the wrapper reported `failed after 4 attempts` and the archive was persisted as a fallback (no 3MF, empty `file_path`). The async wrapper now (a) accepts and uses `timeout` at each call site so `ftp_timeout` controls both the asyncio deadline and the socket deadline, and (b) salvages a post-timeout success: when the executor thread has set an explicit completion flag and the file is on disk, the wrapper returns `True` instead of discarding the result. Also fixes a cosmetic `//` prefix in the directory-search download path (`posixpath.join` replaces string concatenation that produced `"//file.3mf"` when the search dir was `"/"`). Thanks to @MartinNYHC for the report and @PurseChicken for the P1S support bundle. - **SD Card Badge Removed** — After four rounds of fixes the printer-card SD status badge still flipped red on H2D when unrelated activity happened on the network (e.g. powering on an A1 caused every H2D to go red simultaneously). The underlying problem is that Bambu firmware SD-state signaling is not reliably derivable from MQTT: the legacy top-level `sdcard` field is only sent on some pushes with inconsistent typing, and `home_flag` bits 8-9 are cleared on heartbeat pushes even when a card is inserted, with no reliable way to distinguish heartbeats from full status reports. The badge has been removed entirely from the Printers page card and the Printer Info modal. Underlying `state.sdcard` parsing is retained (simplified to a plain truthy read of the `sdcard` field only, no more `home_flag` derivation, no heartbeat latches) because the firmware-update precondition check still needs to know whether a card is inserted before starting an update. Thanks to @MartinNYHC for the extensive reporting across all four rounds. Previously, this entry described the H2D badge flap and its three attempted fixes — kept here for history: The original bug toggled between "inserted" (green) and "not inserted" (red) every few seconds on H2D. Root cause: the MQTT parser used a strict identity check (`data["sdcard"] is True`) on the top-level `sdcard` field, but real firmware ships that field inconsistently — bool on some models, int `1`, or a string enum like `"HAS_SDCARD_NORMAL"` on others — so any message carrying a non-bool value flipped the state to `False`. Fixed by deriving the badge from `home_flag` bits 8–9 (`HAS_SDCARD_NORMAL` / `HAS_SDCARD_ABNORMAL`) when present — the canonical firmware source, same as door and store-to-SD parsing — and falling back to a truthy check on the top-level field for firmwares that only send that. Follow-up: the badge was still flapping because Bambu firmwares send partial MQTT pushes that carry the legacy `sdcard` field alone (without `home_flag`), and the fallback was re-engaging on every such push. The parser now latches `home_flag` as the canonical source for the session once seen, so partial pushes carrying only `sdcard` can no longer flip the badge; the latch resets on reconnect so a firmware change still re-learns. Second follow-up: on H2D the badge still showed red on initial Printers-page navigation and flipped to green on reload, because H2D also sends heartbeat-style `home_flag` pushes where bits 8–9 are clear even when a card is inserted. Downgrades from true→false now require three consecutive clear reads (upgrades false→true still apply immediately), so a single heartbeat no longer turns the badge red. Third follow-up: the three-strike counter still lost the race on idle printers — once an A1 or other printer connecting nearby triggered a burst of MQTT activity, idle H2Ds could accumulate ≥3 heartbeat pushes before the next full status report and all flip to red simultaneously. Reworked the derivation: the legacy top-level `sdcard` field is now authoritative when present (truthy check covers bool/int/string firmware variants), `home_flag` bits 8–9 are only consulted on full `push_status` reports (identified by the presence of multiple state markers like `gcode_state`, `mc_percent`, `nozzle_temper`, `print_type`, `stg_cur`, or `ams`), and bare heartbeat pushes carrying `home_flag` alone no longer affect SD state at all. Thanks to @MartinNYHC for reporting. - **CSP Blocked Sidebar Iframes, Service-Worker Registration, and Google Fonts** — The strict `Content-Security-Policy` header added in 0.2.3b4 broke three things at once: (1) custom sidebar links pointing at external HTTPS URLs (e.g. a Grafana/telemetry dashboard) rendered in `ExternalLinkPage` were blocked because no `frame-src` was declared and iframes fell back to `default-src 'self'`; (2) the inline service-worker registration `