Merge branch 'dev' into feature/slicer-api

This commit is contained in:
MartinNYHC
2026-04-27 17:09:42 +02:00
committed by GitHub
13 changed files with 1294 additions and 31 deletions
+11
View File
@@ -37,6 +37,14 @@ All notable changes to Bambuddy will be documented in this file.
- **Per-request trace ID column on every log line, plumbed through HTTP access log + application logs + response headers** — Builds on the new uvicorn-access-log-into-bambuddy.log change below: the access line tells you *who* called an endpoint, but until now there was no way to tie that line to the application records emitted on the server side while handling that request. A new FastAPI middleware (`trace_id_middleware` in `main.py`, sourced from `backend.app.core.trace`) stamps each request with a fresh 8-char hex ID (or honours a sane inbound `X-Trace-Id` header for cross-system correlation), stores it in a `ContextVar` so any code in the request's call stack can read it, echoes it on the response as `X-Trace-Id`, and a new `TraceIDFilter` injects it into every `LogRecord` so the format string `[%(trace_id)s]` resolves to the right ID for the right request. ContextVars (rather than `request.state`) are the right plumbing here because asyncio copies the current context into every `asyncio.create_task`, so background work spawned from inside a request inherits the trace ID without explicit threading; the logging filter has no access to the FastAPI request object regardless. Records emitted outside any request scope (startup, MQTT callbacks, scheduler) get a stable `-` placeholder so the column stays visually aligned and missing values are obvious in `grep`. Inbound `X-Trace-Id` is hard-validated against a strict whitelist (`[A-Za-z0-9_-]+`, max 64 chars) before being honoured — a hostile or buggy caller cannot smuggle log-injection payloads (newlines, control chars, megabyte blobs) into `bambuddy.log` via the trace-ID column; values that fail the gate silently trigger a freshly minted server-side ID rather than failing the request. Middleware is decorated AFTER `auth_middleware` on purpose: Starlette stacks `@app.middleware` decorators LIFO so the last-decorated runs first inbound, making trace stamp the OUTERMOST layer — auth log lines and every record emitted on the way down to and back from the route handler all carry the same ID. Output now looks like `2026-04-26 09:51:39,152 INFO [uvicorn.access] [a4f3b1e7] 192.168.1.42:54812 - "POST /api/v1/printers/1/print/stop HTTP/1.1" 200` paired with the route handler's `2026-04-26 09:51:39,158 INFO [bambu_mqtt] [a4f3b1e7] [SERIAL] Sent stop print command` — one `grep a4f3b1e7` away from the full causality chain. 30 new tests across `tests/unit/test_trace.py` (placeholder when no request scope, filter copies ContextVar value onto records, ID propagates into spawned tasks via asyncio context copy, concurrent requests don't leak IDs into each other, generator produces unique hex IDs, hostile payloads rejected by validator, max-length boundary, dash/underscore variants accepted) plus `tests/integration/test_trace_middleware.py` (X-Trace-Id header echoed on response, body and header IDs match, each request gets a unique ID, generator format stays short hex, safe inbound IDs honoured, hostile inbound IDs replaced, overlong inbound IDs replaced, ContextVar reset cleanly after request).
### Fixed
- **Moving a file to an external folder updated the DB row but never wrote the bytes to the mount** ([#1112](https://github.com/maziggy/bambuddy/issues/1112) follow-up — confirmed by @Carter3DP after testing 0.2.4b1) — Carter's report read "the file appears in Bambuddy but not physically on the external folder", which traced to `move_files` only updating `file.folder_id` in the DB while leaving the bytes in the internal `library_files_dir`. Direct upload to a writable external folder was already fixed in 0.2.4b1; the move path was not. Cross-boundary moves now physically relocate the bytes through a new `_move_file_bytes` helper. Same-boundary moves (managed → managed) keep the existing DB-only fast path because the file's on-disk location doesn't depend on which managed folder owns it. The helper handles four flows: managed → external (copy bytes to `<external_path>/<filename>`, flip `is_external=True`, store the absolute path, unlink the managed source), external → managed (copy bytes into internal storage with a fresh UUID name, flip `is_external=False`, store the relative path, unlink the external source, recompute `file_hash` since scan-tracked rows historically carry `file_hash=None`), external → external (same as managed → external), and managed → managed (DB-only). Copy-then-unlink ordering means a partial copy followed by a failed unlink leaves both copies on disk rather than losing the source if the target write fails halfway through on a flaky NAS mount. Failed `shutil.copy2` cleans up partial dest before raising. Defence-in-depth checks block: source on a read-only external mount (move = delete-on-source which a RO mount can't fulfil — would copy-then-fail-to-unlink and silently duplicate the file), filename collisions on the target mount (won't silently overwrite a file the user already has on the NAS), traversal-style filenames after `Path.resolve()`, missing source on disk, and `os.access(W_OK)` on the target mount. Each skip carries a structured `{file_id, code, reason}` entry in a new `skipped_reasons` field on the response so the UI can surface "5 of 10 files skipped: 3 had filename collisions on the NAS, 2 are no longer on disk" instead of a blank "skipped: 5". The original `{moved, skipped}` numeric counters are preserved so existing frontend code that only reads those keeps working unchanged. Six new integration tests in `test_external_folders_api.py::TestCrossBoundaryMove` covering: managed → external relocates bytes (the actual #1112 fix — bytes land on mount, internal source removed, DB row matches reality), external → managed relocates bytes (symmetric path including hash recompute), name collision on target external mount skips with `code: "name_collision"` and leaves the pre-existing target file intact, source on read-only external mount skips with `code: "source_readonly"`, managed → managed stays DB-only (file_path doesn't change, no shutil.copy), and `skipped_reasons` is always present (empty list when nothing skipped) so frontend code can treat it as the source of truth without optional-chaining.
- **`bambuddy.log` filling with `Exception terminating connection ... CancelledError` + `database is locked` cascades on long uploads** ([#1112](https://github.com/maziggy/bambuddy/issues/1112) follow-up, surfaced by @Carter3DP's support package) — Two-part fix to a single root cause: Starlette's `BaseHTTPMiddleware` (which FastAPI's `@app.middleware("http")` decorator uses under the hood) cancels the inner task scope when a client disconnects mid-request — common on long multipart uploads where the client times out before the server's response. Pre-fix `get_db` only caught `Exception`, but `CancelledError` is a `BaseException`, so cancellation skipped the rollback path entirely; the SQLite write lock stayed held until the connection was eventually GC'd, producing the `(sqlite3.OperationalError) database is locked` cascade against `runtime_seconds` updates and other tight-loop writers in @Carter3DP's log. Postgres users would see pool exhaustion / "QueuePool limit overflow" instead of file-level lock contention, but the leak shape is identical. **(1)** `get_db` now catches `BaseException` so `CancelledError` triggers rollback, and wraps both `rollback()` and `close()` in `asyncio.shield` so the cleanup completes even when the await itself is being cancelled by the same cancel scope. The SQLite write lock is released promptly; the connection returns to the pool instead of leaking until GC. **(2)** A `CancelledPoolNoiseFilter` (new `logging_filters.py` filter, attached to `sqlalchemy.pool`) drops the residual log noise that pre-existing pools still emit during their own cleanup — both the `Exception terminating connection ... CancelledError` records (matched on prefix + cancellation-driven `exc_info`, including chained `__cause__`/`__context__`) and the symptomatic `garbage collector is trying to clean up non-checked-in connection` records. Real pool problems — broken connections, network hiccups, exhaustion — keep flowing because they carry a different exception chain or a different message prefix; verified by `test_keeps_terminate_with_real_oserror` and `test_keeps_unrelated_pool_message`. 13 new regression tests across `test_get_db_cancel_safety.py` (commit on clean exit, rollback on regular `Exception`, rollback on `CancelledError` — the actual #1112 fix, close runs even if rollback raises, close failure on clean exit doesn't propagate, both rollback + close go through `asyncio.shield`) and `test_cancelled_pool_filter.py` (drops cancellation-driven terminate, drops GC-cleanup, keeps real `OSError` terminate, keeps terminate without `exc_info`, keeps unrelated pool messages, drops chained-cause `CancelledError`, defensive guard against self-referential cause chains). Applies to SQLite and PostgreSQL — `get_db` is dialect-agnostic and the filtered messages come from base `sqlalchemy.pool` not from any specific dialect.
- **Windows install: `bambuddy.log` filling with `WinError 10054 — _ProactorBasePipeTransport._call_connection_lost` tracebacks** ([#1113](https://github.com/maziggy/bambuddy/issues/1113), reported by @cadtoolbox) — Cosmetic-but-noisy. When a printer / MQTT broker / camera RSTs a TCP socket instead of FINing it (offline X1Es in @cadtoolbox's setup, network gear that drops idle TCP, the printer firmware's own watchdog), Windows asyncio's Proactor cleanup path tries `socket.shutdown(SHUT_RDWR)` on the already-dead socket and hits `WinError 10054`. Application-layer reconnect logic (paho-mqtt, httpx) handles the actual disconnect fine — paho retries, MQTT comes back, telemetry resumes — so the traceback is pure asyncio bookkeeping noise, but it fired multiple times per minute on @cadtoolbox's 9-printer setup with 5 offline X1Es and was the first thing in the sanitized log. Adds a custom `loop.set_exception_handler` (new `backend/app/core/asyncio_handlers.py`) installed on Windows only that pattern-matches the specific `_call_connection_lost` cleanup-RST signature (three signals together: `sys.platform == "win32"`, the exception is `ConnectionResetError`, and the asyncio message string contains `_call_connection_lost`) and downgrades it to DEBUG. Real `ConnectionResetError`s raised inside application coroutines (different message string) and other Proactor cleanup errors (`BrokenPipeError`, `ConnectionAbortedError` — same callback site, distinct signal worth keeping visible) all pass through to `loop.default_exception_handler` unchanged. Linux / macOS use the Selector event loop and never hit this codepath, so `install_proactor_reset_filter()` is an explicit no-op there with a `False` return — verified by `test_install_is_no_op_on_non_windows`. 9 unit tests in `test_asyncio_handlers.py` cover: discriminator matches the exact reported signature, rejects unrelated `ConnectionResetError`s, rejects `BrokenPipeError` even on the same callback site, rejects when no exception object is present, install is platform-gated, install wires the handler onto the loop, suppression doesn't reach the default handler, and unrelated exceptions still hit the default handler. Wired from `lifespan` startup before any task can spawn that might trip it.
- **Auto-Print G-code Injection: start snippet landed before printer startup, and `{placeholder}` substitution was silently broken** ([#422](https://github.com/maziggy/bambuddy/issues/422) follow-up) — Two compounding bugs surfaced by @pleite (Swapmod) and @DevScarabyte (multi-height test prints) on the initial #422 ship: **(1)** Start snippets were prepended to the entire `plate_X.gcode` content, which placed them *before* the printer's bed-heat / homing / nozzle-prime sequence — so a Swapmod start snippet that assumed nozzle-at-temp ran on a cold printer. The injection now anchors at `; MACHINE_START_GCODE_END` (the marker sitting at the bottom of every Bambu/Orca slicer's `MACHINE_START_GCODE` block, after `M109` wait-for-temp), matching where a slicer-side custom-start-gcode would land. Files without the marker (older slicer versions) keep the prepend behaviour as a fallback with a warning log. **(2)** Slicer-style placeholders like `G1 Z{max_layer_z} F600` were written verbatim to the output gcode — the printer firmware then parsed `Z{max_layer_z}` as `Z1` and crashed the head into the print on a 60mm-tall model (a real safety issue: prints damaged, top glass + AMS pushed up off the printer when the model was taller than the hard-coded park height). Added a header parser that reads the 3MF's `; HEADER_BLOCK_START..END` block (lowercased keys, `[units]` suffix stripped, spaces → underscores) and a Prusa-style `{name}` substitution pass that runs over both start and end snippets before injection. Supported placeholders: `{max_layer_z}` / `{max_print_height}` (top-layer Z), `{total_layer_number}` / `{total_layers}`, `{total_filament_weight}`, `{total_filament_length}`, plus any other normalised header key from the source file. Unknown placeholders are left in the snippet verbatim with a warning log — a typo never silently expands to an empty string and the firmware never receives a malformed `Z` parameter. 16 new regression tests in `test_gcode_injection.py` covering: start snippet anchored to the marker (printer startup runs first, snippet sits between `M109 S220` and the marker, file head untouched), missing-marker fallback path, end snippet still appended at EOF, `{max_layer_z}` resolved through the alias map, direct-key substitution from the normalised header, unknown-placeholder pass-through, and direct unit tests for each new helper (`_parse_3mf_gcode_header`, `_substitute_placeholders`, `_inject_start_at_marker`). Wiki page documents the supported placeholder list with a safety warning specifically calling out `{max_layer_z}` for park moves.
- **Camera page ignored `?fps=N` URL parameter** ([#1131](https://github.com/maziggy/bambuddy/issues/1131) diagnostic) — `CameraPage.tsx` hard-coded `fps=15` in the stream URL and never read the URL query string, so `/camera/1?fps=5` (and similar diagnostic suggestions for the freeze report) were silent no-ops. The sibling `StreamOverlayPage` already honoured `?fps=` correctly; the bug was that `CameraPage` was the gap. Now reads `searchParams.get('fps')` via `useSearchParams`, parses it, falls back to 15 on missing/non-numeric, clamps to the backend's 1–30 range, and threads the resulting value into the stream URL. Backend `generate_rtsp_mjpeg_stream` already accepted the parameter and re-clamps per-model (chamber-image A1/P1 capped at 5, RTSP capped at 30). 5 new regression tests in `CameraPage.test.tsx::fps URL parameter (#1131)` cover default-15, honoured value, clamp-above-30, clamp-below-1, and non-numeric fallback — same matrix `StreamOverlayPage.test.tsx` already pins. Independent of the underlying freeze investigation in #1131; surfaced while triaging that report.
- **Reprint-from-archive failed with `0500_4003` SD R/W errors after a stuck dispatch, fixable only by restarting the container** ([#1136](https://github.com/maziggy/bambuddy/issues/1136)) — Reported by @smandon: reprinting from archives sometimes fails immediately with MicroSD R/W exception errors, with the printer's MQTT push referencing a 3MF file from a *different unrelated* archive (`WARIO_Wall_decor_-_NO_AMS.3mf` while the user was actually trying to print `Cable_Organiser_Cable_Clip.3mf`). Once it starts happening, every subsequent reprint hits the same error until the container is restarted. Root cause traced from his support package log to paho-mqtt's client-side QoS 1 queue: when the printer's command channel goes half-broken (telemetry still flowing, publishes silently dropped — same #887/#936 pattern), Bambuddy's 15s dispatch deadline expires (`background_dispatch.py:993`) and calls `force_reconnect_stale_session()`. That function was force-closing the underlying socket so paho's auto-reconnect would kick in — but the same `mqtt.Client` instance, same `client_id`, and **same in-process QoS 1 queue** stayed alive across the reconnect. Any unacked publish from the broken session — typically the just-sent `project_file` for the new archive — got replayed verbatim on the new connection. And because the in-process queue accumulates across multiple stuck dispatches within one Python process, by the second or third stuck reprint there were several stale `project_file`/`resume`/`stop`/`clean_print_error` commands queued up and replaying together. The printer received the flood, tried to load whichever stale path the firmware latched onto last, found a file that no longer existed on its SD card → `0500_4003`. Container restart was the only thing that fixed it because it was the only thing that wiped paho's in-process queue. Replaced the socket-close with a context-aware reconnect: `force_reconnect_stale_session()` and `check_staleness()` now go through a routing helper `_reset_client_for_reconnect()` that picks the right teardown strategy based on caller context. **Async-context callers** (the dispatch deadline path — `background_dispatch.py:993` — which is the actual #1136 trigger, plus FastAPI route handlers via `check_staleness`) get the **hard-reset path**: `client.disconnect()` (broker sees DISCONNECT and drops the session immediately, since `clean_session=True`), `client.loop_stop()` (kills the paho network thread, taking its QoS 1 queue with it), nulls out `self._client`, and calls `self.connect()` to construct a fresh `mqtt.Client` with an incremented `client_id`. New connection starts genuinely empty, no replay possible. **Paho-network-thread callers** (the developer-mode probe and `ams_filament_setting` zombie detection inside `_update_state`, lines ~2604 and ~2623) keep the **socket-close fallback** — calling `loop_stop()` from inside the network thread would self-join and deadlock, so the safe pattern there remains "close the socket and let paho's own loop detect it and auto-reconnect on the same client". Theoretical queue replay is still possible on those paths but #1136 specifically traced through the dispatch path, and the legacy socket-close has been battle-tested for the zombie paths since #887. Routing decision is made via `asyncio.get_running_loop()` — paho's callback thread has no loop, every legitimate hard-reset caller does. 7 regression tests across two new test classes: `TestForceReconnectRouting` (3 tests pinning the sync-context → socket-close fallback, async-context → hard-reset path with mock-stubbed `connect()`, and the state-disconnected broadcast firing once on either path) and `TestHardResetClientDirect` (3 tests pinning the helper directly: old client receives `disconnect()` + `loop_stop()`, `_client` reference cleared, failing `disconnect()` doesn't propagate so the await chain in `background_dispatch.py` doesn't break). Existing `TestZombieSessionDetection::test_two_timeouts_force_reconnect` and `TestDeveloperModeProbeTimeout::test_second_timeout_forces_reconnect` updated to assert the socket-close path (matching their paho-thread context), preserving the legacy contract. All 2179 backend unit tests pass. Thanks to @smandon for the precise reproduction logs that made this diagnosable from a single support package.
@@ -77,6 +85,9 @@ All notable changes to Bambuddy will be documented in this file.
- **Queue: active-item progress bar flashed 100% before dropping to 0%** — immediately after a queue item was dispatched, the per-item progress bar on the Queue page showed 100% (or whatever the prior print's final `mc_percent` was) for the few seconds between dispatch and the printer's MQTT state transitioning to `RUNNING`. Frontend `QueuePage.tsx` read `status.progress` directly from the printer's live MQTT snapshot, which carries over the last reported value from the previous print until the new one starts ticking. The progress bar, remaining time, ETA, and layer counter are now gated on `status.state` being `RUNNING` or `PAUSE`; in any other state (including `FINISH` from the prior print, `IDLE`, or `PREPARE` while heating) the bar renders at 0% with no stale ETA/layer values.
- **"Open in Slicer" fails on Windows / Linux for any filename containing spaces or special characters** ([#1059](https://github.com/maziggy/bambuddy/issues/1059)) — clicking "Open in Slicer" from the File Manager or Archives page produced one of three symptoms depending on the file: `.3mf` files opened Bambu Studio / OrcaSlicer but the app showed "Importing to Bambu Studio failed. Please download the file and open it manually" (the file on disk was 0 bytes); `.stl` files greyed the button out; `.step` couldn't be previewed at all. The protocol-handler URL emitted by `frontend/src/utils/slicer.ts` for OrcaSlicer (`orcaslicer://open?file=<URL>`) and Windows/Linux Bambu Studio (`bambustudio://open?file=<URL>`) was built by plain string concatenation with no `encodeURIComponent()` — the macOS `bambustudioopen://<URL>` branch was already encoding correctly, which is why macOS users didn't see this. A stale comment block in the file claimed the browser preserves the URL in the query string so no encoding is needed; that's true for the browser-to-OS handoff but ignores that the slicer itself calls `url_decode()` on the received query (BS `post_init()` calls `url_decode` then `split_str`; OrcaSlicer's Downloader regex-extracts then `url_decode`). Any already-percent-encoded character in the download URL — most commonly `%20` from filenames with spaces, which Bambuddy's archive paths produce naturally — decoded to a literal space and the slicer's subsequent HTTP GET came back 0 bytes or 404. All three URL forms now `encodeURIComponent()` the file URL, so the slicer sees the correctly-encoded URL after its own `url_decode`. The comment block is corrected to document the actual invariant. Regression test in `slicer.test.ts` feeds the exact issue reproduction URL (`Toothpick%20Launcher%20Print-in-Place.3mf`) and asserts `%2520` appears in the generated `orcaslicer://` href — so any future refactor that drops the encoding fails CI. Thanks to @jsapede for the double-encoding diagnosis and @AllanonBrooks and @lunaticds for the original reports.
### Security
- **postcss bumped to 8.5.12 to clear GHSA-qx2v-qp2m-jg93** — moderate-severity advisory: PostCSS < 8.5.10 has an XSS via an unescaped `</style>` sequence in its CSS Stringify output. The caret range in `frontend/package.json` already accepted 8.5.12, so this is a lockfile-only bump; vite, autoprefixer, and `@tailwindcss/postcss` all dedupe onto the same 8.5.12 with no nested copies left in `node_modules`. PostCSS runs at build time only and Bambuddy doesn't pass user-controlled CSS through it at runtime, so the practical impact even on the older version was nil — this is hygiene + clearing the `npm audit` warning.
## [0.2.3.2] - 2020-04-22
+174 -13
View File
@@ -2,6 +2,7 @@
import base64
import binascii
import contextlib
import hashlib
import logging
import os
@@ -186,6 +187,106 @@ def _stored_file_path(abs_path: Path, is_external: bool) -> str:
return str(abs_path) if is_external else to_relative_path(abs_path)
class _MoveSkip(Exception):
"""Signalled by ``_move_file_bytes`` to skip a file with a user-visible reason.
Carries an optional `code` for machine-friendly grouping (the
front-end can localise it) and a fallback English `reason` for logs.
"""
def __init__(self, code: str, reason: str):
super().__init__(reason)
self.code = code
self.reason = reason
def _resolve_source_disk_path(file: LibraryFile) -> Path | None:
"""Return the absolute on-disk path for an existing LibraryFile, or None
if it can't be located (legacy DB row, deleted file, etc.)."""
if file.is_external:
return Path(file.file_path) if file.file_path else None
return to_absolute_path(file.file_path)
def _move_file_bytes(file: LibraryFile, target_folder: LibraryFolder | None) -> str:
"""Physically relocate `file`'s bytes to match `target_folder`.
Used by the move endpoint when source/target straddle the
managed↔external boundary (#1112 follow-up — the prior implementation
updated the DB row's ``folder_id`` but never moved the bytes, so a
file moved to an external SMB folder showed up in Bambuddy's UI but
not on the NAS).
Returns the new ``file_path`` value to persist (relative for managed
targets, absolute for external targets — matches the upload + scan
paths). Raises ``_MoveSkip`` for any condition that would make the
move unsafe (target unwritable, filename collision, source missing).
The copy-then-unlink ordering means a partial copy followed by a
failed unlink leaves both the source and the dest on disk — better
than the symmetric "rename or move" which would lose the source if
the target write didn't complete on a flaky mount. The DB row stays
pointed at the source until the caller commits the new ``file_path``.
"""
src = _resolve_source_disk_path(file)
if not src or not src.exists():
raise _MoveSkip("source_missing", "source file missing on disk")
target_is_external = target_folder is not None and target_folder.is_external
if target_is_external:
if target_folder.external_readonly:
# Already blocked at top level, but defence-in-depth.
raise _MoveSkip("target_readonly", "target external folder is read-only")
if not target_folder.external_path:
raise _MoveSkip("target_misconfigured", "target external folder has no path")
ext_dir = Path(target_folder.external_path)
if not ext_dir.exists() or not ext_dir.is_dir():
raise _MoveSkip("target_inaccessible", f"target path not accessible: {ext_dir}")
if not os.access(ext_dir, os.W_OK):
raise _MoveSkip("target_unwritable", f"target path not writable: {ext_dir}")
dest = (ext_dir / file.filename).resolve()
try:
dest.relative_to(ext_dir.resolve())
except ValueError:
raise _MoveSkip("invalid_filename", f"unsafe filename: {file.filename!r}") from None
if dest.exists():
raise _MoveSkip("name_collision", f"a file named {file.filename!r} already exists in target")
try:
shutil.copy2(src, dest)
except OSError as e:
# Clean up partial dest so a retry can succeed.
with contextlib.suppress(OSError):
dest.unlink(missing_ok=True)
raise _MoveSkip("copy_failed", f"copy failed: {e}") from e
else:
# → managed (root or non-external folder): generate a fresh UUID
# filename in the internal store so we don't collide with another
# file that happens to share `filename`.
ext = src.suffix.lower()
dest = get_library_files_dir() / f"{uuid.uuid4().hex}{ext}"
try:
shutil.copy2(src, dest)
except OSError as e:
with contextlib.suppress(OSError):
dest.unlink(missing_ok=True)
raise _MoveSkip("copy_failed", f"copy failed: {e}") from e
# Copy succeeded — unlink the original. A failure here leaves an
# orphan on disk but the DB row is consistent against the new dest.
try:
src.unlink(missing_ok=True)
except OSError as e:
logger.warning(
"Move: copied %s → %s but couldn't remove source: %s",
src,
dest,
e,
)
return _stored_file_path(dest, is_external=target_is_external)
def _clean_3mf_metadata(obj):
"""Strip bytes and thumbnail-carrier keys so the payload is JSON-storable.
@@ -3265,11 +3366,20 @@ async def move_files(
):
"""Move multiple files to a folder.
Files not owned by the user are skipped (unless user has *_all permission).
Cross-boundary moves (managed ↔ external, or external ↔ external)
physically relocate the bytes — see ``_move_file_bytes``. Same-boundary
moves stay DB-only because the file's on-disk location doesn't depend
on which managed folder owns it.
Files not owned by the user are skipped (unless user has ``*_all``
permission). Each skip carries a structured reason so the UI can
surface "5 of 10 files were skipped: 3 had filename collisions on
the NAS, 2 are no longer on disk" rather than a blank "skipped: 5".
"""
user, can_modify_all = auth_result
# Verify folder exists if specified
target_folder: LibraryFolder | None = None
if data.folder_id is not None:
folder_result = await db.execute(select(LibraryFolder).where(LibraryFolder.id == data.folder_id))
target_folder = folder_result.scalar_one_or_none()
@@ -3278,25 +3388,76 @@ async def move_files(
if target_folder.is_external and target_folder.external_readonly:
raise HTTPException(status_code=403, detail="Cannot move files to a read-only external folder")
# Update files
target_is_external = target_folder is not None and target_folder.is_external
moved = 0
skipped = 0
skipped_reasons: list[dict] = []
for file_id in data.file_ids:
result = await db.execute(LibraryFile.active().where(LibraryFile.id == file_id))
result = await db.execute(
LibraryFile.active().options(selectinload(LibraryFile.folder)).where(LibraryFile.id == file_id)
)
file = result.scalar_one_or_none()
if file:
# Ownership check
if not can_modify_all and file.created_by_id != user.id:
skipped += 1
continue
# Cannot move external files out of their folder
if file.is_external:
skipped += 1
continue
if not file:
continue
# Ownership check
if not can_modify_all and file.created_by_id != user.id:
skipped += 1
skipped_reasons.append({"file_id": file_id, "code": "not_owner", "reason": "not the file owner"})
continue
# No bytes need to move when both ends are managed (same-boundary).
if not file.is_external and not target_is_external:
file.folder_id = data.folder_id
moved += 1
continue
return {"status": "success", "moved": moved, "skipped": skipped}
# Block moves out of a read-only external mount. The user only has
# read access to the source, and a move is semantically a delete on
# the source — which a read-only mount can't fulfil. Without this
# guard we'd succeed at copying to the target, fail to unlink the
# source, and the same file would now exist in two places (with
# the DB pointing at only one).
if file.is_external and file.folder is not None and file.folder.external_readonly:
skipped += 1
skipped_reasons.append(
{"file_id": file_id, "code": "source_readonly", "reason": "source is on a read-only external folder"}
)
continue
# Otherwise relocate the bytes, then update the DB row to match.
try:
new_file_path = _move_file_bytes(file, target_folder)
except _MoveSkip as e:
skipped += 1
skipped_reasons.append({"file_id": file_id, "code": e.code, "reason": e.reason})
continue
file.is_external = target_is_external
file.folder_id = data.folder_id
file.file_path = new_file_path
# External rows historically carry `file_hash=None` (scan skips
# hashing). When pulling an external file into managed storage,
# compute the hash so dedup detection works for future uploads
# of the same content.
if not target_is_external and file.file_hash is None:
try:
abs_path = to_absolute_path(new_file_path)
if abs_path:
file.file_hash = calculate_file_hash(abs_path)
except OSError:
pass # leave hash null; dedup just won't match this row
moved += 1
await db.commit()
return {
"status": "success",
"moved": moved,
"skipped": skipped,
"skipped_reasons": skipped_reasons,
}
@router.post("/bulk-delete", response_model=BulkDeleteResponse)
+73
View File
@@ -0,0 +1,73 @@
"""Asyncio event-loop exception handlers used at app startup.
Currently houses a single Windows-specific filter for the noisy
``_ProactorBasePipeTransport._call_connection_lost`` ``WinError 10054``
that fires every time a printer / MQTT broker / camera RSTs a TCP socket
instead of closing it cleanly. See ``install_proactor_reset_filter`` for
the why and the failure mode it suppresses.
"""
from __future__ import annotations
import asyncio
import logging
import sys
from typing import Any
logger = logging.getLogger(__name__)
def _is_proactor_connection_reset(context: dict[str, Any]) -> bool:
"""True if `context` describes the Windows Proactor cleanup-RST noise.
asyncio's default exception handler is invoked in two distinct cases
we care about — generic uncaught task exceptions, and the specific
`_call_connection_lost` cleanup path — and we only want to suppress
the latter. Match on three signals together so a real
`ConnectionResetError` raised inside an application task still
surfaces normally:
1. The exception is `ConnectionResetError` (or a subclass).
2. asyncio's own message string mentions `_call_connection_lost`
(the Proactor-cleanup callback is the only place Python emits
this exact phrase).
3. We're actually on Windows, where the Proactor is in use.
"""
if sys.platform != "win32":
return False
exc = context.get("exception")
if not isinstance(exc, ConnectionResetError):
return False
message = context.get("message", "")
return "_call_connection_lost" in message
def _proactor_reset_filter(loop: asyncio.AbstractEventLoop, context: dict[str, Any]) -> None:
"""Custom event-loop exception handler.
Handles the Proactor-cleanup `ConnectionResetError` by logging it at
DEBUG instead of ERROR, and delegates everything else to asyncio's
default handler so unrelated bugs are still visible.
"""
if _is_proactor_connection_reset(context):
logger.debug(
"asyncio Proactor: peer reset socket during cleanup (WinError 10054); "
"ignored — application-layer reconnect handles the disconnect"
)
return
loop.default_exception_handler(context)
def install_proactor_reset_filter(loop: asyncio.AbstractEventLoop | None = None) -> bool:
"""Install the filter on `loop` (or the running loop if omitted).
Returns True when the filter was installed (Windows only), False on
every other platform — so callers can branch on the return value if
they want to log the install / skip.
"""
if sys.platform != "win32":
return False
if loop is None:
loop = asyncio.get_running_loop()
loop.set_exception_handler(_proactor_reset_filter)
return True
+17 -3
View File
@@ -141,11 +141,25 @@ async def get_db() -> AsyncSession:
try:
yield session
await session.commit()
except Exception:
await session.rollback()
except BaseException:
# Catch BaseException (not just Exception) so CancelledError —
# raised when Starlette's BaseHTTPMiddleware cancels the inner
# task scope on client disconnect — also triggers rollback.
# `asyncio.shield` keeps the rollback running to completion
# even when the await itself gets cancelled, so the SQLite
# write lock is released promptly instead of being held until
# the connection is GC'd ages later (which was producing the
# "database is locked" cascade in #1112's support package).
try:
await asyncio.shield(session.rollback())
except BaseException: # noqa: BLE001 — rollback failure must not mask the original
pass
raise
finally:
await session.close()
try:
await asyncio.shield(session.close())
except BaseException: # noqa: BLE001 — close failure must not mask the original
pass
async def init_db():
+67 -4
View File
@@ -1,13 +1,16 @@
"""Logging filters for the Bambuddy log pipeline.
Currently houses a single filter that keeps only state-changing HTTP methods
in the file-side uvicorn access log. See ``WriteRequestsOnlyFilter`` for the
why; this lives in its own module so the test suite can import it without
pulling in ``backend.app.main``'s entire startup graph.
Holds two filters: ``WriteRequestsOnlyFilter`` keeps the file-side
uvicorn access log focused on state-changing HTTP methods, and
``CancelledPoolNoiseFilter`` drops SQLAlchemy connection-pool log noise
caused by Starlette's ``BaseHTTPMiddleware`` cancellation propagation
(see the filter's docstring for details). Both live here so tests can
import them without pulling in ``backend.app.main``'s startup graph.
"""
from __future__ import annotations
import asyncio
import logging
@@ -44,3 +47,63 @@ class WriteRequestsOnlyFilter(logging.Filter):
def filter(self, record: logging.LogRecord) -> bool: # noqa: A003 — stdlib API name
message = record.getMessage()
return any(token in message for token in self._WRITE_VERB_TOKENS)
class CancelledPoolNoiseFilter(logging.Filter):
"""Drop SQLAlchemy connection-pool log records driven by request cancellation.
Starlette's ``BaseHTTPMiddleware`` (used under the hood by FastAPI's
``@app.middleware("http")`` decorator) cancels the inner task scope when a
client disconnects mid-request. The cancellation propagates into
SQLAlchemy's connection-pool cleanup and surfaces as two distinct ERROR
records — both expected on disconnect, neither actionable for the user:
1. ``Exception terminating connection ... CancelledError`` — fires every
time ``do_terminate`` is interrupted by the same cancel scope that's
unwinding the request. The ``CancelledError`` traceback always
attributes the cancel to ``BaseHTTPMiddleware.call_next``.
2. ``The garbage collector is trying to clean up non-checked-in
connection`` — fires later when the GC reclaims the session that
couldn't return its connection to the pool because of (1). It's
symptomatic of the cancellation, not a separate bug.
These pile up under heavy upload load (long multipart uploads where the
client times out before the server's response). Real connection-pool
issues — pool exhaustion, broken connections from network hiccups, etc.
— surface through DIFFERENT messages and a non-cancellation
``exc_info`` chain, so they keep flowing through this filter unchanged.
Attach to ``logging.getLogger("sqlalchemy.pool")`` (and only there).
"""
_GC_CLEANUP_PREFIX = "The garbage collector is trying to clean up non-checked-in connection"
_TERMINATE_PREFIX = "Exception terminating connection"
@staticmethod
def _has_cancelled_in_chain(exc: BaseException | None) -> bool:
"""True if `exc` is `CancelledError` or has one in its cause chain."""
seen: set[int] = set()
cur: BaseException | None = exc
while cur is not None and id(cur) not in seen:
seen.add(id(cur))
if isinstance(cur, asyncio.CancelledError):
return True
cur = cur.__cause__ or cur.__context__
return False
def filter(self, record: logging.LogRecord) -> bool: # noqa: A003 — stdlib API name
message = record.getMessage()
# GC-cleanup records have no exc_info — match by prefix only. Always
# symptomatic of the cancellation cascade, never independently useful.
if message.startswith(self._GC_CLEANUP_PREFIX):
return False
# Terminate-connection records carry a traceback; only drop those
# that are cancellation-driven. A real terminate failure (broken
# connection, network hiccup) keeps a non-CancelledError exc_info
# chain and surfaces normally.
if message.startswith(self._TERMINATE_PREFIX) and record.exc_info:
exc = record.exc_info[1]
if self._has_cancelled_in_chain(exc):
return False
return True
+17 -1
View File
@@ -289,7 +289,10 @@ if app_settings.log_to_file:
# for exactly this reason. Filtered to write methods only
# (POST/PUT/PATCH/DELETE) so the high-volume status-poll GETs from the
# frontend don't churn the rotation window faster than it's useful.
from backend.app.core.logging_filters import WriteRequestsOnlyFilter
from backend.app.core.logging_filters import (
CancelledPoolNoiseFilter,
WriteRequestsOnlyFilter,
)
uvicorn_access_logger = logging.getLogger("uvicorn.access")
uvicorn_access_logger.addHandler(file_handler)
@@ -300,6 +303,13 @@ if app_settings.log_to_file:
# ID column as the application logs they correlate with.
uvicorn_access_logger.addFilter(TraceIDFilter())
# Drop SQLAlchemy connection-pool log noise that's caused by Starlette's
# BaseHTTPMiddleware cancelling the inner task scope on client
# disconnect (#1112). The cancel-safe `get_db` already prevents the
# underlying transaction leak; this filter only suppresses the residual
# log records that pre-existing pools still emit during their cleanup.
logging.getLogger("sqlalchemy.pool").addFilter(CancelledPoolNoiseFilter())
# Reduce noise from third-party libraries in production
if not app_settings.debug:
logging.getLogger("sqlalchemy.engine").setLevel(logging.WARNING)
@@ -4161,6 +4171,12 @@ def stop_auth_cleanup() -> None:
@asynccontextmanager
async def lifespan(app: FastAPI):
# Startup
# Install Windows-only asyncio Proactor cleanup-RST filter (#1113) before
# anything else can spawn tasks that might trip it.
from backend.app.core.asyncio_handlers import install_proactor_reset_filter
install_proactor_reset_filter()
await init_db()
# Register an app-scoped httpx client for Bambu Cloud services so
+99 -3
View File
@@ -6,6 +6,7 @@ accurate partial usage reporting for multi-material prints.
"""
import json
import logging
import math
import re
import zipfile
@@ -13,6 +14,8 @@ from pathlib import Path
import defusedxml.ElementTree as ET
logger = logging.getLogger(__name__)
# Default filament properties
DEFAULT_FILAMENT_DIAMETER = 1.75 # mm
DEFAULT_FILAMENT_DENSITY = 1.24 # g/cm³ (PLA)
@@ -442,6 +445,90 @@ def extract_filament_usage_from_3mf(file_path: Path, plate_id: int | None = None
return filament_usage
# Header values exposed as `{placeholder}` substitutions inside snippets.
# Aliases let users write Prusa-style names (`{max_layer_z}`) that map onto
# Bambu/Orca header keys (`max_z_height`).
_HEADER_PLACEHOLDER_ALIASES = {
"max_layer_z": "max_z_height",
"max_print_height": "max_z_height",
"total_layers": "total_layer_number",
}
_HEADER_KEY_RE = re.compile(r"^;\s*([^:]+?)\s*:\s*(.+?)\s*$")
_PLACEHOLDER_RE = re.compile(r"\{([a-zA-Z_][a-zA-Z0-9_]*)\}")
_START_GCODE_END_MARKER = "; MACHINE_START_GCODE_END"
def _parse_3mf_gcode_header(content: str) -> dict[str, str]:
"""Parse the `; HEADER_BLOCK_START..END` block into a normalised dict.
Keys are lowercased, ` [units]` suffixes stripped, and spaces converted
to underscores so callers can look up `total_layer_number` regardless of
whether the source line is `; total layer number: 80` or
`; total filament length [mm] : 12155.34`.
"""
header: dict[str, str] = {}
in_header = False
for raw_line in content.splitlines():
line = raw_line.strip()
if line == "; HEADER_BLOCK_START":
in_header = True
continue
if line == "; HEADER_BLOCK_END":
break
if not in_header:
continue
m = _HEADER_KEY_RE.match(line)
if not m:
continue
key, value = m.group(1), m.group(2)
key = re.sub(r"\s*\[[^\]]*\]\s*$", "", key)
key = key.strip().lower().replace(" ", "_")
header[key] = value
return header
def _substitute_placeholders(snippet: str, header: dict[str, str]) -> str:
"""Replace `{var}` placeholders with header values, leaving unknowns intact."""
def repl(m: re.Match) -> str:
name = m.group(1)
value = header.get(name)
if value is None:
alias = _HEADER_PLACEHOLDER_ALIASES.get(name)
if alias is not None:
value = header.get(alias)
if value is None:
logger.warning(
"G-code injection: placeholder {%s} not found in 3MF header; leaving as-is",
name,
)
return m.group(0)
return value
return _PLACEHOLDER_RE.sub(repl, snippet)
def _inject_start_at_marker(content: str, snippet: str) -> str:
"""Insert snippet immediately before `; MACHINE_START_GCODE_END`.
The marker sits at the bottom of the printer's startup block — bed heat,
homing, and nozzle prime are already done, so injected snippets land in
the same place a slicer-side custom-start-gcode would. Falls back to
prepending if the marker isn't present (older files / non-Bambu slicers).
"""
marker_idx = content.find(_START_GCODE_END_MARKER)
if marker_idx == -1:
logger.warning(
"G-code injection: '%s' not found, prepending start snippet to whole file",
_START_GCODE_END_MARKER,
)
return snippet.rstrip("\n") + "\n" + content
line_start = content.rfind("\n", 0, marker_idx)
line_start = 0 if line_start == -1 else line_start + 1
return content[:line_start] + snippet.rstrip("\n") + "\n" + content[line_start:]
def inject_gcode_into_3mf(
source_path: Path,
plate_id: int,
@@ -450,10 +537,16 @@ def inject_gcode_into_3mf(
):
"""Create a temp copy of a 3MF with G-code injected at start/end.
Snippets support `{placeholder}` substitution against values parsed from
the 3MF G-code header block (e.g. `{max_layer_z}` → `16.00`). Start
snippets are anchored to the `; MACHINE_START_GCODE_END` marker so they
run after the printer's own startup (#422). End snippets are appended
after the last line of the print.
Args:
source_path: Path to the original 3MF file.
plate_id: Plate number (1-indexed) to inject into.
start_gcode: G-code to prepend, or None.
start_gcode: G-code to insert after printer startup, or None.
end_gcode: G-code to append, or None.
Returns:
@@ -486,11 +579,14 @@ def inject_gcode_into_3mf(
# Read and modify gcode content
gcode_content = zf.read(target_gcode).decode("utf-8", errors="ignore")
header = _parse_3mf_gcode_header(gcode_content)
if start_gcode:
gcode_content = start_gcode + "\n" + gcode_content
resolved = _substitute_placeholders(start_gcode, header)
gcode_content = _inject_start_at_marker(gcode_content, resolved)
if end_gcode:
gcode_content = gcode_content.rstrip("\n") + "\n" + end_gcode + "\n"
resolved = _substitute_placeholders(end_gcode, header)
gcode_content = gcode_content.rstrip("\n") + "\n" + resolved + "\n"
# Write modified 3MF to temp file
with tempfile.NamedTemporaryFile(delete=False, suffix=".3mf") as tmp:
@@ -710,3 +710,249 @@ class TestExternalFolderWritableUpload:
assert row.is_external is False
# Internal storage: file_path is UUID-scoped, stored as a relative path.
assert not row.file_path.startswith("/")
class TestCrossBoundaryMove:
"""#1112 follow-up: moving files between managed and external folders
must physically relocate the bytes, not just shuffle the DB ``folder_id``.
Pre-fix symptom (reported by @Carter3DP after testing 0.2.4b1): a file
moved from a managed folder to a NAS-backed external folder showed up
in Bambuddy's UI under the external folder but was never written to
the NAS — so the SMB mount and Bambuddy disagreed about what was
actually there.
"""
@pytest.fixture
def external_dir(self, tmp_path):
ext_dir = tmp_path / "writable_share"
ext_dir.mkdir()
return ext_dir
@pytest.fixture
async def writable_folder(self, async_client, db_session, external_dir):
data = {"name": "Writable NAS", "external_path": str(external_dir), "readonly": False}
response = await async_client.post("/api/v1/library/folders/external", json=data)
assert response.status_code == 200
return response.json()
@pytest.fixture
async def readonly_folder(self, async_client, db_session, tmp_path):
ro_dir = tmp_path / "ro_share"
ro_dir.mkdir()
(ro_dir / "stranded.gcode").write_text("G28")
data = {"name": "Read-only NAS", "external_path": str(ro_dir), "readonly": True}
response = await async_client.post("/api/v1/library/folders/external", json=data)
assert response.status_code == 200
# Populate via scan so the file gets a DB row with is_external=True.
scan = await async_client.post(f"/api/v1/library/folders/{response.json()['id']}/scan")
assert scan.status_code == 200
return response.json()
@pytest.mark.asyncio
@pytest.mark.integration
async def test_managed_to_external_relocates_bytes(
self, async_client: AsyncClient, db_session, writable_folder, external_dir
):
"""The actual #1112 fix: managed → external must write the bytes
to the NAS mount AND drop them from internal storage. Pre-fix the
DB row flipped to the new folder but the bytes stayed put."""
import io
from backend.app.api.routes.library import to_absolute_path
from backend.app.models.library import LibraryFile
upload = await async_client.post(
"/api/v1/library/files",
files={"file": ("ship_me.stl", io.BytesIO(b"original-bytes"), "application/octet-stream")},
)
assert upload.status_code == 200
file_id = upload.json()["id"]
# Snapshot the pre-move on-disk path so we can verify it's gone after.
pre = await db_session.get(LibraryFile, file_id)
await db_session.refresh(pre)
managed_disk_path = to_absolute_path(pre.file_path)
assert managed_disk_path is not None and managed_disk_path.exists()
response = await async_client.post(
"/api/v1/library/files/move",
json={"file_ids": [file_id], "folder_id": writable_folder["id"]},
)
assert response.status_code == 200, response.text
body = response.json()
assert body["moved"] == 1
assert body["skipped"] == 0
# Bytes are on the NAS mount.
on_nas = external_dir / "ship_me.stl"
assert on_nas.exists()
assert on_nas.read_bytes() == b"original-bytes"
# Internal copy is gone.
assert not managed_disk_path.exists(), "managed source must be removed after the move"
# DB row matches reality.
await db_session.refresh(pre)
assert pre.is_external is True
assert pre.folder_id == writable_folder["id"]
assert pre.file_path == str(on_nas.resolve())
@pytest.mark.asyncio
@pytest.mark.integration
async def test_external_to_managed_relocates_bytes(
self, async_client: AsyncClient, db_session, writable_folder, external_dir
):
"""Symmetric direction: external → managed copies the bytes into
internal storage with a UUID name, deletes the source on the
mount, and recomputes the file hash (since scan stores
``file_hash=None`` for external rows)."""
import io
from backend.app.models.library import LibraryFile
# Plant a file on the writable mount and let upload give it a row.
upload = await async_client.post(
f"/api/v1/library/files?folder_id={writable_folder['id']}",
files={"file": ("relocate_me.stl", io.BytesIO(b"nas-bytes"), "application/octet-stream")},
)
assert upload.status_code == 200
file_id = upload.json()["id"]
ext_disk = external_dir / "relocate_me.stl"
assert ext_disk.exists()
response = await async_client.post(
"/api/v1/library/files/move",
json={"file_ids": [file_id], "folder_id": None},
)
assert response.status_code == 200
assert response.json()["moved"] == 1
db_session.expire_all()
row = await db_session.get(LibraryFile, file_id)
assert row.is_external is False
assert row.folder_id is None
assert not row.file_path.startswith("/"), "managed file_path must be relative"
assert not ext_disk.exists(), "external source must be removed after the move"
# Hash filled in for the now-managed row so future dedup works.
assert row.file_hash is not None and len(row.file_hash) == 64
@pytest.mark.asyncio
@pytest.mark.integration
async def test_managed_to_external_collision_skips_with_reason(
self, async_client: AsyncClient, db_session, writable_folder, external_dir
):
"""A name collision on the target external mount must skip the
move with a structured reason — not silently overwrite a file
that's already on the NAS."""
import io
# Pre-existing file on the mount with the same name as the upload.
(external_dir / "duplicate.stl").write_bytes(b"pre-existing")
upload = await async_client.post(
"/api/v1/library/files",
files={"file": ("duplicate.stl", io.BytesIO(b"new-bytes"), "application/octet-stream")},
)
assert upload.status_code == 200
file_id = upload.json()["id"]
response = await async_client.post(
"/api/v1/library/files/move",
json={"file_ids": [file_id], "folder_id": writable_folder["id"]},
)
assert response.status_code == 200
body = response.json()
assert body["moved"] == 0
assert body["skipped"] == 1
reasons = body["skipped_reasons"]
assert len(reasons) == 1
assert reasons[0]["file_id"] == file_id
assert reasons[0]["code"] == "name_collision"
# Pre-existing target file is intact.
assert (external_dir / "duplicate.stl").read_bytes() == b"pre-existing"
@pytest.mark.asyncio
@pytest.mark.integration
async def test_external_readonly_source_skips(self, async_client: AsyncClient, db_session, readonly_folder):
"""A read-only mount allows reading but not deletes, and a move
is semantically a delete on the source. Skip with
``source_readonly`` so the file isn't duplicated by half-moving."""
listing = await async_client.get(f"/api/v1/library/files?folder_id={readonly_folder['id']}")
assert listing.status_code == 200
ext_file_id = listing.json()[0]["id"]
response = await async_client.post(
"/api/v1/library/files/move",
json={"file_ids": [ext_file_id], "folder_id": None},
)
assert response.status_code == 200
body = response.json()
assert body["moved"] == 0
assert body["skipped"] == 1
assert body["skipped_reasons"][0]["code"] == "source_readonly"
@pytest.mark.asyncio
@pytest.mark.integration
async def test_managed_to_managed_remains_db_only(self, async_client: AsyncClient, db_session):
"""Same-boundary moves (managed → managed) keep the existing
DB-only fast path — no shutil.copy, no UUID rename. The original
file_path stays the same, only ``folder_id`` changes."""
import io
from backend.app.models.library import LibraryFile
sub = await async_client.post(
"/api/v1/library/folders",
json={"name": "subfolder", "parent_id": None},
)
assert sub.status_code == 200
target_id = sub.json()["id"]
upload = await async_client.post(
"/api/v1/library/files",
files={"file": ("part.stl", io.BytesIO(b"x"), "application/octet-stream")},
)
assert upload.status_code == 200
file_id = upload.json()["id"]
pre = await db_session.get(LibraryFile, file_id)
await db_session.refresh(pre)
original_path = pre.file_path
response = await async_client.post(
"/api/v1/library/files/move",
json={"file_ids": [file_id], "folder_id": target_id},
)
assert response.status_code == 200
assert response.json()["moved"] == 1
db_session.expire_all()
post = await db_session.get(LibraryFile, file_id)
assert post.folder_id == target_id
assert post.is_external is False
assert post.file_path == original_path # bytes never moved
@pytest.mark.asyncio
@pytest.mark.integration
async def test_skipped_reasons_field_present_even_when_empty(self, async_client: AsyncClient, db_session):
"""Backwards-compatible response shape: ``skipped_reasons`` is
always present (empty list when nothing skipped) so frontend
code can treat it as the source of truth without optional-chain
gymnastics."""
import io
upload = await async_client.post(
"/api/v1/library/files",
files={"file": ("trivial.stl", io.BytesIO(b"x"), "application/octet-stream")},
)
assert upload.status_code == 200
file_id = upload.json()["id"]
response = await async_client.post(
"/api/v1/library/files/move",
json={"file_ids": [file_id], "folder_id": None},
)
assert response.status_code == 200
body = response.json()
assert "skipped_reasons" in body
assert body["skipped_reasons"] == []
+127
View File
@@ -0,0 +1,127 @@
"""Tests for the Windows asyncio Proactor cleanup-RST filter (#1113)."""
from __future__ import annotations
import asyncio
from unittest.mock import patch
import pytest
from backend.app.core.asyncio_handlers import (
_is_proactor_connection_reset,
_proactor_reset_filter,
install_proactor_reset_filter,
)
# `_is_proactor_connection_reset` short-circuits on non-Windows; pretend we're
# on Windows for the discrimination tests so they exercise the actual logic.
@pytest.fixture
def fake_windows():
with patch("backend.app.core.asyncio_handlers.sys.platform", "win32"):
yield
class TestIsProactorConnectionReset:
"""The discriminator that decides whether a context is the noise we silence."""
def test_matches_proactor_cleanup_reset(self, fake_windows):
ctx = {
"exception": ConnectionResetError(10054, "An existing connection was forcibly closed"),
"message": "Exception in callback _ProactorBasePipeTransport._call_connection_lost()",
}
assert _is_proactor_connection_reset(ctx) is True
def test_rejects_when_not_on_windows(self):
# No `fake_windows` fixture — sys.platform reflects the real OS.
ctx = {
"exception": ConnectionResetError(10054, "irrelevant"),
"message": "Exception in callback _ProactorBasePipeTransport._call_connection_lost()",
}
# The whole point of the filter is to be a Windows-only no-op.
with patch("backend.app.core.asyncio_handlers.sys.platform", "linux"):
assert _is_proactor_connection_reset(ctx) is False
def test_rejects_unrelated_connection_reset(self, fake_windows):
"""A real `ConnectionResetError` raised inside an app coroutine —
not from the Proactor cleanup path — must NOT be suppressed.
Otherwise we'd hide genuine connectivity bugs."""
ctx = {
"exception": ConnectionResetError(),
"message": "Task exception was never retrieved",
}
assert _is_proactor_connection_reset(ctx) is False
def test_rejects_other_exception_types(self, fake_windows):
"""Other OSErrors (BrokenPipeError, ConnectionAbortedError) might
share the cleanup path but they're a different signal worth
keeping visible — we only silence the specific 10054 family."""
ctx = {
"exception": BrokenPipeError(),
"message": "Exception in callback _ProactorBasePipeTransport._call_connection_lost()",
}
assert _is_proactor_connection_reset(ctx) is False
def test_rejects_when_no_exception(self, fake_windows):
"""asyncio sometimes invokes the handler with no exception object
(e.g. resource warnings) — those shouldn't blanket-match."""
ctx = {"message": "_call_connection_lost was slow"}
assert _is_proactor_connection_reset(ctx) is False
class TestProactorResetFilter:
"""The handler glue itself — does it suppress the right ones and
pass everything else through to the default handler?"""
@pytest.mark.asyncio
async def test_suppresses_proactor_reset(self, fake_windows):
loop = asyncio.get_running_loop()
with patch.object(loop, "default_exception_handler") as default:
_proactor_reset_filter(
loop,
{
"exception": ConnectionResetError(10054, "forcibly closed"),
"message": "Exception in callback _ProactorBasePipeTransport._call_connection_lost()",
},
)
# Suppression = default handler is never reached.
default.assert_not_called()
@pytest.mark.asyncio
async def test_passes_unrelated_through_to_default(self, fake_windows):
"""A different uncaught exception must go through asyncio's normal
path so it surfaces in logs and tests as an actual problem."""
loop = asyncio.get_running_loop()
ctx = {
"exception": ValueError("real bug"),
"message": "Task exception was never retrieved",
}
with patch.object(loop, "default_exception_handler") as default:
_proactor_reset_filter(loop, ctx)
default.assert_called_once_with(ctx)
class TestInstallation:
"""Wiring: install_proactor_reset_filter only runs on Windows."""
@pytest.mark.asyncio
async def test_install_is_no_op_on_non_windows(self):
"""Linux/macOS use the Selector loop, which doesn't hit this code
path — the install must be inert so the Linux production path
keeps the default exception handler untouched."""
loop = asyncio.get_running_loop()
with (
patch("backend.app.core.asyncio_handlers.sys.platform", "linux"),
patch.object(loop, "set_exception_handler") as setter,
):
installed = install_proactor_reset_filter(loop)
assert installed is False
setter.assert_not_called()
@pytest.mark.asyncio
async def test_install_attaches_handler_on_windows(self, fake_windows):
loop = asyncio.get_running_loop()
with patch.object(loop, "set_exception_handler") as setter:
installed = install_proactor_reset_filter(loop)
assert installed is True
setter.assert_called_once_with(_proactor_reset_filter)
@@ -0,0 +1,76 @@
"""Tests for the SQLAlchemy connection-pool cancellation noise filter (#1112)."""
from __future__ import annotations
import asyncio
import logging
from backend.app.core.logging_filters import CancelledPoolNoiseFilter
def _make_record(message: str, *, exc: BaseException | None = None) -> logging.LogRecord:
"""Build a `LogRecord` carrying `message` (no positional args) and
optionally an `exc_info` tuple holding `exc`."""
record = logging.LogRecord(
name="sqlalchemy.pool.impl.AsyncAdaptedQueuePool",
level=logging.ERROR,
pathname=__file__,
lineno=0,
msg=message,
args=(),
exc_info=(type(exc), exc, exc.__traceback__) if exc is not None else None,
)
return record
class TestCancelledPoolNoiseFilter:
"""Drops the cancellation cascade, keeps real pool errors visible."""
def test_drops_terminate_with_cancelled_exc(self):
cancel = asyncio.CancelledError("Cancelled via cancel scope")
record = _make_record("Exception terminating connection <ABC>", exc=cancel)
assert CancelledPoolNoiseFilter().filter(record) is False
def test_drops_gc_cleanup_record(self):
# GC cleanup messages have no exc_info attached — match by prefix.
record = _make_record("The garbage collector is trying to clean up non-checked-in connection <ABC>")
assert CancelledPoolNoiseFilter().filter(record) is False
def test_keeps_terminate_with_real_oserror(self):
"""A genuine connection-terminate failure (network hiccup, broken
socket) carries a non-cancellation exc_info chain. That's a real
problem the user should see — must NOT be dropped."""
oserr = OSError("broken pipe")
record = _make_record("Exception terminating connection <ABC>", exc=oserr)
assert CancelledPoolNoiseFilter().filter(record) is True
def test_keeps_terminate_without_exc_info(self):
"""If for any reason `exc_info` is missing on a terminate record,
keep it — only filter when we have positive evidence it's the
cancellation cascade."""
record = _make_record("Exception terminating connection <ABC>")
assert CancelledPoolNoiseFilter().filter(record) is True
def test_keeps_unrelated_pool_message(self):
"""Other pool messages (pool size warnings, etc.) keep flowing."""
record = _make_record("Pool size has been exceeded; will spawn overflow")
assert CancelledPoolNoiseFilter().filter(record) is True
def test_drops_when_cancelled_is_in_cause_chain(self):
"""Real-world traceback: SQLAlchemy wraps the CancelledError in a
chained exception. The filter walks `__cause__`/`__context__` so a
chained CancelledError still counts."""
cancel = asyncio.CancelledError()
wrapper = RuntimeError("terminate failed")
wrapper.__cause__ = cancel
record = _make_record("Exception terminating connection <ABC>", exc=wrapper)
assert CancelledPoolNoiseFilter().filter(record) is False
def test_handles_self_referential_cause_chain(self):
"""Defensive: malformed exception chains (rare but possible) must
not loop forever — the `seen` set guards against it."""
a = RuntimeError("a")
a.__cause__ = a # pathological
record = _make_record("Exception terminating connection <ABC>", exc=a)
# Doesn't loop, doesn't raise, returns True (no CancelledError found).
assert CancelledPoolNoiseFilter().filter(record) is True
+221 -3
View File
@@ -4,9 +4,12 @@ import tempfile
import zipfile
from pathlib import Path
import pytest
from backend.app.utils.threemf_tools import inject_gcode_into_3mf
from backend.app.utils.threemf_tools import (
_inject_start_at_marker,
_parse_3mf_gcode_header,
_substitute_placeholders,
inject_gcode_into_3mf,
)
def _make_temp_path(suffix=".3mf") -> Path:
@@ -205,3 +208,218 @@ class TestInjectGcodeInto3mf:
source.unlink(missing_ok=True)
if result:
result.unlink(missing_ok=True)
# Realistic Bambu / Orca header + startup block — the start-gcode marker is the
# anchor point #422 reviewers (DevScarabyte, pleite) reported as the correct
# injection point. Snippets injected before this should land *after* the bed
# heat / homing / nozzle prime sequence, not before it.
_BAMBU_GCODE_TEMPLATE = """\
; HEADER_BLOCK_START
; BambuStudio 02.06.00.51
; total layer number: 80
; total filament length [mm] : 12155.34
; total filament weight [g] : 36.55
; max_z_height: 16.00
; HEADER_BLOCK_END
; MACHINE_START_GCODE_BEGIN
M104 S220 ; preheat
G28 ; home
M109 S220 ; wait for nozzle
G92 E0 ; reset extruder
; MACHINE_START_GCODE_END
G1 X10 Y10 Z0.2
G1 X100 Y100 E5
M104 S0
"""
class TestStartAnchoredInjection:
"""Tests for #422 follow-up: start g-code injected at MACHINE_START_GCODE_END."""
def test_start_lands_after_printer_startup(self):
"""Start snippet sits immediately before MACHINE_START_GCODE_END, not at file head."""
source = _make_test_3mf(_BAMBU_GCODE_TEMPLATE)
try:
result = inject_gcode_into_3mf(source, 1, "; SWAPMOD-START", None)
assert result is not None
with zipfile.ZipFile(result, "r") as zf:
gcode = zf.read("Metadata/plate_1.gcode").decode("utf-8")
# Original file head is preserved — snippet does NOT prepend.
assert gcode.startswith("; HEADER_BLOCK_START\n")
# Snippet sits right above the marker.
marker_idx = gcode.index("; MACHINE_START_GCODE_END")
snippet_idx = gcode.index("; SWAPMOD-START")
assert snippet_idx < marker_idx
# Nothing else between snippet and marker except the trailing newline.
between = gcode[snippet_idx:marker_idx]
assert between == "; SWAPMOD-START\n"
# Printer's own startup commands still come BEFORE the snippet.
startup_idx = gcode.index("M109 S220")
assert startup_idx < snippet_idx
finally:
source.unlink(missing_ok=True)
if result:
result.unlink(missing_ok=True)
def test_no_marker_falls_back_to_prepend(self):
"""Files without MACHINE_START_GCODE_END (older slicers) keep prepend behaviour."""
source = _make_test_3mf("G28\nM400\n")
try:
result = inject_gcode_into_3mf(source, 1, "; LEGACY-START", None)
assert result is not None
with zipfile.ZipFile(result, "r") as zf:
gcode = zf.read("Metadata/plate_1.gcode").decode("utf-8")
assert gcode.startswith("; LEGACY-START\n")
assert "G28" in gcode
finally:
source.unlink(missing_ok=True)
if result:
result.unlink(missing_ok=True)
def test_end_still_appended_at_eof(self):
"""End g-code keeps the existing append-to-EOF behaviour even with marker present."""
source = _make_test_3mf(_BAMBU_GCODE_TEMPLATE)
try:
result = inject_gcode_into_3mf(source, 1, None, "; SWAPMOD-END")
assert result is not None
with zipfile.ZipFile(result, "r") as zf:
gcode = zf.read("Metadata/plate_1.gcode").decode("utf-8")
assert gcode.endswith("; SWAPMOD-END\n")
# Marker anchor is irrelevant for end snippets.
assert gcode.index("; SWAPMOD-END") > gcode.index("; MACHINE_START_GCODE_END")
finally:
source.unlink(missing_ok=True)
if result:
result.unlink(missing_ok=True)
class TestPlaceholderSubstitution:
"""Tests for #422 follow-up: {placeholder} substitution from 3MF header values."""
def test_max_z_height_substituted_in_end_snippet(self):
"""`G1 Z{max_layer_z}` resolves to the model's actual top-layer Z (DevScarabyte safety bug)."""
source = _make_test_3mf(_BAMBU_GCODE_TEMPLATE)
try:
# Prusa-style alias: max_layer_z → max_z_height in the Bambu header
result = inject_gcode_into_3mf(source, 1, None, "G1 Z{max_layer_z} F600")
assert result is not None
with zipfile.ZipFile(result, "r") as zf:
gcode = zf.read("Metadata/plate_1.gcode").decode("utf-8")
# max_z_height in the template is 16.00 — the dangerous Z1 fallback is gone.
assert "G1 Z16.00 F600" in gcode
assert "{max_layer_z}" not in gcode
finally:
source.unlink(missing_ok=True)
if result:
result.unlink(missing_ok=True)
def test_direct_header_key_lookup(self):
"""Snippets can reference normalised header keys directly without going through aliases."""
source = _make_test_3mf(_BAMBU_GCODE_TEMPLATE)
try:
result = inject_gcode_into_3mf(
source, 1, None, "; layers={total_layer_number} weight={total_filament_weight}"
)
assert result is not None
with zipfile.ZipFile(result, "r") as zf:
gcode = zf.read("Metadata/plate_1.gcode").decode("utf-8")
assert "; layers=80 weight=36.55" in gcode
finally:
source.unlink(missing_ok=True)
if result:
result.unlink(missing_ok=True)
def test_unknown_placeholder_left_intact(self):
"""A typo or unsupported placeholder is preserved verbatim instead of becoming empty."""
source = _make_test_3mf(_BAMBU_GCODE_TEMPLATE)
try:
result = inject_gcode_into_3mf(source, 1, None, "; nope={does_not_exist}")
assert result is not None
with zipfile.ZipFile(result, "r") as zf:
gcode = zf.read("Metadata/plate_1.gcode").decode("utf-8")
assert "; nope={does_not_exist}" in gcode
finally:
source.unlink(missing_ok=True)
if result:
result.unlink(missing_ok=True)
def test_no_placeholders_no_header_required(self):
"""Snippets without placeholders inject correctly even when the header is absent."""
source = _make_test_3mf("G28\nM400\n")
try:
result = inject_gcode_into_3mf(source, 1, "; PLAIN", None)
assert result is not None
with zipfile.ZipFile(result, "r") as zf:
gcode = zf.read("Metadata/plate_1.gcode").decode("utf-8")
assert gcode.startswith("; PLAIN\n")
finally:
source.unlink(missing_ok=True)
if result:
result.unlink(missing_ok=True)
class TestHeaderParser:
"""Direct tests for `_parse_3mf_gcode_header`."""
def test_parses_bambu_header_block(self):
header = _parse_3mf_gcode_header(_BAMBU_GCODE_TEMPLATE)
assert header["max_z_height"] == "16.00"
assert header["total_layer_number"] == "80"
# Units suffix is stripped from the key.
assert header["total_filament_length"] == "12155.34"
assert header["total_filament_weight"] == "36.55"
def test_ignores_lines_outside_header_block(self):
content = "; HEADER_BLOCK_START\n; key: in\n; HEADER_BLOCK_END\n; key: out\n"
header = _parse_3mf_gcode_header(content)
assert header == {"key": "in"}
def test_returns_empty_when_no_header(self):
assert _parse_3mf_gcode_header("G28\nG1 X0\n") == {}
class TestPlaceholderHelper:
"""Direct tests for `_substitute_placeholders`."""
def test_substitutes_known_keys(self):
assert _substitute_placeholders("Z={a} F={b}", {"a": "10", "b": "600"}) == "Z=10 F=600"
def test_alias_resolves_to_underlying_key(self):
assert _substitute_placeholders("Z={max_layer_z}", {"max_z_height": "16.00"}) == "Z=16.00"
def test_unknown_left_verbatim(self):
assert _substitute_placeholders("{nope}", {}) == "{nope}"
class TestStartMarkerHelper:
"""Direct tests for `_inject_start_at_marker`."""
def test_inserts_before_marker_line(self):
content = "first\nsecond\n; MACHINE_START_GCODE_END\ntail\n"
result = _inject_start_at_marker(content, "INJECTED")
assert result == "first\nsecond\nINJECTED\n; MACHINE_START_GCODE_END\ntail\n"
def test_marker_at_start_of_file(self):
content = "; MACHINE_START_GCODE_END\nrest\n"
result = _inject_start_at_marker(content, "INJECTED")
assert result == "INJECTED\n; MACHINE_START_GCODE_END\nrest\n"
def test_missing_marker_falls_back_to_prepend(self):
content = "G28\nG1 X0\n"
result = _inject_start_at_marker(content, "INJECTED")
assert result == "INJECTED\nG28\nG1 X0\n"
@@ -0,0 +1,163 @@
"""Tests for `get_db` cancel-safety (#1112).
Starlette's BaseHTTPMiddleware cancels the inner task scope when a
client disconnects mid-request. Pre-fix `get_db` only caught `Exception`
(not `BaseException`), so `CancelledError` skipped the rollback path —
the SQLite write lock stayed held until the connection was eventually
GC'd, producing the "database is locked" cascade in @Carter3DP's
support package on #1112.
The fix:
1. Catch `BaseException` so `CancelledError` triggers rollback.
2. `asyncio.shield` rollback + close so the cleanup completes even
when the await is cancelled by the same cancel scope.
"""
from __future__ import annotations
import asyncio
from unittest.mock import AsyncMock, patch
import pytest
from backend.app.core import database
class _FakeSession:
"""Minimal async-context-manager stand-in for `AsyncSession`.
Records which lifecycle methods were invoked so tests can assert on
the cleanup order without a real engine / DB file.
"""
def __init__(self):
self.commit = AsyncMock(name="commit")
self.rollback = AsyncMock(name="rollback")
self.close = AsyncMock(name="close")
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False # don't suppress
@pytest.fixture
def fake_session_factory(monkeypatch):
"""Patch `database.async_session` to yield a fresh `_FakeSession`."""
session = _FakeSession()
monkeypatch.setattr(database, "async_session", lambda: session)
return session
async def _consume_get_db(action):
"""Drive `get_db` like FastAPI's dependency machinery does:
enter the async generator, run `action(session)`, then advance to
completion. Returns the entered session."""
gen = database.get_db()
session = await gen.__anext__()
try:
await action(session)
except StopAsyncIteration:
return session
# Advance to the end so the generator's finally runs.
try:
await gen.__anext__()
except StopAsyncIteration:
pass
return session
class TestCancelSafety:
"""Pin the cancel-safety contract end-to-end."""
@pytest.mark.asyncio
async def test_commit_on_clean_exit(self, fake_session_factory):
session = fake_session_factory
async def noop(_s):
pass
await _consume_get_db(noop)
session.commit.assert_awaited_once()
session.rollback.assert_not_awaited()
session.close.assert_awaited_once()
@pytest.mark.asyncio
async def test_rollback_on_regular_exception(self, fake_session_factory):
session = fake_session_factory
gen = database.get_db()
await gen.__anext__()
with pytest.raises(ValueError):
await gen.athrow(ValueError("route handler bug"))
session.commit.assert_not_awaited()
session.rollback.assert_awaited_once()
session.close.assert_awaited_once()
@pytest.mark.asyncio
async def test_rollback_on_cancelled_error(self, fake_session_factory):
"""The actual #1112 fix: CancelledError must NOT skip the rollback.
Pre-fix `except Exception` caught nothing because CancelledError
is a BaseException, not an Exception."""
session = fake_session_factory
gen = database.get_db()
await gen.__anext__()
with pytest.raises(asyncio.CancelledError):
await gen.athrow(asyncio.CancelledError("client disconnected"))
session.commit.assert_not_awaited()
session.rollback.assert_awaited_once()
session.close.assert_awaited_once()
@pytest.mark.asyncio
async def test_close_runs_even_if_rollback_raises(self, fake_session_factory):
"""A failing rollback (broken connection during cancellation) must
not prevent `close` from running — otherwise the pool would never
reclaim the connection."""
session = fake_session_factory
session.rollback.side_effect = OSError("broken pipe during rollback")
gen = database.get_db()
await gen.__anext__()
with pytest.raises(asyncio.CancelledError):
await gen.athrow(asyncio.CancelledError())
session.rollback.assert_awaited_once()
session.close.assert_awaited_once()
@pytest.mark.asyncio
async def test_close_failure_does_not_propagate(self, fake_session_factory):
"""A failing close on the clean-exit path must not raise out of
`get_db` — the request already succeeded."""
session = fake_session_factory
session.close.side_effect = OSError("close failed")
async def noop(_s):
pass
# Must not raise.
await _consume_get_db(noop)
session.commit.assert_awaited_once()
session.close.assert_awaited_once()
@pytest.mark.asyncio
async def test_rollback_uses_shield(self, fake_session_factory):
"""Cancellation arriving DURING rollback must not abort the
rollback — `asyncio.shield` keeps it running. Verify the call
path goes through `shield` so future refactors don't silently
drop the protection."""
# The fixture wires the fake session into `database.async_session`;
# we don't need the local handle here.
with patch.object(asyncio, "shield", wraps=asyncio.shield) as shield:
gen = database.get_db()
await gen.__anext__()
with pytest.raises(asyncio.CancelledError):
await gen.athrow(asyncio.CancelledError())
# rollback + close both shielded.
assert shield.call_count == 2
+3 -4
View File
@@ -6381,9 +6381,9 @@
}
},
"node_modules/postcss": {
"version": "8.5.6",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.6.tgz",
"integrity": "sha512-3Ybi1tAuwAP9s0r1UQ2J4n5Y0G05bJkpUIO0/bI9MhwmD70S5aTWbXGBwxHrelT+XM1k6dM0pk+SwNkpTRN7Pg==",
"version": "8.5.12",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.12.tgz",
"integrity": "sha512-W62t/Se6rA0Az3DfCL0AqJwXuKwBeYg6nOaIgzP+xZ7N5BFCI7DYi1qs6ygUYT6rvfi6t9k65UMLJC+PHZpDAA==",
"dev": true,
"funding": [
{
@@ -6399,7 +6399,6 @@
"url": "https://github.com/sponsors/ai"
}
],
"license": "MIT",
"dependencies": {
"nanoid": "^3.3.11",
"picocolors": "^1.1.1",