disconnect() and _abandon_connection() logged nothing, at any level. A
session closed cleanly and a socket genuinely abandoned therefore produced
identical output -- none -- and the only way to tell them apart was to read
the source.
That is how #3009 was filed. Its trace shows a print completion opening two
FTP connections, deleting one file, and then nothing until the printer was
powered off 21 minutes later, read as connections left open and offered as a
mechanism for the 0500-C010 SD-card error that #645 has been chasing since
April. The two connections are the post-print SD cleanup in main.py walking
its candidate filenames, each through delete_file_async, which closes in a
finally; running that against the mock FTPS server shows the server logging
"FTP session closed (disconnect)" for both the 250 and the 550, holding zero
sessions afterwards. Nothing in a support bundle could have shown that.
Both close paths now log one DEBUG line: the printer, whether QUIT was
acknowledged or the socket had to be dropped without it, why, and how long
the session was held. Every connect in a debug log now has a matching close.
The duration comes from a stamp taken when the control socket opens rather
than after login, so a session that dies during login is accounted for too;
where no socket was ever established the line says "held unknown" rather
than claiming a number. The four connect() failure paths pass their own
reason, so a close line stands on its own next to the warning above it.
Nine tests, seven of which fail against the unlogged version. The other two
assert silence -- a bare disconnect(), and a connect skipped by the handshake
cool-off -- where no socket was opened and a close line would pair with no
connect.
When a print's own 3MF cannot be fetched the usage tracker borrows one from the
library or a previous archive, matching on the filename stem. That is far weaker
evidence than it looks: Bambu Studio writes the printer-side filename from the
project's Title metadata, so every plate of a project reaches the printer under
one name however the file was renamed on disk.
The reporter's single-filament job was handed a previous archive's three-filament
plate. Three spools were debited for material that was never extruded, and
nothing on the archive said the numbers were someone else's.
A candidate is now rejected when it positively contradicts the print - a
different plate, or a filament count the slicer's ams_mapping disagrees with -
and the accepted one is logged with both expectations. Only on a contradiction:
the plate needs firmware that echoes it and the count needs a print command
Bambuddy saw, and refusing everything uncorroborated would retire the fallback
recovery this same issue asked for.
The count is scoped to one plate or not compared at all. Unscoped, the filament
reader collects every <filament> in the file, and that sum against one plate's
count would reject every multi-plate library upload on exactly the firmwares
that cannot tell us the plate.
-----
Give a download the time the file needs, and one printer at a time (issue #2957)
ftp_timeout is handed to every download as both the socket inactivity timeout
and the whole-transfer deadline, which makes its 30s default a cap on how big a
file a printer may serve. The reporter measured one 5.4 MB 3MF at 45s off a worn
P1S SD card and 25s off a new one, and a 15.15 MB 3MF at 105s. None of those
links were broken - they were slow, which is what the inactivity timeout exists
to tell apart.
download_to_file already asks for SIZE. It now reports it, and the total
deadline follows the file at the 25 KB/s floor _upload_deadline has used since
do, so #2572's cap on the executor queue wait is untouched. Capped at 300s for a
reason that is not about FTP: on_print_start holds a pooled DB connection across
its whole 3MF hunt.
Downloads also take turns per printer now. He watched Bambu Studio lose its own
connection while Bambuddy pulled a 12 MB 3MF, and a later log caught two
Bambuddy downloads of the same file overlapping at print start. The gate is
soft - whoever cannot have it within 30s goes anyway, because a print losing its
3MF to queueing is worse than the contention, and a soft gate cannot deadlock.
It is also meaningful for the first time. The 90s cap on a multi-path lookup
returned while its worker kept walking the remaining paths, still on the
printer's socket; that walk is now cancelled and waited out before the printer
is handed on.
-----
Look in the shared 3MF cache again before each cover retry (issue #2957)
The cover endpoint and the print-start archive flow share a cache so whichever
fetches the 3MF first hands it to the other (#972). The cover consulted it once
on the way in, then retried for up to two and a half minutes without looking
again.
In the reporter's log the archive flow published the file 42 seconds into that
sequence and the cover's third attempt still pulled its own 5,250,969-byte copy,
off a printer that was mid-print on the same SD card.
A file picked up that way is left alone rather than re-registered under this
endpoint's own name or deleted on the way out. It is the archive flow's.
Assigning a spool to an AMS slot unassigned it again seconds later, and
the slot's colour changed at the same time. It presented as Bambu Studio
and Bambuddy fighting over the slot. The reporter's log shows Bambuddy
losing to itself.
P1S firmware 01.10.00.00 reads every lowercase hex letter in an AMS
tray_color as a zero, and hides it completely: the command response
echoes back the value that was sent and reports result "success", so
only the next AMS push says what was really stored. The spool-assign
path sent spool.rgba verbatim and that column stores lowercase. From the
bundle:
sent 09ff00ff -> AMS reports 09000000
sent ff5100ff -> AMS reports 00510000
sent 090000FF -> AMS reports 090000FF
That is the visible colour change, and it is also what deleted the
assignment. The auto-unlink sweep asks whether the slot still matches
the spool assigned to it; the mangled colour no longer did, so the
assignment Bambuddy had made four seconds earlier was removed.
colors_similar('09000000', '09FF00FF') is False, which is the whole of
it.
Re-assigning could not recover, because the Configure Slot dialog seeds
its colour from whatever the printer currently reports. It wrote the
mangled colour back and cemented it, which is the loop the report
describes in its steps 4 and 5.
Colours are now uppercased where the command is assembled rather than in
each of the four routes that configure a slot. A caller that forgets is
exactly how this arrived. Nothing else changes: no padding, no invented
alpha, no six-to-eight widening, and tray_type and tray_sub_brands keep
their case, where it carries meaning -- "PLA Matte" is a product line,
"PLA MATTE" is not.
Two paths deliberately left alone. The developer-mode probe re-sends the
colour the printer itself just reported so that the probe is inert;
uppercasing there would turn it into a write. And the Virtual Printer
forwards the slicer's own command verbatim -- Studio could in principle
hit the same firmware bug, but nothing here evidences that it sends
lowercase, and rewriting a slicer payload inside a transparent proxy is
not a change to make on a hunch.
Two more defects from the same log.
A spool with a brand and no subtype was configured with the string
"None" in its name: the branded branch interpolated spool.subtype
without checking it while the unbranded branch guarded it, so
"Sunlu PLA Matte None" went on the wire and into Studio's display.
And the FTP log is readable again. A 426 whose bytes Bambuddy has
already verified against the printer is how Bambu FTPS normally ends a
transfer, not a fault, so it drops from WARNING to INFO. It fired 54
times in this one bundle, every one followed by a completed upload, and
it was burying the 26 TLS handshake failures in the same log that
actually cost the reporter two prints. A 426 whose bytes do not verify
is still an error and still fails the upload.
The handshake failures themselves are printer-side FTPS cool-off under
load and are not touched here.
@grolmus measured a 9-printer farm and the numbers settle what this
failure is not. Reproduced here, three results:
cleartext "421" banner on the TLS port
-> [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1032)
1.2-only server, client forced to 1.3
-> [SSL: TLSV1_ALERT_PROTOCOL_VERSION]
1.2-only server, an uncapped client
-> negotiates 1.2 and connects
The first is byte-for-byte what the farm logs. So WRONG_VERSION_NUMBER
means the printer's first bytes were not a TLS record, a version
mismatch cannot produce it, and reaching a 1.2-only peer needs no cap.
What it still does not say is WHICH cleartext message, and that is the
part that would name the fault. OpenSSL has eaten those bytes by the
time the exception surfaces, so on this error the client now opens one
plain connection and reads them. The log then carries the printer's own
words -- an FTP refusal such as "421 Too many connections" would settle
it outright -- marked as the line to quote in a report. This gets the
answer from every affected install rather than from the one farm able
to take a packet capture.
Three things keep it from making the suspected fault worse:
- The failed socket is closed BEFORE the probe opens its connection.
Holding a dead handshake open across a second connect to a printer
that may be out of connection slots is the leak #2780's own cleanup
was added to stop.
- It asks once per cool-off window, not once per attempt. Checked
before the new deadline is written, so a live entry means an earlier
failure already asked -- which matters because a dispatch ignores the
cool-off (#2898) and reaches this branch four times.
- Connect and read share one timeout budget rather than getting one
each.
Only WRONG_VERSION_NUMBER is probed. A protocol-version alert means the
peer did speak TLS, so there is nothing in the clear to read and the
probe would only sit out its timeout. A vsFTPd answering its connection
limit by accepting and staying silent -- the other half of the standing
theory -- arrives as a handshake timeout and lands on that branch
instead; there is a test saying so, because widening the trigger later
would look like an improvement.
The profile registry is corrected to what was measured. Its docstring
claimed "the P2S evidently does offer 1.3"; six P2S units refuse it.
Worse, the X2D (#1638) and H2C (#2582) entries were capped on the
reading that WRONG_VERSION_NUMBER came from a TLS-1.3 ClientHello,
which cannot happen -- so the cap is not what changed those outcomes
and both are now marked RE-TEST WANTED. They are kept rather than
removed: their reporters saw the symptom clear, nobody here has that
hardware, and the entry costs nothing on a printer that does not offer
1.3 anyway. The P2S entry (#1401) is a different symptom -- a 426
truncation mid-transfer -- and is the only one a session-ticket problem
could explain, though grolmus's firmware refuses 1.3 there too.
Both measurements are pinned by tests, so the explanation stays
falsifiable instead of becoming the next set of confident wrong
comments. Two existing cool-off tests now count two connections where
they counted one; the promise they exist for -- contacted twice, not
~110 -- is unchanged, and they say why rather than carrying a new
number.
Every dispatch upload that failed carried one sentence: "Failed to
upload file to printer. Check if SD card is inserted and properly
formatted (FAT32/exFAT)." The reporter got it after a TLS handshake
failure and restarted the printer on the strength of it. That could not
have helped. The handshake never reached the printer's filesystem, and
the cool-off that made the next dispatch fail the same way lives in
Bambuddy's own memory, where power-cycling a printer does not reach.
because the advice was known not to work. It survived in the string
people actually read, so the failure mode that fix closed was still
reachable through the UI.
reachable through the UI.
The information was never missing. connect() separates five failure
classes and upload_file() separates 553/552/550, each with its own log
line -- 553 even logs a spelled-out list of storage causes -- and then
both returned a bare False. The dispatch had nothing left to work with
and guessed storage for all of them.
So the reason now travels with the result. The client records an
FtpFailure (kind, detail, and the reply code where the server gave one)
on every failure branch, and upload_file_async fills in a report object
the CALLER owns. Not a per-IP dict beside _mode_cache: those describe a
printer and are right to share, while this describes one operation, and
a background timelapse fetch running beside a dispatch would overwrite
the dispatch's reason with its own -- reporting the wrong cause with
total confidence, which is this bug again rather than a fix for it.
describe_upload_failure() picks the wording, and lives next to the kinds
so the two cannot drift. A 553 or 552 keeps the card advice, which is
the case it was written for, and quotes the reply code so a queue entry
and a support bundle line up. A handshake failure says the file service
answered without TLS and that the card is not involved. A refusal points
at the access code, a timeout at the network, and anything unclassified
says so and points at the log rather than picking a plausible cause. A
wrong instruction costs more than a vague one: it sends someone to work
on hardware that is fine. Nothing prescribes a power cycle.
The failure notification now carries the same sentence the queue shows,
instead of its own fixed "Failed to upload file to printer", so a push
and the screen cannot disagree about what happened.
Tests cover the classification, the report reaching the caller through
the retry loop, two callers not crossing, the queue entry itself, and
one line per message branch -- without that last one, deleting the
access-code or timeout branch left every other assertion passing, since
they only check that the card is not named and the generic message
satisfies that too. The card-advice test keys on the instruction rather
than the words "SD card", because the handshake message names the card
in order to rule it out.
A failed TLS handshake arms a 300s per-IP cool-off, and connect()
consulted it for every caller. A print dispatch retries after 2s, so
once the cool-off was armed all four attempts were answered from the
gate rather than the network, and every further job queued for that
printer failed the same way for the rest of the window. The reporter's
farm lost three jobs to one handshake error, with the retry budget
contributing nothing to any of them.
The gate was serving two callers that want opposite things from it. The
background sweeps -- the post-print 3MF, cover and timelapse fetches --
walk ~110 candidate paths against one wedged printer with nobody
waiting, and backing off for minutes is right for them. A dispatch is
one delete plus at most four upload attempts with someone watching a
progress bar. So the split is by caller: a client built with
respect_handshake_cooloff=False goes to the printer regardless, and the
dispatch's delete and upload -- and a firmware upload, same shape --
opt out. Everything else keeps #2780's behaviour untouched.
In the reported trace it is the pre-upload delete that takes the SSL
error and arms the cool-off, 8ms before the upload's first attempt, so
exempting the upload alone would have left one dispatch's worth of the
problem in place.
Callers that do respect the cool-off no longer sleep out a retry loop
against it: with_ftp_retry takes the printer's IP and stops at the
attempt that armed the gate, instead of spending three more attempts
and six seconds on connections that cannot happen. It also reports the
attempts it really made -- "failed after 4 attempts" for one attempt is
part of how this read as a network problem.
Two diagnosis fixes go with it. The cool-off skip was the one connect()
failure path that reported without naming its cause, and at DEBUG, so
four identical reason-free warnings were all the operator saw. It now
says at WARNING that nothing was sent and how long the printer has
left, once per cool-off rather than once per attempt -- not every
caller is gated, and a download-zip of 200 files would otherwise repeat
the sentence 200 times, which is the flood #2780 set out to stop.
And a dispatch that fails this way no longer tells anyone to check
whether the SD card is inserted and formatted -- nothing reached the
printer's filesystem, so the card is the one part of the machine that
was working. The message names the file service and rules the card out.
It is used only when a handshake failed during the dispatch itself,
read from the cool-off deadline MOVING rather than merely being armed:
the dispatch ignores the gate, so it can be running underneath one an
unrelated background fetch left behind, and blaming TLS for an upload
that really hit a full disk would repeat the mistake in the other
direction.
Tests count sockets rather than return values, since "returned False"
looks identical whether or not anything was attempted -- which is what
made the original report a log dive. Reverting any one of the five
behaviours above fails a distinct test.
When the sliced file stays somewhere Bambuddy cannot read, the archive is
built from the printer's report alone and carries no weight. Nothing could
supply one afterwards: rescan reads the figure out of the 3MF, and that
archive has no file to read. The reporter's H2S print left 46.16 g on the
spool with nothing recording it, and he corrected Spoolman by hand.
Edit Archive now has a Filament used (g) field. It is written to the
archive's most recent run as well, because the Projects roll-up and the
Prometheus counter sum PrintLogEntry rather than the cards - correcting
only the archive would fix the display and leave every aggregate reading
the old figure, or none at all.
But not over a figure the run measured for itself. A run's grams come from
the tracked spool delta when there is one and only fall back to copying
the archive's estimate when there is not, so mirroring unconditionally
would overwrite a measurement with a typed estimate. The mirror now takes
a run that has no figure, or one holding exactly what this archive held -
which also makes the undo complete, since clearing the archive clears the
copy it made and leaves a measured run alone.
The field is text rather than a number input. A number input reports an
empty string for anything the browser judges malformed, a decimal comma in
a locale that does not expect one included, and that reads here as "the
user cleared it" - it would have wiped a good figure while the field still
showed what was typed. Filtering on the way in keeps what is displayed and
what would be sent the same string, and clamps it to the range the API
accepts: this modal has no error surface, so a refused save looks like
nothing happened at all.
Saving also invalidates the archive's runs query. The Print Log this modal
renders at its top reads them separately and kept serving the pre-edit row,
so a correction looked like it had not taken - true for the status and
failure-reason mirrors since #1444 as well.
Second half of the same report: the internal-storage probe (#2856) logged
which file it found but not where. On a printer that keeps uploads for
weeks - the reporter has months of them in /cache - a reprint of a name
that was re-sliced but never re-sent can match an older copy, and without
the directory that mismatch is invisible rather than merely rare. The
download helper returns the path that served the file instead of a bare
flag; every caller only ever tested it for truth.
A print's dispatch says where the printer put the sliced file:
ftp://<name> for external storage, brtc://emmc/<name> for internal. Since
that there is then no file to find at any path.
That is where the printer chose to put it, which is not the same as where
port 990 can read it. The reporter's H2D - firmware 01.03.00.00, card in
the slot - reports brtc://emmc and keeps the same file under /cache: his
log has every print from 08-12 downloading from there, 19 MB included,
until the skip landed and two days of archives came out as a name and
nothing else. #2780's P2S and H2C really did 550 on every path, so both
are true and the URL alone cannot tell them apart.
So ask the printer rather than the model. The dispatch names the exact
file, which turns the question into one connection walking five
directories - against the sweep's ~110, which is the cost that made
skipping worth doing. A hit archives normally and is shared with the
cover endpoint; a miss keeps #2780's fallback archive and its reason, so
the archives banner still explains itself. Not probed when the printer
reports an empty slot, or while its file service is in TLS cool-off:
both have already answered the question.
The connection diagnostic asked the same question off the URL and warned
that the last print was out of reach. On this reporter's printer that
warning would have sent him to a setting that was already right, so it
now probes too - by directory listing, since the file it is asking about
can be tens of megabytes and the answer is a yes or a no. Capped at 6s to
stay inside the support bundle's per-printer budget, and "could not
check" leaves the warning standing.
The probe filename arrives over MQTT and becomes both a remote path and a
local temp filename, so names carrying separators, traversal or control
characters are declined rather than cleaned.
Bambuddy reads a print's 3MF, cover and timelapse over FTPS on port 990,
which on every Bambu model serves external storage only. Under some
configurations H2-series and P2S firmware keeps the sliced file on
internal storage, where Bambu Studio put it over the port-6000 service,
and then no path on 990 can find it.
The print command has always said which of the two it used -- `url` reads
ftp://<name> or brtc://emmc/<name>. We discarded it and swept anyway:
~110 connections per print, all certain to fail, ending in an archive
card with nothing on it and no stated reason. In the reporter's bundle
all 35 dispatches to their H2C and P2S said internal storage, all 25 to
their X1C said external, and all 44 empty cards belonged to the first two.
Read the field, skip the sweep when it cannot succeed, and record which
reason applied. A printer that uses the card is unaffected, and so is one
we have no answer for -- silence is not evidence, and reading it as bad
news would break archives that work today.
The answer is held per print and dropped when that print ends, rather
than kept as a standing fact about the printer. Plenty of prints never
announce themselves: 14 of the 79 print starts in that bundle arrived
with nothing on the request topic, started from the printer's own screen
or picked up after a restart. Left standing, one slicer print to internal
storage would suppress the lookup for every screen-started print after
it, on a printer whose files really are on the card. The sticky reading
is kept for the connection diagnostic alone, which is run after the print
that prompted it and would otherwise have nothing to report.
Two things that pointed the wrong way go with it. The archives banner
told everyone to enable "Store sent files on external storage"; the
reporter had it on for the whole three weeks and it would not have
helped. The diagnostic passed a printer whose slot was empty, because it
read only the toggle -- an empty slot is now a failure naming the slot,
and a printer that has storage and still used its own is a warning. On
P1-series that empty-slot failure yields to the existing unsupported-model
skip: the toggle cannot be switched on there at all, so telling the
operator to insert a card would promise a fix inserting a card does not
deliver (#2524).
Also close FTP sockets on the failure paths, which dropped them for the
garbage collector -- 1813 in a day in that bundle -- and drop the advice
to restart the printer, which the reporter tried twice while a single
manual connection to the same printer handshook cleanly.
This does not make the affected prints archive in full; that needs the
port-6000 protocol tracked in #2762.
Two printers went on printing while every archive they produced held nothing
but a filename. Bambuddy opened port 990, the printer accepted the connection
and answered with something that was not TLS, and connect() logged a warning
and returned False -- indistinguishable, to every caller, from "the file is
not at this path". So the 3MF lookup walked all six filename variants across
five directories with four retries each, the cover endpoint ran its own
sixteen-path sweep, and the timelapse scan added four more, all against a
sixteen-path sweep, and the timelapse scan added four more, all against a
printer that could not have answered any of them. One reporter's log carried
1813 identical handshake failures, another's 3511.
The evidence says this is the printer's own file service getting stuck, not a
model, firmware or TLS-configuration problem. In #2780's bundle the same two
printers ran clean from 22 July to 4 August and failed again from the 5th; a
second bundle shows an X2D serving files for five days, flipping on 19 July,
then failing every connection for eight days with zero successes. The same
models and firmware appear in roughly twenty other bundles with no occurrences
at all. Both bundles show it happening with cap_tls_v1_2 in effect -- the X2D
and H2C entries in ftp_profiles were added on analogy with P2S to fix exactly
this symptom, and the reporter's own debug line proves they do not.
An ssl.SSLError from connect() now opens a five-minute cool-off for that
printer. Subsequent connects return False without touching the network, so a
wedged printer is contacted twice an hour instead of hundreds of times a
minute, and the single warning that is logged names the remedy. The cool-off
is dropped on expiry rather than kept, so the map holds one key per currently
wedged printer. ftps_handshake_blocked() lets the sweeps stop: the 3MF lookup
abandons the remaining paths and skips the directory-walk fallback, the cover
endpoint returns 503 naming the file service instead of a 404 that reads as
"this print has no thumbnail", and the timelapse scan separates 503 (cannot
reach the printer) from 404 (no timelapse directory) -- one 500 used to cover
both, which is what the reporter hit when reproducing.
The Connection Diagnostic completed a bare TCP connect to 990, which is why it
reported the port green throughout: the port is open, it is what is behind it
that is broken. It now completes a real implicit-TLS handshake using the
model's own ftp_profiles cap, so a pass means the FTP client would also get
through. An open port that cannot negotiate reports warn with reason no_tls,
selecting a new message in all 13 locales that points at a printer restart
rather than at the firewall. No login is attempted, so this stays valid in the
pre-save Add Printer flow.
The cool-off tests run against a real socket that accepts on 990 and replies
with a plaintext FTP banner, reproducing WRONG_VERSION_NUMBER rather than
mocking ssl. The autouse fixture clearing _mode_cache now clears the cool-off
map too -- every test here talks to 127.0.0.1, so one left behind would make
the next test's connect() a no-op.
Timelapse was on, the video never reached the archive, and Scan for Timelapse
found nothing afterwards. Across 247 support bundles this was the norm, not an
edge case: 457 automatic scans scheduled, 262 attached.
The scan looked four times over ~65s. The attempt that found the video was #1
272 times, then 17 / 13 / 13 — flat against the cutoff, not decaying, i.e.
files were still arriving when we stopped. What ran afterwards searched for the
print name inside the filename; Bambu only writes "video_<timestamp>", so it
fired 159 times and matched zero.
The manual Scan had no baseline at all and matched on filename timestamp, FTP
mtime, or "there is only one video" — all reading a clock a LAN-only printer
cannot sync. The reporter's P1S was six and a half days out.
- Poll for minutes instead of ~65s; drop the name-match fallback.
- Persist the print-start baseline on the archive, so the diff survives a
restart mid-print and the manual Scan runs the same comparison. With a
baseline present the clock-based strategies are skipped entirely — they can
only turn an honest "pick one" into a confident wrong answer.
- When several files are new (a previous print's video landing late), exclude
the ones already attached to another archive instead of ordering the
candidates. Ordering could only be done on the printer's clock.
- Delete the video from the printer once archived. Keeps /timelapse to
unclaimed files, which is what makes the diff unambiguous, and stops P1S
cards filling with AVIs.
- Gate that delete on a verified transfer: download_file now compares against
the size from the listing. An FTPS connection closing early does not always
raise, so a partial buffer was being attached as a complete video — which
would also have been the one case where deleting the source lost data.
Bounded twice on purpose: wall-clock deadline plus a derived round cap, since
the deadline stops bounding the loop as soon as the sleeps are shortened.
Per-round logging only speaks when the listing changed — 31 rounds of full
listings would bury the interesting line in the support bundle.
Migration adds print_archives.timelapse_baseline as JSON, spelled the same on
both dialects so a migrated database matches a fresh one.
-----------
fix(finish-photo): add the timelapse frame to the archive after the notification (#2704)
When a print records a timelapse, its last frame is the better finish photo:
the firmware stops recording with the toolhead parked and before the end
G-code drops the bed, where a live grab at that moment catches a lowered
plate. Bambuddy waited 60s for the video and then gave up, because the
print-complete notification blocks on that photo and holding a notification
for minutes is worse than sending it with the live grab.
P1-series printers write MJPEG AVI rather than H.264 MP4 and serve it slowly.
Measured over 261 attaches in the support bundles: P1S median 33s, p90 167s,
worst 546s, while every other model finished inside 26s. So the printers that
most needed the better framing were the ones that never got it.
Keep the notification on the same bound, and keep waiting off to the side.
_capture_finish_photo_from_timelapse now reports whether it ran out of time or
concluded — a video that landed and failed extraction is not worth retrying,
one that never arrived is. On the first, schedule a background task that waits
up to 15 minutes and inserts the extracted frame at the front of the archive's
photo list, where the gallery opens.
The live grab stays on disk: the notification already links to that exact
file, so removing it would leave a broken image in Discord or Telegram.
The length check proves we received what the listing said, not that the file
was finished. The first look happens ~5s after the print ends, while the
printer may still be writing, so a growing file can be listed short, served
short, and pass. Re-list after the download and only accept the video once its
size has stopped changing — a failed re-list counts as not settled, since
"could not check" must not mean "safe to delete".
The remaining routes of the idle-in-transaction class: the file-manager,
storage, camera-snapshot and timelapse routes each took their printer row
via Depends(get_db) and then talked FTP/camera on the same held session, so
a farm dashboard polling cover/snapshot tiles (offline printers included)
crept the pool to exhaustion over ~23h. They now read in a short session and
release before the I/O; timelapse re-opens a fresh session only for the write.
Also caps the four bare-executor FTP helpers with asyncio.wait_for so a
saturated 48-worker pool can't pin a caller (and its DB connection)
indefinitely, and runs the synchronous smtplib send off the event loop with
an explicit timeout so a wedged relay can't freeze the loop.
The reporter's 19-printer farm started prints "one by one", up to an hour apart.
check_queue awaited each dispatch inline, and a dispatch includes the FTP upload,
so every printer queued behind every other printer's transfer despite being an
independent machine. His logs give the arithmetic: 40978500 bytes in 254.1s,
157 KB/s - a Bambu printer's SD write, not the network, is the bottleneck. Nineteen
of those in series is ~80 minutes, and the next upload started 131 ms after the
previous one finished. The delay is linear in fleet size, which is why it got worse
the more printers he selected.
Dispatch is now collected during the (still sequential) selection loop and run
concurrently afterwards, capped by queue_max_concurrent_uploads - Settings ->
Workflow -> Queue & Dispatch, default 4, 1 restores the old behaviour. Every gate
is untouched; only the transfers overlap. The pass still awaits its uploads before
returning: _start_print flips the row pending -> printing only after the upload,
so an early return would let the next tick re-dispatch the same rows.
FTP work moves to its own thread pool. It was on asyncio's default executor -
min(32, cpu+4), six threads on a 2-core NAS, shared with everything else - which
was survivable only while uploads were serial.
Two problems the same bundle exposed:
A printer that accepts project_file but never starts (#1678) was retried forever:
270s watchdog, revert to pending, re-upload the whole file, repeat. Hence his
"printer who, since the morning, still not launch" - and on a farm each lap also
eats an upload slot the other printers are waiting on. Attempts are now counted on
the queue item; after three it fails with a message pointing at the printer instead
of queueing a fourth re-upload.
The debug bundle we asked him for held 4m49s of history. The push_status dumps fired
on every frame rather than on change - several while their own comment claimed
otherwise - which is 27,727 of the bundle's 29,830 lines and rolls 5 MB in under five
minutes on 19 printers. They now log transitions only. The bundle also read just the
live log while three rotated backups sat next to it, under a byte budget four times
larger than the file it was reading.
Migration verified on SQLite and Postgres: idempotent, backfills legacy NULLs
(dispatch_attempts + 1 is NULL for a NULL row, which would silently disable the cap).
Tests: 6 on concurrent dispatch (overlap, cap honoured, 1 == serial, default applies
with no settings row, a failed printer does not cancel its siblings, no early return),
4 on the retry budget, 6 on the bundle's rotated-log span, 7 on the debug gating.
Each verified to fail against the unfixed code - the first end-to-end log assertion I
wrote passed without the fix and had to be tightened.
upload_file_async carried a flat 600s wall-clock deadline and ran the
transfer via asyncio.wait_for(run_in_executor(...)). wait_for cancels the
future, not the executor thread. A 96 MB 3MF to an A1 over WiFi sustains
~75 KB/s and needs ~20 minutes, so the await gave up at ~70 MB, returned
False, and with_ftp_retry started a second STOR of the same file onto the
same printer while the first was still streaming. The reporter filmed two
transfers of one job climbing in parallel at 2% and 72%; the print never
landed and the printer read as having a flaky network.
The deadline is now derived from the file size against a 25 KB/s floor, so a
slow-but-healthy transfer can finish — a link that has actually died is
caught within socket_timeout by the blocking sendall, which is what should be
detecting failure. A deadline expiry now stops the transfer for real: the
worker is signalled, raises UploadCancelled from its progress callback, and
upload_file's existing cancel path breaks the send loop and deletes the
partial file. with_ftp_retry never retries that, and a per-printer lock makes
overlapping uploads impossible however they were triggered.
ssl.create_default_context() leaves minimum_version at MINIMUM_SUPPORTED,
so the floor came from the OpenSSL build rather than from Bambuddy. On
identical OpenSSL 3.5.6, python:3.13-slim-trixie reports TLSv1_2 while a
bare-metal venv reports MINIMUM_SUPPORTED -- Docker installs were floored
at 1.2, bare-metal and appliance installs were not.
Set minimum_version explicitly in ImplicitFTP_TLS and the MQTT client. On
the P2S/X2D profiles that also cap maximum_version this becomes an exact
TLS 1.2 pin. Probed against an X1C and an H2D on :990 and :8883: both
complete only on TLS 1.2 and reject 1.0, 1.1 and 1.3; live FTPS login
through the new path succeeds on both.
Also correct a stale comment in ftp_profiles.py -- X1C and H2D refuse
TLS 1.3, so cap_tls_v1_2 is a no-op there, contrary to what it claimed.
Two warnings polluting every A1 support bundle on healthy prints, both
unrelated to the timelapse-default behaviour the issue actually reports.
1. mqtt_bridge.py's post-bind nudge calls request_status_update on the
real printer's MQTT client to populate the bridge cache without
waiting for the next periodic pushall. The bind frequently races the
TLS handshake, especially on A1 firmware. Skip the nudge when
state.connected is False — the periodic pushall fills the cache
anyway. The WARNING in bambu_mqtt.py stays for the genuinely-
actionable callers (refresh-status API, bug reporter).
2. Post-finish SD-card cleanup (and the symmetric forced-timelapse dir
walk) used delete_file_async's bool return to drive a WARNING when
all candidates failed. A1 firmware self-cleans the SD card before
our cleanup runs — every candidate FTP-DELE returns 550, we burn
the retry budget, then WARN on a successful print. Introduce
DeleteResult.{DELETED,NOT_FOUND,FAILED} so the helpers only WARN
on real network/auth/transient failures. NOT_FOUND advances to the
next candidate without consuming the 2s backoff. User-facing delete
endpoint returns 404 on NOT_FOUND.
Python 3.13 negotiates TLS 1.3 by default. The P2S firmware 01.02.00.00
vsFTPd build doesn't tolerate TLS 1.3's async session-ticket model on
the FTPS data channel — session resumption races, the data channel gets
torn down mid-stream, uploads land truncated at a chunk boundary, and
the printer replies 426 instead of 226. Visible to the user as "unable
to parse 3mf file" 30 s into the print.
Capping the SSL context's maximum_version to TLS 1.2 makes session
resumption synchronous and uploads complete normally.
Follow the per-model pattern established by camera_profiles.py in the
#1395 follow-up: add backend/app/services/ftp_profiles.py with a frozen
FTPProfile dataclass and a per-model registry. Only P2S (display name
+ N7 SSDP code) gets the cap today. X1C, H2D, P1S, A1 stay on negotiated
TLS 1.3 — the maintainer's dogfooded printers see zero behaviour change.
Previous daily build (1fac0276) tightened the post-STOR voidresp
handler to fail on any ftplib.Error, stopping Bambuddy from
sending a print command for a truncated 3MF. Reporter
(@enjoylifenow on a P2S) then confirmed — after a clean SD-card
filesystem check, reformat, and power cycle — that v0.2.4.1
worked on the same hardware. That proves the 426 returned by
this firmware revision is noise: the TLS data-channel close
races the 226 confirmation, server reports failure, file is in
fact on the SD card.
Reverting wholesale would re-introduce the silent-truncation
bug from the original fix. Narrow the rule instead: after an
ftplib.Error from voidresp, run an FTP SIZE against the upload
path. SIZE matches the local file size → warn and proceed
(the reporter's case). SIZE mismatch, or SIZE itself raises →
fail loudly with full diagnostics (the original tightened
behavior — preserved).
Applied identically to upload_file() and upload_bytes() so the
A1-compatibility manual-transfer path is covered.
Tests: two regressions from the previous round renamed and
split into intact / truncated / size-check-fails. Intact-file
tests inject SIZE explicitly because pyftpdlib only flushes on
a clean voidresp — which can't happen when we monkeypatch
voidresp to raise. Docstring spells that out. 87 FTP unit tests
green; 118 FTP-touching tests across unit+integration green;
ruff clean.
The View-Timelapse-greyed-out behavior #1417 was originally
about stays untouched; once the reporter confirms upload
reliability is back, that diagnosis continues on a healthy
install.
bambu_ftp.upload_file (and upload_bytes) wrapped the voidresp() call in a
broad "except Exception: log warning and proceed" because H2D printers
can take 30+ seconds to send the 226 and we don't want to fail on that.
But the same handler was swallowing ftplib.error_temp (e.g. 426 "Failure
reading network stream") from buggy printer firmware, which explicitly
means the data stream was cut mid-transfer and the file on the SD card
is partial.
Bambuddy then sent the print command anyway, and the printer surfaced a
generic "unable to parse 3mf file" error 30 seconds into the print
attempt -- with nothing in the log on the user side to suggest the
upload had actually failed.
Split the catch: ftplib.Error subclasses (server-reported failure)
re-raise so the outer handler returns False; everything else (socket
timeout etc.) keeps the existing proceed-with-warning behaviour so the
H2D 226 tolerance survives.
Two regression tests patch _ftp.voidresp to raise error_temp("426 ...")
and assert both upload_file() and upload_bytes() return False.
The underlying P2S firmware / TLS-data-channel issue that triggers the
426 for the reporter is separate -- this change just stops Bambuddy from
hiding it.
Daily builds since 889c8bd8 (Apr 29) silently destroyed archive copies
and library file bytes on every print completion. Reprint / View G-code
later returned 404 with no log line explaining why; the DB row was
intact and the archive grid kept showing the entry, but the file
behind archive.file_path no longer existed on disk.
Root cause: #1166 added three dispatch sites that cache the live
archive copy (and library file bytes for Direct-Print) in the shared
3MF download cache, so /cover could skip a redundant FTP transfer
mid-print. The cache was originally designed for transient downloads
under archive_dir/temp/, and clear_3mf_cache(printer_id) — called
from on_print_complete to keep that temp dir from accumulating —
happily unlink()'d every cached path. Path.exists() guarded the
unlink, so no exception, no warning, just silent destruction. Listing
didn't change; only acting on the archive surfaced the 404.
Fix: clear_3mf_cache._maybe_unlink refuses to unlink any path outside
archive_dir/temp. Cache dict is still cleared (so re-cache continues
to work and /cover hits a fresh path next print), only the on-disk
delete is gated. Persistent locations — archive/<printer_id>/...,
archive/unassigned/... (VP-archived prints with printer_id=None),
library_files/..., is_external library mounts — all survive.
Regression test test_clear_does_not_delete_persistent_files pins the
contract end-to-end: archive 3mf, library 3mf, and temp 3mf all
cached for the same printer; after clear, all three cache entries
are dropped from the dict, but only the temp file is unlinked from
disk. Two existing tests updated to put fixtures under
archive_dir/temp.
Second wave of #972 — reproducer on a 37.5 MB BambuStudio print to an A1
showed three stacking root causes when Bambuddy restarts mid-print.
1. Archive start_time lost on container restart. The name-based dedup
cancelled any "printing" archive older than 4h and recreated it with
started_at=now(), so a 13h print that saw a restart 10h in ended up
showing ~1.5h duration. Persist MQTT subtask_id on every archive and
match on that first, regardless of age — same id means same print,
resume in place. Also revives Stale-cancelled rows for users
upgrading mid-print.
2. 3MF FTP search tried non-existent paths for ~48 min. Order was
/cache → /model → /data → /data/Metadata → / with 11×30s retries
each; BambuStudio actually pushes to / on A1, so the real path was
tested last. Reorder to / first, and raise a new FileNotOnPrinterError
sentinel from download_to_file on 550 so with_ftp_retry short-circuits
via non_retry_exceptions. 425 / SSL EOF / connection resets still
retry as before.
3. Cover endpoint and archive flow downloaded the same 36 MB twice and
competed for the printer's single FTP socket, producing 425 errors
that fed cause-2's retry storm. Add an in-memory _threemf_path_cache
keyed on (printer_id, normalized filename); whichever flow fetches
first populates it, the other reuses the file read-only. Eviction
runs on on_print_complete and deletes the temp file.
Backend: 14 new tests across test_bambu_ftp.py and a new
test_subtask_archive_resume.py. Existing suite: 2737 pass. ruff clean,
frontend build clean.
Two-part root cause for missing photos/filament/cost on large prints
(#972). The configured ftp_timeout was only plumbed through as the FTP
socket timeout; the asyncio.wait_for wrapping run_in_executor stayed on
its 60s hardcoded default, so the user's 300s setting never applied.
Worse, asyncio.wait_for cannot cancel run_in_executor threads — after
the 60s outer timeout fired, the executor thread kept running
ftplib.retrbinary and frequently completed the download ~30–60s later,
but by then the async wrapper had returned False. with_ftp_retry kept
re-attempting the same path, each retry truncating the file the zombie
thread had just written, and the archive was ultimately persisted as a
fallback with no 3MF.
download_file_async now accepts timeout at each call site (plumbed from
ftp_timeout) and salvages post-timeout success via an explicit
completion flag the executor thread sets only after download_to_file
returns True. Per-attempt completion dict so a prot_p zombie can't
flip the flag for a later prot_c attempt. A cosmetic // prefix in the
directory-search download path is also fixed by replacing string
concatenation with posixpath.join.
The FTP upload skipped reading the server's "Transfer complete"
confirmation, sending the print command before the file was flushed
to the SD card. Now waits for the 226 response with a 60s timeout.
FTP upload: reduce chunk size from 1MB to 64KB for smooth progress bar
updates (~1s intervals instead of 20+ second gaps). Skip voidresp() for
all printer models — H2D delays the 226 response by 30+ seconds after
data transfer, causing a hang at 100%. Add transfer speed and TLS
handshake timing to logs for diagnosing slow connections.
Print/Schedule modal: widen from max-w-lg (512px) to max-w-2xl (672px)
to accommodate long filament profile names like "PLA Support for PETG
PETG Basic @Bambu Lab H2D 0.4 nozzle".
This allows users sending particularly large prints to slow printers such as the P1-series to start a print and then move onto another task in Bambuddy immediately (such as starting more prints on more printers). It also gives the user visibility into what's happening instead of a loading indicator appearing for an indefinite period of time.
The new toast-based UI uses websockets to update in real time. It will also appear for other users / instances of Bambuddy, not just the user who started the prints, allowing more transparency and handling cases where the user closes the page and then comes back wanting to know the status of the dispatching.
fix: review fixes for background dispatch PR #408
- Restore missing imports in main.py (inventory, print_log, virtual_printers, mqtt_smart_plug_service)
- Guard voidresp() for A1 printers to prevent hang after upload
- Don't fail upload on voidresp() error since data transfer already completed
- Add ams_mapping to register_expected_print calls for Spoolman usage tracking
- Fix cancel_job TOCTOU race by using single lock acquisition
- Fix batch counter reset TOCTOU by re-checking condition inside second lock
- Add backgroundDispatch translations to fr.ts and pt-BR.ts
- Remove dead upload_progress_callback definitions
- Skip redundant "Print queued" toast in reprint mode (dispatch toast handles it)
- 5 backend tests: cancel_job single-lock TOCTOU, batch reset re-check,
job lifecycle
- 2 FTP regression tests: voidresp error handling (upload-loop fix),
A1 model voidresp skip
- 1 frontend test: reprint toast suppression
- CHANGELOG: background dispatch feature + test coverage entries
- README: add background dispatch to Scheduling & Automation
- Website: add feature item to Automation section
- Wiki: add Background Print Dispatch section to print-queue.md
Dual-nozzle H2D/H2D Pro: filament matching now respects nozzle assignments
from the 3MF file. Each AMS unit feeds a specific nozzle (L/R), and the
scheduler/frontend constrain matching to only trays on the correct nozzle.
Falls back to unfiltered matching when no trays exist on the target nozzle.
L/R badges shown in the filament mapping UI. Translated in en/de/ja/it.
Fix AMS slot config overwritten on startup: on_ams_change unconditionally
unlinked BL spool assignments on every MQTT pushall, then re-assigned them
sending ams_filament_setting without setting_id — clearing the printer's
filament preset. Now compares spool RFID identifiers before unlinking.
Fix BL spool detection false positives: removed tray_info_idx from detection
logic in both backend is_bambu_lab_spool() and frontend isBambuLabSpool().
Third-party spools using Bambu generic presets had GF-prefixed tray_info_idx
values, causing misidentification. Now uses only tray_uuid and tag_uid.
SQLite WAL mode with 5s busy timeout reduces "database is locked" errors.
download_to_file() returned True when retrbinary transferred 0 bytes
without raising an exception. This caused the /cover endpoint to hit
the empty file check and raise HTTP 500 instead of retrying or
returning 404.
Now treats 0-byte downloads as failures, allowing the cover endpoint's
retry logic to work and falling back to 404 if all attempts fail.
The CodeQL cleanup in "Housekeeping" (2b11efd) bulk-narrowed except
clauses across 50+ files, breaking FTP uploads on ALL printer models.
ftplib.error_perm (550 errors) is not a subclass of ftplib.error_reply,
so diagnose_storage() CWD failures escaped the handler and prevented
STOR from ever executing — causing 100% upload failure and HTTP 500s
on /api/v1/archives/{id}/reprint and /api/v1/library/files/{id}/print.
FTP fixes:
- Remove diagnose_storage() from upload hot path
- Change all except (OSError, ftplib.error_reply) to
except (OSError, ftplib.Error) across bambu_ftp.py
Exception handling reverts (9 files):
- Revert narrowed except clauses back to except Exception in route
handlers and service code where broad catches are intentional
defensive programming (archive parsing, HTTP clients, 3MF/ZIP
processing, Home Assistant, firmware checks)
- Keep narrow exceptions only where safe (single-op blocks like
int(), file.unlink(), socket.close())
- Remove unused XMLParseError imports from archive.py, threemf_tools.py
Closes#287
The A1 printer's FTP server hangs when Python's storbinary() calls
voidresp() to wait for the server's completion response. This caused
upload timeouts on A1 and A1 Mini printers.
Fix contributed by an A1 user - replaces storbinary() with manual
chunked transfer using transfercmd() + sendall():
- Uses 1MB chunks (CHUNK_SIZE constant) for better throughput
- Sets explicit 120s socket timeout on data connection
- Manually closes connection after transfer, avoiding voidresp() hang
Applied to all printer models since the manual approach is compatible
with X1C/P1S/P1P as well (transfercmd is what storbinary uses internally).
User feedback indicated A1 Mini with current firmware works with prot_p
(protected/SSL data channel), not prot_c as previously assumed. Different
A1 firmware versions have different FTP SSL behavior.
Changes:
- Remove hardcoded assumption that A1 models need prot_c
- Try prot_p first for all models (including A1/A1 Mini)
- If upload/download fails on A1 models, automatically retry with prot_c
- Cache working mode per printer IP for subsequent operations
- Add force_prot_c parameter for explicit mode control
This makes FTP work across A1 firmware versions:
- New firmware: prot_p succeeds, cached
- Old firmware: prot_p fails → prot_c fallback succeeds, cached
The FTP code called prot_p() (protected data channel) for all printers,
but for A1/A1 Mini it didn't wrap the data connection in SSL. This
mismatch caused an immediate EOFError - server expected encrypted data
but received plain data.
Fix:
- Use prot_c() (clear/unencrypted data channel) for A1/A1 Mini
- Use prot_p() (protected/encrypted data channel) for X1C/P1S/etc
- Removed non-functional ftplib._SSLSocket = None workaround
A1/A1 Mini: control channel encrypted (implicit TLS), data channel clear
X1C/P1S/etc: both channels encrypted with SSL session reuse
Closes#271
Removed P1S and P1P from SKIP_SESSION_REUSE_MODELS
- These printers use vsFTPd which requires SSL session reuse on data channel
- Only A1/A1 Mini should skip session reuse (they have issues with SSL on data channel)
- P1S/P1P were incorrectly added in commit 9969005, causing EOFError on FTP upload
Closes#266
The fix for A1/P1S FTP uploads (commit 82a6025) was accidentally broken in
commit 9969005 which removed the skip_session_reuse parameter from the
ImplicitFTP_TLS constructor. This caused P2S (and other models in
SKIP_SESSION_REUSE_MODELS) to still use SSL on the data channel, resulting
in "426 Failure reading network stream" errors.
The fix was implemented in commit b96ecfa on test/issue_174 branch but
never merged to main. This cherry-picks that fix.
Also includes:
- Storage diagnostics for debugging upload issues
- Better FTP error logging with specific error codes (553, 550, 552)
- Improved error messages in print scheduler
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Printers without an SD card store files in the root folder `/` instead of
`/cache`. Added root folder to search paths in both main.py and bambu_ftp.py
so 3MF files can be found regardless of storage configuration.
Closes#146
P2S printer creates fallback archives without thumbnails/metadata because
FTP couldn't find 3MF files. The P2S uses different directory structure
(/data/Metadata/) and doesn't have a /cache directory.
Changes:
- Add P2S to SKIP_SESSION_REUSE_MODELS for SSL compatibility
- Add /data/ and /data/Metadata/ to 3MF download paths
- Update fallback search to try multiple directories instead of just /cache
- Add /data paths to FTP storage scan directories
Closes#146
The previous fix only skipped SSL session reuse for A1 printers but
still wrapped the data channel in SSL. VREmma's testing (issue #31)
showed that A1 printers need the data channel to not be SSL-wrapped
at all - they timeout waiting for transfer completion responses.
The fix:
- X1C/P1S: SSL with session reuse on data channel (required by vsFTPd)
- A1/A1 Mini: No SSL on data channel (control channel remains encrypted)
This matches VREmma's working fix (c723f25) which set ftplib._SSLSocket
to None, but implemented properly within the ImplicitFTP_TLS class.
Fixes#31
Thanks VREmma!
Resolves FTP transfer failures on A1/A1 Mini printers caused by SSL
session reuse incompatibility. Also adds configurable FTP timeout
setting for slow WiFi connections.
Changes:
- Add SSL session reuse bypass for A1/A1 Mini printers (auto-detected)
- Add printer_model parameter to FTP functions for model detection
- Add configurable FTP timeout (10-120s, default 30s) in Settings
- Update all FTP callers: archiving, reprint, timelapse, firmware upload
Technical details:
- A1/A1 Mini printers don't support vsFTPd's SSL session reuse
- X1C/P1S continue to use session reuse as required by their FTP server
- ImplicitFTP_TLS now accepts skip_session_reuse flag
- BambuFTPClient auto-detects A1/A1 Mini and sets flag accordingly
maziggy