ForgejoBackend.test_connection asked GET /user who the token belonged to
before asking whether the token could reach the repository, and treated a 403
there as fatal. A Forgejo v15 repository-scoped token may only carry
read/write on issues and repositories, so it 403s on /user -- and was rejected
despite reaching its own repository fine, which is all a backup needs: the push
path uses the Contents API and restore reads commits, trees and blobs, all
under /repos/{owner}/{repo}. That /user call was the only one in the whole
provider layer.
The probe stays, because a 401 from it is genuinely conclusive and names a bad
token before the repo call has to guess -- Forgejo v15+ hides a private repo
behind 404 rather than 403, so the repo call cannot always tell those apart.
Every other status now falls through to the repo check.
Two additions keep the messages as sharp as before: the repo call's own 401 is
mapped to "Invalid access token" instead of a generic API error, and the 404
names write:repository and the scoped-to-another-repository case, mentioning a
possibly-invalid token only when /user did not confirm the identity.
The token hint under the field was one shared string reading "fine-grained
token with Contents read/write" -- GitHub's advice, shown to Gitea, Forgejo and
GitLab users too. It is now per provider via PROVIDER_TOKEN_HINT_I18N_KEY,
following the existing repo-URL placeholder map, translated in all 13 locales.
Tests pin the repository-scoped token connecting, a transient /user status not
blocking the repo call, both 404 wordings, and the repo-call 401; a frontend
test switches providers and asserts the hint follows.
`if not isinstance(total, int) or seen >= total or not entries: return blobs, ""`
— the first arm short-circuited the page loop into a **success** holding page 1
only. Gitea clamps `per_page` to `MAX_RESPONSE_ITEMS` (default 50), so that is
50 entries of an arbitrarily large tree returned as a complete listing.
The restore then reports genuinely-present categories as "Not present in this
backup commit". That silent skip is the exact failure this override exists to
prevent, and the same class as E7 and G2 — G2 fixed the arithmetic here and
left the shape. GitHub and GitLab both hard-fail in the equivalent spot; only
Gitea guessed, and it guessed in the one direction that loses data quietly.
Whether Gitea always sends `total_count` on this route is beside the point: the
code was defending against a response shape it did not trust, and then trusting
it.
Now a missing or non-int `total_count` means "page until a short or empty
page". A page shorter than the first one is the last one, floored at Gitea's
default clamp so a genuinely small tree still costs exactly one request — the
reason G2 rejected paging-until-short in the `total_count`-present case, which
is unchanged and still stops on the count. The existing `page <= 50` ceiling
gives the correct hard failure for a tree that really is over cap, so this
cannot truncate.
Residual, and deliberately not widened into a `return None` on the first
ambiguous response — that would break single-page trees, the common case: an
instance whose `MAX_RESPONSE_ITEMS` is set *below* 50 *and* which omits
`total_count` would still stop at page 1. Both halves have to be true.
Tests: +6 (paged to the end with no count, on both Gitea and Forgejo; a short
page ends it; a non-int count is treated as no count; the page ceiling still
fails). Fail-pre-fix 5, control that passes either way 1 (a small tree is one
request). These don't match `-k github`, so 274 -> 280 across the three restore
files but `-k github` is unmoved.
The pager computed `seen = (page - 1) * 1000 + len(entries)`, taking the
requested `per_page` as fact. Gitea clamps `per_page` to
`MAX_RESPONSE_ITEMS`, which defaults to 50. So on a default install page 1
returns 50 entries and sets `seen` to 50, then page 2 sets it to 1050 —
which clears any `total_count` under 1050. The loop returns `success: true`
holding the first 100 entries of a much larger tree.
The restore then reads every missing path as "category not present in this
commit" and skips it silently, which is precisely the failure this override
was written to prevent. Same class as the GitLab pager fix, in the one
direction that got left behind.
Fix: accumulate `seen += len(entries)`. A genuinely over-cap tree still
hard-fails rather than truncating; the cap is a page count, not a file
count, because the page size is the server's choice.
Test: a 120-entry tree served 50 at a time reaches its last entry, in three
requests. Confirmed failing against the pre-fix backend — it stopped after
two pages and reported 100 entries as the whole tree.
The three remaining review items, all in the read path.
E1 — four provider calls where two would do. preview() called list_commits
twice: once inside _resolve_ref to turn HEAD into a SHA, once more at limit=20
purely to find the entry describing that same SHA. And list_tree's recursive
tree GET was thrown away, so fetch_files immediately fetched the identical tree
again to map path -> blob SHA. _resolve_ref now returns the entry it already
has, and list_tree returns its blob_shas map for fetch_files to take as an
optional argument. GitLab reads files by path and ignores it.
E2 — `commit: null` for a ref outside the 20 most recent. Two causes, and the
second is the one that actually bit: REF_PATTERN accepts a 7-character ref while
providers return the full 40, so the exact `==` in the scan never matched an
abbreviated SHA *even when the commit was in the window*. Fixed by prefix
comparison, plus a get_commit(ref) on the GitHub and GitLab backends for the
genuinely-outside-the-window case. Gitea and Forgejo inherit GitHub's. Still
best-effort: it is a subject line and a date, so a failed lookup renders the
preview without them rather than failing it.
E7 — the two tree readers disagreed, and each was wrong in the other's
direction. GitHub's recursive trees endpoint is not paginated and signals
overflow with truncated=true, which _blob_shas_at hard-fails on. Gitea and
Forgejo *do* page that endpoint, and inherited that single GET unchanged — so a
large backup repo returned only the first page and every category beyond it
looked absent from the commit. GiteaBackend now has its own paging
_blob_shas_at. GitLab had the mirror-image bug the review did not name: at its
50-page cap it exited through the while condition and returned success: True
with a silently partial path list. Both now fail loudly, which is what the
GitHub version was always doing.
Both halves of E7 are the same failure the module already refuses to allow: a
restore that skips categories and calls it "not present in this backup commit".
24 new or changed tests, all failing against this commit's parent.
The Git backup feature was push-only: there was no equivalent of the local
backup's Restore button, so recovering meant hand-downloading JSON files from
the repository. This adds the read side.
Providers gain list_commits / list_tree / fetch_files on the GitProviderBackend
ABC. GitHub implements them against the Git Data API and Gitea/Forgejo inherit
that unchanged; GitLab overrides for its own REST shape, including tree
pagination and subgroup path encoding. fetch_files is batched so the path ->
blob SHA lookup happens once per restore rather than once per file, and uses the
blobs API rather than contents because contents silently inlines only the first
1 MB.
The new GitHubRestoreService resolves HEAD to a concrete SHA up front, so a
preview and the restore that follows act on the same commit even if a scheduled
backup lands in between. Categories are applied archives -> spools -> settings
-> kprofiles: archives first because spool usage history references archive_id,
K-profiles last because they leave the database and publish over MQTT.
Restores never reuse the backup's primary keys. spool.id and print_archives.id
are bare autoincrement columns, so ids from an old backup very likely belong to
unrelated rows today; rows are matched on natural keys (tag_uid, then
tray_uuid, then a descriptive composite for spools; content_hash or filename
plus started_at for archives), inserted without an explicit id, and an
old_id -> new_id map rewrites the foreign keys in spool usage history.
created_at is carried across on insert so restoring the same backup twice
matches instead of duplicating. Dangling printer/project links are cleared and
reported rather than failing the row.
Settings restore re-applies the collector's credential denylist on the read
side, plus a pattern guard, because a backup taken before that denylist existed
can still contain secrets. Restored archives are metadata-only: the 3MF and
thumbnail bytes are not in a Git backup and print_archives.file_path is NOT
NULL, so inserted rows get an empty path and the UI says so.
Backup and restore take a mutex against each other; both write the same tables
and talk to the same printers. Restores are logged as GitHubBackupLog rows with
trigger="restore", which needs no migration and surfaces them in the existing
History card.
Cloud profiles are deliberately not a restore category. The collector never
actually writes cloud_profiles/*.json - it reads a "setting" list key the Bambu
Cloud API does not return - and the preset list it would write carries no
setting payload. Filed separately.
Permission github:restore already existed and is granted to Administrators, so
no permission changes were needed.
Tests: 125 new backend tests (provider reads across all four providers, the
per-category appliers, the API endpoints) and 13 frontend tests. Full suites
pass with no regressions; the 35 backend failures on Windows are byte-identical
with and without this branch.
Gitea/Forgejo served under a ROOT_URL path prefix (e.g. https://host/gitea)
place repos at /<prefix>/owner/repo. The Gitea backend assumed a host-root
layout: parse_repo_url required exactly two path segments (so subpath URLs
raised "Cannot parse repository URL") and get_api_base dropped the prefix,
yielding https://host/api/v1 instead of https://host/gitea/api/v1.
Treat the final two path segments as owner/repo and keep leading segments as
a base-path prefix; derive the API base as {scheme}://{host}{prefix}/api/v1.
Root-hosted instances are unaffected. Forgejo inherits the fix; GitHub/GitLab
are untouched.
While auditing real-world Bambuddy backup repos on GitHub I found
several left public. That's a serious leak: the settings backup only
filters bambu_cloud_token and auth_secret_key, so mqtt_username,
mqtt_password, ha_token, prometheus_token, bambu_cloud_email,
external_url, and the printer access codes (via K-profiles) were going
to whatever visibility the user picked.
Hard guard at every save and re-checked on every push:
- POST /github-backup/config and PATCH /github-backup/config (when URL,
token, or provider changes) run a connection test internally and
return 400 unless is_private comes back True.
- run_backup() re-checks before each scheduled or manual push, so a
repository that flipped from private to public gets a clear
"Backup aborted: the target repository is no longer private" failure.
Each provider's test_connection now returns is_private (GitHub /
Gitea / Forgejo read data.private, GitLab reads visibility=="private";
"internal" is treated as non-private). None means "couldn't determine"
and is also rejected -- safer to fail closed.
Frontend renders visibility inline on Test Connection: green check when
private, red warning panel listing every credential at risk when public,
yellow when unknown.
---
ui(github-backup): show save-failure messages inline on the card
The new "repository is not private" rejection message is ~250 characters
listing every credential the backup carries (MQTT password, HA token,
Prometheus token, Bambu Cloud email, printer access codes), which clips
badly in a toast.
Both the initial-setup save and the debounced autosave now stash the
backend's error message into a saveError state and render it as a red
inline banner above the test-result block, with whitespace-pre-wrap so
the full message stays readable. The banner clears on success, on the
next save attempt, and when the user starts editing URL / token / provider
-- the three fields whose changes invalidate the privacy check -- so it
doesn't linger after the user has already addressed the cause.
Short success toasts (Settings saved, Token updated, Backup enabled) are
unchanged.
feat(#1239): first cut at Gitea backups silently failing after 1st run
feat(#1239): Added Token Scope for Forgejo edge case. Also included: test coverage for fixes
Subsequent backups against Gitea 1.24+ failed with the opaque
"Backup failed: 'tree'" message after the initial-backup fix landed in
7ee89b56. Root cause: Gitea's GET /repos/{owner}/{repo}/git/commits/{sha}
returns the wrapped Commit schema where the tree lives at
data["commit"]["tree"]["sha"], whereas GitHub's same-named Git Database
endpoint returns the unwrapped GitCommit schema with tree at the top
level. The bare commit_response.json()["tree"]["sha"] lookup at
gitea.py:109 raised KeyError: 'tree' and the broad except in push_files
surfaced it as the opaque "Backup failed: 'tree'" string — masking the
real shape mismatch.
Adds a _commit_tree_sha() helper that tries the flat shape first
(GitHub-compatible / older Gitea) and falls back to the wrapped shape
(Gitea 1.24+, Forgejo). Returns None on truly malformed responses;
push_files maps that to a clear "Failed to extract tree SHA from commit
response" instead of leaking a KeyError repr. Keeps the existing-files
diff working on both shapes so subsequent backups don't re-upload every
blob — preferred over the .get()-and-skip approach which would have
required also dropping base_tree from the tree POST and re-uploading
unchanged files on every backup.
Two interacting bugs in the Gitea/Forgejo backend, both inherited from
GitHubBackend because PR #1160 assumed Gitea's Git Data API was fully
GitHub-compatible. It isn't, on two specific points:
1. List-shaped ref response. Gitea/Forgejo's
GET /api/v1/repos/{owner}/{repo}/git/refs/heads/{branch} returns a
GET /api/v1/repos/{owner}/{repo}/git/refs/heads/{branch} returns a
list of matching refs even when only one matches; GitHub returns a
single object. The inherited push paths did
ref_response.json()["object"]["sha"] and crashed with
"list indices must be integers or slices, not str" against any
populated Gitea repo.
2. Empty-repo writes refused. GitHub accepts blob/tree/commit POSTs
against a brand-new empty repo and creates the initial commit
implicitly. Gitea refuses every blob POST with 404 until the repo
has at least one commit, so _create_initial_commit silently failed:
blobs returned 404, tree_items stayed empty, the tree POST then
also 404'd ("Failed to create tree").
Fix lives entirely in GiteaBackend — github.py is untouched so the
proven GitHub path takes zero risk. GiteaBackend now overrides
push_files, _create_branch_and_push, and _create_initial_commit:
- _ref_sha() helper accepts both list and dict shapes; called at the
two SHA extraction sites in push_files and _create_branch_and_push.
- _create_initial_commit posts to Gitea's Contents API
(POST /api/v1/repos/{owner}/{repo}/contents with a files array plus
branch + new_branch) which seeds the initial commit + branch in
one transaction and is documented to work on empty repos.
ForgejoBackend extends GiteaBackend with no overrides and inherits
both fixes; tests pin that.
maziggy