The '========== Updated Releases ==========' header was a fixed 38
chars regardless of the table width below it. Now the divider line
is extended to match the table's visual width, with the title text
centered within the '=' borders.
- Add TableVisualWidth() to measure table width via runewidth
- Add HeaderDividerCentered() and HeaderDividerCenteredStyled()
for centered dividers with optional bold+blue ANSI styling
- Refactor DisplayAffectedReleases to build the table first, then
compute its width before logging the header
- Update all test snapshots and integration test output files
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat: parallel kubedog tracking with progress printer and safety valves
Rework kubedog integration so resource tracking runs in parallel with
helm upgrade/install, giving live progress output and recovering from
known helm/kubedog wedge conditions.
Core:
- kubedogTrackingHandle runs tracking in a background goroutine alongside
the helm subprocess (startBackgroundKubedogTracking); helm output is
buffered and replayed as a single block so it no longer interleaves
with progress ticks.
- Capture UID+generation baselines before handing off to helm so each
tracker waits until the resource actually changes (freshness gate).
Progress printer (pkg/kubedog/printer.go):
- Styled, auto-sized progress table with a heartbeat flusher, child (pod)
status roll-up, pre-ready pod-phase handling, multi-namespace support,
and optional color. PreviewBreakdown summarizes kept/filtered resources.
Safety valves (verify cluster state via the live API):
- Tracker-race valve (always on): when helm succeeds but a dyntracker
goroutine is wedged, poll the API and cancel the tracker so wait()
returns success instead of blocking until --track-timeout.
- Helm-stuck killer (opt-in via helmStuckGrace): if the cluster stays
converged while helm v4's hook waiter is wedged, SIGINT the helm
subprocess to recover.
- Failure watchdog (pkg/kubedog/watchdog.go): surface failing pods that
never made it into dyntracker's resource graph.
Options: trackFailedLogs, helmStuckGrace, trackTimeout, color
(Color/NoColor), and resource filtering (trackKinds/skipKinds/
trackResources). PersistentVolumeClaim support in resource classification.
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
* test: add unit tests for kubedog tracking
Cover the progress printer, resource classification, the failure
watchdog, helm-output trimming/dedup, the release hard-timeout helper,
and the color/track option plumbing.
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
* test: update golden logs, e2e snapshots, and values-id fixtures
Refresh pkg/app testapply/testdestroy golden logs, e2e template
snapshots, and the TestGenerateID values-id golden hashes for the new
kubedog progress output and the merged release struct layout.
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
* refactor: remove dead kubedog display code and dedupe tracker setup
Deep-review pass on the parallel kubedog tracking changes:
- Remove pkg/kubedog/display.go (308 lines) and display_test.go (453 lines).
These rendered progress for the legacy per-kind trackers that this PR
replaces; in the merged tree every function is unreferenced outside its
own tests. The new progressPrinter (printer.go) supersedes them.
TestMain (color.ForceColor for deterministic ANSI in tests) is preserved
in a new main_test.go.
- Dedupe trackWithKubedog: the post-helm fallback rebuilt the exact same
tracker options as buildReleaseTracker. Reuse buildReleaseTracker instead,
dropping ~40 lines of duplicated timeout/log/filter/tracker construction.
No behavior change; build, go vet, golangci-lint, and the kubedog/state
unit tests all pass.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: stop waitForFreshness busy-looping the API after helm finishes
Once upstreamDoneCh closes it is always ready, so the select in
waitForFreshness stopped blocking on the ticker and re-ran probe() (a
live GET) as fast as the round-trip allowed for the whole 3s grace
window — hammering the API server once per tracked resource.
Track a local view of the channel and nil it out on first delivery so
the first hit records the timestamp (one fast retry, as intended) and
all subsequent polls are ticker-throttled. Functional behavior is
unchanged: return nil when fresh, errUpstreamDoneNoChange after grace.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: drop trailing blank line from helm4 OCI pull snapshots
The trailing-newline trim in helmexec.info() removes the blank line helm
prints after the OCI chart "Digest:" line. The helm3 (output.yaml)
snapshots never captured that line, but the helm4 (output-helm4.yaml)
snapshots for oci_chart_pull{,_direct,_once,_once2} and
issue_473_oci_chart_url_fetch still expected it, so they failed under
helm 4. Remove the blank line so the snapshots match the trimmed output.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: refresh diff-args integration goldens for styled release headers
DisplayAffectedReleases now emits a styled "========== Updated Releases
==========" header (matching the app/e2e goldens already updated by this
PR) and helmexec.info() trims the trailing blank after helm's install
status. Update the diff-args apply-stderr{,-helm4} and apply-live-
stderr{,-helm4} goldens accordingly so they match the actual stderr.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: drop trimmed blank line from v1-subhelmfile template golden
The trailing-newline trim in helmexec.info() removes the blank line helm
prints after '"incubator" has been added to your repositories'. Update
the v1-subhelmfile-multi-bases-with-array-values result and result-live
goldens so the template stdout comparison matches.
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: Roman Mykhailiuk <romanm@cybellum.com>
* fix: Fix broken `trackLogs` functionality in Kubedog tracker
When using helmfile with the following setup
```
releases:
- name: my-app
chart: ./mychart
trackMode: kubedog
trackLogs: true
trackFailOnError: true
```
We don't actually see any logs from the container being printed.
This is because when building the options for the Kubedog tracker, we never
specify `SaveLogsOnlyForNumberOfReplicas` which means this defaults to 0.
Looking at the logic in `pkg/tracker/deployment/tracker.go` we see
```
ignoreLogs := job.ignoreLogs || job.savingLogsReplicas >= job.SaveLogsOnlyForNumberOfReplicas
```
With job.SaveLogsOnlyForNumberOfReplicas always defaulting to 0, this will always ignore logs
This change sets it to a reasonable default of tracking logs from up to 10 pods.
Signed-off-by: Graeme Gillies <ggillies@gitlab.com>
* fix formatting
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Graeme Gillies <ggillies@gitlab.com>
* feat: Refactor out building tracker opts into it's own function, add tests
Signed-off-by: Graeme Gillies <ggillies@gitlab.com>
---------
Signed-off-by: Graeme Gillies <ggillies@gitlab.com>
Co-authored-by: Graeme Gillies <ggillies@gitlab.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* build(deps): replace werf/kubedog-for-werf-helm with werf/kubedog
Replace the fork github.com/werf/kubedog-for-werf-helm with the upstream
github.com/werf/kubedog. The fork was a temporary compatibility shim; the
upstream repository now includes the necessary k8s API fixes.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix(kubedog): initialize InformerFactory to prevent nil pointer panic
The upstream werf/kubedog now requires an InformerFactory for its resource
trackers (deployment, statefulset, daemonset, job, canary), but the
multitrack layer still passes nil. Bypass the broken multitrack feed layer
by creating resource trackers directly with a properly initialized
InformerFactory.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix(lint): correct misspelling of canceled
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
Add support for trackMode: helm-legacy to use Helm v4's --wait=legacy flag,
which maintains compatibility with Helm v3's wait behavior during migration.
Helm v4 changed the default --wait behavior from polling to a watcher-based
approach. This can cause issues with charts that have broken livenessProbe
configurations without startupProbe. The --wait=legacy flag preserves the
Helm v3 polling behavior for smoother migration.
Changes:
- Add TrackModeHelmLegacy constant in pkg/kubedog/options.go
- Use kubedog.TrackMode constants instead of raw strings in helmx.go
- Enhance appendWaitFlags to use --wait=legacy for Helm v4 when trackMode
is helm-legacy
- Add nil check for logger before logging warning
- Add version check with warning when helm-legacy is used with Helm v3
- Update validation in pkg/config to accept helm-legacy track mode
- Update command-line flags in cmd/apply.go and cmd/sync.go
- Add comprehensive documentation in docs/advanced-features.md
- Add thorough test coverage including warning message verification
Behavior:
- Helm v4 + helm-legacy: Uses --wait=legacy
- Helm v3 + helm-legacy: Falls back to --wait with warning
- Helm v4 + helm: Uses --wait (watcher mode)
- Any + kubedog: Skips --wait flag
Fixes#2464
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: Copilot <copilot@github.com>
yxxhero