mirror of
https://github.com/helmfile/helmfile.git
synced 2026-09-30 18:00:42 +02:00
* feat: parallel kubedog tracking with progress printer and safety valves Rework kubedog integration so resource tracking runs in parallel with helm upgrade/install, giving live progress output and recovering from known helm/kubedog wedge conditions. Core: - kubedogTrackingHandle runs tracking in a background goroutine alongside the helm subprocess (startBackgroundKubedogTracking); helm output is buffered and replayed as a single block so it no longer interleaves with progress ticks. - Capture UID+generation baselines before handing off to helm so each tracker waits until the resource actually changes (freshness gate). Progress printer (pkg/kubedog/printer.go): - Styled, auto-sized progress table with a heartbeat flusher, child (pod) status roll-up, pre-ready pod-phase handling, multi-namespace support, and optional color. PreviewBreakdown summarizes kept/filtered resources. Safety valves (verify cluster state via the live API): - Tracker-race valve (always on): when helm succeeds but a dyntracker goroutine is wedged, poll the API and cancel the tracker so wait() returns success instead of blocking until --track-timeout. - Helm-stuck killer (opt-in via helmStuckGrace): if the cluster stays converged while helm v4's hook waiter is wedged, SIGINT the helm subprocess to recover. - Failure watchdog (pkg/kubedog/watchdog.go): surface failing pods that never made it into dyntracker's resource graph. Options: trackFailedLogs, helmStuckGrace, trackTimeout, color (Color/NoColor), and resource filtering (trackKinds/skipKinds/ trackResources). PersistentVolumeClaim support in resource classification. Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com> * test: add unit tests for kubedog tracking Cover the progress printer, resource classification, the failure watchdog, helm-output trimming/dedup, the release hard-timeout helper, and the color/track option plumbing. Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com> * test: update golden logs, e2e snapshots, and values-id fixtures Refresh pkg/app testapply/testdestroy golden logs, e2e template snapshots, and the TestGenerateID values-id golden hashes for the new kubedog progress output and the merged release struct layout. Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com> * refactor: remove dead kubedog display code and dedupe tracker setup Deep-review pass on the parallel kubedog tracking changes: - Remove pkg/kubedog/display.go (308 lines) and display_test.go (453 lines). These rendered progress for the legacy per-kind trackers that this PR replaces; in the merged tree every function is unreferenced outside its own tests. The new progressPrinter (printer.go) supersedes them. TestMain (color.ForceColor for deterministic ANSI in tests) is preserved in a new main_test.go. - Dedupe trackWithKubedog: the post-helm fallback rebuilt the exact same tracker options as buildReleaseTracker. Reuse buildReleaseTracker instead, dropping ~40 lines of duplicated timeout/log/filter/tracker construction. No behavior change; build, go vet, golangci-lint, and the kubedog/state unit tests all pass. Signed-off-by: yxxhero <aiopsclub@163.com> * fix: stop waitForFreshness busy-looping the API after helm finishes Once upstreamDoneCh closes it is always ready, so the select in waitForFreshness stopped blocking on the ticker and re-ran probe() (a live GET) as fast as the round-trip allowed for the whole 3s grace window — hammering the API server once per tracked resource. Track a local view of the channel and nil it out on first delivery so the first hit records the timestamp (one fast retry, as intended) and all subsequent polls are ticker-throttled. Functional behavior is unchanged: return nil when fresh, errUpstreamDoneNoChange after grace. Signed-off-by: yxxhero <aiopsclub@163.com> * test: drop trailing blank line from helm4 OCI pull snapshots The trailing-newline trim in helmexec.info() removes the blank line helm prints after the OCI chart "Digest:" line. The helm3 (output.yaml) snapshots never captured that line, but the helm4 (output-helm4.yaml) snapshots for oci_chart_pull{,_direct,_once,_once2} and issue_473_oci_chart_url_fetch still expected it, so they failed under helm 4. Remove the blank line so the snapshots match the trimmed output. Signed-off-by: yxxhero <aiopsclub@163.com> * test: refresh diff-args integration goldens for styled release headers DisplayAffectedReleases now emits a styled "========== Updated Releases ==========" header (matching the app/e2e goldens already updated by this PR) and helmexec.info() trims the trailing blank after helm's install status. Update the diff-args apply-stderr{,-helm4} and apply-live- stderr{,-helm4} goldens accordingly so they match the actual stderr. Signed-off-by: yxxhero <aiopsclub@163.com> * test: drop trimmed blank line from v1-subhelmfile template golden The trailing-newline trim in helmexec.info() removes the blank line helm prints after '"incubator" has been added to your repositories'. Update the v1-subhelmfile-multi-bases-with-array-values result and result-live goldens so the template stdout comparison matches. Signed-off-by: yxxhero <aiopsclub@163.com> --------- Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com> Signed-off-by: yxxhero <aiopsclub@163.com> Co-authored-by: Roman Mykhailiuk <romanm@cybellum.com>
83 lines
7.8 KiB
Go
83 lines
7.8 KiB
Go
package cmd
|
|
|
|
import (
|
|
"github.com/spf13/cobra"
|
|
|
|
"github.com/helmfile/helmfile/pkg/app"
|
|
"github.com/helmfile/helmfile/pkg/config"
|
|
)
|
|
|
|
// NewSyncCmd returns sync subcmd
|
|
func NewSyncCmd(globalCfg *config.GlobalImpl) *cobra.Command {
|
|
syncOptions := config.NewSyncOptions()
|
|
|
|
cmd := &cobra.Command{
|
|
Use: "sync",
|
|
Short: "Sync releases defined in state file",
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
syncImpl := config.NewSyncImpl(globalCfg, syncOptions)
|
|
err := config.NewCLIConfigImpl(syncImpl.GlobalImpl)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
if err := syncImpl.ValidateConfig(); err != nil {
|
|
return err
|
|
}
|
|
|
|
a := app.New(syncImpl)
|
|
return toCLIError(syncImpl.GlobalImpl, a.Sync(syncImpl))
|
|
},
|
|
}
|
|
|
|
f := cmd.Flags()
|
|
f.StringVar(&globalCfg.GlobalOptions.Args, "args", "", "pass args to helm sync")
|
|
f.StringVar(&syncOptions.SyncArgs, "sync-args", "", "pass args to helm upgrade")
|
|
f.StringArrayVar(&syncOptions.Set, "set", nil, "additional values to be merged into the helm command --set flag")
|
|
f.StringArrayVar(&syncOptions.Values, "values", nil, "additional value files to be merged into the helm command --values flag")
|
|
f.IntVar(&syncOptions.Concurrency, "concurrency", 0, "maximum number of concurrent helm processes to run, 0 is unlimited")
|
|
f.BoolVar(&syncOptions.Validate, "validate", false, "validate your manifests against the Kubernetes cluster you are currently pointing at. Note that this requires access to a Kubernetes cluster to obtain information necessary for validating, like the sync of available API versions")
|
|
f.BoolVar(&syncOptions.SkipNeeds, "skip-needs", true, `do not automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided. Defaults to true when --include-needs or --include-transitive-needs is not provided`)
|
|
f.BoolVar(&syncOptions.SkipCRDs, "skip-crds", false, "if set, no CRDs will be installed on sync. By default, CRDs are installed if not already present")
|
|
f.BoolVar(&syncOptions.IncludeNeeds, "include-needs", false, `automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided`)
|
|
f.BoolVar(&syncOptions.IncludeTransitiveNeeds, "include-transitive-needs", false, `like --include-needs, but also includes transitive needs (needs of needs). Does nothing when --selector/-l flag is not provided. Overrides exclusions of other selectors and conditions.`)
|
|
f.BoolVar(&syncOptions.EnforceNeedsAreInstalled, "enforce-needs-are-installed", false, "enforce that all 'needs' dependencies are installable before applying changes")
|
|
f.BoolVar(&syncOptions.HideNotes, "hide-notes", false, "add --hide-notes flag to helm")
|
|
f.BoolVar(&syncOptions.TakeOwnership, "take-ownership", false, `add --take-ownership flag to helm`)
|
|
f.StringVar(&syncOptions.ServerSide, "server-side", "", `add --server-side flag to helm upgrade (Helm 4 only). Must be "true", "false", or "auto"`)
|
|
f.BoolVar(&syncOptions.SyncReleaseLabels, "sync-release-labels", false, "sync release labels to the target release")
|
|
f.BoolVar(&syncOptions.Wait, "wait", false, `Override helmDefaults.wait setting "helm upgrade --install --wait"`)
|
|
f.BoolVar(&syncOptions.WaitForJobs, "wait-for-jobs", false, `Override helmDefaults.waitForJobs setting "helm upgrade --install --wait-for-jobs"`)
|
|
f.IntVar(&syncOptions.Timeout, "timeout", 0, `Override helmDefaults.timeout in seconds for "helm upgrade --install --timeout" (default 0, which uses helmDefaults.timeout or helm's default if not set)`)
|
|
f.BoolVar(&syncOptions.ReuseValues, "reuse-values", false, `Override helmDefaults.reuseValues "helm upgrade --install --reuse-values"`)
|
|
f.BoolVar(&syncOptions.ResetValues, "reset-values", false, `Override helmDefaults.reuseValues "helm upgrade --install --reset-values"`)
|
|
f.StringVar(&syncOptions.PostRenderer, "post-renderer", "", `pass --post-renderer to "helm template" or "helm upgrade --install"`)
|
|
f.StringArrayVar(&syncOptions.PostRendererArgs, "post-renderer-args", nil, `pass --post-renderer-args to "helm template" or "helm upgrade --install"`)
|
|
f.BoolVar(&syncOptions.SkipSchemaValidation, "skip-schema-validation", false, `pass --skip-schema-validation to "helm template" or "helm upgrade --install"`)
|
|
f.StringVar(&syncOptions.Cascade, "cascade", "", "pass cascade to helm exec, default: background")
|
|
f.StringVar(&syncOptions.TrackMode, "track-mode", "", "Track mode for releases: 'helm' (default), 'helm-legacy' (Helm v4 only), or 'kubedog'")
|
|
f.IntVar(&syncOptions.TrackTimeout, "track-timeout", 0, `Timeout in seconds for kubedog tracking (0 to use default 300s timeout)`)
|
|
f.BoolVar(&syncOptions.TrackLogs, "track-logs", false, "Enable log streaming with kubedog tracking (all pods)")
|
|
f.BoolVar(&syncOptions.TrackFailedLogs, "track-failed-logs", false, "Enable log streaming with kubedog tracking, but only emit logs for pods that enter a failed state. Overridden by --track-logs when both are set")
|
|
f.IntVar(&syncOptions.HelmStuckGrace, "helm-stuck-grace", 0, "When using --track-mode kubedog: if the cluster confirms all tracked resources have converged but the helm subprocess is still running, wait this many seconds before sending SIGINT to helm. Recovers from helm v4 hook waiter wedges. May leave the release secret in pending-install state requiring manual cleanup. 0 disables.")
|
|
f.BoolVar(&syncOptions.TrackFailOnError, "track-fail-on-error", false, "Fail with non-zero exit code when kubedog tracking fails")
|
|
f.StringVar(&syncOptions.Description, "description", "", `Set description for all releases. If set, overrides descriptions in helmfile.yaml. Will be passed to "helm upgrade --description"`)
|
|
|
|
// Diff-related flags for --interactive mode
|
|
f.IntVar(&syncOptions.Context, "context", 0, "output NUM lines of context around changes (interactive preview only)")
|
|
f.StringVar(&syncOptions.DiffOutput, "output", "", "output format for diff plugin (interactive preview only)")
|
|
f.StringVar(&syncOptions.DiffArgs, "diff-args", "", "pass args to helm-diff (interactive preview only)")
|
|
f.StringArrayVar(&syncOptions.Suppress, "suppress", nil, "suppress specified Kubernetes objects in the diff output (interactive preview only). Can be provided multiple times. For example: --suppress KeycloakClient --suppress VaultSecret")
|
|
f.BoolVar(&syncOptions.SuppressSecrets, "suppress-secrets", false, "suppress secrets in the diff output (interactive preview only). highly recommended to specify on CI/CD use-cases")
|
|
f.BoolVar(&syncOptions.ShowSecrets, "show-secrets", false, "do not redact secret values in the diff output (interactive preview only). should be used for debug purpose only")
|
|
f.BoolVar(&syncOptions.NoHooks, "no-hooks", false, "do not diff changes made by hooks (interactive preview only)")
|
|
f.BoolVar(&syncOptions.SuppressDiff, "suppress-diff", false, "suppress diff in the output (interactive preview only). Usable in new installs")
|
|
f.BoolVar(&syncOptions.SkipDiffOnInstall, "skip-diff-on-install", false, "Skips running helm-diff on releases being newly installed on this sync (interactive preview only). Useful when the release manifests are too huge to be reviewed, or it's too time-consuming to diff at all")
|
|
f.BoolVar(&syncOptions.IncludeTests, "include-tests", false, "enable the diffing of the helm test hooks (interactive preview only)")
|
|
f.BoolVar(&syncOptions.DetailedExitcode, "detailed-exitcode", false, "return a non-zero exit code 2 instead of 0 when releases are synced (use --interactive to also see a diff preview)")
|
|
f.BoolVar(&syncOptions.StripTrailingCR, "strip-trailing-cr", false, "strip trailing carriage return on input (interactive preview only)")
|
|
f.StringArrayVar(&syncOptions.SuppressOutputLineRegex, "suppress-output-line-regex", nil, "a list of regex patterns to suppress output lines from diff output (interactive preview only)")
|
|
|
|
return cmd
|
|
}
|