mirror of
https://github.com/helmfile/helmfile.git
synced 2026-09-30 09:33:41 +02:00
* feat: parallel kubedog tracking with progress printer and safety valves Rework kubedog integration so resource tracking runs in parallel with helm upgrade/install, giving live progress output and recovering from known helm/kubedog wedge conditions. Core: - kubedogTrackingHandle runs tracking in a background goroutine alongside the helm subprocess (startBackgroundKubedogTracking); helm output is buffered and replayed as a single block so it no longer interleaves with progress ticks. - Capture UID+generation baselines before handing off to helm so each tracker waits until the resource actually changes (freshness gate). Progress printer (pkg/kubedog/printer.go): - Styled, auto-sized progress table with a heartbeat flusher, child (pod) status roll-up, pre-ready pod-phase handling, multi-namespace support, and optional color. PreviewBreakdown summarizes kept/filtered resources. Safety valves (verify cluster state via the live API): - Tracker-race valve (always on): when helm succeeds but a dyntracker goroutine is wedged, poll the API and cancel the tracker so wait() returns success instead of blocking until --track-timeout. - Helm-stuck killer (opt-in via helmStuckGrace): if the cluster stays converged while helm v4's hook waiter is wedged, SIGINT the helm subprocess to recover. - Failure watchdog (pkg/kubedog/watchdog.go): surface failing pods that never made it into dyntracker's resource graph. Options: trackFailedLogs, helmStuckGrace, trackTimeout, color (Color/NoColor), and resource filtering (trackKinds/skipKinds/ trackResources). PersistentVolumeClaim support in resource classification. Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com> * test: add unit tests for kubedog tracking Cover the progress printer, resource classification, the failure watchdog, helm-output trimming/dedup, the release hard-timeout helper, and the color/track option plumbing. Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com> * test: update golden logs, e2e snapshots, and values-id fixtures Refresh pkg/app testapply/testdestroy golden logs, e2e template snapshots, and the TestGenerateID values-id golden hashes for the new kubedog progress output and the merged release struct layout. Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com> * refactor: remove dead kubedog display code and dedupe tracker setup Deep-review pass on the parallel kubedog tracking changes: - Remove pkg/kubedog/display.go (308 lines) and display_test.go (453 lines). These rendered progress for the legacy per-kind trackers that this PR replaces; in the merged tree every function is unreferenced outside its own tests. The new progressPrinter (printer.go) supersedes them. TestMain (color.ForceColor for deterministic ANSI in tests) is preserved in a new main_test.go. - Dedupe trackWithKubedog: the post-helm fallback rebuilt the exact same tracker options as buildReleaseTracker. Reuse buildReleaseTracker instead, dropping ~40 lines of duplicated timeout/log/filter/tracker construction. No behavior change; build, go vet, golangci-lint, and the kubedog/state unit tests all pass. Signed-off-by: yxxhero <aiopsclub@163.com> * fix: stop waitForFreshness busy-looping the API after helm finishes Once upstreamDoneCh closes it is always ready, so the select in waitForFreshness stopped blocking on the ticker and re-ran probe() (a live GET) as fast as the round-trip allowed for the whole 3s grace window — hammering the API server once per tracked resource. Track a local view of the channel and nil it out on first delivery so the first hit records the timestamp (one fast retry, as intended) and all subsequent polls are ticker-throttled. Functional behavior is unchanged: return nil when fresh, errUpstreamDoneNoChange after grace. Signed-off-by: yxxhero <aiopsclub@163.com> * test: drop trailing blank line from helm4 OCI pull snapshots The trailing-newline trim in helmexec.info() removes the blank line helm prints after the OCI chart "Digest:" line. The helm3 (output.yaml) snapshots never captured that line, but the helm4 (output-helm4.yaml) snapshots for oci_chart_pull{,_direct,_once,_once2} and issue_473_oci_chart_url_fetch still expected it, so they failed under helm 4. Remove the blank line so the snapshots match the trimmed output. Signed-off-by: yxxhero <aiopsclub@163.com> * test: refresh diff-args integration goldens for styled release headers DisplayAffectedReleases now emits a styled "========== Updated Releases ==========" header (matching the app/e2e goldens already updated by this PR) and helmexec.info() trims the trailing blank after helm's install status. Update the diff-args apply-stderr{,-helm4} and apply-live- stderr{,-helm4} goldens accordingly so they match the actual stderr. Signed-off-by: yxxhero <aiopsclub@163.com> * test: drop trimmed blank line from v1-subhelmfile template golden The trailing-newline trim in helmexec.info() removes the blank line helm prints after '"incubator" has been added to your repositories'. Update the v1-subhelmfile-multi-bases-with-array-values result and result-live goldens so the template stdout comparison matches. Signed-off-by: yxxhero <aiopsclub@163.com> --------- Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com> Signed-off-by: yxxhero <aiopsclub@163.com> Co-authored-by: Roman Mykhailiuk <romanm@cybellum.com>
340 lines
9.0 KiB
Go
340 lines
9.0 KiB
Go
package config
|
|
|
|
import (
|
|
"fmt"
|
|
"slices"
|
|
)
|
|
|
|
// SyncOptions is the options for the build command
|
|
type SyncOptions struct {
|
|
// Set is the set flag
|
|
Set []string
|
|
// Values is the values flag
|
|
Values []string
|
|
// Concurrency is the concurrency flag
|
|
Concurrency int
|
|
// Validate is the validate flag
|
|
Validate bool
|
|
// IncludeCRDs is the include crds flag
|
|
SkipNeeds bool
|
|
// IncludeNeeds is the include needs flag
|
|
IncludeNeeds bool
|
|
// IncludeTransitiveNeeds is the include transitive needs flag
|
|
IncludeTransitiveNeeds bool
|
|
// EnforceNeedsAreInstalled indicates whether to error when there are unmeetable dependencies
|
|
EnforceNeedsAreInstalled bool
|
|
// SkipCrds is the skip crds flag
|
|
SkipCRDs bool
|
|
// Wait is the wait flag
|
|
Wait bool
|
|
// WaitRetries is the wait retries flag
|
|
WaitRetries int
|
|
// WaitForJobs is the wait for jobs flag
|
|
WaitForJobs bool
|
|
// Timeout is the timeout flag in seconds
|
|
Timeout int
|
|
// ReuseValues is true if the helm command should reuse the values
|
|
ReuseValues bool
|
|
// ResetValues is true if helm command should reset values to charts' default
|
|
ResetValues bool
|
|
// Propagate '--post-renderer' to helmv3 template and helm install
|
|
PostRenderer string
|
|
// Propagate '--post-renderer-args' to helmv3 template and helm install
|
|
PostRendererArgs []string
|
|
// Propagate '--skip-schema-validation' to helmv3 template and helm install
|
|
SkipSchemaValidation bool
|
|
// Cascade '--cascade' to helmv3 delete, available values: background, foreground, or orphan, default: background
|
|
Cascade string
|
|
// SyncArgs is the list of arguments to pass to the helm upgrade command.
|
|
SyncArgs string
|
|
// HideNotes is the hide notes flag
|
|
HideNotes bool
|
|
// TakeOwnership is the take ownership flag
|
|
TakeOwnership bool
|
|
// ServerSide controls the helm 4 --server-side flag. Must be "true", "false", or "auto".
|
|
ServerSide string
|
|
// SyncReleaseLabels is the sync release labels flag
|
|
SyncReleaseLabels bool
|
|
// TrackMode specifies whether to use 'helm' or 'kubedog' for tracking resources
|
|
TrackMode string
|
|
// TrackTimeout specifies timeout for kubedog tracking (in seconds)
|
|
TrackTimeout int
|
|
// TrackLogs enables log streaming with kubedog
|
|
TrackLogs bool
|
|
// TrackFailedLogs streams logs only for pods that enter a failed state.
|
|
TrackFailedLogs bool
|
|
// HelmStuckGrace, when > 0, enables the helm-killer safety valve. See
|
|
// ReleaseSpec.HelmStuckGrace for details. Value is in seconds.
|
|
HelmStuckGrace int
|
|
// TrackFailOnError controls whether kubedog tracking failures cause a non-zero exit code
|
|
TrackFailOnError bool
|
|
// Description is the description that will be passed to helm upgrade --description
|
|
Description string
|
|
|
|
// Diff-related options for --interactive mode
|
|
SuppressOutputLineRegex []string
|
|
IncludeTests bool
|
|
Suppress []string
|
|
SuppressSecrets bool
|
|
ShowSecrets bool
|
|
NoHooks bool
|
|
SuppressDiff bool
|
|
SkipDiffOnInstall bool
|
|
DiffArgs string
|
|
DetailedExitcode bool
|
|
StripTrailingCR bool
|
|
Context int
|
|
DiffOutput string
|
|
}
|
|
|
|
// NewSyncOptions creates a new Apply
|
|
func NewSyncOptions() *SyncOptions {
|
|
return &SyncOptions{}
|
|
}
|
|
|
|
// SyncImpl is impl for applyOptions
|
|
type SyncImpl struct {
|
|
*GlobalImpl
|
|
*SyncOptions
|
|
}
|
|
|
|
// NewSyncImpl creates a new SyncImpl
|
|
func NewSyncImpl(g *GlobalImpl, t *SyncOptions) *SyncImpl {
|
|
return &SyncImpl{
|
|
GlobalImpl: g,
|
|
SyncOptions: t,
|
|
}
|
|
}
|
|
|
|
// Concurrency returns the concurrency
|
|
func (t *SyncImpl) Concurrency() int {
|
|
return t.SyncOptions.Concurrency
|
|
}
|
|
|
|
// IncludeNeeds returns the include needs
|
|
func (t *SyncImpl) IncludeNeeds() bool {
|
|
return t.SyncOptions.IncludeNeeds || t.IncludeTransitiveNeeds()
|
|
}
|
|
|
|
// IncludeTransitiveNeeds returns the include transitive needs
|
|
func (t *SyncImpl) IncludeTransitiveNeeds() bool {
|
|
return t.SyncOptions.IncludeTransitiveNeeds
|
|
}
|
|
|
|
// EnforceNeedsAreInstalled errors if the transitive dependencies are not installable
|
|
func (t *SyncImpl) EnforceNeedsAreInstalled() bool {
|
|
return t.SyncOptions.EnforceNeedsAreInstalled
|
|
}
|
|
|
|
// Set returns the Set
|
|
func (t *SyncImpl) Set() []string {
|
|
return t.SyncOptions.Set
|
|
}
|
|
|
|
// SkipNeeds returns the skip needs
|
|
func (t *SyncImpl) SkipNeeds() bool {
|
|
if !t.IncludeNeeds() {
|
|
return t.SyncOptions.SkipNeeds
|
|
}
|
|
|
|
return false
|
|
}
|
|
|
|
// Validate returns the validate
|
|
func (t *SyncImpl) Validate() bool {
|
|
return t.SyncOptions.Validate
|
|
}
|
|
|
|
// Values returns the values
|
|
func (t *SyncImpl) Values() []string {
|
|
return t.SyncOptions.Values
|
|
}
|
|
|
|
// SkipCRDS returns the skip crds
|
|
func (t *SyncImpl) SkipCRDs() bool {
|
|
return t.SyncOptions.SkipCRDs
|
|
}
|
|
|
|
// Wait returns the wait
|
|
func (t *SyncImpl) Wait() bool {
|
|
return t.SyncOptions.Wait
|
|
}
|
|
|
|
// WaitRetries returns the wait retries
|
|
func (t *SyncImpl) WaitRetries() int {
|
|
return t.SyncOptions.WaitRetries
|
|
}
|
|
|
|
// WaitForJobs returns the wait for jobs
|
|
func (t *SyncImpl) WaitForJobs() bool {
|
|
return t.SyncOptions.WaitForJobs
|
|
}
|
|
|
|
// Timeout returns the timeout
|
|
func (t *SyncImpl) Timeout() int {
|
|
return t.SyncOptions.Timeout
|
|
}
|
|
|
|
// ReuseValues returns the ReuseValues.
|
|
func (t *SyncImpl) ReuseValues() bool {
|
|
if !t.ResetValues() {
|
|
return t.SyncOptions.ReuseValues
|
|
}
|
|
return false
|
|
}
|
|
func (t *SyncImpl) ResetValues() bool {
|
|
return t.SyncOptions.ResetValues
|
|
}
|
|
|
|
// PostRenderer returns the PostRenderer.
|
|
func (t *SyncImpl) PostRenderer() string {
|
|
return t.SyncOptions.PostRenderer
|
|
}
|
|
|
|
// PostRendererArgs returns the PostRendererArgs.
|
|
func (t *SyncImpl) PostRendererArgs() []string {
|
|
return t.SyncOptions.PostRendererArgs
|
|
}
|
|
|
|
// SkipSchemaValidation returns the SkipSchemaValidation.
|
|
func (t *SyncImpl) SkipSchemaValidation() bool {
|
|
return t.SyncOptions.SkipSchemaValidation
|
|
}
|
|
|
|
// Cascade returns cascade flag
|
|
func (t *SyncImpl) Cascade() string {
|
|
return t.SyncOptions.Cascade
|
|
}
|
|
|
|
// SyncArgs returns the sync args
|
|
func (t *SyncImpl) SyncArgs() string {
|
|
return t.SyncOptions.SyncArgs
|
|
}
|
|
|
|
// HideNotes returns the hide notes
|
|
func (t *SyncImpl) HideNotes() bool {
|
|
return t.SyncOptions.HideNotes
|
|
}
|
|
|
|
// TakeOwnership returns the take ownership
|
|
func (t *SyncImpl) TakeOwnership() bool {
|
|
return t.SyncOptions.TakeOwnership
|
|
}
|
|
|
|
// ServerSide returns the server-side value.
|
|
func (t *SyncImpl) ServerSide() string {
|
|
return t.SyncOptions.ServerSide
|
|
}
|
|
|
|
func (t *SyncImpl) SyncReleaseLabels() bool {
|
|
return t.SyncOptions.SyncReleaseLabels
|
|
}
|
|
|
|
// TrackMode returns the track mode.
|
|
func (t *SyncImpl) TrackMode() string {
|
|
return t.SyncOptions.TrackMode
|
|
}
|
|
|
|
// TrackTimeout returns the track timeout.
|
|
func (t *SyncImpl) TrackTimeout() int {
|
|
return t.SyncOptions.TrackTimeout
|
|
}
|
|
|
|
// TrackLogs returns the track logs flag.
|
|
func (t *SyncImpl) TrackLogs() bool {
|
|
return t.SyncOptions.TrackLogs
|
|
}
|
|
|
|
// TrackFailedLogs returns the track-failed-logs flag.
|
|
func (t *SyncImpl) TrackFailedLogs() bool {
|
|
return t.SyncOptions.TrackFailedLogs
|
|
}
|
|
|
|
// HelmStuckGrace returns the helm-stuck-grace value (seconds, 0 = disabled).
|
|
func (t *SyncImpl) HelmStuckGrace() int {
|
|
return t.SyncOptions.HelmStuckGrace
|
|
}
|
|
|
|
// TrackFailOnError returns whether kubedog tracking failures should cause a non-zero exit code.
|
|
func (t *SyncImpl) TrackFailOnError() bool {
|
|
return t.SyncOptions.TrackFailOnError
|
|
}
|
|
|
|
// Description returns the description.
|
|
func (t *SyncImpl) Description() string {
|
|
return t.SyncOptions.Description
|
|
}
|
|
|
|
// SuppressOutputLineRegex returns the SuppressOutputLineRegex.
|
|
func (t *SyncImpl) SuppressOutputLineRegex() []string {
|
|
return t.SyncOptions.SuppressOutputLineRegex
|
|
}
|
|
|
|
// IncludeTests returns the IncludeTests.
|
|
func (t *SyncImpl) IncludeTests() bool {
|
|
return t.SyncOptions.IncludeTests
|
|
}
|
|
|
|
// Suppress returns the Suppress.
|
|
func (t *SyncImpl) Suppress() []string {
|
|
return t.SyncOptions.Suppress
|
|
}
|
|
|
|
// SuppressSecrets returns the SuppressSecrets.
|
|
func (t *SyncImpl) SuppressSecrets() bool {
|
|
return t.SyncOptions.SuppressSecrets
|
|
}
|
|
|
|
// ShowSecrets returns the ShowSecrets.
|
|
func (t *SyncImpl) ShowSecrets() bool {
|
|
return t.SyncOptions.ShowSecrets
|
|
}
|
|
|
|
// NoHooks returns the NoHooks.
|
|
func (t *SyncImpl) NoHooks() bool {
|
|
return t.SyncOptions.NoHooks
|
|
}
|
|
|
|
// SuppressDiff returns the SuppressDiff.
|
|
func (t *SyncImpl) SuppressDiff() bool {
|
|
return t.SyncOptions.SuppressDiff
|
|
}
|
|
|
|
// SkipDiffOnInstall returns the SkipDiffOnInstall.
|
|
func (t *SyncImpl) SkipDiffOnInstall() bool {
|
|
return t.SyncOptions.SkipDiffOnInstall
|
|
}
|
|
|
|
// DiffArgs returns the DiffArgs.
|
|
func (t *SyncImpl) DiffArgs() string {
|
|
return t.SyncOptions.DiffArgs
|
|
}
|
|
|
|
// DetailedExitcode returns the DetailedExitcode.
|
|
func (t *SyncImpl) DetailedExitcode() bool {
|
|
return t.SyncOptions.DetailedExitcode
|
|
}
|
|
|
|
// StripTrailingCR returns the StripTrailingCR.
|
|
func (t *SyncImpl) StripTrailingCR() bool {
|
|
return t.SyncOptions.StripTrailingCR
|
|
}
|
|
|
|
// Context returns the Context.
|
|
func (t *SyncImpl) Context() int {
|
|
return t.SyncOptions.Context
|
|
}
|
|
|
|
// DiffOutput returns the DiffOutput.
|
|
func (t *SyncImpl) DiffOutput() string {
|
|
return t.SyncOptions.DiffOutput
|
|
}
|
|
|
|
func (t *SyncImpl) ValidateConfig() error {
|
|
validTrackModes := []string{"helm", "helm-legacy", "kubedog"}
|
|
if t.SyncOptions.TrackMode != "" && !slices.Contains(validTrackModes, t.SyncOptions.TrackMode) {
|
|
return fmt.Errorf("--track-mode must be 'helm', 'helm-legacy', or 'kubedog', got: %s", t.SyncOptions.TrackMode)
|
|
}
|
|
return t.GlobalImpl.ValidateConfig()
|
|
}
|