* feat: add --template-args to enable helm lookup() during template/apply/sync (#1833)
Add a --template-args flag to the template, apply, and sync subcommands so
extra args (most notably --dry-run=server) can be passed to the helm template
invocation, enabling Helm's lookup() function to resolve live cluster values.
- template: --template-args reaches both chartify's pre-render helm template
and the final helm template output (flagsForTemplate).
- apply/sync: --template-args reaches chartify's pre-render helm template.
apply/sync already inject --dry-run=server automatically for cluster
operations; the flag is an explicit opt-in for the template subcommand or
for passing additional flags.
- When --dry-run is present in template args, kube-context/kubeconfig are also
injected into chartify so lookup() can actually reach the cluster.
- Resolves the long-stale PR #1833 rebased onto current main, which already
contains the cluster-connectivity infrastructure (issues #2271, #2309,
#2355, #2444).
- Includes integration test (lookup.sh) covering both chartify and
non-chartify scenarios.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: make lookup template nil-safe to fix integration CI
The lookup() function returns an empty map when the chart is rendered
without a cluster connection (notably the helm-diff phase of `helmfile
apply`). The original fixture chained `index` over the lookup result,
panicking with "index of untyped nil" during apply's diff rendering.
Guard every index with `default dict` so the template falls back to
"overwritten" when lookup is empty, while still resolving to the live
value ("init") when cluster access is available (--dry-run=server via
--template-args, or a real helm upgrade).
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat: enable lookup() during apply/diff via --template-args in helm-diff
Thread --template-args into the helm-diff rendering path so that
`helmfile apply`/`diff --template-args="--dry-run=server"` resolves
Helm's lookup() function during the diff phase too. helm-diff supports
`--dry-run=server`, which explicitly "enables the cluster access ...
and the lookup template function".
Previously --template-args only reached chartify's pre-render (which is a
no-op for plain charts due to chartify's early-return when there is no
forceNamespace/patches/injections) and the final `helm template` of the
`template` subcommand. As a result `helmfile apply` on a lookup chart
rendered client-side during the diff phase.
Changes:
- pkg/state: add TemplateArgs to DiffOpts; append it in appendExtraDiffFlags
(reaches every helm-diff invocation: apply, standalone diff, interactive
sync), mirroring the existing flagsForTemplate handling.
- pkg/config + cmd: add --template-args to the diff/doctor commands and to
DiffConfigProvider, so lookup works for `helmfile diff` as well.
- pkg/app: populate DiffOpts.TemplateArgs from apply/diff/sync-interactive.
- docs/cli.md: correct the previous overpromising wording and document diff
support plus the nil-safe lookup guidance.
- tests: unit-test the TemplateArgs handling in appendExtraDiffFlags and
flagsForTemplate; integration lookup.sh now exercises apply with
--template-args="--dry-run=server".
Signed-off-by: yxxhero <aiopsclub@163.com>
* refactor: de-duplicate chartify template-args logic, add helmDefaults.templateArgs
Address review feedback on #2666:
1. Eliminate stale duplicated test helpers (issue_2444_test.go, issue_2355_test.go).
Both files intentionally copied the processChartification flag-building logic
with explicit SYNC WARNING comments, then drifted out of sync when #2666
refactored the production code (needsKubeConnection gate, user-args merge).
Extract the real logic into pure, unit-tested helpers
(buildChartifyTemplateArgs, commandRequiresCluster) and delete the copies.
2. Add unit coverage for the new chartify merge path: template +
--template-args=--dry-run=server now triggers kubeconfig/kube-context
injection (TestTemplateArgsDryRunTriggersKubeInjection,
TestTemplateArgsMergedBeforeInjection) — previously only covered by the
cluster-dependent integration test.
3. Add a negative integration case (lookup.sh assert_template_fallback)
verifying lookup() falls back to the default value WITHOUT --template-args,
guarding against a regression that silently always connects to the cluster.
4. Add helmDefaults.templateArgs for parity with diffArgs/syncArgs, so users
can enable lookup() support permanently instead of passing the flag on every
invocation. CLI --template-args overrides (does not merge with) the default.
Resolved via effectiveTemplateArgs, wired into the chartify, flagsForTemplate,
and appendExtraDiffFlags paths.
5. Minor: capitalize --template-args help text to match surrounding flags;
document helmDefaults.templateArgs precedence in docs/cli.md.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: cover helmDefaults->chartify composition; fix helm helm-diff typo
Address remaining review nits on #2666:
- Add TestHelmDefaultsTemplateArgsReachesChartify, a belt-and-suspenders test
for the processChartification composition (effectiveTemplateArgs ->
buildChartifyTemplateArgs), closing the last unit-level coverage gap for
helmDefaults.templateArgs reaching the chartify path.
- Fix pre-existing typo in cmd/bind_diff_flags.go: 'pass args to helm helm-diff'
-> 'Pass args to helm-diff' (doubled 'helm', lowercase).
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: correct 'helm helm-diff' typo in apply --diff-args help text
Sibling of the bind_diff_flags.go fix; the same doubled-'helm' typo and
lowercase help existed in cmd/apply.go's --diff-args registration, leaving
the apply and diff/doctor help strings inconsistent.
Signed-off-by: yxxhero <aiopsclub@163.com>
* docs: add helmDefaults.templateArgs to configuration reference
The complete helmfile.yaml schema in docs/configuration.md documents
diffArgs and syncArgs under helmDefaults but was missing the new
templateArgs field added in #2666. Add it beside syncArgs for
discoverability, noting the --template-args CLI override.
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat: parallel kubedog tracking with progress printer and safety valves
Rework kubedog integration so resource tracking runs in parallel with
helm upgrade/install, giving live progress output and recovering from
known helm/kubedog wedge conditions.
Core:
- kubedogTrackingHandle runs tracking in a background goroutine alongside
the helm subprocess (startBackgroundKubedogTracking); helm output is
buffered and replayed as a single block so it no longer interleaves
with progress ticks.
- Capture UID+generation baselines before handing off to helm so each
tracker waits until the resource actually changes (freshness gate).
Progress printer (pkg/kubedog/printer.go):
- Styled, auto-sized progress table with a heartbeat flusher, child (pod)
status roll-up, pre-ready pod-phase handling, multi-namespace support,
and optional color. PreviewBreakdown summarizes kept/filtered resources.
Safety valves (verify cluster state via the live API):
- Tracker-race valve (always on): when helm succeeds but a dyntracker
goroutine is wedged, poll the API and cancel the tracker so wait()
returns success instead of blocking until --track-timeout.
- Helm-stuck killer (opt-in via helmStuckGrace): if the cluster stays
converged while helm v4's hook waiter is wedged, SIGINT the helm
subprocess to recover.
- Failure watchdog (pkg/kubedog/watchdog.go): surface failing pods that
never made it into dyntracker's resource graph.
Options: trackFailedLogs, helmStuckGrace, trackTimeout, color
(Color/NoColor), and resource filtering (trackKinds/skipKinds/
trackResources). PersistentVolumeClaim support in resource classification.
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
* test: add unit tests for kubedog tracking
Cover the progress printer, resource classification, the failure
watchdog, helm-output trimming/dedup, the release hard-timeout helper,
and the color/track option plumbing.
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
* test: update golden logs, e2e snapshots, and values-id fixtures
Refresh pkg/app testapply/testdestroy golden logs, e2e template
snapshots, and the TestGenerateID values-id golden hashes for the new
kubedog progress output and the merged release struct layout.
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
* refactor: remove dead kubedog display code and dedupe tracker setup
Deep-review pass on the parallel kubedog tracking changes:
- Remove pkg/kubedog/display.go (308 lines) and display_test.go (453 lines).
These rendered progress for the legacy per-kind trackers that this PR
replaces; in the merged tree every function is unreferenced outside its
own tests. The new progressPrinter (printer.go) supersedes them.
TestMain (color.ForceColor for deterministic ANSI in tests) is preserved
in a new main_test.go.
- Dedupe trackWithKubedog: the post-helm fallback rebuilt the exact same
tracker options as buildReleaseTracker. Reuse buildReleaseTracker instead,
dropping ~40 lines of duplicated timeout/log/filter/tracker construction.
No behavior change; build, go vet, golangci-lint, and the kubedog/state
unit tests all pass.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: stop waitForFreshness busy-looping the API after helm finishes
Once upstreamDoneCh closes it is always ready, so the select in
waitForFreshness stopped blocking on the ticker and re-ran probe() (a
live GET) as fast as the round-trip allowed for the whole 3s grace
window — hammering the API server once per tracked resource.
Track a local view of the channel and nil it out on first delivery so
the first hit records the timestamp (one fast retry, as intended) and
all subsequent polls are ticker-throttled. Functional behavior is
unchanged: return nil when fresh, errUpstreamDoneNoChange after grace.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: drop trailing blank line from helm4 OCI pull snapshots
The trailing-newline trim in helmexec.info() removes the blank line helm
prints after the OCI chart "Digest:" line. The helm3 (output.yaml)
snapshots never captured that line, but the helm4 (output-helm4.yaml)
snapshots for oci_chart_pull{,_direct,_once,_once2} and
issue_473_oci_chart_url_fetch still expected it, so they failed under
helm 4. Remove the blank line so the snapshots match the trimmed output.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: refresh diff-args integration goldens for styled release headers
DisplayAffectedReleases now emits a styled "========== Updated Releases
==========" header (matching the app/e2e goldens already updated by this
PR) and helmexec.info() trims the trailing blank after helm's install
status. Update the diff-args apply-stderr{,-helm4} and apply-live-
stderr{,-helm4} goldens accordingly so they match the actual stderr.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: drop trimmed blank line from v1-subhelmfile template golden
The trailing-newline trim in helmexec.info() removes the blank line helm
prints after '"incubator" has been added to your repositories'. Update
the v1-subhelmfile-multi-bases-with-array-values result and result-live
goldens so the template stdout comparison matches.
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: Roman Mykhailiuk <romanm@cybellum.com>
* feat: add support for helm 4 --server-side upgrade flag
Add support for the helm 4 upgrade flag --server-side which accepts
"true", "false", or "auto" (default "auto"). This allows users to
explicitly control server-side apply behavior, which is needed for
releases originally installed with Helm 3 and being managed with Helm 4.
The flag can be configured via:
- CLI: --server-side flag on sync, apply, and diff commands
- helmDefaults.serverSide in helmfile.yaml
- releases[].serverSide per-release override
Precedence: release-level > CLI flag > helmDefaults.
Errors are returned when serverSide is set but running Helm 3, or when
an invalid value is provided.
Closes#2640
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: update TestGenerateID expected hashes for new ServerSide field
Adding ServerSide *string to ReleaseSpec changes spew's %#v output and
shifts the FNV hash used by generateValuesID. Update the hard-coded want
values to the new deterministic hashes.
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: add trackFailOnError option to control kubedog exit code behavior
When kubedog release tracking fails (e.g. pod ImagePullBackOff), helmfile
exits with code 0 instead of a non-zero exit code. Add a trackFailOnError
configuration option (default: false) that when set to true, propagates
kubedog tracking failures to the exit code.
The option is available as:
- Per-release YAML: trackFailOnError: true
- CLI flag: --track-fail-on-error (sync and apply commands)
Extract trackReleaseIfEnabled helper to consolidate kubedog tracking logic
from two duplicated call sites into a single maintainable method.
Fixes#2507
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: add //go:build ignore to server.go to fix go test CI failure
The test/integration/test-cases/issue-2103/input/server.go is a
package main helper binary used by the issue-2103 integration test.
When go test -coverprofile runs on this package, it fails with
"go: no such tool covdata" in the CI environment.
Adding //go:build ignore excludes the file from go list ./... (and
therefore from PKGS in the Makefile), while still allowing the
integration test to build it explicitly via file path:
go build -o server ./path/to/server.go
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/8a7000af-72b7-48f8-8a82-24813b5df341
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: update TestGenerateID expected hashes after adding TrackFailOnError field
Adding TrackFailOnError *bool to ReleaseSpec changed the spew
serialization of the struct, which changed the FNV-32a hash values
produced by generateValuesID. Update temp_test.go with the new
expected hash strings.
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/caa86cd9-73d1-4894-b745-fd70c0811fd6
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Add support for trackMode: helm-legacy to use Helm v4's --wait=legacy flag,
which maintains compatibility with Helm v3's wait behavior during migration.
Helm v4 changed the default --wait behavior from polling to a watcher-based
approach. This can cause issues with charts that have broken livenessProbe
configurations without startupProbe. The --wait=legacy flag preserves the
Helm v3 polling behavior for smoother migration.
Changes:
- Add TrackModeHelmLegacy constant in pkg/kubedog/options.go
- Use kubedog.TrackMode constants instead of raw strings in helmx.go
- Enhance appendWaitFlags to use --wait=legacy for Helm v4 when trackMode
is helm-legacy
- Add nil check for logger before logging warning
- Add version check with warning when helm-legacy is used with Helm v3
- Update validation in pkg/config to accept helm-legacy track mode
- Update command-line flags in cmd/apply.go and cmd/sync.go
- Add comprehensive documentation in docs/advanced-features.md
- Add thorough test coverage including warning message verification
Behavior:
- Helm v4 + helm-legacy: Uses --wait=legacy
- Helm v3 + helm-legacy: Falls back to --wait with warning
- Helm v4 + helm: Uses --wait (watcher mode)
- Any + kubedog: Skips --wait flag
Fixes#2464
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: Copilot <copilot@github.com>
* Initial plan
* Implement --timeout flag for helmfile sync command
- Add Timeout field to SyncOptions struct in pkg/config/sync.go
- Add --timeout flag to sync command in cmd/sync.go
- Add Timeout field to SyncOpts struct in pkg/state/state.go
- Modify timeoutFlags() function to prioritize CLI timeout over release and default configs
- Add test case to verify CLI timeout overrides other timeout settings
- Follow same pattern as existing --wait and --wait-for-jobs flags
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Fix lint issues: format test struct fields properly
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Update docs: Add --timeout flag documentation for helmfile sync command
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* docs: add skipSchemaValidation to index.md and update related structs
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat: add SkipSchemaValidation to config and state handling
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
* Add "PostRendererArgs" option to be passed to helm
This allows using PowerShell scripts on Windows as Post Renderer.
Signed-off-by: Maarten Boekhold <maarten.boekhold@finastra.com>
Ref https://github.com/helmfile/helmfile/discussions/593
* implemented --reset-values flag overriding .helmDefaults.reuseValues=true
* tests for --reset-values flag
* fixed pkg/app tests
* added tests for both reset & reuse flags given
* added appendValuesControlMode method for --reuse/reset-values flag processing
* updated code comments & docs
Signed-off-by: Karol Ossowski <k@koralsky.pl>
1. only implement post-renderer flags this patch
2. As mumoshu advise, add helmfile flags `--post-render` and add the
postRenderer config in helmDefaults and release. the priority is
helmfile flags > release > helmDefaults.
3. fix the test case in state_test.go and some other tests.
Signed-off-by: guofutan <guofutan@tencent.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
* add interactive in sync & remove --interactive in global options
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix unittest
Signed-off-by: yxxhero <aiopsclub@163.com>
* same behave as apply when in interactive
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
yxxhero