* feat: parallel kubedog tracking with progress printer and safety valves
Rework kubedog integration so resource tracking runs in parallel with
helm upgrade/install, giving live progress output and recovering from
known helm/kubedog wedge conditions.
Core:
- kubedogTrackingHandle runs tracking in a background goroutine alongside
the helm subprocess (startBackgroundKubedogTracking); helm output is
buffered and replayed as a single block so it no longer interleaves
with progress ticks.
- Capture UID+generation baselines before handing off to helm so each
tracker waits until the resource actually changes (freshness gate).
Progress printer (pkg/kubedog/printer.go):
- Styled, auto-sized progress table with a heartbeat flusher, child (pod)
status roll-up, pre-ready pod-phase handling, multi-namespace support,
and optional color. PreviewBreakdown summarizes kept/filtered resources.
Safety valves (verify cluster state via the live API):
- Tracker-race valve (always on): when helm succeeds but a dyntracker
goroutine is wedged, poll the API and cancel the tracker so wait()
returns success instead of blocking until --track-timeout.
- Helm-stuck killer (opt-in via helmStuckGrace): if the cluster stays
converged while helm v4's hook waiter is wedged, SIGINT the helm
subprocess to recover.
- Failure watchdog (pkg/kubedog/watchdog.go): surface failing pods that
never made it into dyntracker's resource graph.
Options: trackFailedLogs, helmStuckGrace, trackTimeout, color
(Color/NoColor), and resource filtering (trackKinds/skipKinds/
trackResources). PersistentVolumeClaim support in resource classification.
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
* test: add unit tests for kubedog tracking
Cover the progress printer, resource classification, the failure
watchdog, helm-output trimming/dedup, the release hard-timeout helper,
and the color/track option plumbing.
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
* test: update golden logs, e2e snapshots, and values-id fixtures
Refresh pkg/app testapply/testdestroy golden logs, e2e template
snapshots, and the TestGenerateID values-id golden hashes for the new
kubedog progress output and the merged release struct layout.
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
* refactor: remove dead kubedog display code and dedupe tracker setup
Deep-review pass on the parallel kubedog tracking changes:
- Remove pkg/kubedog/display.go (308 lines) and display_test.go (453 lines).
These rendered progress for the legacy per-kind trackers that this PR
replaces; in the merged tree every function is unreferenced outside its
own tests. The new progressPrinter (printer.go) supersedes them.
TestMain (color.ForceColor for deterministic ANSI in tests) is preserved
in a new main_test.go.
- Dedupe trackWithKubedog: the post-helm fallback rebuilt the exact same
tracker options as buildReleaseTracker. Reuse buildReleaseTracker instead,
dropping ~40 lines of duplicated timeout/log/filter/tracker construction.
No behavior change; build, go vet, golangci-lint, and the kubedog/state
unit tests all pass.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: stop waitForFreshness busy-looping the API after helm finishes
Once upstreamDoneCh closes it is always ready, so the select in
waitForFreshness stopped blocking on the ticker and re-ran probe() (a
live GET) as fast as the round-trip allowed for the whole 3s grace
window — hammering the API server once per tracked resource.
Track a local view of the channel and nil it out on first delivery so
the first hit records the timestamp (one fast retry, as intended) and
all subsequent polls are ticker-throttled. Functional behavior is
unchanged: return nil when fresh, errUpstreamDoneNoChange after grace.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: drop trailing blank line from helm4 OCI pull snapshots
The trailing-newline trim in helmexec.info() removes the blank line helm
prints after the OCI chart "Digest:" line. The helm3 (output.yaml)
snapshots never captured that line, but the helm4 (output-helm4.yaml)
snapshots for oci_chart_pull{,_direct,_once,_once2} and
issue_473_oci_chart_url_fetch still expected it, so they failed under
helm 4. Remove the blank line so the snapshots match the trimmed output.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: refresh diff-args integration goldens for styled release headers
DisplayAffectedReleases now emits a styled "========== Updated Releases
==========" header (matching the app/e2e goldens already updated by this
PR) and helmexec.info() trims the trailing blank after helm's install
status. Update the diff-args apply-stderr{,-helm4} and apply-live-
stderr{,-helm4} goldens accordingly so they match the actual stderr.
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: drop trimmed blank line from v1-subhelmfile template golden
The trailing-newline trim in helmexec.info() removes the blank line helm
prints after '"incubator" has been added to your repositories'. Update
the v1-subhelmfile-multi-bases-with-array-values result and result-live
goldens so the template stdout comparison matches.
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: Roman Mykhailiuk <romanm@cybellum.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: Roman Mykhailiuk <romanm@cybellum.com>
Add appendServerSideFlagsForDiff to validate helm-diff plugin version
(v3.15.10+) before passing the --server-side flag to helm diff upgrade.
Extract resolveServerSideValue helper to share precedence logic between
upgrade and diff paths. Bump helm-diff recommended version to v3.15.10
across Dockerfiles, CI, and integration scripts.
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat: add support for helm 4 --server-side upgrade flag
Add support for the helm 4 upgrade flag --server-side which accepts
"true", "false", or "auto" (default "auto"). This allows users to
explicitly control server-side apply behavior, which is needed for
releases originally installed with Helm 3 and being managed with Helm 4.
The flag can be configured via:
- CLI: --server-side flag on sync, apply, and diff commands
- helmDefaults.serverSide in helmfile.yaml
- releases[].serverSide per-release override
Precedence: release-level > CLI flag > helmDefaults.
Errors are returned when serverSide is set but running Helm 3, or when
an invalid value is provided.
Closes#2640
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: update TestGenerateID expected hashes for new ServerSide field
Adding ServerSide *string to ReleaseSpec changes spew's %#v output and
shifts the FNV hash used by generateValuesID. Update the hard-coded want
values to the new deterministic hashes.
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
Update helm-diff plugin version from v3.15.8 to v3.15.9 across
Dockerfiles, recommended version constant, CI matrix, and
integration test default.
Signed-off-by: yxxhero <aiopsclub@163.com>
* chore: bump helm release pins
* chore: align helm module metadata
* chore: finalize helm patch bumps
* fix: add --plain-http flag for Helm 3.21+ OCI push in tests
Helm 3.21.1 introduced stricter security checks that reject HTTP
scheme downgrades when pushing to OCI registries, with the error:
"blob upload Location downgrades scheme from https"
Previously only Helm 4 required --plain-http for HTTP-only OCI
registries. Now Helm 3.21+ also requires this flag.
Add a new requiresPlainHTTPForOCI() helper that returns true for
both Helm 4.x and Helm 3.21+, and use it in execHelmPush() instead
of isHelm4().
* fix: safe fallback in requiresPlainHTTPForOCI when version detection fails
Default to true (require --plain-http) when helm version detection
fails, since any Helm version that supports helm push also supports
the --plain-http flag. This avoids the inconsistent HELMFILE_HELM4
env var fallback which only covered Helm 4.
* fix: update snapshot tests for Helm 4.2.1 OCI pull output
Helm 4.2.1 now outputs additional 'Pulled:' and 'Digest: sha256:...'
lines after each OCI chart pull. The SHA256 digest is non-deterministic
because helm packages include build timestamps, so normalize it with
a regex placeholder.
- Add ociDigestRegex to normalize non-deterministic OCI digest values
- Create output-helm4.yaml for 5 tests that lacked Helm 4 snapshots
- Update output-helm4.yaml for oci_need and postrenderer to include
the new Pulled/Digest lines from Helm dependency pull operations
* fix: update ociDigestRegex to match empty digest in Helm 4.2.1 OCI pull output
Helm 4.2.1 outputs "Digest: sha256:" (empty hash) when pulling OCI charts.
The regex required at least one hex char ([0-9a-f]+), so it did not match
and the digest was not normalized to $DIGEST in snapshot tests.
Also fix the replacement string: Go regex ReplaceAllString interprets $DIGEST
as a capture group reference (resolving to empty). Use $$DIGEST to produce
a literal $DIGEST in the output.
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yxxhero <aiopsclub@163.com>
This changes the behaviour of the cleanup event during sync to be
triggered right before the function exits and matches the behaviour of
apply
Signed-off-by: Niklas Ott <niklas.ott@unwired.at>
Co-authored-by: Raphael Luba <raphael@leanbyte.com>
This commit deduplicates the preparation logic for sync and apply by
moving it to a common function.
Signed-off-by: Niklas Ott <niklas.ott@unwired.at>
Co-authored-by: Raphael Luba <raphael@leanbyte.com>
* feat: expose WithPreparedCharts and SyncRun
This exposes the previously internal functions withPreparedCharts and
sync to be used by tooling built on top of helmfile
Co-authored-by: Raphael Luba <raphael@leanbyte.com>
Signed-off-by: Niklas Ott <niklas.ott@unwired.at>
* feat: expose Helm interface and HelmState
This adds a function to get the current HelmState
Co-authored-by: Raphael Luba <raphael@leanbyte.com>
Signed-off-by: Niklas Ott <niklas.ott@unwired.at>
* feat: return an error instead of panicking on multiple calls on WithPreparedCharts
Signed-off-by: Niklas Ott <niklas.ott@unwired.at>
---------
Signed-off-by: Niklas Ott <niklas.ott@unwired.at>
Co-authored-by: Raphael Luba <raphael@leanbyte.com>
* feat(state): add mergeStrategy field to EnvironmentSpec
Introduces a per-environment mergeStrategy with valid values "override"
(default, current behavior) and "fallback". This commit only adds the
field, the constants, and a parse-time validator; the loader still
ignores the value, so behavior is unchanged.
Subsequent commits thread the value through the values loader and
implement the fallback semantics.
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* refactor(state): thread mergeStrategy through values loader
Adds a mergeStrategy string parameter to LoadEnvironmentValues,
loadValuesEntries, and mapMerge so the value can flow from
EnvironmentSpec down to the merge call site. Behavior is unchanged in
this commit; mapMerge ignores the strategy and the next commit
implements the fallback semantics.
Top-level state.DefaultValues and the --state-values-file/-set loaders
are passed an empty strategy ("") since they have no per-environment
spec to consult and stay on the default override behavior.
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* feat(state): implement fallback merge strategy
Adds a hand-rolled fallbackDeepMerge that, unlike mergo, preserves
keys present in the destination even when their value is the zero
value (false, 0, "", nil, empty list/map). mapMerge dispatches to it
when mergeStrategy == "fallback"; "override" and the empty default
keep using mergo with WithOverride so existing behaviour is unchanged.
Validation lives at the entry of LoadEnvironmentValues so a single
chokepoint guards the field. Invalid values produce an error naming
both the offending value and the valid options.
Tests cover: first-file-wins precedence, gap filling, deep nested
merge, three-file chains, explicit zero-value preservation (the case
naïve mergo gets wrong), explicit nil preservation, inline map
entries, override regression, default-equals-override equivalence,
and invalid-strategy errors.
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* feat(state): expose prior-file values in fallback template context
Under mergeStrategy: fallback, .gotmpl values files can now reference
values from earlier files in the same `values:` list via .Values
(e.g. `service.domain: "service.{{ .Values.cluster.domain }}"`).
The accumulated result is layered under env.GetMergedValues so env
defaults, env values, and CLI overrides still win on overlap. Override
mode keeps the historical template context — unchanged — so this is
strictly opt-in via the mergeStrategy field.
Together with the precedence flip from the previous commit, this lets
users replace the brittle two-stage `merged-values.yaml.gotmpl`
workaround with native helmfile syntax.
Tests cover the headline cross-file template reference case and pin
the override-mode contract that prior-file values stay invisible.
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* docs: document mergeStrategy and fallback semantics
Adds a new section to values-and-merging.md describing the override vs
fallback strategies, the explicit-zero-value preservation guarantee,
and the cross-file template reference behavior. Adds a brief pointer
to environments.md so users land on the new field from the
environment values discussion.
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* refactor(state): reuse maputil.MergeMaps for fallback merge
Replaces the hand-rolled fallbackDeepMerge with a single call to
maputil.MergeMaps, swapping its arguments so the accumulated dest wins
over the new src file. Same first-file-wins semantic, fewer lines, and
the fallback path now inherits the same slice merge strategies the
rest of helmfile already uses.
The one observable behavior shift is for explicit nil values: under
fallback, nil in an earlier file no longer 'wins' over a non-nil value
in a later file — instead it falls through (matching MergeMaps' rule
that nil from the override side only fills missing keys). This is
internally consistent: nil-overwrites is an mergo.WithOverride quirk
that lives only in the override path. The renamed test
NilFallsThroughToFallback pins the new behavior with a comment
referencing the contrast with override mode (Issue1154).
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
---------
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* fix: add trackFailOnError option to control kubedog exit code behavior
When kubedog release tracking fails (e.g. pod ImagePullBackOff), helmfile
exits with code 0 instead of a non-zero exit code. Add a trackFailOnError
configuration option (default: false) that when set to true, propagates
kubedog tracking failures to the exit code.
The option is available as:
- Per-release YAML: trackFailOnError: true
- CLI flag: --track-fail-on-error (sync and apply commands)
Extract trackReleaseIfEnabled helper to consolidate kubedog tracking logic
from two duplicated call sites into a single maintainable method.
Fixes#2507
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: add //go:build ignore to server.go to fix go test CI failure
The test/integration/test-cases/issue-2103/input/server.go is a
package main helper binary used by the issue-2103 integration test.
When go test -coverprofile runs on this package, it fails with
"go: no such tool covdata" in the CI environment.
Adding //go:build ignore excludes the file from go list ./... (and
therefore from PKGS in the Makefile), while still allowing the
integration test to build it explicitly via file path:
go build -o server ./path/to/server.go
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/8a7000af-72b7-48f8-8a82-24813b5df341
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: update TestGenerateID expected hashes after adding TrackFailOnError field
Adding TrackFailOnError *bool to ReleaseSpec changed the spew
serialization of the struct, which changed the FNV-32a hash values
produced by generateValuesID. Update temp_test.go with the new
expected hash strings.
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/caa86cd9-73d1-4894-b745-fd70c0811fd6
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* feat: add 'create' subcommand to scaffold helmfile deployment projects
Add 'helmfile create [NAME]' command that generates a best-practice
helmfile project structure with:
- helmfile.yaml with commented examples (helmDefaults, repositories,
environments, releases)
- environments/default.yaml for environment-specific values
- values/.gitkeep placeholder for release values
Supports --output-dir/-o for custom output path and --force to
overwrite existing files. Validates project name to prevent path
traversal.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: add overwrite protection for all scaffold files and unit tests for create command
- pkg/app/create.go: extract writeFileIfNotExists helper that respects the
--force flag; all three scaffold files (helmfile.yaml,
environments/default.yaml, values/.gitkeep) now refuse to overwrite
without --force
- pkg/config/create.go: ValidateConfig now checks all three scaffold paths
and reports every already-existing file in a single error before
proceeding, instead of only checking helmfile.yaml
- pkg/app/create_test.go: add unit tests covering new directory, current
directory, per-file overwrite rejection without --force, and full
overwrite with --force
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/eb6d9e4b-0f72-4e26-b841-e1e39a2b2e83
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: remove redundant absDir from ValidateConfig error message
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/eb6d9e4b-0f72-4e26-b841-e1e39a2b2e83
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: address create command review feedback
- cmd/create.go: add config.NewCLIConfigImpl() call for consistency with other
subcommands; update --force flag help text to list all overwritten files
- pkg/config/create.go: delegate to c.GlobalImpl.ValidateConfig() at end of
ValidateConfig() for global option validation (--color/--no-color)
- pkg/config/create_test.go: add unit tests for CreateImpl.ValidateConfig()
covering path separator rejection, '..' rejection, existing-file detection
per-file and with --force, and global color conflict delegation
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/6327d657-5888-4b94-85fb-def80c0a193f
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: clarify test helper name and comment in create_test.go
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/6327d657-5888-4b94-85fb-def80c0a193f
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: atomic preflight check in App.Create before any writes
Refactor Create to collect all conflicting scaffold paths up front
before writing anything. When --force is not set and any scaffold file
already exists, the command returns a single error listing all
conflicts without touching the filesystem.
Also removes the now-unnecessary writeFileIfNotExists helper and adds a
test (TestCreate_PreflightAtomicOnLaterConflict) verifying that a
conflict on a later file (e.g. environments/default.yaml) prevents even
the first file (helmfile.yaml) from being created.
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/aae6f2e6-7f9e-42b8-afa3-78edd3215127
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: handle non-IsNotExist Stat errors in preflight check; add whitespace name test; fix gci formatting
- pkg/app/create.go: treat os.Stat errors that are NOT os.IsNotExist as
hard errors in the preflight scan, surfacing permission/IO issues before
any writes happen; remove trailing blank line that caused gci failure
- pkg/config/create.go: same non-IsNotExist error handling in ValidateConfig
- pkg/config/create_test.go: add TestCreateImpl_ValidateConfig_WhitespaceOnlyName
covering the " " (whitespace-only) name rejection branch
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/d6574f56-f46d-46f7-99d9-e0b0b897b3b5
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* refactor: eliminate duplicated scaffold existence check; use O_EXCL for TOCTOU protection
- pkg/config/create.go: remove file-existence check from ValidateConfig
(duplicate of App.Create's preflight); ValidateConfig now only validates
the project name and delegates to GlobalImpl.ValidateConfig. Remove unused
os/path/filepath imports.
- pkg/app/create.go: add writeScaffoldFile helper that uses O_CREATE|O_EXCL
when force=false, so a file appearing between the preflight check and the
actual write is caught rather than silently overwritten (TOCTOU protection).
- pkg/config/create_test.go: remove four file-existence tests that tested the
now-deleted ValidateConfig logic; file-conflict coverage remains in
pkg/app/create_test.go. Simplify ValidName and GlobalColorConflict tests.
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/82f82e72-934f-416c-8662-5060e92284fa
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: wrap O_EXCL error with --force hint; add writeScaffoldFile unit tests
- pkg/app/create.go: wrap os.IsExist error from writeScaffoldFile with a
message that names the conflicting file and suggests --force, so the user
gets actionable output even in the TOCTOU case
- pkg/app/create_test.go: add TestWriteScaffoldFile_CreatesNewFile,
TestWriteScaffoldFile_ExistingFileNoForce, and
TestWriteScaffoldFile_ExistingFileWithForce to cover the helper directly
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/82f82e72-934f-416c-8662-5060e92284fa
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: wrap App.Create errors in *app.Error; reject '.' as project name; add '.' name test
- pkg/app/create.go: wrap all App.Create fmt.Errorf returns with appError("", ...)
so toCLIError produces a clean user-friendly message instead of
"unexpected error: *fmt.wrapError: ..."
- pkg/config/create.go: reject "." as a NAME alongside ".." to prevent
accidentally scaffolding into the current directory via a named argument
- pkg/config/create_test.go: add TestCreateImpl_ValidateConfig_NameDot
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/6d64508e-2d66-47e9-a02a-7669a2f481b7
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: drop unused outputDir param from test helper to fix unparam lint error
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/11cd65e9-c5ef-4195-9375-bc929169616b
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: drop unused force param from test helper to fix unparam lint error
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/0e1bdac5-708f-4615-ae6d-e22fc1e921f2
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* fix: use --post-renderer-args=VALUE format to prevent Helm flag parsing failure
When postRendererArgs contains values like short flags (e.g. -v), passing
--post-renderer-args and the value as separate arguments causes Helm to
interpret the value as its own flag. Using the --post-renderer-args=VALUE
format unambiguously binds the value to the flag.
Fixes#2563
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: update hasFlagWithValue doc/errors and add -v short-flag test cases
- Update hasFlagWithValue doc comment to describe both '--flag value' and
'--flag=value' forms
- Update t.Errorf messages in app_test.go to reflect both accepted formats
- Add 'post-renderer-args-short-flag-value' test case (-v) to both
TestHelmState_flagsForUpgrade and TestHelmState_flagsForTemplate to
verify --post-renderer-args=-v emission (core regression from #2563)
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/dd95f046-358b-4867-9069-9432c1b5318e
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* fix: skip subhelmfiles when selectors conflict with CLI selectors (#2544)
When CLI selectors are provided (e.g. -l name=b), subhelmfiles whose
explicit selectors are provably incompatible are now skipped entirely,
avoiding unnecessary YAML loading and template rendering.
Two selector sets are incompatible when every pair has a positive label
conflict: same key with different values (e.g. name=b vs name=a).
Negative labels are not compared.
Fixes#2544
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: address PR review comments - use CLI selectors, fix doc comment, add malformed selector test
Agent-Logs-Url: https://github.com/helmfile/helmfile/sessions/1f1c33ce-e50d-4781-85b8-d606b5d4ca54
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* fix: add debug logging, unit tests, docs, and fix integration test for subhelmfile selector skip
- Add debug log when skipping subhelmfile due to selector conflict
- Add TestSubhelmfileSelectorsConflict with 11 cases for direct unit coverage
- Document the selector-based subhelmfile skip optimization in docs/index.md
- Fix integration test: use 'app' label key instead of reserved 'name' key
(GetReleasesWithLabels overwrites labels["name"] with the release name)
Signed-off-by: yxxhero <aiopsclub@163.com>
* refactor: avoid map allocation in positiveLabelsCompatibleWith
Compare positive label slices directly instead of allocating a map per
comparison, as label counts are typically small (1-3 entries).
Addresses Copilot review comment on PR #2545.
Signed-off-by: yxxhero <aiopsclub@163.com>
* docs: clarify subhelmfile selector docs per Copilot review feedback
Reword the first two bullets to avoid the contradiction between
'CLI selectors are ignored' and the new skip optimization.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: address Copilot review comments round 3
- Log parse errors from SelectorsAreCompatible at debug level instead of
silently discarding them
- Hoist regex compilation to package-level vars in ParseLabels to avoid
repeated compilation per selector
- Replace EXIT traps with explicit cleanup calls in integration test to
avoid interfering with the parent runner's trap
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
This commit only renames some variables to match other places in the
code, so the variable names are less confusing, it does not add or
change any functionality
Signed-off-by: Niklas Ott <niklas.ott@unwired.at>
Co-authored-by: Raphael Luba <raphael@leanbyte.com>
* fix: helmfile list now reflects version from helmfile.lock
The list command now resolves locked dependencies before returning
release information, ensuring the version field reflects the pinned
version from helmfile.lock when present.
Fixes#1953
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: address PR review comments
- Remove redundant maps.Copy in list() - labels already merged by GetReleasesWithLabels()
- Fix default lockfile path to use basePath for multi-file mode
- Update test to expect basePath-joined lockfile path
- Add multi-file test for lockfile resolution in helmfile.d directory
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix more test
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix tests
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: propagate errors instead of panic in list()
When skipCharts=false, errors from list() now properly propagate instead
of causing a crash. Uses a closure variable to capture the error and
propagates it after withPreparedCharts completes.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix tests
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix tests
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix tests
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
This commit makes the apply logic exit early in the event there are no
changes to releases. I believe this effectively reverts helmfile#522.
Updates relevant snapshots
Clarify conditions under which preapply hooks are triggered to include that they will no longer fire if there is a no-op.
Docs as requested by the maintainer from a copilot request made by them.
Fixes: helmfile#679
Signed-off-by: Thomas Arrow <thomas.arrow@wikimedia.de>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
PR #2367 introduced CLIOverrides to give --state-values-set element-by-element
array merge semantics. However, nested helmfile values (helmfiles[].values:)
were also routed into CLIOverrides, causing their arrays to merge instead of
replace. This broke the pre-v1.3.0 behavior where passing an array via
helmfiles[].values: would fully replace the child's default array.
Add OverrideValuesAreCLI flag to SubhelmfileEnvironmentSpec so the loader can
distinguish CLI flags from nested helmfile values. CLI values continue using
CLIOverrides (element-by-element merge); nested helmfile values now use Values
(Sparse merge strategy → full array replacement).
Fixes#2451
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: use absolute baseDir in sequential helmfiles for correct values path resolution (#2424)
PR #2410 introduced a regression where a relative directory was passed as
baseDir instead of an absolute one, causing values and secrets file paths
to resolve incorrectly when using --sequential-helmfiles with helmfile.d/.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: mirror reporter's bases/templates/inherit setup in issue-2424 integration test
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
---------
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
fix: helmDefaults.skipRefresh ignored in runHelmDepBuilds (#2269)
`runHelmDepBuilds()` only checked the CLI flag (`opts.SkipRefresh`) when
deciding whether to run `helm repo update` before building dependencies.
This meant that setting `helmDefaults.skipRefresh: true` in helmfile.yaml
had no effect on the repo update call inside dep builds.
Add `!st.HelmDefaults.SkipRefresh` to the guard condition so that
`helmDefaults.skipRefresh: true` is respected alongside the CLI flag.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: eliminate os.Chdir in sequential helmfiles to fix relative path resolution
The sequential code path used within() → os.Chdir() to change the
process-wide working directory when processing helmfile.d files.
This broke relative environment variable paths (e.g. KUBECONFIG=kubeconfig.yaml)
because they resolved from the wrong directory after chdir.
Replace the chdir-based approach with the same baseDir parameter pattern
used by the parallel code path, passing explicit directory context through
loadDesiredStateFromYamlWithBaseDir() instead of mutating global process state.
Closes#2409
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: restore within() for single-file sequential to preserve chart path format
The previous approach used baseDir for all sequential processing, which
changed chart path format in output (e.g. from "../../../../charts/raw"
to "test/integration/charts/raw"). This broke integration tests that
compare chart paths in expected output.
Now the sequential branch uses two strategies:
- Single file: use os.Chdir via within() to preserve backward-compatible
relative chart paths in output
- Multiple files with --sequential-helmfiles: use baseDir parameter to
avoid os.Chdir, fixing relative env var paths like KUBECONFIG (#2409)
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: revert e2e snapshot outputs to match within() behavior
The previous commit restored within() for single-file sequential
processing, which produces relative chart paths (e.g. ../../charts/raw)
and filename-only FilePath. Revert the e2e snapshot expected outputs
to match main branch since single-file behavior is now identical.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: restructure integration test for multi-file sequential processing
- Point -f at helmfile.d/ directly (not parent dir) so findDesiredStateFiles
discovers the yaml files
- Add second helmfile to trigger baseDir path (len > 1)
- Inline environment config to avoid base file relative path issues
- Verify both releases appear in output instead of comparing with parallel
(which may differ in ordering)
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: reduce cognitive complexity and improve accuracy of sequential helmfiles
Replace inline visitSubHelmfiles closure with calls to the existing
processNestedHelmfiles() method, matching the parallel path. This
eliminates duplicated nested logic and reduces gocognit complexity
below the CI threshold of 110. Also fixes help text and docs to
accurately describe that single-file processing still uses within(),
and adds kubeContext verification to the integration test.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* test: validate kubeContext resolution in sequential helmfiles integration test
Restructure the integration test to replicate the exact user scenario
from issue #2409:
- Multiple files in helmfile.d/ using bases: with relative paths
(../bases/) for environments and defaults
- Environment values set kubeContext via .Environment.Values
- helmDefaults.kubeContext rendered from gotmpl
- Local chart references (../../../../charts/raw) from helmfile.d/
- Run diff against the minikube cluster to exercise kubeContext
resolution, which would fail with "context does not exist" if
os.Chdir() broke relative path resolution
- Also verify template output for both releases and relative values
file (values/common.yaml) resolution
Fix normalizeChart() in util.go to be idempotent — skip re-prefixing
when the chart path already starts with basePath. This prevents
double-prefixing of local chart paths (e.g. helmfile.d/test/.../raw)
when normalizeChart is called multiple times (once during chart
preparation and again during diff/sync).
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
---------
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: helmBinary setting ignored in multi-document YAML files
The helmBinary setting in helmfile.yaml was being ignored when using
multi-document YAML files (files with --- separators).
Root Cause:
When processing multi-document YAML files, the load() function splits
the file into parts and processes each part separately. Each part was
calling applyDefaultsAndOverrides() which would set an empty helmBinary
to the default 'helm'. When merging parts, the default value from a
later part would override the correct value from an earlier part.
Fix:
- Added a new applyDefaults parameter to ParseAndLoad() to control when
defaults are applied
- Modified rawLoad() to pass applyDefaults=false when processing
individual parts
- Added a call to ApplyDefaultsAndOverrides() after all parts are merged
to apply defaults once on the final merged state
- Exported ApplyDefaultsAndOverrides() method for use by the app package
Fixes: #2319
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: update comment per PR review
Change 're-apply' to 'apply' since defaults are never applied during
part processing (applyDefaults=false is passed), so this is the first
and only time defaults are applied to the merged state.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: clarify applyDefaults logic in test LoadFile callbacks
Add explicit applyDefaults variable with comment explaining why it
equals evaluateBases: base files shouldn't apply defaults, only the
main file should after all parts/bases are merged.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: address PR review comments
- Remove applyDefaults parameter from rawLoad() since it's always false
- Add regression test for multi-document YAML with helmBinary (issue #2319)
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: add integration test for helmBinary in multi-document YAML
Add TestHelmBinaryPreservedInMultiDocumentYAML that exercises the full
loadDesiredStateFromYaml path to ensure helmBinary from the first
document is preserved when merging multi-document YAML files.
This is a regression test at the load() orchestration level for issue #2319.
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
toCLIError() panics on unhandled error types (e.g. helmexec.ExitError
from a failed helm plugin install). On Windows, plugin install hooks
often fail due to missing 'sh', causing helmfile init to crash even
when the plugin binary was placed correctly.
- Add helmexec.ExitError case to toCLIError and replace panic in the
default case with a graceful error return
- After AddPlugin/UpdatePlugin errors, verify whether the plugin is
actually present before failing; log a warning and continue if so
Fixes#1983
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
Adds a new `helmfile unittest` command that integrates the helm-unittest
plugin, allowing users to define unit test paths per release and run them
via helmfile.
Closes#2376
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* feat: upgrade Helm version to v3.20.0 and v4.1.0
This commit updates the recommended Helm version from v3.19.5/v4.0.5 to
v3.20.0/v4.1.0 across all workflows, Dockerfiles, and application constants.
Changes:
- Update CI matrix to test with Helm v3.20.0 and v4.1.0
- Update .github/workflows/Makefile HELM_VERSION to v4.1.0
- Update Dockerfiles with new version and SHA256 checksums
- Update pkg/app/init.go HelmRecommendedVersion to v4.1.0
- Update go.mod helm.sh/helm/v3 to v3.20.0 and helm.sh/helm/v4 to v4.1.0
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: remove source field from e2e test helm plugin configs
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: remove source field from integration test helm plugin config
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: array merge regression - layer arrays now replace defaults (#2353)
PR #2288 introduced element-by-element array merging to fix#2281, but this
caused a regression where layer/environment arrays were merged instead of
replacing base arrays entirely.
This fix uses automatic sparse array detection:
- Arrays with nil values (from --state-values-set) merge element-by-element
- Arrays without nils (from layer YAML) replace entirely
This follows Helm's documented behavior where arrays replace rather than merge.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: use separate CLIOverrides field for element-by-element array merging
The previous approach using ArrayMergeStrategySparse detection didn't work
for --state-values-set array[0]=value because setting index 0 produces no
nils in the array.
This fix adds a CLIOverrides field to Environment that keeps CLI values
separate from layer values. CLI overrides are merged last using
ArrayMergeStrategyMerge (always element-by-element), while layer values
use the default strategy (arrays replace).
This ensures:
- --state-values-set array[0]=x only changes index 0, preserving other elements
- Layer/environment file arrays still replace base arrays entirely
- Issue #2281 fix is preserved (--state-values-set array[1].field=x works)
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: correct comment about array merge strategy in test
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: propagate Defaults in multi-part helmfiles and fix merge order
- Add Defaults field merging from ctxEnv to preserve base values across
helmfile parts separated by ---
- Fix merge order: current part values now correctly override previous
parts (was reversed, causing older values to win)
- Update 147 snapshot test files for new Environment log format with
CLIOverrides field
This completes the fix for issue #2353 by ensuring:
1. Layer arrays replace entirely (not element-by-element merge)
2. CLI --state-values-set sparse arrays still merge element-by-element
3. Multi-part helmfiles properly inherit and override values
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: address Copilot review comments
- Initialize EmptyEnvironment with empty maps to match New() constructor
- Update test comment to accurately describe ArrayMergeStrategySparse
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: ensure templates access merged values via .Environment.Values
This commit fixes a regression in the CLIOverrides integration where
templates accessing .Environment.Values couldn't see CLI override values.
Changes:
- Remove CLIOverrides-into-Values merge from Merge() to keep proper
layering order (Defaults → Values → CLIOverrides) in GetMergedValues()
- Update NewEnvironmentTemplateData to set envCopy.Values to the merged
values, ensuring templates see the same values via both .Values and
.Environment.Values
This ensures:
- Issue #2353: Layer arrays still replace entirely (Sparse strategy)
- Issue #2281: CLI sparse arrays still merge element-by-element
- Templates can access CLI overrides via .Environment.Values
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* docs: improve mergeSlices documentation per Copilot review
Address Copilot review comments on PR #2367:
- Document empty array edge case: explicitly setting [] clears base array
- Document recursive strategy propagation for nested map merging
- Add comprehensive behavior description for all array merge strategies
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: use merged values when rendering environment value files
Environment value files (*.yaml.gotmpl) can reference CLI values via
.Values. Previously, only env.Values was passed to template rendering,
which didn't include CLIOverrides.
Now we call env.GetMergedValues() to get Defaults + Values + CLIOverrides
before rendering, so templates can access CLI values like:
--state-values-set foo=bar
This fixes the state-values-set-cli-args-in-environments integration test.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
---------
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
fix: pass --kube-context to helm template when using jsonPatches (#2309)
When using jsonPatches or strategicMergePatches in helmfile, the
`helm template` command was not receiving the `--kube-context` flag.
This caused issues when `--dry-run=server` was used (introduced in
PR #2271 to support lookup() functions), because helm would connect
to the wrong cluster context.
Root Cause:
1. `flagsForTemplate()` did not call `appendConnectionFlags()`, unlike
`flagsForUpgrade()` and `flagsForDiff()` which both include this call.
2. `processChartification()` did not include `--kube-context` when
setting `chartifyOpts.TemplateArgs` for internal helm template calls.
Fix:
1. Added `appendConnectionFlags()` call to `flagsForTemplate()` to ensure
kube-context and other connection flags are passed to helm template.
2. Added `getKubeContext()` helper function that resolves kube-context
with proper priority: release > environment > helmDefaults.
3. Modified `processChartification()` to include `--kube-context` in
chartifyOpts.TemplateArgs when chartify needs to run helm template.
4. Added compatibility check for `--validate` flag to avoid Helm 4
mutual exclusion error between --validate and --dry-run (Issue #2355).
Fixes#2309
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* Initial plan
* Fix helmfile.d parallel processing to respect alphabetical order
Changed multiple helmfile.d file processing from parallel to sequential
to match the documented behavior. Files are now processed in alphabetical
order as stated in the documentation, ensuring predictable deployment order.
This fixes issues where deployments would break because parallel processing
did not respect the intended ordering (e.g., databases before applications).
- Modified visitStatesWithContext to process files sequentially
- Removed unused processStateFileParallel function
- Updated test snapshots to reflect deterministic ordering
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Add --sequential-helmfiles flag for opt-in sequential processing
Based on feedback, added a flag to control processing mode instead of
forcing sequential behavior. Parallel processing is now the default for
performance, with an opt-in --sequential-helmfiles flag for when order matters.
Changes:
- Added SequentialHelmfiles field to GlobalOptions and App
- Added --sequential-helmfiles CLI flag
- Modified visitStatesWithContext to check flag and choose parallel vs sequential
- Updated documentation to clarify parallel is default, sequential is opt-in
- Restored original parallel processing as default behavior
This gives users control over the behavior based on their needs.
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Add documentation and improve code readability
- Added documentation to processStateFileParallel function
- Extracted shouldProcessInParallel variable for clarity
- All tests pass, security scan clean
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Fix lint issue: remove trailing whitespace
Removed trailing whitespace from line 983 in pkg/app/app.go to fix
formatting lint error.
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* build(deps): update Helm v4 from 4.0.0 to 4.0.1
Update Helm v4 binary and Go library dependency to version 4.0.1.
Changes:
- Update helm.sh/helm/v4 Go module from v4.0.0 to v4.0.1
- Update Helm binary version in all Dockerfiles (alpine, ubuntu, debian)
- Update SHA256 checksums for linux/amd64 and linux/arm64
- Update CI workflow matrix to test against v4.0.1
- Update HelmRecommendedVersion constant in pkg/app/init.go
- Update test mocks to return v4.0.1 version string
- Update test plugin fixture version
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* build(deps): update helm-secrets from 4.7.0 to 4.7.4
Update helm-secrets plugin version across all configurations:
- Docker images (all 3 variants) - use ARG variable for version
- CI test matrix
- Integration test defaults
- Unit test fixtures and expectations
- HelmSecretsRecommendedVersion constant
- Dynamic plugin installation in exec.go
Also update plugin filename format from helm-secrets-*.tgz to
secrets-{version}.tgz to match the new release naming convention.
Update suppress-output-line-regex test expected output for Helm 4.0.1
which now suppresses Service diff after ipFamily normalization.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
---------
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
