* feat(state): add mergeStrategy field to EnvironmentSpec
Introduces a per-environment mergeStrategy with valid values "override"
(default, current behavior) and "fallback". This commit only adds the
field, the constants, and a parse-time validator; the loader still
ignores the value, so behavior is unchanged.
Subsequent commits thread the value through the values loader and
implement the fallback semantics.
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* refactor(state): thread mergeStrategy through values loader
Adds a mergeStrategy string parameter to LoadEnvironmentValues,
loadValuesEntries, and mapMerge so the value can flow from
EnvironmentSpec down to the merge call site. Behavior is unchanged in
this commit; mapMerge ignores the strategy and the next commit
implements the fallback semantics.
Top-level state.DefaultValues and the --state-values-file/-set loaders
are passed an empty strategy ("") since they have no per-environment
spec to consult and stay on the default override behavior.
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* feat(state): implement fallback merge strategy
Adds a hand-rolled fallbackDeepMerge that, unlike mergo, preserves
keys present in the destination even when their value is the zero
value (false, 0, "", nil, empty list/map). mapMerge dispatches to it
when mergeStrategy == "fallback"; "override" and the empty default
keep using mergo with WithOverride so existing behaviour is unchanged.
Validation lives at the entry of LoadEnvironmentValues so a single
chokepoint guards the field. Invalid values produce an error naming
both the offending value and the valid options.
Tests cover: first-file-wins precedence, gap filling, deep nested
merge, three-file chains, explicit zero-value preservation (the case
naïve mergo gets wrong), explicit nil preservation, inline map
entries, override regression, default-equals-override equivalence,
and invalid-strategy errors.
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* feat(state): expose prior-file values in fallback template context
Under mergeStrategy: fallback, .gotmpl values files can now reference
values from earlier files in the same `values:` list via .Values
(e.g. `service.domain: "service.{{ .Values.cluster.domain }}"`).
The accumulated result is layered under env.GetMergedValues so env
defaults, env values, and CLI overrides still win on overlap. Override
mode keeps the historical template context — unchanged — so this is
strictly opt-in via the mergeStrategy field.
Together with the precedence flip from the previous commit, this lets
users replace the brittle two-stage `merged-values.yaml.gotmpl`
workaround with native helmfile syntax.
Tests cover the headline cross-file template reference case and pin
the override-mode contract that prior-file values stay invisible.
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* docs: document mergeStrategy and fallback semantics
Adds a new section to values-and-merging.md describing the override vs
fallback strategies, the explicit-zero-value preservation guarantee,
and the cross-file template reference behavior. Adds a brief pointer
to environments.md so users land on the new field from the
environment values discussion.
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* refactor(state): reuse maputil.MergeMaps for fallback merge
Replaces the hand-rolled fallbackDeepMerge with a single call to
maputil.MergeMaps, swapping its arguments so the accumulated dest wins
over the new src file. Same first-file-wins semantic, fewer lines, and
the fallback path now inherits the same slice merge strategies the
rest of helmfile already uses.
The one observable behavior shift is for explicit nil values: under
fallback, nil in an earlier file no longer 'wins' over a non-nil value
in a later file — instead it falls through (matching MergeMaps' rule
that nil from the override side only fills missing keys). This is
internally consistent: nil-overwrites is an mergo.WithOverride quirk
that lives only in the override path. The renamed test
NilFallsThroughToFallback pins the new behavior with a comment
referencing the contrast with override mode (Issue1154).
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
---------
Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>
* fix: array merge regression - layer arrays now replace defaults (#2353)
PR #2288 introduced element-by-element array merging to fix#2281, but this
caused a regression where layer/environment arrays were merged instead of
replacing base arrays entirely.
This fix uses automatic sparse array detection:
- Arrays with nil values (from --state-values-set) merge element-by-element
- Arrays without nils (from layer YAML) replace entirely
This follows Helm's documented behavior where arrays replace rather than merge.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: use separate CLIOverrides field for element-by-element array merging
The previous approach using ArrayMergeStrategySparse detection didn't work
for --state-values-set array[0]=value because setting index 0 produces no
nils in the array.
This fix adds a CLIOverrides field to Environment that keeps CLI values
separate from layer values. CLI overrides are merged last using
ArrayMergeStrategyMerge (always element-by-element), while layer values
use the default strategy (arrays replace).
This ensures:
- --state-values-set array[0]=x only changes index 0, preserving other elements
- Layer/environment file arrays still replace base arrays entirely
- Issue #2281 fix is preserved (--state-values-set array[1].field=x works)
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: correct comment about array merge strategy in test
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: propagate Defaults in multi-part helmfiles and fix merge order
- Add Defaults field merging from ctxEnv to preserve base values across
helmfile parts separated by ---
- Fix merge order: current part values now correctly override previous
parts (was reversed, causing older values to win)
- Update 147 snapshot test files for new Environment log format with
CLIOverrides field
This completes the fix for issue #2353 by ensuring:
1. Layer arrays replace entirely (not element-by-element merge)
2. CLI --state-values-set sparse arrays still merge element-by-element
3. Multi-part helmfiles properly inherit and override values
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: address Copilot review comments
- Initialize EmptyEnvironment with empty maps to match New() constructor
- Update test comment to accurately describe ArrayMergeStrategySparse
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: ensure templates access merged values via .Environment.Values
This commit fixes a regression in the CLIOverrides integration where
templates accessing .Environment.Values couldn't see CLI override values.
Changes:
- Remove CLIOverrides-into-Values merge from Merge() to keep proper
layering order (Defaults → Values → CLIOverrides) in GetMergedValues()
- Update NewEnvironmentTemplateData to set envCopy.Values to the merged
values, ensuring templates see the same values via both .Values and
.Environment.Values
This ensures:
- Issue #2353: Layer arrays still replace entirely (Sparse strategy)
- Issue #2281: CLI sparse arrays still merge element-by-element
- Templates can access CLI overrides via .Environment.Values
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* docs: improve mergeSlices documentation per Copilot review
Address Copilot review comments on PR #2367:
- Document empty array edge case: explicitly setting [] clears base array
- Document recursive strategy propagation for nested map merging
- Add comprehensive behavior description for all array merge strategies
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: use merged values when rendering environment value files
Environment value files (*.yaml.gotmpl) can reference CLI values via
.Values. Previously, only env.Values was passed to template rendering,
which didn't include CLIOverrides.
Now we call env.GetMergedValues() to get Defaults + Values + CLIOverrides
before rendering, so templates can access CLI values like:
--state-values-set foo=bar
This fixes the state-values-set-cli-args-in-environments integration test.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
---------
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* feat: inject cli state values (--state-values-set) into environment templating context
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
* test: added envvals_loader unit test for environment values
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
* test: added 'state values set cli args in environments' integration test
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
* fix: merge environments before loadValuesEntries
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
* fix: 'state values set cli args in environments' integration test
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
---------
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
* fix env value lost in environment values
Signed-off-by: yxxhero <aiopsclub@163.com>
* add more test
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
This is a successor to #596. We need a smooth migration path from `gopkg.in/yaml.v2`, and this pull request moves it forward with `goccy/go-yaml` instead of `gopkg.in/yaml.v3`. Merging this unblocks users stuck in Helmfile v0.146.x or earlier due to #435, so that they can upgrade to 0.147.x or greater without updating their helmfile configs.
We previously tried to upgrade to `yaml.v3` (https://github.com/helmfile/helmfile/issues/394) in Helmfile v0.x, presuming it won't break anything. Apparently, it broke use-cases where you want to layer release's `values` field over three or more release templates and releases (#435).
We then tried to bring back `yaml.v2` for Helmfile v0.x and keep `yaml.v3` for the upcoming Helmfile v1. However, it failed due to incompatibility in the Unmarshaller interface between `yaml.v2` and `yaml.v3` (https://github.com/helmfile/helmfile/pull/596).
`goccy/go-yaml` is, from my observation, a well-maintained alternative to `yaml.v2`. One of its premises is that it enables us to swap the implementation from `gopkg.in/yaml.v2` to `goccy/go-yaml` just by replacing the import directive. It seems to use the same `Unmarshaller` interface as yaml.v2 too.
Once this PR gets merged, I'd like to follow-up with adding a new build-time variable and an envvar to set the proper default for the yaml parser Helmfile uses and the ability to switch the parser at runtime. All in all, the next Helmfile release, v0.150.0 will get reverted to use `gopkg.in/yaml.v2` by default which resolves#435.
New users who started using Helmfile since any of v0.148.0, v0.148.1, and v0.149.0 might be already relying on the new behavior, They might need to specify a new envvar to enable `goccy/go-yaml`.
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: yxxhero <aiopsclub@163.com>
Currently it's not possible to use `.Environment` values in `*.gomtpl` files. The documentation states the opposite:
https://github.com/roboll/helmfile#environment (2nd paragraph).
The problem is already described in #1090.
This PR fixes this bug.
Fixes#1090
Co-authored-by: Peter Aichinger <petera@topdesk.com>
since mergo had been bumped to 0.3.9 an environment value like:
```
affinity: {}
```
was not merged properly (not merged at all) instead it threw an error
that it cannot find the key "affinity" in the environment values even
though it was outputted in debug output as read in properly.
Probably since #647 helmfile has been unable to merge nested maps in environment values if they were loaded from files. This fixes it.
The relevant test is also enhanced so that no further regression like this happens.
Fixes#677
Dominik Schmidt
Aditya Menon
Vince-Chenal
yxxhero
xtphate
Quan TRAN
Yusuke Kuoka
yxxhero
Arkaitz Jimenez
austin ce
pjotre86
Kevin J. Qiu
Daniel Elo
Erik Forsberg