feat(state): add mergeStrategy: fallback for first-file-wins env values (#2578)

* feat(state): add mergeStrategy field to EnvironmentSpec

Introduces a per-environment mergeStrategy with valid values "override"
(default, current behavior) and "fallback". This commit only adds the
field, the constants, and a parse-time validator; the loader still
ignores the value, so behavior is unchanged.

Subsequent commits thread the value through the values loader and
implement the fallback semantics.

Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>

* refactor(state): thread mergeStrategy through values loader

Adds a mergeStrategy string parameter to LoadEnvironmentValues,
loadValuesEntries, and mapMerge so the value can flow from
EnvironmentSpec down to the merge call site. Behavior is unchanged in
this commit; mapMerge ignores the strategy and the next commit
implements the fallback semantics.

Top-level state.DefaultValues and the --state-values-file/-set loaders
are passed an empty strategy ("") since they have no per-environment
spec to consult and stay on the default override behavior.

Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>

* feat(state): implement fallback merge strategy

Adds a hand-rolled fallbackDeepMerge that, unlike mergo, preserves
keys present in the destination even when their value is the zero
value (false, 0, "", nil, empty list/map). mapMerge dispatches to it
when mergeStrategy == "fallback"; "override" and the empty default
keep using mergo with WithOverride so existing behaviour is unchanged.

Validation lives at the entry of LoadEnvironmentValues so a single
chokepoint guards the field. Invalid values produce an error naming
both the offending value and the valid options.

Tests cover: first-file-wins precedence, gap filling, deep nested
merge, three-file chains, explicit zero-value preservation (the case
naïve mergo gets wrong), explicit nil preservation, inline map
entries, override regression, default-equals-override equivalence,
and invalid-strategy errors.

Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>

* feat(state): expose prior-file values in fallback template context

Under mergeStrategy: fallback, .gotmpl values files can now reference
values from earlier files in the same `values:` list via .Values
(e.g. `service.domain: "service.{{ .Values.cluster.domain }}"`).

The accumulated result is layered under env.GetMergedValues so env
defaults, env values, and CLI overrides still win on overlap. Override
mode keeps the historical template context — unchanged — so this is
strictly opt-in via the mergeStrategy field.

Together with the precedence flip from the previous commit, this lets
users replace the brittle two-stage `merged-values.yaml.gotmpl`
workaround with native helmfile syntax.

Tests cover the headline cross-file template reference case and pin
the override-mode contract that prior-file values stay invisible.

Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>

* docs: document mergeStrategy and fallback semantics

Adds a new section to values-and-merging.md describing the override vs
fallback strategies, the explicit-zero-value preservation guarantee,
and the cross-file template reference behavior. Adds a brief pointer
to environments.md so users land on the new field from the
environment values discussion.

Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>

* refactor(state): reuse maputil.MergeMaps for fallback merge

Replaces the hand-rolled fallbackDeepMerge with a single call to
maputil.MergeMaps, swapping its arguments so the accumulated dest wins
over the new src file. Same first-file-wins semantic, fewer lines, and
the fallback path now inherits the same slice merge strategies the
rest of helmfile already uses.

The one observable behavior shift is for explicit nil values: under
fallback, nil in an earlier file no longer 'wins' over a non-nil value
in a later file — instead it falls through (matching MergeMaps' rule
that nil from the override side only fills missing keys). This is
internally consistent: nil-overwrites is an mergo.WithOverride quirk
that lives only in the override path. The renamed test
NilFallsThroughToFallback pins the new behavior with a comment
referencing the contrast with override mode (Issue1154).

Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>

---------

Signed-off-by: Dominik Schmidt <dev@dominik-schmidt.de>

docs/environments.md

@@ -104,6 +104,21 @@ releases:
   ...
 ```
 
+#### Merge strategy
+
+By default, when several files are listed under `values:`, later files override earlier files. Set `mergeStrategy: fallback` on the environment to flip the precedence so earlier files win and later files only fill missing keys:
+
+```yaml
+environments:
+  production:
+    mergeStrategy: fallback
+    values:
+      - cluster-specific.yaml   # wins on every key it defines
+      - shared-defaults.yaml.gotmpl
+```
+
+Under `fallback`, explicit non-nil values in the earlier file (including zero values like `false`, `0`, `""`, and empty list) are preserved against any later file, while maps are deep-merged so later files may still add nested keys. A later `.gotmpl` file can also reference values from earlier files via `.Values`. See [Merge Strategy: override vs fallback](values-and-merging.md#4a-merge-strategy-override-vs-fallback) for the full semantics, including how explicit `null` is handled.
+
 #### HCL specifications
 
 Since Helmfile v0.164.0, HCL language is supported for environment values only.

docs/values-and-merging.md

@@ -116,6 +116,54 @@ environments:
       - secrets.yaml       # Merged last (step 3) - highest priority
 ```
 
+### 4a. Merge Strategy: `override` vs `fallback`
+
+By default, when an environment lists multiple files under `values:`, **later files override earlier files** (the historical helmfile behavior, equivalent to `mergeStrategy: override`).
+
+You can flip this per environment so that **earlier files take precedence** and later files only fill in missing keys:
+
+```yaml
+environments:
+  production:
+    mergeStrategy: fallback
+    values:
+      - cluster-specific.yaml   # wins on every key it defines
+      - shared-defaults.yaml    # only fills gaps
+```
+
+Under `fallback`:
+
+- An explicit non-nil value in an earlier file is preserved against any later file, including the zero values `false`, `0`, `""`, and empty list. An explicit `enabled: false` in `cluster-specific.yaml` is *not* silently overwritten by `enabled: true` from `shared-defaults.yaml`.
+- Maps are deep-merged. An earlier map does not block later files from adding nested keys it didn't set; only the keys an earlier file explicitly defines win on conflict.
+- An explicit `null` in an earlier file falls through to a later file's value, matching how `MergeMaps` treats nil from the override side elsewhere in helmfile.
+- Within a single `values:` entry that expands to multiple files (e.g. via a glob), the **first** file in the expansion wins.
+- A later `.gotmpl` values file can reference values from earlier files via `.Values`, so derived defaults work natively:
+
+  ```yaml
+  # cluster-specific.yaml
+  cluster:
+    domain: prod.example.com
+  ```
+
+  ```yaml
+  # shared-defaults.yaml.gotmpl
+  service:
+    domain: "service.{{ .Values.cluster.domain }}"
+  ```
+
+  → `service.domain: service.prod.example.com`. Under `mergeStrategy: override` this cross-file template reference is not available.
+
+Valid values: `override` (default) and `fallback`. Any other value is rejected at load time.
+
+#### Interaction with `.hcl` values files
+
+`.hcl` files are evaluated as a single unit so that HCL `locals` and `values` blocks can reference each other across files. Helmfile collects every `.hcl` entry from the `values:` list, renders them together, and merges the combined result after the YAML pass. Two consequences worth knowing:
+
+- `mergeStrategy` does not reshuffle the position of HCL within the list. HCL's combined output is always merged after the YAML pass. Under `override` it overrides YAML on conflicts (the historical behavior); under `fallback` it fills gaps only.
+- Among multiple `.hcl` files, the precedence is HCL's own (last-file-wins within HCL), independent of `mergeStrategy`. The strategy applies at the YAML-vs-HCL boundary, not within HCL.
+
+If you need first-file-wins precedence between specific HCL files, restructure them into one HCL file (or split the values into YAML).
+
 ### 5. CLI Overrides
 
 The highest priority values come from CLI flags: