* fix: resolve issues #2281, #2270, #2269, and #2247
This commit addresses four critical bugs in helmfile:
1. **Issue #2281**: Fix array merging in --state-values-set
- Problem: Arrays were being replaced entirely instead of merged element-by-element
- Root cause: MergeMaps() didn't handle arrays, and mergo.Merge was used in some places
- Solution:
* Enhanced MergeMaps() with mergeSlices() and toInterfaceSlice() functions
* Replaced mergo.Merge calls with MergeMaps in environment.go and create.go
* Arrays now merge element-by-element, with nested maps merged recursively
- Files changed:
* pkg/maputil/maputil.go - Added array merging logic
* pkg/maputil/maputil_test.go - Added comprehensive unit tests
* pkg/environment/environment.go - Use MergeMaps instead of mergo.Merge
* pkg/state/create.go - Use MergeMaps instead of mergo.Merge
* test/integration/test-cases/issue-2281-array-merge/ - Integration test
* test/integration/run.sh - Added new integration test
2. **Issue #2270**: Suppress AWS SDK debug logging
- Problem: AWS SDK debug logs exposing sensitive information (tokens, auth headers)
- Root cause: vals.New() called without LogOutput option
- Solution: Set LogOutput to io.Discard in ValsInstance()
- Files changed:
* pkg/plugins/vals.go - Added LogOutput: io.Discard option
3. **Issue #2269**: Fix helmDefaults.skipDeps and helmDefaults.skipRefresh being ignored
- Problem: skipRefresh only checked CLI flags, not helmDefaults or release settings
- Root cause: Incomplete calculation at line 1559 in state.go
- Solution: Added proper skipRefresh calculation mirroring skipDeps logic
- Files changed:
* pkg/state/state.go - Fixed skipRefresh calculation (lines 1522-1525, 1564)
* pkg/state/skip_test.go - Added unit tests for skipDeps and skipRefresh
4. **Issue #2247**: Allow OCI charts without explicit version
- Problem: OCI charts without version defaulted to "latest" which was then rejected
- Root cause: getOCIQualifiedChartName() defaulted chartVersion to "latest"
- Solution: Use release.Version directly without defaulting, only reject explicit "latest"
- Files changed:
* pkg/state/state.go - Remove default to "latest", use empty string
* pkg/state/oci_chart_version_test.go - Added comprehensive unit tests
* test/integration/test-cases/issue-2247/ - Integration test with registry
* test/integration/run.sh - Added new integration test
Fixes#2281, #2270, #2269, #2247
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: correct integration test for issue #2281 array merging
The helmfile template needed to pass the 'top' values to the chart
so that .Values.top is accessible in the template context.
Changes:
- Pass state values to chart values using toYaml
- Adjusted indentation for proper YAML structure
- Template now correctly accesses .Values.top for array data
Test output now matches expected output with proper element-by-element
array merging.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: make Helm version parsing more robust in issue-2247 test
Improved version parsing to handle edge cases in CI environments:
- Added fallback to 3.8 if version parsing fails
- Added default values for HELM_MAJOR and HELM_MINOR
- Prevents test failures due to version detection issues
This ensures the test runs correctly across different environments
and Helm versions.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* debug: add diagnostic output for issue-2247 test failure
Added debug logging to show:
- helmfile command output when it succeeds unexpectedly
- Helm version being used by the test
This will help diagnose why the validation isn't triggering in CI.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: make OCI 'latest' validation work for all Helm versions
The validation for explicit 'latest' in OCI charts was depending on
helm.IsVersionAtLeast("3.8.0") which could fail if Helm version
detection has issues in CI environments.
Changes:
- Remove Helm version check from validation
- Always reject explicit 'latest' for OCI charts
- Remove Helm version check from integration test
- Update unit tests to expect 'latest' to fail for all Helm versions
This ensures consistent behavior across all environments and
Helm versions, fixing the CI failure where helm version detection
was problematic.
Fixes integration test failure in CI.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: remove unused helm parameter from getOCIQualifiedChartName
Since the Helm version check was removed from the OCI validation,
the helm parameter is no longer needed in getOCIQualifiedChartName.
Changes:
- Removed helm parameter from function signature
- Updated all callers to not pass helm argument
- Removed unused mockHelmExec test implementation
- Removed unused imports (testutil, helmexec, chart)
This resolves the golangci-lint unparam error.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* test: update TestGetOCIQualifiedChartName to expect 'latest' rejection
Updated test case for Helm 3.7.0 to expect error when using 'latest'
since we now reject explicit 'latest' for all Helm versions, not just
>= 3.8.0.
This aligns the test with the updated validation logic that ensures
consistent behavior across all Helm versions.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: handle set -e in issue-2247 integration test
The integration test script is sourced by run.sh which has `set -e`
enabled. When helmfile commands fail (as expected for validation tests),
the script would exit immediately before capturing the exit code.
This fix temporarily disables `set -e` around each helmfile command that
may fail, allowing proper exit code capture and validation.
This resolves the persistent CI test failure where the test would exit
at Test 1.1 without showing any error message.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: add set -e handling for helm commands in issue-2247 test
Extends the previous set -e fix to cover helm package and push commands
in the registry tests (Test 2.2). These commands can fail and need proper
error handling without triggering immediate script exit.
This ensures:
- helm package failures are caught and handled gracefully
- helm push failures are caught and handled gracefully
- Test can skip registry tests and pass with validation-only results
- set -e is properly re-enabled after each command sequence
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
---------
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* feat: make environment context available
This feature adds the "{{.Environment.KubeContext}}" variable.
Discussion #829
Signed-off-by: sewieland <sebastian.wieland@iav.de>
* chore: fix tests which compare logging outputs
This commit adds an addtional space wherever needed to the expected log outputs due to the added "KubeContext" in the environment struct.
Discussion #829
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
* docs: added documentation for `Environment.KubeContext`
Discussion #829
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
* test: make sure the `Environment.KubeContext` is mapped out correctly
Discussion #829
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
---------
Signed-off-by: sewieland <sebastian.wieland@iav.de>
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
Co-authored-by: sewieland <sebastian.wieland@iav.de>
* fix env value lost in environment values
Signed-off-by: yxxhero <aiopsclub@163.com>
* add more test
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
This is a successor to #596. We need a smooth migration path from `gopkg.in/yaml.v2`, and this pull request moves it forward with `goccy/go-yaml` instead of `gopkg.in/yaml.v3`. Merging this unblocks users stuck in Helmfile v0.146.x or earlier due to #435, so that they can upgrade to 0.147.x or greater without updating their helmfile configs.
We previously tried to upgrade to `yaml.v3` (https://github.com/helmfile/helmfile/issues/394) in Helmfile v0.x, presuming it won't break anything. Apparently, it broke use-cases where you want to layer release's `values` field over three or more release templates and releases (#435).
We then tried to bring back `yaml.v2` for Helmfile v0.x and keep `yaml.v3` for the upcoming Helmfile v1. However, it failed due to incompatibility in the Unmarshaller interface between `yaml.v2` and `yaml.v3` (https://github.com/helmfile/helmfile/pull/596).
`goccy/go-yaml` is, from my observation, a well-maintained alternative to `yaml.v2`. One of its premises is that it enables us to swap the implementation from `gopkg.in/yaml.v2` to `goccy/go-yaml` just by replacing the import directive. It seems to use the same `Unmarshaller` interface as yaml.v2 too.
Once this PR gets merged, I'd like to follow-up with adding a new build-time variable and an envvar to set the proper default for the yaml parser Helmfile uses and the ability to switch the parser at runtime. All in all, the next Helmfile release, v0.150.0 will get reverted to use `gopkg.in/yaml.v2` by default which resolves#435.
New users who started using Helmfile since any of v0.148.0, v0.148.1, and v0.149.0 might be already relying on the new behavior, They might need to specify a new envvar to enable `goccy/go-yaml`.
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: yxxhero <aiopsclub@163.com>
This commit proposes a potential solution for
https://github.com/roboll/helmfile/issues/1201
The gist is that, if prestate rendering fails, for any reason,
we do not populate the .Values in the second pass renderer.
I think that what have been expected in this case is to populate the
.Values irregardless.
pkg/app/two_pass_renderer.go
- Migrated to use finalEnv.GetMergedValues()
pkg/environment/environment.go
- Introduced GetMergedValues, which merges the environment's defaults
and current values, and then casts the keys to string;
This was previously defined in HelmState.Values() - however, as this
method is only concerned with the environment, I think it's more
appropriate for it to sit here.
pkg/state/state_exec_tmpl.go
- Extracted out HelmState.Values() to environment.go, see above
This adds `values` to state files as proposed in #640.
```yaml
values:
- key1: val1
- defaults.yaml
environments:
default:
- values:
- environments/default.yaml
production:
- values:
- environments/production.yaml
```
`{{ .Valuese.key1 }}` evaluates to `val1` if and only if it is not overrode via the production or the default env, or command-line args.
Resolves#640
Aditya Menon
yxxhero
SeWieland
Yusuke Kuoka
yxxhero
austin ce
Danny Shemesh