This PR fixes four related bugs affecting chart preparation, caching,
and OCI registry authentication.
Issue #2295: OCI chart cache conflicts with parallel helmfile processes
- Added filesystem-level locking using flock for cross-process sync
- Implements double-check locking pattern for efficiency
- Retry logic with 5-minute timeout and 3 retries
- Refactored into reusable acquireChartLock() helper function
- Added refresh marker coordination for cross-process cache management
Issue #2296: helmDefaults.skipDeps and helmDefaults.skipRefresh ignored
- Check both CLI options AND helmDefaults when deciding to skip repo sync
Issue #2297: Local chart + transformers causes panic
- Normalize local chart paths to absolute before calling chartify
OCI Registry Login URL Fix:
- Added extractRegistryHost() to extract just the registry host from URLs
- Fixed SyncRepos to use extracted host for OCI registry login
- e.g., "account.dkr.ecr.region.amazonaws.com/charts" ->
"account.dkr.ecr.region.amazonaws.com"
Test Plan:
- Unit tests for issues #2295, #2296, #2297
- Unit tests for OCI registry login (extractRegistryHost, SyncRepos_OCI)
- Integration tests for issues #2295 and #2297
- All existing unit tests pass (including TestLint)
Fixes#2295Fixes#2296Fixes#2297
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: resolve issues #2281, #2270, #2269, and #2247
This commit addresses four critical bugs in helmfile:
1. **Issue #2281**: Fix array merging in --state-values-set
- Problem: Arrays were being replaced entirely instead of merged element-by-element
- Root cause: MergeMaps() didn't handle arrays, and mergo.Merge was used in some places
- Solution:
* Enhanced MergeMaps() with mergeSlices() and toInterfaceSlice() functions
* Replaced mergo.Merge calls with MergeMaps in environment.go and create.go
* Arrays now merge element-by-element, with nested maps merged recursively
- Files changed:
* pkg/maputil/maputil.go - Added array merging logic
* pkg/maputil/maputil_test.go - Added comprehensive unit tests
* pkg/environment/environment.go - Use MergeMaps instead of mergo.Merge
* pkg/state/create.go - Use MergeMaps instead of mergo.Merge
* test/integration/test-cases/issue-2281-array-merge/ - Integration test
* test/integration/run.sh - Added new integration test
2. **Issue #2270**: Suppress AWS SDK debug logging
- Problem: AWS SDK debug logs exposing sensitive information (tokens, auth headers)
- Root cause: vals.New() called without LogOutput option
- Solution: Set LogOutput to io.Discard in ValsInstance()
- Files changed:
* pkg/plugins/vals.go - Added LogOutput: io.Discard option
3. **Issue #2269**: Fix helmDefaults.skipDeps and helmDefaults.skipRefresh being ignored
- Problem: skipRefresh only checked CLI flags, not helmDefaults or release settings
- Root cause: Incomplete calculation at line 1559 in state.go
- Solution: Added proper skipRefresh calculation mirroring skipDeps logic
- Files changed:
* pkg/state/state.go - Fixed skipRefresh calculation (lines 1522-1525, 1564)
* pkg/state/skip_test.go - Added unit tests for skipDeps and skipRefresh
4. **Issue #2247**: Allow OCI charts without explicit version
- Problem: OCI charts without version defaulted to "latest" which was then rejected
- Root cause: getOCIQualifiedChartName() defaulted chartVersion to "latest"
- Solution: Use release.Version directly without defaulting, only reject explicit "latest"
- Files changed:
* pkg/state/state.go - Remove default to "latest", use empty string
* pkg/state/oci_chart_version_test.go - Added comprehensive unit tests
* test/integration/test-cases/issue-2247/ - Integration test with registry
* test/integration/run.sh - Added new integration test
Fixes#2281, #2270, #2269, #2247
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: correct integration test for issue #2281 array merging
The helmfile template needed to pass the 'top' values to the chart
so that .Values.top is accessible in the template context.
Changes:
- Pass state values to chart values using toYaml
- Adjusted indentation for proper YAML structure
- Template now correctly accesses .Values.top for array data
Test output now matches expected output with proper element-by-element
array merging.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: make Helm version parsing more robust in issue-2247 test
Improved version parsing to handle edge cases in CI environments:
- Added fallback to 3.8 if version parsing fails
- Added default values for HELM_MAJOR and HELM_MINOR
- Prevents test failures due to version detection issues
This ensures the test runs correctly across different environments
and Helm versions.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* debug: add diagnostic output for issue-2247 test failure
Added debug logging to show:
- helmfile command output when it succeeds unexpectedly
- Helm version being used by the test
This will help diagnose why the validation isn't triggering in CI.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: make OCI 'latest' validation work for all Helm versions
The validation for explicit 'latest' in OCI charts was depending on
helm.IsVersionAtLeast("3.8.0") which could fail if Helm version
detection has issues in CI environments.
Changes:
- Remove Helm version check from validation
- Always reject explicit 'latest' for OCI charts
- Remove Helm version check from integration test
- Update unit tests to expect 'latest' to fail for all Helm versions
This ensures consistent behavior across all environments and
Helm versions, fixing the CI failure where helm version detection
was problematic.
Fixes integration test failure in CI.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: remove unused helm parameter from getOCIQualifiedChartName
Since the Helm version check was removed from the OCI validation,
the helm parameter is no longer needed in getOCIQualifiedChartName.
Changes:
- Removed helm parameter from function signature
- Updated all callers to not pass helm argument
- Removed unused mockHelmExec test implementation
- Removed unused imports (testutil, helmexec, chart)
This resolves the golangci-lint unparam error.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* test: update TestGetOCIQualifiedChartName to expect 'latest' rejection
Updated test case for Helm 3.7.0 to expect error when using 'latest'
since we now reject explicit 'latest' for all Helm versions, not just
>= 3.8.0.
This aligns the test with the updated validation logic that ensures
consistent behavior across all Helm versions.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: handle set -e in issue-2247 integration test
The integration test script is sourced by run.sh which has `set -e`
enabled. When helmfile commands fail (as expected for validation tests),
the script would exit immediately before capturing the exit code.
This fix temporarily disables `set -e` around each helmfile command that
may fail, allowing proper exit code capture and validation.
This resolves the persistent CI test failure where the test would exit
at Test 1.1 without showing any error message.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: add set -e handling for helm commands in issue-2247 test
Extends the previous set -e fix to cover helm package and push commands
in the registry tests (Test 2.2). These commands can fail and need proper
error handling without triggering immediate script exit.
This ensures:
- helm package failures are caught and handled gracefully
- helm push failures are caught and handled gracefully
- Test can skip registry tests and pass with validation-only results
- set -e is properly re-enabled after each command sequence
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
---------
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
This commit adds comprehensive support for Helm 4 while maintaining
full backward compatibility with Helm 3. The implementation includes:
- Updated helm version detection to support both Helm 3 and Helm 4
- Added HELMFILE_HELM4 environment variable to control Helm version
- Modified helm execution paths to handle version-specific binaries
- Updated helm plugin installation to support split architecture
- Helm 4: Uses split plugin architecture (3 separate .tgz files)
- helm-secrets.tgz
- helm-secrets-getter.tgz
- helm-secrets-post-renderer.tgz
- Helm 3: Continues using single plugin installation
- Updated Dockerfiles, CI workflows, and core installation code
- Helm 4 requires post-renderers to be plugins, not executable scripts
- Created Helm plugin structure for integration tests
- Updated helmfile.yaml templates to dynamically select renderer type
- Added test plugins: add-cm, add-cm1, add-cm2
- Updated integration tests for Helm 3/4 compatibility
- Created Helm 4 variant expected output files
- Fixed test determinism issues (repo cleanup between iterations)
- Added version-specific output filtering for warnings/messages
- Updated workflows to test both Helm 3 and Helm 4
- Matrix testing across Helm versions
- Updated helm-diff to v3.14.0 for compatibility
- Updated README and docs with Helm 4 information
- Added migration guidance
- Updated version requirements
All changes are backward compatible - existing Helm 3 users will
see no behavior changes.
fix: update Helm 4 lint expected output to match filtered output
The grep filter removes the semver warning, so the expected output
should not include it. Updated lint-helm4 files to match the filtered
output (warning removed, no extra blank line).
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* feat: Add updateStrategy option in the state file with 'reinstall'/'reinstallIfForbidden' choices to uninstall and apply the specific release(s) (if forbidden to update)
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Fix unit tests related to the new updateStrategy feature
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Fix unit tests related to the new updateStrategy feature
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Resolve linter issue due to cognitive complexity
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Updated index.md to describe the possible values of updateStrategy
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Add validation of updateStrategy parameter and unit test
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Updated unit test
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Removed 'reinstall' update strategy option to only have reinstallIfForbidden, cleanup of pre-sync changes, adapted unit tests
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Display affected releases that were reinstalled
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Make sure to add --wait when deleting a release to be reinstalled due to reinstallIfForbidden
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Apply suggestions from Copilot code review
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
---------
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Initial plan
* Add enableDNS flag support to diff command
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Extract EnableDNS flag logic into reusable function
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Initial plan
* Implement --timeout flag for helmfile sync command
- Add Timeout field to SyncOptions struct in pkg/config/sync.go
- Add --timeout flag to sync command in cmd/sync.go
- Add Timeout field to SyncOpts struct in pkg/state/state.go
- Modify timeoutFlags() function to prioritize CLI timeout over release and default configs
- Add test case to verify CLI timeout overrides other timeout settings
- Follow same pattern as existing --wait and --wait-for-jobs flags
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Fix lint issues: format test struct fields properly
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Update docs: Add --timeout flag documentation for helmfile sync command
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Feat: reuseValues in release
Adding properties to set reuseValues flag on release-level.
Signed-off-by: Adam Blasko <adam.blasko1@gmail.com>
* feat: fixing tests
Most of the tests had issues with flag order, which changed due to moving the value control flags out of the "common flags" for diff
Signed-off-by: Adam Blasko <adam.blasko1@gmail.com>
* fix: fixing lint issue
Signed-off-by: Adam Blasko <adam.blasko1@gmail.com>
---------
Signed-off-by: Adam Blasko <adam.blasko1@gmail.com>
* eat: adding ability for for charts to be pulled without HTTPS
accomplished by:
- Adding PlainHttp attribute to RepositorySpec., HelmDefault, ReleaseSpec
- Adding UnitTests for getOCIChart Flags.
- Adding funciton and unitTests for getChartDownload
- Changing and refactoring how Flags are added to getOCIChart.
Resolves#1224
Signed-off-by: Peter Halliday <peter.halliday@servicenow.com>
* Pass PlainHttp to OCI repo options, fix unit test
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Fix doc
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Use repository fields in non-OCI chart download options
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Update hashes in TestGenerateID
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Make sure repo exists when using its options
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Do not add TLS options if PlainHttp is set, adapt unit tests
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Fix doc
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Remove 'else if' from appendChartDownloadFlags
Signed-off-by: Pascal Rivard <privard@rbbn.com>
---------
Signed-off-by: Peter Halliday <peter.halliday@servicenow.com>
Signed-off-by: Pascal Rivard <privard@rbbn.com>
Co-authored-by: Peter Halliday <peter.halliday@servicenow.com>
Co-authored-by: Pascal Rivard <privard@rbbn.com>
* fix: inject KubeVersion and ApiVersions in Chartify rendering
fixes#1623
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
* test: appendApiVersionsFlags
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
* test: add case for appendApiVersionsFlags
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
---------
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
* add the SyncArgs option
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
* add syncArgs to helmDefaults and update index.md
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
* add --sync-args flags to helmfile sync
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
* add tests for appendExtraDiffFlags and appendExtraSyncFlags
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
---------
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
Ref https://github.com/helmfile/helmfile/discussions/593
* implemented --reset-values flag overriding .helmDefaults.reuseValues=true
* tests for --reset-values flag
* fixed pkg/app tests
* added tests for both reset & reuse flags given
* added appendValuesControlMode method for --reuse/reset-values flag processing
* updated code comments & docs
Signed-off-by: Karol Ossowski <k@koralsky.pl>
1. only implement post-renderer flags this patch
2. As mumoshu advise, add helmfile flags `--post-render` and add the
postRenderer config in helmDefaults and release. the priority is
helmfile flags > release > helmDefaults.
3. fix the test case in state_test.go and some other tests.
Signed-off-by: guofutan <guofutan@tencent.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Allow configuring the lockfile in the state. This makes it possible for
example maintain a lock per environment.
Signed-off-by: Lassi Pölönen <lassi.polonen@iki.fi>
Signed-off-by: Lassi Pölönen <lassi.polonen@iki.fi>
Aditya Menon
Simon Bouchard
Copilot
Adam Blasko
yxxhero
Purple Clay
Felipe Santos
ennekein
Henrik Huitti
Thomas Loubiou
Nick Van Dyck
Tim Ramlot
Eduardo Naves
Virgínia Tavares
WrenIX
Dejan Benedik
William Lahti
Vincent Stchu
Soh Takada
koralsky
xiaomudk
guofutan
koralsky
yxxhero
Lassi Pölönen
Helge Eichelberg
Quan TRAN