* fix: resolve OCI version constraints before deriving the shared chart cache path
When an OCI release uses a semver constraint (e.g. `~1`, `^2.0.0`, `*`),
`getOCIChartPath` currently derives the on-disk cache directory from the
raw constraint string via `safeVersionPath`, which substitutes constraint
characters (`~`, `^`, `>`, `<`, `!`, `|`, `=`, ` `, `,`, `*`) with `_`.
So `version: ~1` becomes `.../mychart/_1/` on disk. `acquireChartLock`
then refuses to refresh anything under the shared cache dir to avoid
race conditions between concurrent processes, so once the constraint is
first resolved and written to `_1/`, every subsequent render on that
machine (or that container replica) returns the pinned tarball
regardless of newer matching tags being published.
In multi-pod deployments like ArgoCD's argocd-repo-server this shows up
as intermittent stale renders: different pods populate their caches at
different moments and serve different snapshots of the same `~1`
release forever.
Fix: for OCI releases whose `version` looks like a constraint, run
`helm show chart <ref> --version <constraint> [flags]` and use the
returned metadata.Version as the effective version for all downstream
cache-key and path derivation. Helm already resolves the constraint
against the registry and returns the concrete matching Chart.yaml.
Callers get a content-addressable cache path (`.../mychart/1.0.1/`)
that naturally invalidates when the constraint resolves to a new
version. Exact-version releases and non-OCI releases skip the extra
call.
Adds an opt-out `resolveOCIVersions` field on `helmDefaults` and
`ReleaseSpec` (both default true). If the resolution call fails
transiently, the resolver logs a warning and falls back to the
pre-fix behavior so a network hiccup doesn't break rendering.
Adds `ShowChartWithFlags` to helmexec.Interface so the existing
`ShowChart` API stays backwards compatible.
Resolves#2766
Signed-off-by: Samuel Archambault <samuel.archambault@getmaintainx.com>
* refactor: move ShowChartWithFlags to a ChartInspector capability interface
Address Copilot review feedback on PR #2768: adding a method to the
exported helmexec.Interface is a source-breaking change for every
third-party implementation and mock of that interface, even though
ShowChart itself stayed backward-compatible.
Move ShowChartWithFlags off Interface and onto a new capability
interface, helmexec.ChartInspector, following the same pattern used
by the existing DependencyUpdater capability interface. The concrete
execer and the exectest.Helm test stub still satisfy it (they
already have the method); the OCI resolver in state.HelmState now
type-asserts and falls back to the pre-fix caching behavior when the
capability is absent, so downstream callers with their own
helmexec.Interface implementations keep compiling untouched.
Adds TestResolveOCIConstraintVersion_ChartInspectorFallback that
exercises the type-assertion path with a helm value that satisfies
Interface but deliberately does not satisfy ChartInspector.
Reverts ShowChartWithFlags additions from testutil.noCallHelmExec
and app_test.mockHelmExec since Interface no longer requires them.
Signed-off-by: Samuel Archambault <samuel.archambault@getmaintainx.com>
* fix: detect wildcard-segment semver constraints (1.x, 1.X) as constraints
Address Copilot review feedback on PR #2768: the previous
isVersionConstraint implementation scanned the input for operator
characters (~, ^, >, <, !, |, =, space, comma, *). Masterminds/semver
also accepts wildcard-segment constraints like "1.x", "1.X", "1.x.x",
and "1.2.X" that contain no operator characters. Those would slip past
the classifier, bypass OCI constraint resolution, and remain cached
forever under the raw ".../mychart/1.x/" path — the same stale-cache
bug the PR is meant to fix.
Replace the character scan with a semver-parser-based check: a value
is a constraint iff Masterminds/semver rejects it as a NewVersion but
accepts it as a NewConstraint. This correctly:
- Recognizes wildcard forms (1.x, 1.X, 1.x.x, 1.2.x, v1.x).
- Preserves exact versions where "x" appears in prerelease metadata
("1.0.0-alpha.x") or build metadata ("1.0.0+x", "1.0.0+build.x.1")
without misclassifying them, which a naive "add x to the scanned
charset" fix would have gotten wrong.
- Continues to classify values that are neither a version nor a
constraint (empty string, "latest", junk) as non-constraints; helm
handles those elsewhere.
Removes the now-unused versionConstraintChars string constant.
Expands TestIsVersionConstraint with 8 wildcard cases and 3 prerelease
/build metadata cases containing "x", plus 2 non-parseable inputs.
Adds a "wildcard segment constraint resolves to concrete version"
subtest to TestResolveOCIConstraintVersion so the end-to-end pipeline
is exercised for a version string that has no operator characters.
Signed-off-by: Samuel Archambault <samuel.archambault@getmaintainx.com>
* test: add getOCIChart integration test proving cache-path/pull-flag wiring
Address Copilot review feedback on PR #2768. The existing unit test
exercised resolveOCIConstraintVersion in isolation but did not prove
that its output was propagated into the downstream cache key, cache
path, and `helm chart pull --version` flag. Add a targeted
integration test that:
1. Calls getOCIChart with a constraint release (`~1`) and a helm
mock whose ShowChartWithFlags returns Chart.yaml version 1.0.1.
2. Asserts helm chart pull receives `--version 1.0.1`, not `~1`.
3. Asserts the destination path passed to helm chart pull contains
the resolved-version segment (`/1.0.1/`) and does NOT contain
the raw-constraint segment (`/_1/`).
4. Reads back the on-disk Chart.yaml under the cache path to
confirm resolved version, path, and flag agree end to end.
Add a second test that runs the same release twice with different
resolver outputs (1.0.1, then 1.0.2 — simulating a newly published
matching tag) and asserts the two resolutions land in distinct cache
directories. This is the promise of the fix: once the raw constraint
is out of the path, a new matching tag stops silently reusing the
previously-resolved cache entry.
The integration test flushed out a real correctness gap in the
initial fix: getOCIChart resolved release.Version and chartVersion
but did NOT recompute the qualified OCI ref that
getOCIQualifiedChartName built pre-resolution. Helm was therefore
receiving `oci://<repo>/<chart>:<constraint>` alongside a
`--version <resolved>` flag — at best redundant, at worst rejected
by future Helm versions. Fixed by re-invoking
getOCIQualifiedChartName on the mutated release copy so the
embedded tag also carries the resolved value.
Isolates the shared helmfile cache via `t.Setenv(HELMFILE_CACHE_HOME,
t.TempDir())` so the OutputDirTemplate == "" code path (which writes
into remote.CacheDir) does not touch the user's real
`~/.cache/helmfile` during test runs.
Signed-off-by: Samuel Archambault <samuel.archambault@getmaintainx.com>
* refactor: flatten OCI constraint-resolution wiring in getOCIChart
Address review feedback on PR #2768:
- Extract the inline resolve/requalify block from getOCIChart into
applyOCIConstraintResolution, keeping getOCIChart flat (guard-clause
style) and making the resolution wiring independently testable. The
helper returns the (possibly updated) release, qualified chart name,
and chart version; every failure mode returns its inputs unchanged.
- On a re-qualify failure after a successful resolution, fall back to
the pre-fix behavior entirely (raw constraint in cache key, ref, AND
--version flag) instead of the previous half-resolved mix (resolved
version in the cache key, raw constraint in the path and flag), which
could desynchronize the in-process cache key from the on-disk path.
- Build the 'helm show chart' ref by reusing parseOCIChartRef instead of
re-implementing its last-slash/last-colon tag-splitting inline. Same
behavior for all realistic refs (registry ports preserved), and it
also handles the digest suffix should one ever reach this point.
- Drop --devel from the resolver flags: helm documents --devel as
ignored whenever --version is set, and --version is always passed on
this path.
No behavior change intended beyond the requalify-failure fallback
(which cannot realistically trigger) and the removal of the inert
--devel flag.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: classify partial semver versions (1, 1.2) as OCI constraints
Address review feedback on PR #2768: Masterminds' lenient parser accepts
partial versions like "1" or "1.2" as versions, so the previous
classifier (NewVersion fails && NewConstraint succeeds) treated them as
exact pins. But helm's OCI resolution — registry.GetTagMatchingVersionOrConstraint
— honors a version string as an exact pin ONLY when a registry tag
literally equals it; otherwise it parses the string as a constraint, and
"1"/"1.2" float across 1.x.y/1.2.y tags. Caching those under their raw
spelling reproduces the stale-cache bug of issue #2766, just with a
narrower trigger.
Replace the NewVersion probe with isFullSemver, which additionally
requires the whole major.minor.patch triple to be spelled out
(optional v prefix, prerelease, and build metadata all still count as
exact when the core is fully qualified). When a registry does carry a
literal tag equal to the version string, the resolver's
metadata.Version == chartVersion path reports no change, so literal-tag
pins keep today's behavior.
TestIsVersionConstraint: "1"/"1.0" flip to constraints, joined by new
v1.2/0/v1 cases and a 1.2.3 exact case. TestResolveOCIConstraintVersion
gains a "partial version resolves" subtest.
Docs updated to describe the parser-based classification instead of
"constraint characters".
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: skip OCI constraint resolution under skipRefresh
Address review feedback on PR #2768: the resolver ran even under
--skip-refresh, so offline and cache-only workflows gained a
'helm show chart' registry attempt per constraint-versioned OCI release.
It degraded gracefully (warn + fallback), but added registry-timeout
latency and warning noise per release.
skipOCIConstraintResolution now suppresses resolution when any of the
skipRefresh levels is set — CLI --skip-refresh (forced), per-release
skipRefresh, or helmDefaults.skipRefresh — with the same precedence the
other skipRefresh consumers in prepareChartForRelease use. Skipped runs
fall back to the constraint-keyed cache path, i.e. they reuse whatever a
previous non-skipped run resolved, which is what 'skip checking for
updates to cached charts' means for constraint versions.
The existing issue #2766 integration tests flip their opts to
SkipRefresh: false since they assert resolution happens. New coverage:
TestSkipOCIConstraintResolution (tri-state precedence table) and
TestGetOCIChart_SkipRefreshSkipsConstraintResolution (no inspector call,
raw constraint in --version and cache path).
Signed-off-by: yxxhero <aiopsclub@163.com>
* perf: memoize OCI constraint resolution per chart+constraint
Address review feedback on PR #2768: resolution ran before the
in-process chart-cache fast path and was not memoized, so every
constraint-versioned OCI release paid its own 'helm show chart' registry
round-trip on every render — including N releases sharing the same
chart+constraint, whose parallel workers could even resolve to different
versions if the registry changed between their lookups.
Memoize successful resolutions in resolvedOCIConstraints keyed by
(chart ref, constraint), mirroring the downloadedCharts pattern:
- Releases sharing a chart+constraint cost one round-trip per process
and consistently use one resolved version per run.
- Only successful resolutions are memoized; failures may be transient.
- Flags are not part of the key: they govern TLS/verification/registry
credentials, not which tag a constraint matches (--devel is already
omitted as it is ignored whenever --version is set).
- Concurrent misses may both hit the registry; last write wins,
harmlessly.
resetResolvedOCIConstraintsForTest is added alongside the existing
resetChartCacheForTest and wired into the issue #2766 tests — notably
ResolvesToDifferentVersionsPicksSeparateCachePaths, which reuses the
same chart+constraint across its two runs and would otherwise be served
the first resolution from the memo (which is exactly the intended
per-process semantics).
New coverage: TestResolveOCIConstraintVersion_Memoized (memo hit skips
the registry, different constraint is a different key) and
TestGetOCIChart_SharedConstraintResolvedOncePerProcess (two releases,
one inspector call, one pull, same path).
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: cover URL-embedded OCI constraint resolution
Address review feedback on PR #2768: the existing integration tests only
exercised the repo-aliased spelling (chart: myrepo/mychart, version:
'~1') and the version-field spelling. The chart-URL spelling
(chart: oci://<registry>/<chart>:~1) takes a different branch in
getOCIQualifiedChartName — the URL version is deliberately NOT embedded
into the qualified ref and flows through --version only — so its
re-qualification after constraint resolution (release.Version mutated to
the resolved value, versionInURL still the constraint) was untested.
TestGetOCIChart_URLEmbeddedConstraintResolves asserts the resolver
receives the URL-embedded constraint, helm chart pull receives the
resolved version via --version with a tag-less ref, and the cache path
carries the resolved version segment instead of the raw constraint.
Also gofmt-aligns the test tables added in earlier commits and drops a
redundant 1.2.3 test case that tripped goconst.
Signed-off-by: yxxhero <aiopsclub@163.com>
* docs: note empty-version OCI releases are unaffected by resolveOCIVersions
Releases with no version: at all keep their pre-existing semantics: helm
picks the latest tag at pull time and helmfile caches it under a
version-less shared-cache path. Document the limitation alongside the
other resolveOCIVersions scope notes.
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: Samuel Archambault <samuel.archambault@getmaintainx.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: Samuel Archambault <samuel.archambault@getmaintainx.com>
Co-authored-by: yxxhero <aiopsclub@163.com>
Adds a new `helmfile unittest` command that integrates the helm-unittest
plugin, allowing users to define unit test paths per release and run them
via helmfile.
Closes#2376
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
This commit adds comprehensive support for Helm 4 while maintaining
full backward compatibility with Helm 3. The implementation includes:
- Updated helm version detection to support both Helm 3 and Helm 4
- Added HELMFILE_HELM4 environment variable to control Helm version
- Modified helm execution paths to handle version-specific binaries
- Updated helm plugin installation to support split architecture
- Helm 4: Uses split plugin architecture (3 separate .tgz files)
- helm-secrets.tgz
- helm-secrets-getter.tgz
- helm-secrets-post-renderer.tgz
- Helm 3: Continues using single plugin installation
- Updated Dockerfiles, CI workflows, and core installation code
- Helm 4 requires post-renderers to be plugins, not executable scripts
- Created Helm plugin structure for integration tests
- Updated helmfile.yaml templates to dynamically select renderer type
- Added test plugins: add-cm, add-cm1, add-cm2
- Updated integration tests for Helm 3/4 compatibility
- Created Helm 4 variant expected output files
- Fixed test determinism issues (repo cleanup between iterations)
- Added version-specific output filtering for warnings/messages
- Updated workflows to test both Helm 3 and Helm 4
- Matrix testing across Helm versions
- Updated helm-diff to v3.14.0 for compatibility
- Updated README and docs with Helm 4 information
- Added migration guidance
- Updated version requirements
All changes are backward compatible - existing Helm 3 users will
see no behavior changes.
fix: update Helm 4 lint expected output to match filtered output
The grep filter removes the semver warning, so the expected output
should not include it. Updated lint-helm4 files to match the filtered
output (warning removed, no extra blank line).
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
All the dependencies get correctly installed when dealing with remote
charts.
If there's a local chart that depends on remote dependencies then those
don't get automatically installed. See #526. They end up with this
error:
```
Error: no cached repository for helm-manager-b6cf96b91af4f01317d185adfbe32610179e5246214be9646a52cb0b86032272 found. (try 'helm repo update'): open /root/.cache/helm/repository/helm-manager-b6cf96b91af4f01317d185adfbe32610179e5246214be9646a52cb0b86032272-index.yaml: no such file or directory
```
One workaround for that would be to add the repositories from the local
charts. Something like this:
```
cd local-chart/ && helm dependency list $dir 2> /dev/null | tail +2 | head -n -1 | awk '{ print "helm repo add " $1 " " $3 }' | while read cmd; do $cmd; done
```
This however is not trivial to parse and implement.
An easier fix which I did here is just to not allow doing
`--skip-refresh` for local repositories.
Fixes#526
Signed-off-by: Indrek Juhkam <indrek@urgas.eu>
Signed-off-by: Indrek Juhkam <indrek@urgas.eu>
Signed-off-by: yxxhero <aiopsclub@163.com>
1. only implement post-renderer flags this patch
2. As mumoshu advise, add helmfile flags `--post-render` and add the
postRenderer config in helmDefaults and release. the priority is
helmfile flags > release > helmDefaults.
3. fix the test case in state_test.go and some other tests.
Signed-off-by: guofutan <guofutan@tencent.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat: show live output from the Helm binary
Signed-off-by: Rodrigo Fior Kuntzer <rodrigo@miro.com>
* fixup! Merge branch 'main' into enable-live-output
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Parses a new field in repositories named `skipTLSVerify` and if set to `true`, it appends `--insecure-skip-tls-verify` in `helm repo add` command.
This should be useful with internal self-signed repos, mitm proxies etc.
Resolves#1871
This adds the ability to include the --pass-credentials flag to the helm add repo command by:
- Adding repo.passCredentials to the helmfile yaml
- Changing state, helmexec, and app to include RepositorySpec.PassCredentials
Resolves#1898
Co-authored-by: almed4 <alexandre.meddin@ingka.ikea.com>
Adds a basic support for Helm repositories hosted on Azure Container Registry (not OCI but classic ones). Add a new field to RepositorySpec to state that is externally managed and runs the `az-cli` command instead of the helm one to manage the repository.
* Parse and process helm version using github.com/Masterminds/semver/v3.
* Add --force-update only when Helm version >= 3.3.2, < 3.3.4.
See: https://github.com/helm/helm/pull/8777.
* Add test cases.
- createNamespace is a new attribute that can be added to helmDefaults
or an individual release to enforce the creation of a release namespace
during sync if the namespace does not exist. This leverages helm's
(3.2+) --create-namespace flag for the install/upgrade command. If
running helm < 3.2, the createNamespace attribute has no effect.
Resolves#891Resolves#1140
* Add option to suppress diff on apply
Add --supress-diff option on apply. Usable for fresh installs when a
lot of output is produces by diff.
Resolves#458
* fix tests for suppress-diff
Runs `helm version` in helmexec.New, and exposes a method on Interface to allow other packages to use the detected version. Preserves compatibility with previous HELMFILE_HELM3 mechanism.
Resolves#923
