Extend git-getter style remote references (git::, s3::, https://, ...) used
in release and environment values/secrets to support glob patterns in the
file selector, e.g.:
git::https://github.com/org/repo.git@config/*.yaml?ref=main
Remote.Fetch already downloads the whole repository/directory and joins the
"@<file>" selector onto it verbatim, so a wildcard selector already survives
untouched; the only missing piece was that Storage.resolveFile checked the
result with FileExistsAt instead of expanding it as a glob.
- pkg/remote/remote.go: add HasGlobPattern to detect a wildcard in the file
selector (checking only the selector, not the raw URL, so "?ref=main" and
IPv6/placeholder brackets elsewhere are not mistaken for wildcards). Reject
wildcards in Fetch for getter shapes that can never expand one: plain
http(s)/s3 (single object), non-archive forced s3:: (single object), and
any getter used without an explicit "@" selector (Dir/File cannot be
reliably split from the pattern otherwise).
- pkg/state/storage.go: resolveFile now globs the fetched cache path with the
same st.fs.Glob/sort.Strings used for local values-file globs when the
selector is a pattern, filtering out directory matches. A literal, existing
path is still resolved directly. Fixed an existing err-shadowing hazard in
the same code path while restructuring it.
- docs/environments.md: document the new wildcard support, its syntax
(filepath.Match, no recursive **), and its getter/selector requirements.
- Tests: new cases in pkg/remote/remote_test.go (glob detection, Fetch
wildcard expansion and cache-key sharing, rejected getter shapes) and
pkg/state/storage_test.go (a real end-to-end wildcard fetch against a
pinned upstream tag, plus a hermetic fan-out/sorting/missing-file test with
no network access).
Release values/secrets keep their existing "glob patterns ... not supported
yet" restriction for multi-file matches (pkg/state/state.go), unchanged by
this commit and applying equally to local and remote globs. helmfiles: entries
are out of scope.
Signed-off-by: Vincent Cui <privat@vincentcui.de>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: array merge regression - layer arrays now replace defaults (#2353)
PR #2288 introduced element-by-element array merging to fix#2281, but this
caused a regression where layer/environment arrays were merged instead of
replacing base arrays entirely.
This fix uses automatic sparse array detection:
- Arrays with nil values (from --state-values-set) merge element-by-element
- Arrays without nils (from layer YAML) replace entirely
This follows Helm's documented behavior where arrays replace rather than merge.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: use separate CLIOverrides field for element-by-element array merging
The previous approach using ArrayMergeStrategySparse detection didn't work
for --state-values-set array[0]=value because setting index 0 produces no
nils in the array.
This fix adds a CLIOverrides field to Environment that keeps CLI values
separate from layer values. CLI overrides are merged last using
ArrayMergeStrategyMerge (always element-by-element), while layer values
use the default strategy (arrays replace).
This ensures:
- --state-values-set array[0]=x only changes index 0, preserving other elements
- Layer/environment file arrays still replace base arrays entirely
- Issue #2281 fix is preserved (--state-values-set array[1].field=x works)
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: correct comment about array merge strategy in test
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: propagate Defaults in multi-part helmfiles and fix merge order
- Add Defaults field merging from ctxEnv to preserve base values across
helmfile parts separated by ---
- Fix merge order: current part values now correctly override previous
parts (was reversed, causing older values to win)
- Update 147 snapshot test files for new Environment log format with
CLIOverrides field
This completes the fix for issue #2353 by ensuring:
1. Layer arrays replace entirely (not element-by-element merge)
2. CLI --state-values-set sparse arrays still merge element-by-element
3. Multi-part helmfiles properly inherit and override values
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: address Copilot review comments
- Initialize EmptyEnvironment with empty maps to match New() constructor
- Update test comment to accurately describe ArrayMergeStrategySparse
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: ensure templates access merged values via .Environment.Values
This commit fixes a regression in the CLIOverrides integration where
templates accessing .Environment.Values couldn't see CLI override values.
Changes:
- Remove CLIOverrides-into-Values merge from Merge() to keep proper
layering order (Defaults → Values → CLIOverrides) in GetMergedValues()
- Update NewEnvironmentTemplateData to set envCopy.Values to the merged
values, ensuring templates see the same values via both .Values and
.Environment.Values
This ensures:
- Issue #2353: Layer arrays still replace entirely (Sparse strategy)
- Issue #2281: CLI sparse arrays still merge element-by-element
- Templates can access CLI overrides via .Environment.Values
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* docs: improve mergeSlices documentation per Copilot review
Address Copilot review comments on PR #2367:
- Document empty array edge case: explicitly setting [] clears base array
- Document recursive strategy propagation for nested map merging
- Add comprehensive behavior description for all array merge strategies
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: use merged values when rendering environment value files
Environment value files (*.yaml.gotmpl) can reference CLI values via
.Values. Previously, only env.Values was passed to template rendering,
which didn't include CLIOverrides.
Now we call env.GetMergedValues() to get Defaults + Values + CLIOverrides
before rendering, so templates can access CLI values like:
--state-values-set foo=bar
This fixes the state-values-set-cli-args-in-environments integration test.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
---------
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
I had been unhappy with the fact that our go-test output had a lot of debug log messages which obfuscated test results.
I'm finally removeing all those by directing the test log output to io.Discard.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
If remote file isn't present in repo we will add it anyway to the files list and check `if len(files) == 0 {` never will be true. That leads to missing section with `MissingFileHandler`.
That fix check that cloned file actually exists. In that case if we add a link to non-existing remote file `MissingFileHandler` will be called as expected.
Signed-off-by: Vladimir Kuznichenkov <kuzaxak.tech@gmail.com>
