feat(remote): support wildcards in remote values/secrets file selectors (#2787)

Extend git-getter style remote references (git::, s3::, https://, ...) used
in release and environment values/secrets to support glob patterns in the
file selector, e.g.:

  git::https://github.com/org/repo.git@config/*.yaml?ref=main

Remote.Fetch already downloads the whole repository/directory and joins the
"@<file>" selector onto it verbatim, so a wildcard selector already survives
untouched; the only missing piece was that Storage.resolveFile checked the
result with FileExistsAt instead of expanding it as a glob.

- pkg/remote/remote.go: add HasGlobPattern to detect a wildcard in the file
  selector (checking only the selector, not the raw URL, so "?ref=main" and
  IPv6/placeholder brackets elsewhere are not mistaken for wildcards). Reject
  wildcards in Fetch for getter shapes that can never expand one: plain
  http(s)/s3 (single object), non-archive forced s3:: (single object), and
  any getter used without an explicit "@" selector (Dir/File cannot be
  reliably split from the pattern otherwise).
- pkg/state/storage.go: resolveFile now globs the fetched cache path with the
  same st.fs.Glob/sort.Strings used for local values-file globs when the
  selector is a pattern, filtering out directory matches. A literal, existing
  path is still resolved directly. Fixed an existing err-shadowing hazard in
  the same code path while restructuring it.
- docs/environments.md: document the new wildcard support, its syntax
  (filepath.Match, no recursive **), and its getter/selector requirements.
- Tests: new cases in pkg/remote/remote_test.go (glob detection, Fetch
  wildcard expansion and cache-key sharing, rejected getter shapes) and
  pkg/state/storage_test.go (a real end-to-end wildcard fetch against a
  pinned upstream tag, plus a hermetic fan-out/sorting/missing-file test with
  no network access).

Release values/secrets keep their existing "glob patterns ... not supported
yet" restriction for multi-file matches (pkg/state/state.go), unchanged by
this commit and applying equally to local and remote globs. helmfiles: entries
are out of scope.

Signed-off-by: Vincent Cui <privat@vincentcui.de>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
hoppla20
2026-09-13 09:16:26 +08:00
committed by GitHub
co-authored by Claude Opus 5
parent 6e7aafd5e5
commit daa43e1081
5 changed files with 531 additions and 29 deletions
+28
View File
@@ -222,6 +222,14 @@ environments:
- http://$HOSTNAME/artifactory/example-repo-local/test.tgz@environments/production.secret.yaml
```
Wildcards are supported in the file selector, the same way as for [remote values files](#loading-remote-environment-values-files):
```yaml
environments:
staging:
secrets:
- git::https://{{ env "GITHUB_PAT" }}@github.com/org/repo.git@environments/staging/secrets/*.yaml?ref=main
```
### Loading remote Environment values files
Since Helmfile v0.118.8, you can use `go-getter`-style URLs to refer to remote values files:
@@ -261,6 +269,26 @@ For more information about the supported protocols see: [go-getter Protocol-Spec
This is particularly useful when you co-locate helmfiles within your project repo but want to reuse the definitions in a global repo.
##### Wildcards in remote values files
The file selector (the part after `@`) can be a glob pattern, so a whole directory of values files can be referenced at once, without listing every file individually:
```yaml
environments:
production:
values:
- git::https://github.com/org/repo.git@config/production/*.yaml?ref=main
```
All matches are fetched from a single clone of the repository, then merged in the same sorted order as a local `values:` glob (see [Precedence](#environment-values-precedence)).
Notes and limitations:
- The pattern is matched with the same syntax as local values file globs (`filepath.Match`: `*`, `?`, `[abc]`), against a single path segment — there is no recursive `**`.
- Wildcards require an explicit `@` selector to mark the repository root, as in the example above.
- Wildcards are only supported with getters that download a whole directory, such as `git::`. Plain `https://`/`s3://` references and a non-archive `s3::` reference each fetch a single file and cannot be expanded; use `git::` (or an `s3::` archive URL) instead.
- A raw `?` in the file selector can never work as a wildcard: it starts the URL's query string (e.g. `?ref=main`), so anything after it is parsed as part of the query, not the file selector. A `?` wildcard must be percent-encoded as `%3F` to survive as part of the path, e.g. `@dir/v%3F.yaml?ref=main`.
- This applies to environment `values:`/`secrets:` only. In release-level `values:`/`secrets:`, a wildcard that matches exactly one file also works, but a wildcard matching more than one file still fails with "glob patterns in release values and secrets is not supported yet" — the same restriction that already applies to local glob patterns there.
### Environment values precedence
With the introduction of HCL, a new value precedence was introduced over environment values.
Here is the order of precedence from least to greatest (the last one overrides all others)