mirror of
https://github.com/helmfile/helmfile.git
synced 2026-10-09 02:22:45 +02:00
feat(remote): support wildcards in remote values/secrets file selectors (#2787)
Extend git-getter style remote references (git::, s3::, https://, ...) used in release and environment values/secrets to support glob patterns in the file selector, e.g.: git::https://github.com/org/repo.git@config/*.yaml?ref=main Remote.Fetch already downloads the whole repository/directory and joins the "@<file>" selector onto it verbatim, so a wildcard selector already survives untouched; the only missing piece was that Storage.resolveFile checked the result with FileExistsAt instead of expanding it as a glob. - pkg/remote/remote.go: add HasGlobPattern to detect a wildcard in the file selector (checking only the selector, not the raw URL, so "?ref=main" and IPv6/placeholder brackets elsewhere are not mistaken for wildcards). Reject wildcards in Fetch for getter shapes that can never expand one: plain http(s)/s3 (single object), non-archive forced s3:: (single object), and any getter used without an explicit "@" selector (Dir/File cannot be reliably split from the pattern otherwise). - pkg/state/storage.go: resolveFile now globs the fetched cache path with the same st.fs.Glob/sort.Strings used for local values-file globs when the selector is a pattern, filtering out directory matches. A literal, existing path is still resolved directly. Fixed an existing err-shadowing hazard in the same code path while restructuring it. - docs/environments.md: document the new wildcard support, its syntax (filepath.Match, no recursive **), and its getter/selector requirements. - Tests: new cases in pkg/remote/remote_test.go (glob detection, Fetch wildcard expansion and cache-key sharing, rejected getter shapes) and pkg/state/storage_test.go (a real end-to-end wildcard fetch against a pinned upstream tag, plus a hermetic fan-out/sorting/missing-file test with no network access). Release values/secrets keep their existing "glob patterns ... not supported yet" restriction for multi-file matches (pkg/state/state.go), unchanged by this commit and applying equally to local and remote globs. helmfiles: entries are out of scope. Signed-off-by: Vincent Cui <privat@vincentcui.de> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
6e7aafd5e5
commit
daa43e1081
@@ -222,6 +222,14 @@ environments:
|
||||
- http://$HOSTNAME/artifactory/example-repo-local/test.tgz@environments/production.secret.yaml
|
||||
```
|
||||
|
||||
Wildcards are supported in the file selector, the same way as for [remote values files](#loading-remote-environment-values-files):
|
||||
```yaml
|
||||
environments:
|
||||
staging:
|
||||
secrets:
|
||||
- git::https://{{ env "GITHUB_PAT" }}@github.com/org/repo.git@environments/staging/secrets/*.yaml?ref=main
|
||||
```
|
||||
|
||||
### Loading remote Environment values files
|
||||
|
||||
Since Helmfile v0.118.8, you can use `go-getter`-style URLs to refer to remote values files:
|
||||
@@ -261,6 +269,26 @@ For more information about the supported protocols see: [go-getter Protocol-Spec
|
||||
|
||||
This is particularly useful when you co-locate helmfiles within your project repo but want to reuse the definitions in a global repo.
|
||||
|
||||
##### Wildcards in remote values files
|
||||
|
||||
The file selector (the part after `@`) can be a glob pattern, so a whole directory of values files can be referenced at once, without listing every file individually:
|
||||
|
||||
```yaml
|
||||
environments:
|
||||
production:
|
||||
values:
|
||||
- git::https://github.com/org/repo.git@config/production/*.yaml?ref=main
|
||||
```
|
||||
|
||||
All matches are fetched from a single clone of the repository, then merged in the same sorted order as a local `values:` glob (see [Precedence](#environment-values-precedence)).
|
||||
|
||||
Notes and limitations:
|
||||
- The pattern is matched with the same syntax as local values file globs (`filepath.Match`: `*`, `?`, `[abc]`), against a single path segment — there is no recursive `**`.
|
||||
- Wildcards require an explicit `@` selector to mark the repository root, as in the example above.
|
||||
- Wildcards are only supported with getters that download a whole directory, such as `git::`. Plain `https://`/`s3://` references and a non-archive `s3::` reference each fetch a single file and cannot be expanded; use `git::` (or an `s3::` archive URL) instead.
|
||||
- A raw `?` in the file selector can never work as a wildcard: it starts the URL's query string (e.g. `?ref=main`), so anything after it is parsed as part of the query, not the file selector. A `?` wildcard must be percent-encoded as `%3F` to survive as part of the path, e.g. `@dir/v%3F.yaml?ref=main`.
|
||||
- This applies to environment `values:`/`secrets:` only. In release-level `values:`/`secrets:`, a wildcard that matches exactly one file also works, but a wildcard matching more than one file still fails with "glob patterns in release values and secrets is not supported yet" — the same restriction that already applies to local glob patterns there.
|
||||
|
||||
### Environment values precedence
|
||||
With the introduction of HCL, a new value precedence was introduced over environment values.
|
||||
Here is the order of precedence from least to greatest (the last one overrides all others)
|
||||
|
||||
Reference in New Issue
Block a user