* feat: Add updateStrategy option in the state file with 'reinstall'/'reinstallIfForbidden' choices to uninstall and apply the specific release(s) (if forbidden to update)
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Fix unit tests related to the new updateStrategy feature
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Fix unit tests related to the new updateStrategy feature
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Resolve linter issue due to cognitive complexity
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Updated index.md to describe the possible values of updateStrategy
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Add validation of updateStrategy parameter and unit test
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Updated unit test
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Removed 'reinstall' update strategy option to only have reinstallIfForbidden, cleanup of pre-sync changes, adapted unit tests
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Display affected releases that were reinstalled
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Make sure to add --wait when deleting a release to be reinstalled due to reinstallIfForbidden
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Apply suggestions from Copilot code review
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
---------
Signed-off-by: Simon Bouchard <sbouchard@rbbn.com>
* Fix helmBinary and kustomizeBinary being ignored when using bases
- Add mergo.WithOverride to merge operations for proper precedence
- Move default binary setting after base loading
- Add comprehensive tests for various base scenarios
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Fix code formatting in create_test.go
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Remove duplicate comment block in create_test.go
Removed duplicate comment lines (530-532) as identified by code review.
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
The build command is intended to be a read-only inspection command that
outputs the helmfile state. However, when releases use jsonPatches,
strategicMergePatches, or transformers, the chart preparation step
triggers chartify, which runs helm template and requires dependencies to
be built.
This causes two issues:
1. helm template is executed unnecessarily for a simple state inspection
2. Missing chart dependencies cause errors even with SkipDeps enabled
This change modifies PrepareCharts to filter out releases that require
chartify when the command is "build". These releases are excluded from
chart preparation, preventing helm template from being invoked.
The state output will still include these releases, but their charts
won't be processed during the build operation.
Signed-off-by: Shane Starcher <shanestarcher@gmail.com>
* cleanup disk in release ci
Signed-off-by: yxxhero <aiopsclub@163.com>
* cleanup disk in release ci
Signed-off-by: yxxhero <aiopsclub@163.com>
* cleanup disk in release ci
Signed-off-by: yxxhero <aiopsclub@163.com>
* cleanup disk in release ci
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
* Initial plan
* Add enableDNS flag support to diff command
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Extract EnableDNS flag logic into reusable function
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Initial plan
* Implement --timeout flag for helmfile sync command
- Add Timeout field to SyncOptions struct in pkg/config/sync.go
- Add --timeout flag to sync command in cmd/sync.go
- Add Timeout field to SyncOpts struct in pkg/state/state.go
- Modify timeoutFlags() function to prioritize CLI timeout over release and default configs
- Add test case to verify CLI timeout overrides other timeout settings
- Follow same pattern as existing --wait and --wait-for-jobs flags
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Fix lint issues: format test struct fields properly
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Update docs: Add --timeout flag documentation for helmfile sync command
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Initial plan
* Fix panic in helmfile init when parsing invalid helm versions
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Fix parseHelmVersion to handle versions without v prefix
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Simplify parseHelmVersion function to be more readable
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
- Bump HelmDiffRecommendedVersion from v3.12.3 to v3.12.5 in pkg/app/init.go
- Bump default HELM_DIFF_VERSION from 3.12.3 to 3.12.5 in test/integration/run.sh
- Update HelmRecommendedVersion from v3.18.4 to v3.18.5 in pkg/app/init.go
Signed-off-by: yxxhero <aiopsclub@163.com>
* Allow caching of remote files to be disabled
Make it possible to automatically update the cache of remote
resources by disabling the caching of those resources using a query
string parameter (`cache=false`).
Signed-off-by: Jess <jess@ros.io>
* Fix test that broke
Because query parameters are being re-encoded, = is being encoded to %3D.
Signed-off-by: Jess <jess@ros.io>
* Add test for disabling caching of remote resources
Signed-off-by: Jess <jess@ros.io>
* Include example usage in docs
Signed-off-by: Jess <jess@ros.io>
---------
Signed-off-by: Jess <jess@ros.io>
* docs: update status section with May 2025 release information
Signed-off-by: yxxhero <aiopsclub@163.com>
* docs: Update Helmfile v1 proposal link in index.md
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
* Feat: reuseValues in release
Adding properties to set reuseValues flag on release-level.
Signed-off-by: Adam Blasko <adam.blasko1@gmail.com>
* feat: fixing tests
Most of the tests had issues with flag order, which changed due to moving the value control flags out of the "common flags" for diff
Signed-off-by: Adam Blasko <adam.blasko1@gmail.com>
* fix: fixing lint issue
Signed-off-by: Adam Blasko <adam.blasko1@gmail.com>
---------
Signed-off-by: Adam Blasko <adam.blasko1@gmail.com>
* fix: Check needs with context and namespace
Signed-off-by: André Arnqvist <andrearnqvist@gmail.com>
* fix: Ensure releases have overrides
Signed-off-by: André Arnqvist <andrearnqvist@gmail.com>
* fix: Run go fmt
Signed-off-by: André Arnqvist <andrearnqvist@gmail.com>
* fix: Add tests checking needs with same name in different namespaces
Signed-off-by: André Arnqvist <andrearnqvist@gmail.com>
* fix: Simplify setting overrides
Signed-off-by: André Arnqvist <andrearnqvist@gmail.com>
---------
Signed-off-by: André Arnqvist <andrearnqvist@gmail.com>
When the evaluated release has installed: false it's perfectly fine that
a needed release is installed: false as well.
fixes#1464
Signed-off-by: jayme-github <jayme-github@users.noreply.github.com>
* build: update golang version to 1.24 and golangci-lint to v1.64.5
Signed-off-by: yxxhero <aiopsclub@163.com>
* build: update golang version to 1.24 in Dockerfiles
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix more issues
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
* docs: add skipSchemaValidation to index.md and update related structs
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat: add SkipSchemaValidation to config and state handling
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat: inject cli state values (--state-values-set) into environment templating context
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
* test: added envvals_loader unit test for environment values
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
* test: added 'state values set cli args in environments' integration test
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
* fix: merge environments before loadValuesEntries
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
* fix: 'state values set cli args in environments' integration test
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
---------
Signed-off-by: Vincent Chenal <vincent.chenal@protonmail.com>
This commit is supposed to add template support to post renderer args.
Also, to make it possible to template arguments that are added to helm
defaults, during the load, I'm removing default post renderer args from
the state and putting them to each release, unless custom args are
defined for the release.
Signed-off-by: Nikolai Rodionov <allanger@badhouseplants.net>
When having multiple helmfiles in one helmfile.d folder and multiple of
them having repositories defined, the errors mention in #1749 still
existed.
Fixes#1749 (again)
Signed-off-by: Matthias Baur <m.baur@syseleven.de>
* fix: inject global values in Chartify
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
* test: add integration tests for template --values
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
---------
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
update example chart URL
Signed-off-by: David Neeley <david.neeley@motorolasolutions.com>
Co-authored-by: David Neeley <david.neeley@motorolasolutions.com>
* feat: add HELMFILE_INTERACTIVE env var to enable interactive mode
This commit adds the `HELMFILE_INTERACTIVE` environment variable
to enable the interactive mode by default.
Anything other than `true` will disable the interactive mode.
The precedence has the `--interactive` flag.
Signed-off-by: Krzysztof Łuczak <krzysztof.luczak.pro@gmail.com>
* Trim trailing whitespaces
Signed-off-by: Krzysztof Łuczak <krzysztof.luczak.pro@gmail.com>
---------
Signed-off-by: Krzysztof Łuczak <krzysztof.luczak.pro@gmail.com>
* Add integration tests for #1749
Signed-off-by: Matthias Baur <m.baur@syseleven.de>
* Reset extra args on a higher level to only affect subsequent helmfiles
With the implementation before, extra args has been reset after each
helm.exec which leads to problems with multiple charts in a helmfile
since the correct args are only set once in Template(). But Template()
calls helm.exec(template) multiple times.
Signed-off-by: Matthias Baur <m.baur@syseleven.de>
---------
Signed-off-by: Matthias Baur <m.baur@syseleven.de>
--skip-refresh right now disables adding new repos, but in addition it
should pass down to helm deps update to not refresh the actual values
Signed-off-by: Shane Starcher <shane.starcher@gmail.com>
Co-authored-by: Shane Starcher <shane.starcher@gmail.com>
* chore: Update Ubuntu image to LTS version 24.04 (#1696)
Signed-off-by: Patrick Hobusch <patrick@hobusch.net>
Signed-off-by: Vedran Kolka <vedran.kolka@syntio.net>
* Fixed typo in configuration section
The comment for setting the password for a Helm repository said it was the "username". Changed it to "password"
Signed-off-by: Vedran Kolka <vedran.kolka@syntio.net>
---------
Signed-off-by: Patrick Hobusch <patrick@hobusch.net>
Signed-off-by: Vedran Kolka <vedran.kolka@syntio.net>
Co-authored-by: Patrick Hobusch <pathob@users.noreply.github.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* eat: adding ability for for charts to be pulled without HTTPS
accomplished by:
- Adding PlainHttp attribute to RepositorySpec., HelmDefault, ReleaseSpec
- Adding UnitTests for getOCIChart Flags.
- Adding funciton and unitTests for getChartDownload
- Changing and refactoring how Flags are added to getOCIChart.
Resolves#1224
Signed-off-by: Peter Halliday <peter.halliday@servicenow.com>
* Pass PlainHttp to OCI repo options, fix unit test
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Fix doc
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Use repository fields in non-OCI chart download options
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Update hashes in TestGenerateID
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Make sure repo exists when using its options
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Do not add TLS options if PlainHttp is set, adapt unit tests
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Fix doc
Signed-off-by: Pascal Rivard <privard@rbbn.com>
* Remove 'else if' from appendChartDownloadFlags
Signed-off-by: Pascal Rivard <privard@rbbn.com>
---------
Signed-off-by: Peter Halliday <peter.halliday@servicenow.com>
Signed-off-by: Pascal Rivard <privard@rbbn.com>
Co-authored-by: Peter Halliday <peter.halliday@servicenow.com>
Co-authored-by: Pascal Rivard <privard@rbbn.com>
build(deps): update helm plugins to latest versions
Update the versions of the helm plugins in the Dockerfile to the latest stable versions:
- helm-s3: v0.16.0 -> v0.16.2
This update ensures that the Docker image is using the most up-to-date versions of the helm plugins s3.
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: inject KubeVersion and ApiVersions in Chartify rendering
fixes#1623
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
* test: appendApiVersionsFlags
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
* test: add case for appendApiVersionsFlags
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
---------
Signed-off-by: Thomas Loubiou <thomas.loubiou@mirakl.com>
* Update README.md
Broken links in Status section
Signed-off-by: Jonathan Manton <jonathan@manton.com>
* Update README.md
Broken links in Status section
Signed-off-by: Jonathan Manton <jonathan@manton.com>
* Update README.md
Broken links in Status section
Signed-off-by: Jonathan Manton <jonathan@manton.com>
---------
Signed-off-by: Jonathan Manton <jonathan@manton.com>
* use logger for helm output
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
* update integration test output
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
* make logging output configurable
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
* also compare stderr in integration tests
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
---------
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
Add a `--show-only` parameter to the `helmfile template` command to pass
on to the `helm template` command.
Signed-off-by: Jim Barber <jim.barber@healthengine.com.au>
* add the SyncArgs option
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
* add syncArgs to helmDefaults and update index.md
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
* add --sync-args flags to helmfile sync
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
* add tests for appendExtraDiffFlags and appendExtraSyncFlags
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
---------
Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
`isReleaseInstalled` will call `helm list` which can take a bit of time.
in order to allow parallelism, we need to use a finer mutex lock.
Signed-off-by: Steven Davidovitz <steven.davidovitz@dominodatalab.com>
* disable version check for sops while building container image
Signed-off-by: Quan TRAN <account@itscaro.me>
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: prevent preparing chart for disabled releases
Previously, PrepareCharts does not filter any releases whose condition
is disabled with no selectors.
Prevent following things from
- using any unnecessary computing resources for disabled charts
- throwing any error from wrong configurations for disabled charts
Signed-off-by: Seonghoi lee <mighty1231@gmail.com>
* fix: working for integration test about list and build
Some tests require that PrepareCharts without any selector
may not add any labels on the release.
make markExcludedReleases do not add any label without any selectors
Signed-off-by: Seonghoi lee <mighty1231@gmail.com>
* fix: prevent resolved chart version loss
State loss the resolved chart version info from st.Releases
when st.GetSelectedReleases() be called
update st.Releases after st.GetSelectedReleases() in prepareCharts
Signed-off-by: Seonghoi lee <mighty1231@gmail.com>
* fix: preserve resolved version from resolveDeps
In PrepareCharts, the version, resolved from resolveDeps,
is removed after invoking GetSelectedReleases.
Do updateDeps at the first before GetSelectedReleases call
Signed-off-by: Seonghoi lee <mighty1231@gmail.com>
---------
Signed-off-by: Seonghoi lee <mighty1231@gmail.com>
* Resolve dependencies before releases are created
Signed-off-by: vlpav030 <vpav.030@gmail.com>
* Add test
Signed-off-by: vlpav030 <vpav.030@gmail.com>
---------
Signed-off-by: vlpav030 <vpav.030@gmail.com>
This pr fixes auto-wrapping of booleans and integers into quotes when using --state-values-set by:
- Adding: --state-values-set-string flag for intentional string set of boolean or integer
- Changing: --state-values-set flag not wrapping now
- Removing -
Resolves https://github.com/roboll/helmfile/issues/1347
Signed-off-by: Tunahan Sezen <sezentunahan@outlook.com>
* Allow secrets to be excluded from inheritance, small docs update
Signed-off-by: Alex Vorona <alex@vorona.com.ua>
* Fix typo in docs
Signed-off-by: Alex Vorona <alex@vorona.com.ua>
---------
Signed-off-by: Alex Vorona <alex@vorona.com.ua>
The [readthedocs build](https://readthedocs.org/projects/helmfile/builds/) is failing due to a missing required `build.os`
```
Error
Problem in your project's configuration. Invalid configuration option "build.os": build not found
```
Signed-off-by: Joseph Gardner <josephgardner@users.noreply.github.com>
* Add "PostRendererArgs" option to be passed to helm
This allows using PowerShell scripts on Windows as Post Renderer.
Signed-off-by: Maarten Boekhold <maarten.boekhold@finastra.com>
* fix: support large output with --enable-live-ouput
This replaces Scanner with ReadString to handle large amount of data returned by helm ouptut when executing diff action
- Changing pkg/helmexec/runner_test.go TestLiveOutput test, adding test with large amount of data, reproducing the issue before applying this fix
- Changing pkg/helmexec/runner.go Scanner with Readstring
Resolves https://github.com/helmfile/helmfile/issues/893
Signed-off-by: Franck Labatut <franck.labatut@ubisoft.com>
* fix: prevent data race
Signed-off-by: Franck Labatut <franck.labatut@ubisoft.com>
---------
Signed-off-by: Franck Labatut <franck.labatut@ubisoft.com>
* Fix symlink behaviour
Make sure to evaluate symlinks when the path is not absolute and not
local. Otherwise just fall back to regular absolute paths.
Signed-off-by: Zoltán Reegn <zoltan.reegn@gmail.com>
Removes reference to non-existent Helm 3 images
Updates version
Corrects bind mounts for current Dockerfile
Signed-off-by: Daniel Key <daniel@pointyshinyburning.org>
* feat: make environment context available
This feature adds the "{{.Environment.KubeContext}}" variable.
Discussion #829
Signed-off-by: sewieland <sebastian.wieland@iav.de>
* chore: fix tests which compare logging outputs
This commit adds an addtional space wherever needed to the expected log outputs due to the added "KubeContext" in the environment struct.
Discussion #829
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
* docs: added documentation for `Environment.KubeContext`
Discussion #829
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
* test: make sure the `Environment.KubeContext` is mapped out correctly
Discussion #829
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
---------
Signed-off-by: sewieland <sebastian.wieland@iav.de>
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
Co-authored-by: sewieland <sebastian.wieland@iav.de>
... by using plain indented code blocks instead of indented fenced
code blocks.
While here also fix list rendering and add missing Go template escaping
in inline `values` map example.
Signed-off-by: Stoned Elipot <stoned.elipot@gmail.com>
* feat: add/expose cli flags
Signed-off-by: Hans Song <hans.m.song@gmail.com>
* fix tests
Signed-off-by: Hans Song <hans.m.song@gmail.com>
* remove skipdeps from subcommand options
Signed-off-by: Hans Song <hans.m.song@gmail.com>
* remove skip-deps from subcommand flags
Signed-off-by: Hans Song <hans.m.song@gmail.com>
* remove SkipDeps from subcommand implementations
Signed-off-by: Hans Song <hans.m.song@gmail.com>
* update doco with new flags
Signed-off-by: Hans Song <hans.m.song@gmail.com>
---------
Signed-off-by: Hans Song <hans.m.song@gmail.com>
* Use goccy/go-yaml fork to not break dynamic values on helmfile v1
I forked goccy/go-yaml to https://github.com/helmfile/go-yaml and added a commit 259d2ed450 to let it produce YAML data compatible with other legacy YAML parsers that do not support YAML 1.2.
Ref https://github.com/helmfile/helmfile/discussions/656
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Add test for goccy/go-yaml fork behavior
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
---------
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Disable double-rendering in V1
We are going to force users to separate environments and releases sections in every helmfile.yaml in the [Helmfile V1](https://github.com/helmfile/helmfile/blob/main/docs/proposals/towards-1.0.md). The goal of separation was to make the helmfile.yaml rendering result not dependent on the double-rendering and therefore the feature should be safe to be turned off in V1.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix helmfile template rendering error log for v1
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Add test for helmfile template debug log differences between v0 and v1 modes
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix helmfile template render debug log to not mention "first-pass" in v1 mode
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
The go-getter driver (named `Remote`) is used to implement remote values. I found some unused functions in its implementation. This commit removes those unused functions so that it becomes more maintainable.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Ref https://github.com/helmfile/helmfile/discussions/593
* implemented --reset-values flag overriding .helmDefaults.reuseValues=true
* tests for --reset-values flag
* fixed pkg/app tests
* added tests for both reset & reuse flags given
* added appendValuesControlMode method for --reuse/reset-values flag processing
* updated code comments & docs
Signed-off-by: Karol Ossowski <k@koralsky.pl>
This should fix#435 for Helmfile v0.x releases since the next v0.150.0.
We introduce a new envvar to opt-in to the new YAML library, so that you can give it a shot before upgrading your Helmfile to v1. The same envvar can be used to opt-out of the new YAML library after you upgrade to Helmfile v1, giving you a more flexible migration story.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Previously a values template rendering failure resulted in a cryptic error message like:
```
g release templates in "input.yaml": failed executing templates in release "input.yaml"."foo1": failed executing template expressions in release "foo1".values[0] = "[98 97 115 101 76 97 98 101 108 58 32 34 123 123 32 103 101 116 32 46 82 101 108 101 97 115 101 46 76 97 98 101 108 115 32 92 34 98 97 115 101 92 34 32 92 34 78 79 84 95 73 78 72 69 82 73 84 69 68 92 34 32 125 125 34 10]": template: stringTemplate:1: unexpected "\\" in operand
```
The issue was that the values template at the value position of the YAML dict was printed as a formatted byte array. We now print it as a standard string so that it is easily readable.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* fix env value lost in environment values
Signed-off-by: yxxhero <aiopsclub@163.com>
* add more test
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
This is a successor to #596. We need a smooth migration path from `gopkg.in/yaml.v2`, and this pull request moves it forward with `goccy/go-yaml` instead of `gopkg.in/yaml.v3`. Merging this unblocks users stuck in Helmfile v0.146.x or earlier due to #435, so that they can upgrade to 0.147.x or greater without updating their helmfile configs.
We previously tried to upgrade to `yaml.v3` (https://github.com/helmfile/helmfile/issues/394) in Helmfile v0.x, presuming it won't break anything. Apparently, it broke use-cases where you want to layer release's `values` field over three or more release templates and releases (#435).
We then tried to bring back `yaml.v2` for Helmfile v0.x and keep `yaml.v3` for the upcoming Helmfile v1. However, it failed due to incompatibility in the Unmarshaller interface between `yaml.v2` and `yaml.v3` (https://github.com/helmfile/helmfile/pull/596).
`goccy/go-yaml` is, from my observation, a well-maintained alternative to `yaml.v2`. One of its premises is that it enables us to swap the implementation from `gopkg.in/yaml.v2` to `goccy/go-yaml` just by replacing the import directive. It seems to use the same `Unmarshaller` interface as yaml.v2 too.
Once this PR gets merged, I'd like to follow-up with adding a new build-time variable and an envvar to set the proper default for the yaml parser Helmfile uses and the ability to switch the parser at runtime. All in all, the next Helmfile release, v0.150.0 will get reverted to use `gopkg.in/yaml.v2` by default which resolves#435.
New users who started using Helmfile since any of v0.148.0, v0.148.1, and v0.149.0 might be already relying on the new behavior, They might need to specify a new envvar to enable `goccy/go-yaml`.
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: yxxhero <aiopsclub@163.com>
This is a successor to #442 rebuilt on top of #594 so that we can merge this while we are still at Hemlfile v0.x without worrying any backward-incompatibility.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
This is a successor to #440 rebuilt on top of #594 so that we can merge this while we are still at Hemlfile v0.x without worrying any backward-incompatibility. Much appreciation to @yxxhero for the original work!
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* feat: Helmfile V1 mode
We add a new "V1 mode" to Helmfile so that you can seemlessly upgrade Helmfile from the current v0.x to the upcoming v1.0.
The idea is that we build both v0 and v1 binaries from the same tagged commit within the main branch, with different defaults for the "V1 mode"- the V1 mode is disabled by default for v0.x binaries, while it is enabled by default for v1.x binaries.
The V1 mode can be overrode at runtime via envvar. That is, even after upgrading the binary to v1, you will not see any backward-incompatible changes while you explicitly set an envvar, `HELMFILE_V1MODE=true`, at runtime.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
All the dependencies get correctly installed when dealing with remote
charts.
If there's a local chart that depends on remote dependencies then those
don't get automatically installed. See #526. They end up with this
error:
```
Error: no cached repository for helm-manager-b6cf96b91af4f01317d185adfbe32610179e5246214be9646a52cb0b86032272 found. (try 'helm repo update'): open /root/.cache/helm/repository/helm-manager-b6cf96b91af4f01317d185adfbe32610179e5246214be9646a52cb0b86032272-index.yaml: no such file or directory
```
One workaround for that would be to add the repositories from the local
charts. Something like this:
```
cd local-chart/ && helm dependency list $dir 2> /dev/null | tail +2 | head -n -1 | awk '{ print "helm repo add " $1 " " $3 }' | while read cmd; do $cmd; done
```
This however is not trivial to parse and implement.
An easier fix which I did here is just to not allow doing
`--skip-refresh` for local repositories.
Fixes#526
Signed-off-by: Indrek Juhkam <indrek@urgas.eu>
Signed-off-by: Indrek Juhkam <indrek@urgas.eu>
Signed-off-by: yxxhero <aiopsclub@163.com>
1. only implement post-renderer flags this patch
2. As mumoshu advise, add helmfile flags `--post-render` and add the
postRenderer config in helmDefaults and release. the priority is
helmfile flags > release > helmDefaults.
3. fix the test case in state_test.go and some other tests.
Signed-off-by: guofutan <guofutan@tencent.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
All the dependencies get correctly installed when dealing with remote
charts.
If there's a local chart that depends on remote dependencies then those
don't get automatically installed. See #526. They end up with this
error:
```
Error: no cached repository for helm-manager-b6cf96b91af4f01317d185adfbe32610179e5246214be9646a52cb0b86032272 found. (try 'helm repo update'): open /root/.cache/helm/repository/helm-manager-b6cf96b91af4f01317d185adfbe32610179e5246214be9646a52cb0b86032272-index.yaml: no such file or directory
```
One workaround for that would be to add the repositories from the local
charts. Something like this:
```
cd local-chart/ && helm dependency list $dir 2> /dev/null | tail +2 | head -n -1 | awk '{ print "helm repo add " $1 " " $3 }' | while read cmd; do $cmd; done
```
This however is not trivial to parse and implement.
An easier fix which I did here is just to not allow doing
`--skip-refresh` for local repositories.
Fixes#526
Signed-off-by: Indrek Juhkam <indrek@urgas.eu>
Signed-off-by: Indrek Juhkam <indrek@urgas.eu>
* Fix presync hooks are not called on no diff when run apply subcommand
Signed-off-by: xiaomudk <xiaomudk@gmail.com>
* Update docs/index.md
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: xiaomudk <xiaomudk@gmail.com>
Signed-off-by: xiaomudk <xiaomudk@gmail.com>
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
* fetch checksum when possible
* use sha256sum -c to validate checksum
* use tar features to extract artifacts
* validate installed pkg in-place
Signed-off-by: Cyril Jouve <jv.cyril@gmail.com>
Signed-off-by: Cyril Jouve <jv.cyril@gmail.com>
* refactor integrations
Signed-off-by: yxxhero <aiopsclub@163.com>
* Update the integration test directory structure to better correlate the test script with testdata
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Undo the test-cases directory renaming to make the diff more easy to understand
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* fix ci
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
* Enhance `make fmt` to cover running gci for golangci-lint compat
I have been using `make fmt` for formatting code before submitting a pull request. It turned out not to work in some cases where you added a new import because the default go-fmt obviously do not organize imports as the golangci-lint and gci expect.
This fixes that, by adding a gci command to the `make fmt` target.
Please note that this does not cover installing gci. If you need, please submit another pull request to add something like a `make install-gci` and make the `make fmt` depenedent on that target, so that one can get automatic gci installation when running `make fmt` for first time.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* We don't need running gci via make on CI as we already run it as part of golangci-lint run
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Use the new runWithLogCapture helper instead of the long boilerplate to capture the log for snapshot testing.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Allow configuring the lockfile in the state. This makes it possible for
example maintain a lock per environment.
Signed-off-by: Lassi Pölönen <lassi.polonen@iki.fi>
Signed-off-by: Lassi Pölönen <lassi.polonen@iki.fi>
This improves the `helmfile sync` performance.
From the code: `BuildDeps` is used only by `runHelmDepBuilds`, which
only is used by `PrepareCharts` which is finally only used by
`withPreparedCharts`.
`withPreparedCharts` already does `SyncReposOnce` which means we do not
have to refresh the local repository cache on each chart build.
This is only supported in Helm v3.
This seems to be mostly affecting helmfiles which have a lot of releases
and those release charts use sub dependencies.
I saw significant performance improvements for a helmfile with 45
releases, 2 repositories, and most of the charts also had their own
dependencies. Results:
Before the patch:
* real 9m10.565s
* real 9m38.335s
* real 9m14.941s
* real 5m13.106s (with cache)
After the patch:
* real 6m51.965s
* real 6m36.605s
* real 6m31.685s
* real 3m0.271s (with cache)
These were tested with:
```
rm -rf ~/.cache/helmfile ~/.cache/helm ~/.config/helm/repositories.* && helmfile sync ...
```
The result with `(with cache)` was without deleting the caches first.
From these metrics it seems that the sync duration decreased 20-45%
depending on the run, release count, dependencies and if the cache was
used or not.
As far as I understand, this should be backward-compatible change.
Signed-off-by: Indrek Juhkam <indrek@urgas.eu>
Signed-off-by: Indrek Juhkam <indrek@urgas.eu>
I had been unhappy with the fact that our go-test output had a lot of debug log messages which obfuscated test results.
I'm finally removeing all those by directing the test log output to io.Discard.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* fix: child process not exit when recive term signal
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: wait for clean done
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
Most shells do not support hyphens in environment variables.
However, there are cases where you may want to include hyphens in the repository name.
Therefore, I have included a process in `gatherOCIUsernamePassword` to replace hyphens with underbar.
Signed-off-by: mugioka <okamugi0722@gmail.com>
* feat: show live output from the Helm binary
Signed-off-by: Rodrigo Fior Kuntzer <rodrigo@miro.com>
* fixup! Merge branch 'main' into enable-live-output
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
This changes list command so it doesn't run withPreparedCharts,
and just lists releases instead
Signed-off-by: Viktor Oreshkin <imselfish@stek29.rocks>
* add interactive in sync & remove --interactive in global options
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix unittest
Signed-off-by: yxxhero <aiopsclub@163.com>
* same behave as apply when in interactive
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
We would like to add back --set flag to apply subcommand, which was
accidentally removed after switching to cobra (#234)
Signed-off-by: Michal Jura <mjura@suse.com>
Fixing releases being included which do not match the environment
requested, which is a regression introduced by #234. The issue remains
when Helmfile state values are supplied, which is not a regression and
will be addressed separately.
Partial resolution for #271
Signed-off-by: David Ackroyd <dackroyd@fairfaxmedia.com.au>
Signed-off-by: David Ackroyd <dackroyd@fairfaxmedia.com.au>
If remote file isn't present in repo we will add it anyway to the files list and check `if len(files) == 0 {` never will be true. That leads to missing section with `MissingFileHandler`.
That fix check that cloned file actually exists. In that case if we add a link to non-existing remote file `MissingFileHandler` will be called as expected.
Signed-off-by: Vladimir Kuznichenkov <kuzaxak.tech@gmail.com>
Until now, the 'images' workflow was separated into two different jobs,
one for just building the images in e.g. pull requests and the other
one for building and pushing the images e.g. after a merge to the 'main'
branch, which resulted in code repetitions. Also, both jobs used
different approaches, one (build) using a 'matrix strategy' based on
the file name of the Dockerfile, the other one (build and push) having a
seperate build and push step for each Dockerfile.
With this change, both jobs have been unified into a single "build and
optionally push" job to remove the repetitions, which now also shares
the same approach - a matrix strategy based on the file names of the
Dockerfiles.
The package naming now follows a clear schema based on the file name of
the Dockerfile. 'Dockerfile' will result in a 'helmfile' package,
'Dockerfile.ubuntu' will result in a 'helmfile-ubuntu' package and so
on. In order to keep the 'helmfile-debian-stable-slim' image package
name, the 'Dockerfile.debian' had to be renamed to
'Dockerfile.debian-stable-slim' accordingly.
Furthermore, the evaluation of the condition whether a push is intended
(or not) has been moved directly to the 'push' flag of the
'docker/build-push-action'.
Signed-off-by: Patrick Hobusch <patrick.hobusch@gmail.com>
This similar to #239, but different in that it involves a local helm chart repo server instead of a local OCI registry as the container of the chart used in tests.
This also adds a new `localChartRepoServer.enabled` config to the `config.yaml` of each test case directory to enable the local helm repo server in the test, which might be useful when you want to add another test case that involes a remote chart(which is hosted in the local chart repo server).
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: Tsubasa Nagasawa <toversus2357@gmail.com>
This adds OCI registry support for adhoc dependencies.
The users can now add their OCI hosted charts as dependencies to helm releases:
* Bump chartify version to v0.10.0
* Add an example helmfile to advanced usage section
so that it might be even more clear if the reported problem has been introduced by the prev release or a long-standing one which existed for longer time period.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: Adam Gardner <adam.gardner@magicmemories.com>
chore: fix test case broken by change of helm subcommand
Signed-off-by: Adam Gardner <adam.gardner@magicmemories.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Make a few helmfile sub-commands to consistently support needs-related flags
* helmfile-diff adds support for --include-transitive-needs
* helmfile-template adds support for --skip-needs
* helmfile-lint adds support for --skip-needs, --include-needs, and --include-transitive-needs
Ref https://github.com/roboll/helmfile/issues/2055
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix a few helmfile-lint needs related bugs and add tests
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Is include-transitive-needs realy working as intended? 🤔
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Confirm that it does fail on unselected need by default
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Add missing testdata
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Test helmfile-template for include/skip needs support
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix a few terms
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Add more tests to better know the current helmfile-diff behavior around needs
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix failing tests
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix helmfile-diff to consistently handle skip/include-needs
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Extract testhelper.RequireLog for reusing
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix all bugs and test cases for TestDiff and TestDiff_2
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix TestDiff_2
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix TestDiff
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix TestDiffWithNeeds
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Unify behavior on including disabled releases as needs for lint and template
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix bug that --include-transitive-needs does not imply include-needs
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
We will be using this automation which is triggered on every new tag created in this repo for the upcoming helmfile 0.145.0 release.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Our CI workflows and docker builds already use 1.18 but our go.mod was left with 1.17. This bumps it to 1.18.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Probably this is the best we can currently offer. Any suggestions are welcomed though.
Ref https://github.com/roboll/helmfile/issues/2147
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
It turned out we lack some important information that is necessary to make it a real bug without guesswork.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
I've asked a few times by @yxxhero if he deserves a seat as a maintainer of the Helmfile project.
Seeing the amount and the number of his recent contributions, and the nature of his contributions (add tests and fix bugs and CI issues, review PRs), I think he is in a relatively unique position compared to existing maintainers, and we can work better together!
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* introduce DISABLE_INSECURE_FEATURES to disable insecure executions
Signed-off-by: Quan TRAN <account@itscaro.me>
* disable remote sources when DISABLE_INSECURE_FEATURES is set to "true"
Signed-off-by: Quan TRAN <account@itscaro.me>
* refactor envvar package
Signed-off-by: Quan TRAN <account@itscaro.me>
* (test) fix test fixtures
Signed-off-by: Quan TRAN <account@itscaro.me>
* use absolute path to avoid unit test failure
Signed-off-by: Quan TRAN <account@itscaro.me>
* Fix conflicts
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
* Adds feature to fetch environment values from remote
The releases and environment section allow for values files on the local
disk.
This enhancement allows for referencing remote (go-getter) files to be
fetched, cached and referenced.
In addition when fetching a remote git source with a ssh key the ssh key
will not be part of the caching folder name. This avoids two problems:
1. Don't leak sensitive information in the name of the caching folder
2. Base64 encoded SSH keys are very long. On some file systems the max
lenght of the directory name is hit when using the full base64
information in the path name.
The sshkey informations are reducted. Because of this fixed string
there is a change of colloding cache names. The likelihood of this
collision is very low. The git repo and git reference need to be the
same, but the sshkey can change. This will result in the same source to
be checkout out and referenced.
Signed-off-by: Lüchinger Dominic <dev@snowgarden.ch>
* Update pkg/state/storage.go
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
Since helm-diff has added an ability to auto-detect the term to decide if it should output with color or not, helmfile had been defaulted to no-color.
This resoloves that, by adding a term-detection logic that is same as helm-diff.
As a part of this work, I have also implemented a new global flag `--color`, which is used for forcing color without relying on the term-detection logic implemented in helmfile or explicitly setting the HELM_DIFF_COLOR envvar. I hope it is useful for folks.
Ref https://github.com/roboll/helmfile/issues/2043
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
When chartify is involved due to the use of `forceNamespace`, `strategicMergePatches`, `jsonPatches`, and so on, We had been internally mutating the Release.Chart with the path to the local temporary directory that contains the modified version of the chart.
This resulted in us unintentionally making `helmfile deps` to remove entries for the chart being modified out of helmfile.lock file, which resulted in issues like #2110.
To be clear, although the original issue is reported to occur for `strategicMergePatches`, I believe that it occurered also for any remote charts using `jsonPatches` and `forceNamespace` too.
I also believe this has been the issue since our introduction of chartify (maybe a year or so ago??), and I guess why it took so much time to be found and reported is that not so many people with chartify in combination with `helmfile deps` 🤔
Lastly, this changes chart names surfaced in the various log output from Helmfile, from temporary chart paths to the chart name/path declared in the helmfile.yaml. I think this is generally a good change, no fear of being a breaking change. But if anyone has any concern about that, please feel free to comment/report/etc.
Ref https://github.com/roboll/helmfile/issues/2110
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
This adds support for `kube-version` and `api-versions` to be available to `chartify` so that it works even if your release requires `chartify` due to that you use features like `forceNamespace`, `jsonPatches`, `strategicMergePatches`, and so on.
This also enhances `ReleaseSpec` which corresponds to each item of `releases[]` in your `helmfile.yaml` to also accept `kubeVersion` and `apiVersions`, in addition to the top-level `kubeVersion` and `apiVersions` we have today.
The top-level ones works as the default values for release-specific ones. If you have been using the top-level ones, keep using it. It is backward-compatible. If you want to specify it per release, because, for example, your releases are deployed across clusters(in case you differentiate `kubeContext` fields), try the new fields added to the release spec.
Resolves#1864
This release of chartify fixes a single bug in chartify that resulted in a few issues when your chart had dependencies. One of issues is the mysterious "no cached repository for helm-manager-HASH found" error mentioned in https://github.com/variantdev/chartify/pull/31. Another is #2117 which was due to the bug resulted in rendering some resources, CRDs in the case, twice.
See https://github.com/variantdev/chartify/releases/tag/v0.9.5 for more information.
This release fixes an issue when you tried to chartify a local chart whose directory name does not match the name of the chart defined in Chart.yaml.
See https://github.com/variantdev/chartify/releases/tag/v0.9.4 for more information.
This should fix a few issues, most notably that adhoc dependencies breaks when some of original chart dependencies are unresolvable with `helm dep up` and/or `helm dep build`.
https://github.com/variantdev/chartify/releases/tag/v0.9.2
It turned out that Helmfile has never had support for release template on `needs`.
This adds that, along with the new end-to-end test suite to verify helmfile template output with snapshot testing involving a real `helmfile build` command.
Ref #2098
Use the value of the `condition` field instead of the `installed` field of a release in the `enabled` column of helmfile list.
The value of the `installed` field is shown in a new `installed` column.
Fixes#1920
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
Adds a new templating function called `readDir`. With `readDir` users can read the contents of a specified directory and get a list of all contained files. This is useful when reading a bunch of files from a directory. The following example shows a snippet of a values file for configuring a Logstash deployment. Using only `readFile`, a user must specify each file by hand and adjust this list as the number of files to be read grows.
```yaml
logstash:
configs:
logstash.yml: |
{{- tpl (readFile "config/logstash.yml.gotmpl") . | nindent 6 }}
jvm.options: |
{{- readFile "config/jvm.options" | nindent 6 }}
pipelines.yml: |
{{- readFile "config/pipelines.yml" | nindent 6 }}
pipelines:
beats-log4j.conf: |
{{- readFile "config/pipelines/beats-log4j.conf" | nindent 6 }}
nginx-access.conf: |
{{- readFile "config/pipelines/nginx-access.conf" | nindent 6 }}
nginx-error.conf: |
{{- readFile "config/pipelines/nginx-error.conf" | nindent 6 }}
syslog-logs.conf: |
{{- readFile "config/pipelines/syslog-logs.conf" | nindent 6 }}
tcp-logs.conf: |
{{- readFile "config/pipelines/tcp-logs.conf" | nindent 6 }}
udp-debug.conf: |
{{- readFile "config/pipelines/udp-debug.conf" | nindent 6 }}
udp-logs.conf: |
{{- readFile "config/pipelines/udp-logs.conf" | nindent 6 }}
certificates:
ca.crt: |
{{- readFile "config/certificates/ca.crt" | nindent 6 }}
logstash.crt: |
{{- readFile "config/certificates/logstash.crt" | nindent 6 }}
logstash.key: |
{{- readFile "config/certificates/logstash.key" | nindent 6 }}
```
With `readDir` the above snippet can be rewritten as follows:
```yaml
logstash:
configs:
{{- range readDir "config" }}
{{ base . }}: |
{{- if hasSuffix "gotmpl" . }}
{{- tpl (readFile .) $ | nindent 6 }}
{{- else }}
{{- readFile . | nindent 6 }}
{{- end }}
{{- end }}
pipelines:
{{- range readDir "config/pipelines" }}
{{ base . }}: |
{{- readFile . | nindent 6 }}
{{- end }}
certificates:
{{- range readDir "config/certificates" }}
{{ base . }}: |
{{- readFile . | nindent 6 }}
{{- end }}
```
Fixes#1646
This will allow us to use the new `fromJson` and `mustFromJson` functions
among others.
Co-authored-by: Graeme Gillies <ggillies@gitlab.com>
Parses a new field in repositories named `skipTLSVerify` and if set to `true`, it appends `--insecure-skip-tls-verify` in `helm repo add` command.
This should be useful with internal self-signed repos, mitm proxies etc.
Resolves#1871
* Do fail on a possible typo in `needs` entries
Helmfile kindly fails with a friendly error when you made a typo in a `needs` entry, i.e. a `needs` entry included a reference to a release that is not defined in the helmfile config.
Example Output:
```
in ./helmfile.needs.yaml: release(s) "app" depend(s) on an undefined release "infrastructure/cert-manager2". Perhaps you made a typo in `needs` or forgot defining a release named "cert-manager2" with appropriate `namespace` and `kubeContext`?
```
This prevents issues like #1959
* Fix regression in helmfile-diff (This may break when you had two or more duplicated releases that are intended to be de-duplicated before DAG calculation using selectors
* Fix regression when you used selector to deduplicate releases before DAG calculation
* Comments
* Fix regressions in helmfile-apply and helmfile-sync
* Fix regression in duplicate release detection
Currently it's not possible to use `.Environment` values in `*.gomtpl` files. The documentation states the opposite:
https://github.com/roboll/helmfile#environment (2nd paragraph).
The problem is already described in #1090.
This PR fixes this bug.
Fixes#1090
Co-authored-by: Peter Aichinger <petera@topdesk.com>
* Guard collectNeeds-method against infinite recursion.
* Also check for namespace and kubecontext when collecting needs.
Co-authored-by: Peter Aichinger <petera@topdesk.com>
It turned out the Helm 2.17.0 binary is nowhere now. I considered a bit about checking for a newer Helm 2.x releases as an alternative but I resisted as it is almost a year since EoL of Helm 2. Thanks Helm 2. Long live Helm 3!
Apparently we needed to pass `--validate` on helm-template run by chartify when the targeted chart contains Capabilities.APIVersions in a chart template. Otherwise, you can never make such chart work with chartify, as at apply time helm template expressions that involved Capabilities.APIVersions are already nowhere.
- Add debian image based on `stable-slim`, desire for this is largely
around my use case using Azure DevOps which makes it challenging to
use images which are not glibc based.
- Drop support for helm2 in the docker images. This is a tricky one but
given that I was having errors during the docker build for helm2 and
the fact that it has been EoL for a long time now made me think that
this was the correct move.
- As a "while I'm in here" I've upgraded kubectl and helm. I've popped
on the most current patch of the last release (v1.20.3) to give a
slightly broader support for different Kubernetes versions.
- Reworked CI to support pushing a debian and alpine base, and dropped
support for the helm2 versions.
This adds the ability to include the --pass-credentials flag to the helm add repo command by:
- Adding repo.passCredentials to the helmfile yaml
- Changing state, helmexec, and app to include RepositorySpec.PassCredentials
Resolves#1898
Co-authored-by: almed4 <alexandre.meddin@ingka.ikea.com>
This release includes new `ref+tfstates3://` and `ref+tfstategs://` backends for reading terraform states stored in s3 and google cloud storage, and features a new aws-sdk-go that supports AWS SSO, and the fix to allow referencing a local tfstate file by the absolute path.
See https://github.com/variantdev/vals/releases/tag/v0.14.0 for all the changes.
#1772 broke `--selector` with `needs` in many ways.
The two biggest problems I've encountered were:
- duplicate releases even if you've provided a proper `selector` to deduplicate
- sync/deletion ordering broken when you have `needs`
For the first issue, we had to update `getSelectedReleases` function to also calculate the "selected releases and releases needed by the selected releases", and use that to calculate the DAG. That should have been done in #1772.
The latter started happening after I've fixed the first issue. The source of the issue was that `needs` turned out to be ambiguous in a few cases.
Previously, `needs: ["foo/bar"]` had two meanings. One for "needs release bar in kubecontext foo", another for "needs release bar in namespace foo".
Moreover, `needs: ["foo/bar/baz"]` had three meanings.
- `needs release baz in tiller namespace foo and namespace baz`
- `needs release baz in namespace bar in kubecontext foo`
- `needs release baz in tiller namespace bar in kubecontext foo`.
Especially, the first meaning doesn't make sense at all. Helm 2 solely use tillerNamespace for namespacing the release and Helm 3 uses namespace for that.
This fix sorts all the bugs and issues I've found so far around that, by changing the meanings of the above two examples as follows:
- `foo/bar` means `namespace=foo,name=bar` for Helm 3 and `tillerNamespace=foo,name=bar` for Helm 2
- `needs release bar in kubecontext foo` is now `foo//bar`. Notice the extra `/` between `foo` and `bar`.
- `foo/bar/baz` means `kubecontext=foo,namespace=bar,name=baz` for Helm 3 and `kubecontext=foo,tillerNamespace=bar,name=baz` in Helm 2
Fixes#1818
* tests: fix vagrant test run
* feat: added an option to specify the different diff output format
* renamed diff-output to output
* renamed diff-output to output
Co-authored-by: Andrey Tuzhilin <andrey@zelf.co>
* Bump chartify to 0.8.2
This version fixes charitfy not to fail when you used the combination of (1)helm 3 and (2)strategicMergePatches/jsonPatches/transformers etc that triggers chartify on (3)a chart that contains CRDs.
See https://github.com/roboll/helmfile/issues/1778#issuecomment-824451990 for details of the issue.
The chartify-side of this fix is 55b23f9e9dFixes#1778
* Add test for helmfile-destroy with selector
I was curious if destroy with a selector is working at all while reading https://github.com/roboll/helmfile/issues/1750#issuecomment-823521572. So I added this test for verification. From the test result, it is working, though.
* Add one more test case for destroying disabled but installed release
* Add --{include,skip}-needs to helmfile-sync and helmfile-apply
* Add --include-needs to helmfile-template
* Add TODO related to #1018
* Add a few new test files to cover new functionalities
* Update apply test to incorporate the change that the destroy and sync steps target affected releases only
Helmfile has been providing a feature called "adhoc chart dependency" that basially enabled you to add Chart.yaml `dependencies` entry adhocly without forking or modifying the chart.
It was missing the support for using a local chart as the adhoc dependency. This patch adds that.
Usage:
`releases[].dependencies[].chart` is enhanced to accept the fs path to the local chart:
```
releases:
- name: foo
chart: ./path/to/foo
dependencies:
- chart: ./path/to/bar
```
Resolves#1762
Ref https://github.com/roboll/helmfile/issues/809#issuecomment-814423653
This is not what had been requested originally in #809 but anyway- This enables you to limit helm-dep-up run by helmfile-deps by selector. For example, `helmfile -l name=foo deps` should result in only the release named `foo` to be helm-dep-uped.
We use kustomize in two places. One for turning kustomize into a chart, and another for patching manifests and the chart.
This fixes the former to work with kustomize v4.
Fixes#1688
Related to #494
This feature is mostly a built-in alternative to the `incubator/raw` chart without external dependency and has
access to helmfile's own template functions and template data.
The expected use-case of this feature is to add arbitrary K8s resources to your deployment.
Unlike the original issue raised in #494 this doesn't enable you to add arbitary resources to a release. That's another story. But this would be a good foundation for that, too.
Secret files ending with .gotmpl are now also rendered as a gotemplate.
```
releases:
- name: myapp
secrets:
- secrets.yaml.gotmpl
```
Note that currently, .gotmpl files must be valid YAML files as well.
The expected use-case of this feature is to compose a YAML array from values and encrypted secrets.
Without this feature, you would have tried to do something like the below, which didn't work.
**Example (doesn't work!)**
`values.yaml.gotmpl`:
```
environment:
- name: MY_EXTERNAL_IP
value: |
{{ exec "./get-external-ip.sh" (list "") }}
```
`secrets.yaml`:
```
_sops:
#...
environment:
- name: MY_SECRET_VALUE
value: (encrypted by sops)
```
`helmfile.yaml`:
```
releases:
- name: foo
values:
- values.yaml
secrets:
- secrets.yaml
```
This doesn't work because `values.yaml` and the decrypted `secrets.yaml` are passed to `helm` to be merged, and helm overrides the array instead of merging or concatenating the arrays.
**Example (works!)**
Instead of `values.yaml` and `secrets.yaml`, you provide a single `secrets.yaml.gotmpl` that is a valid YAML and encrypted by sops:
```
_sops:
#...
environment:
- name: MY_EXTERNAL_IP
value: |
{{ exec "./get-external-ip.sh" (list "") }}
- name: MY_SECRET_VALUE
value: (encrypted by sops)
```
`helmfile.yaml`:
```
releases:
- name: foo
secrets:
- secrets.yaml.gotmpl
```
Helmfile decrypts the gotmpl by handing it over to helm-secrets and then renders the result as a gotmpl file. The end result is that you have a two-element array `environments` that can be just passed to helm.
Resolves#1700
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
Adds `--chart` flag for overriding the selected release's chart ad-hoc-ly like `helmfile --chart $CHART template`.
This is handy when e.g. you want to have an ArgoCD application per each release in your helmfile.yaml, while also providing the ability to customize the release's chart without touching helmfile.yaml.
See https://github.com/roboll/helmfile/issues/1690#issuecomment-812321354 for more context.
Closes#1690
This enables you to write a `kubectl-apply` hook more declaratively than writing `command` and `args`:
```
releases:
- name: myapp
chart: mychart
hooks:
- events: ["presync"]
kubectlApply:
filename: path/to/manifests
#kustomize: path/to/kustomize
```
This would allow cli flag `--kube-context` to override value in helmDefaults allowing to use different values in local development and CI context.
Co-authored-by: Andrey Tuzhilin <andrey@3adigital.ru>
* Update referance for helm-secrets to latest maintained fork
* Missed a referance in the docs to an older edition of helm secrets
* Removed docker2 changes back to original method
Co-authored-by: Nathan Flynn <nflynn@williamhill.co.uk>
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
- replace deprecated helm-secrets with the currently active helm-secrets
```plain
Deprecation Info
Please note, this project is no longer being maintained.
Link to active helm-secret plugin could be found in helm documentation: https://helm.sh/docs/community/related/#helm-plugins
```
* Fix OCI support
I have seen various issues related to the OCI repository support recently added to Helmfile.
This is the patch that should fix all the issues Im aware of until now.
Adding documentation for this feature: https://github.com/helm/helm/pull/6819
not being aware of this, make turn helm2 to 3 migrations very complicated.
This finishes the work started at #1619 by introducing the following changes:
- You can now set `HELMFILE_TEMPDIR` and `CHARTIFY_TEMPDIR` as the directory to persist temporary values files rendered by helmfile and temporary charts generated by chartify. Note that chartify is used internally when you use kustomize integration or raw K8s manifests as a chart.
- Helmfile uses the hash sum of the release config and the values file content in the file name of the temporary values file.
In combination with `HELMFILE_TEMPDIR` and `CHARTIFY_TEMPDIR`, this enables helmfile to use the stable file names for temporary values files, which contribute to stabilize log messages like `Comparing release=appset, chart=PATH/TO/TEMPORARY/CHART`, where the `PATH/TO/TEMPORARY/CHART` had been randomized due to formerly random helmfile tempdir and temporary values file names, and random chartify tempdir.
You can try this feature with a script like:
```
tempdir=$(mktemp -d -t helmfile)
HELMFILE_TEMPDIR=${tempdir} CHARTIFY_TEMPDIR=${tempdir} helmfile diff
rm -rf ${tempdir}
```
`helmfile-diff` sorts multiple and concurrent helm-diff outputs and stabilizes writes to stdout.
It's required to use the stdout from helmfile-diff to detect if there was another change(s) between 2 points in time.
For example, terraform-provider-helmfile runs a helmfile-diff on `terraform plan` and another on `terraform apply`. `terraform`, by design, fails when helmfile-diff outputs were not equivalent. Stabilized helmfile-diff output rescues that.
This improves helmfile-apply with two things:
- Some users had timing-out issues or annoyed by huge output from helm-diff run as part of helmfile-apply on first install. `--skip-diff-on-install` skips running helm-diff for releases being newly installed, so that you can avoid those issues.
- Some users had difficultly or found it not straight-forward to install CRDs and custom resources from separate charts in one helmfile-apply (#1353). The new helmfile.yaml release field `disableValidationOnInstall: true` adds `--disable-validation` to helm-diff only for releases being newly released, which should mostly resolve the issue.
Resolves#1353
Please see the updated `advanced-features.md` for more details.
This is often used for adding common labels and annotations to any resources rendered from a Helm chart.
* feat: enable EnableBashCompletion to get --generate-bash-completion command line flag
* feat: add scripts for bash and zsh
* feat: document shell completion
* Fix `helmfile template --include-crds` not to break with `chartify`
This bumps variantdev/chartify to 0.4.9 so that we can incorporate fed8bb2953
* Fix integration test
Small update to the readme repository examples since Google is no longer officially supporting hosting the stable/incubator helm charts. The official helm docs are using these repos now, eg. https://helm.sh/docs/intro/quickstart/.
Bumps dependencies mainly to fix possible go mod error that is fixed via 0edd534322, but also bumps all the other dependencies because they looked very outdated :)
This deprecates the old `--retain-values` which was not working as intended.
Also see #1570 - there's now `--skip-cleanup` for `helmfile-template`, too.
This allows you to use helmfile-template output as a GitOps source, when the template output contains CRDs and you use Helm 3.
Helm 3 by default removes CRDs from the template output. If you want to git-commit helmfile-template containing CRDs for GitOps and you use Helm 3 for templating, the only way is provide this newly added `--include-crds` flag.
I had mistakenly inverted the necessary flag to turn off the legacy processing on selectors that should be used only for helmfile commands that does not support DAGs/needs.
Fixes#1552
`helm dep up` is now skipped while running helm-x/chartify when the chart/directory is obtained by running go-getter, or `skipDeps` is configured using a command-line flag, helmDefaults, or release configuration.
Resolves#1547
Changes:
* Bump helm to v3.3.4.
* Bump kubectl to v1.18.9
* Update Dockerfile to be a little more verbose to help understand which command in a chain failed.
Signed-off-by: Jake Hill <jake@naphta.uk>
While using the `--debug` option, the output of helm seems to be printed triple times:
```
Templating release=test, chart=charts/test
exec: helm template test charts/test --namespace staging --values /tmp/values266548685 --output-dir outputs//helmfile-test-staging-ed5e3c94-test --debug
helm:MRAjW> install.go:172: [debug] Original chart version: ""
helm:MRAjW> install.go:189: [debug] CHART PATH: /app/charts/test
helm:MRAjW> wrote outputs//helmfile-test-staging-ed5e3c94-test/test/templates/secret.yaml
helm:MRAjW> wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/configmap.yaml
helm:MRAjW> wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/health-configmap.yaml
helm:MRAjW> wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/headless-svc.yaml
helm:MRAjW> wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/metrics-svc.yaml
helm:MRAjW> wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/redis-master-svc.yaml
helm:MRAjW> wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/redis-master-statefulset.yaml
helm:MRAjW> wrote outputs//helmfile-test-staging-ed5e3c94-test/test/templates/ingress.yaml
helm:MRAjW> wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/metrics-prometheus.yaml
helm:MRAjW>
exec: helm template test charts/test --namespace staging --values /tmp/values266548685 --output-dir outputs//helmfile-test-staging-ed5e3c94-test --debug: wrote outputs//helmfile-test-staging-ed5e3c94-test/test/templates/secret.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/configmap.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/health-configmap.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/headless-svc.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/metrics-svc.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/redis-master-svc.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/redis-master-statefulset.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/templates/ingress.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/metrics-prometheus.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/templates/secret.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/configmap.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/health-configmap.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/headless-svc.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/metrics-svc.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/redis-master-svc.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/redis-master-statefulset.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/templates/ingress.yaml
wrote outputs//helmfile-test-staging-ed5e3c94-test/test/charts/redis/templates/metrics-prometheus.yaml
```
- The first with the prefixes `helm:MRAjW>` comes from <563fce4adf/pkg/helmexec/runner.go (L59-L65)>
- The second one with the format `exec: helm ... --debug: ...` comes from <563fce4adf/pkg/helmexec/exec.go (L369)>
- And the last are the logs of non-debug mode.
It sometimes makes the log a bit confusing, while I first time read the log, I thought helmfile executed helm command multiple times 😅.
Since we already piped helm outputs to stdout (the first), we remove the second logs.
Adds a basic support for Helm repositories hosted on Azure Container Registry (not OCI but classic ones). Add a new field to RepositorySpec to state that is externally managed and runs the `az-cli` command instead of the helm one to manage the repository.
This fixes the bug that generated Chart.yaml misses the `apiVersion` field which resulted in `helm lint` always failing on the generated chart.
Fixes#1527
* Parse and process helm version using github.com/Masterminds/semver/v3.
* Add --force-update only when Helm version >= 3.3.2, < 3.3.4.
See: https://github.com/helm/helm/pull/8777.
* Add test cases.
* Bump sprig to v3.1.0
test for mergeOverwrite
* Let mergo not (accidentally) try to merge unexported fields
This is also a good chance separate `HelmState` with the config loaded from YAML, which I had been wanting to do for a long time.
Co-authored-by: Johannes Alkjær <johannes.alkjaer@wunderman.com>
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
This is an experimental feature to support a potential use-case that you need to set namespaces in manifests rendered by `helmfile template`, WHEN the chart is unconventional hence does not have `namespace: {{ .Namespace }}`.
Rather than using this, you should usually fork/maintain or update/pull-request the chart to have `namespace: {{ .Namespace }}`.
Use this only when you CAN NOT do so, but still need to use `helmfile template`.
This is an attempt to log helm exec output as the helm exec command is running. It only prints if the --debug flag is set.
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
This adds `comonLabels` option to helmfile by:
- Adding `CommonLabels` to HelmState
- Changing `markExcludedReleases` and `ListReleases` functions to merge common labels into release labels
Resolves#1266
To fix the issue that adhoc json patches were not working on kustomize/raw manifests.
Note that regular kustomize project was working. In other words, this only affetcts `chart: path/to/dir` combined with `jsonPatches: ...` when the `path/to/dir` points to a kustomize project or a local directory containing raw K8s manifests.
Ref https://github.com/roboll/helmfile/issues/1434#issuecomment-682247709
This fixes the regression introduced by #1406 that resulted in `helmfile template` with helm v2 failing when there is any release with `version`.
Fixes#1414
This reverts a part of #1383 so that repository updates are done in the pre-0.125.0 way, which tries to update any repositories only once regardless of they are referenced by selected releases or not.
Ref #1404
In #1172, we accidentally changed the meaning of prepare hook that is intended to be called BEFORE the pathExists check. It broke the scenario where one used a prepare hook for generating the local chart dynamically. This fixes Helmfile not to fetch local chart generated by prepare hook.
In addition to that, this patch results in the following fixes:
- Fix an issue that `helmfile template` without `--skip-deps` fails while trying to run `helm dep build` on `helm fetch`ed chart, when the remote chart has outdated dependencies in the Chart.lock file. It should be up to the chart maintainer to update Chart.lock and the user should not be blocked due to that. So, after this patch `helm dep build` is run only on the local chart, not on fetched remote chart.
- Skip fetching chart on `helmfile template` when using Helm v3. `helm template` in helm v3 does support rendering remote charts so we do not need to fetch beforehand.
Fixes#1328
May relate to #1341
Changes:
- Prevent Helmfile from unnecessarily running `helm repo add` and `helm repo up` against repositories for unused repositories(repositories of releases filtered out by selector)
- Fixes#1330
This, in combination with #1172, allows you to use `go-getter`-supported URL for K8s manifests on `chart`, so that Helmfile automatically fetches it and then turning it into a temporary local chart, which is then installed by Helmfile as similar as standard Helm charts.
An example usecase of this is to install cert-manager CRDs which is distributed separately from the chart:
```
releases:
- name: cert-manager-crds
chart: git::http://github.com/jetstack/cert-manager.git@deploy/crds?ref=v0.15.2
```
I'm adding this based on discussion with @lukasmrtvy. He was trying to install cert-manager and prometheus-opreator with Helmfile, and this combined with #1373 should do the job. Thanks for the input!
`disableOpenAPIValidation: true` might be useful for workaround for broken CRDs that is known to be exist in older OpenShift versions, and `disableValidation: true` is confirmed to allow installing charts like prometheus-operator that tries to install CRDs and CRs in the same chart.
Strictly speaking, for the latter case I believe you only need `disableValidation: true` set during the first installation, but for the ease of operation I shall suggest you to always set it.
Obviously turning validation mostly(disableOpenAPIValidation) or entirely(disableValidation) result in deferring any real error until sync time. We need completely client-side validation that is able to read CRDs and use it for validating any CRs to catch any error before sync. But it worth an another (big) issue.
Fixes#1124
This is useful for e.g. removing state file names and their hash values out of output dirs so that it can be used easily in a gitops setup. For example, `--output-dir-template mybasedir/{{.Release.Name}}` produces `mybasedir/RELEASE/CHART/templates/*.yaml` for each release in your helmfile.yaml.
* Move kube-context from helmDefaults to 1'st argument to let kube-context per release override it
* fix tests
Co-authored-by: Alex Vorona <av@dysnix.com>
- Add `version` parameter for Vault provider
- Add `profile` parameter for AWS SSM/SecretsManager provider
= Add `version` parameter for AWS SSM Parameter Store provider
- Add support for app-role authentication when using Vault provider
Summary of changes:
* Output any error from Mkdir in `helmfile template`
* Add failing test for .Release.Name interpolation
* Add golden files for testing
* Parse resources with kustomize to compare them structure by structure
* Decode resources into plain maps
The RNode type from kustomize uses yaml.Node under the hood,
which carries extra information like line numbers, which
become noisy when comparing with deep.Equal.
Changes:
* Add global hooks
* Add top level hooks field to yaml spec
* Add functions for global prepare and cleanup events
* Call global prepare and cleanup events in withPreparedCharts function
* Update README
* Add helmfileCommand variable to withPreparedCharts
Pass the information on what helmfileCommand has been run down from the
top level functions through withReposAndPreparedCharts and withPreparedCharts.
* New output flag for list command
Support output as json
Add new formatters file to handle extrac formatting to its own concern
New config interface to support list command specification
* Fix usage message
* Add error handling for formatters
* Replace usage of getOrNil with get function
* Add nil in get function
Co-authored-by: Rene Hernandez Remedios <rene.hernandez@fullscript.com>
Probably this has been affecting strategicMergePatches and dependencies as well, as the source of the problem is the underlying common feature called helm-x that previsouly used `helm template --output-dir`, which turned out to be not working as intended.
Ref #1279
A kustomization has not been properly chartified when a kustomization is specified on `releases[].chart` but there were no `releases[].strategicMergePatches` and `jsonPatches` specified.
This fixes that.
This is the GA version of the helm-x integration #673 developed last year.
You get all the following benefits without an extra helm plugin:
- Ability to add ad-hoc chart dependencies/aliases, without forking the chart (Fixes#876 )
- Ability to patch resulting K8s resources before installing the helm chart
- Ability to install a kustomization as a chart (Requires `kustomize` binary to be available in `$PATH`
- Ability to install a directory of K8s manifests as a chart
- etc.
Fixes https://github.com/roboll/helmfile/issues/1142
desired_state_file_loader.go
- Will now normalize the content before splitting it to parts
context:
Me & and a fellow dev have tried to figure out why helmfile didn't fill in certain values on his machine;
turns out, he'd mistakenly checked out our project w/ CRLF line endings, which had caused part splitting to not work (as it's hard coded to look for '\n').
The following was acted on as a single part, causing values from the bases not to be available in the next yaml part:
```
bases:\r\n
- base.yaml\r\n
---\r\n
releases:
- name: external-secrets-crd
... some templated yaml ...
```
I've thought about regex-ing it out instead of replace-all, but benchmarks had shown that a plain replace is faster.
I've also considered splitting by "\n---" instead of "\n---", but that would break if the dashes were to continue with some other text.
This prevents typos in helmfiles from silently preventing operations
from running. The default behavior is to print a warning, to preserve
backwards compatibility.
Add a global flag --debug that sets the logging level to debug and passes the global flag --debug to Helm through the implementation of subcommands’ --args.
Resolves#1104
Co-authored-by: ayoub mrini <amrini@wiremind.fr>
Resolves#1232
CHANGES:
* Disable repo update when installed is false
When install is false, we do not need to update the repositories and get
the chart.
Signed-off-by: Guillaume Perrin <guillaume28.perrin@gmail.com>
- createNamespace is a new attribute that can be added to helmDefaults
or an individual release to enforce the creation of a release namespace
during sync if the namespace does not exist. This leverages helm's
(3.2+) --create-namespace flag for the install/upgrade command. If
running helm < 3.2, the createNamespace attribute has no effect.
Resolves#891Resolves#1140
* feat(tmpl): added fetchSecretValue template function
This adds a tmpl `fetchSecretValue` and `expandSecretRefs` function by:
- Adding:
- `expandSecretRefs` function in tmpl package that uses vals
package to fetch secrets
- `fetchSecretValue` function in tmpl package like below but for
single string value
- gomock for tests purpose
- Changing:
- move init of vals package to function (so the same instance can be used for template values and rendering the whole template)
* doc(secret): added doc how to use new tmpl methods
Added example usage of `fetchSecretValue` and `expandSecretRefs`
Prior to this change, these two examples suggested enclosing a release spec in
an if expression, which breaks the lock file when switching environments.
This change parameterizes `installed` instead of the entire release, thus
preserving the dependency locking functionality.
* New output flag for list command
Support output as json
Add new formatters file to handle extrac formatting to its own concern
New config interface to support list command specification
* Fix usage message
* Add error handling for formatters
This commit proposes a potential solution for
https://github.com/roboll/helmfile/issues/1201
The gist is that, if prestate rendering fails, for any reason,
we do not populate the .Values in the second pass renderer.
I think that what have been expected in this case is to populate the
.Values irregardless.
pkg/app/two_pass_renderer.go
- Migrated to use finalEnv.GetMergedValues()
pkg/environment/environment.go
- Introduced GetMergedValues, which merges the environment's defaults
and current values, and then casts the keys to string;
This was previously defined in HelmState.Values() - however, as this
method is only concerned with the environment, I think it's more
appropriate for it to sit here.
pkg/state/state_exec_tmpl.go
- Extracted out HelmState.Values() to environment.go, see above
Put tiller binary on the path in helm v2.X container image so that helm-tiller plugin will use it instead of duplicating the download of helm tarball and performing version check.
vals v0.3.0 introduces the terraform output source that can be accessed by `ref+tfstate://path/to/tfstatefile/type.resourcename.prop` syntax. Please see the updated README section of vals for more info.
since mergo had been bumped to 0.3.9 an environment value like:
```
affinity: {}
```
was not merged properly (not merged at all) instead it threw an error
that it cannot find the key "affinity" in the environment values even
though it was outputted in debug output as read in properly.
Enhances Helmfile to print more helpful message on error while calling `exec` template function.
Helmfile has been printing error messages like the below:
```
in ./helmfile.yaml: error during helmfile.yaml.part.0 parsing: template: stringTemplate:5:8: executing "stringTemplate" at <exec "./exectest.sha" (list)>: error calling exec: exit status 1
```
Adding captured stdout and stderr, with some indentation to make it readable, it now produces the following message on missing executable:
```
$ make build && ./helmfile build
go build
in ./helmfile.yaml: error during helmfile.yaml.part.0 parsing: template: stringTemplate:5:8: executing "stringTemplate" at <exec "./exectest.sha" (list)>: error calling exec: fork/exec ./exectest.sha: no such file or directory
COMMAND:
./exectest.sha
ERROR:
fork/exec ./exectest.sha: no such file or directory
```
On non-zero exit status without output:
```
$ make build && ./helmfile build
go build
in ./helmfile.yaml: error during helmfile.yaml.part.0 parsing: template: stringTemplate:5:8: executing "stringTemplate" at <exec "./exectest.sh" (list)>: error calling exec: exit status 1
COMMAND:
./exectest.sh
ERROR:
exit status 1
```
On non-zero exit status with output:
```
$ make build && ./helmfile build
go build
in ./helmfile.yaml: error during helmfile.yaml.part.0 parsing: template: stringTemplate:5:8: executing "stringTemplate" at <exec "./exectest.sh" (list)>: error calling exec: exit status 2
COMMAND:
./exectest.sh
ERROR:
exit status 2
COMBINED OUTPUT:
out1
err1
```
Resolves#1158
`--retain-values-files` prevents temporary values files that were passed to Helm commands run by Helmfile for debugging purpose.
With that, you can manually rerun helm commands that were logged when `--log-level=debug` is enabled.
Resolves ##1117
* Add option to suppress diff on apply
Add --supress-diff option on apply. Usable for fresh installs when a
lot of output is produces by diff.
Resolves#458
* fix tests for suppress-diff
When chart is not set, an exception without any hints occure due
normalizeChart tries to access the first character of a 0 length string.
Properly inform user of missing chart.
This makes it possible to pass the API Capabilities to helmfile when executing a task that does not render against an actual cluster (diff, template, apply).
Resolves#1014
Adds support for indexed key arguments and escaped dot arguments to `--state-values-set`.
Things like `--state-values-set config\.yml.something=abc` or `--state-values-set config.something[0]=abc` can be passed in the command line.
Resolves#899
It looks like during the beta phase the testing functionality was tweaked / changed back during the beta phase but has since been returned back to how it worked originally.
RE: a6f4bc1bc0
Signed-off-by: Jake Hill <jake@naphta.uk>
When helmfile is run with `--environment NAME` and there was a base hemlfile that misses `environments`, helmfile had been trying to load env values for NAME and failing.
A base helmfile is allowed to reference values from within itself, but that's optional. In other words, a base helmfile that misses the env is okay as long as it doesn't self-reference env values.
So, this change allows missing env and env values while loading base helmfile. After loading, a base helmfile can fail due to referencing missing env values, but that's okay.
Fixes#1008
Those are not actually random but would have looked like so. We use an external go pkg `variantdev/vals` to expand urls like `ref+vault://foo/bar` contained in release values into their respective secret values.
There was a bug in `vals` that it tries to expand unintended types of strings which resulted in confusing errors like reported in #973.
`vals` fixed the issue in ba4c7a2987. This commit upgrades `vals` to accomodate that.
Fixes#973
Runs `helm version` in helmexec.New, and exposes a method on Interface to allow other packages to use the detected version. Preserves compatibility with previous HELMFILE_HELM3 mechanism.
Resolves#923
* Fix regression since 0.90.0 that Helmfile becomes too slow when there are many releases
Fixes#959
* Ensure that the up-to-date helm-diff is installed and used in integration tests
The problem was that `--namespace NS` had been not taken into account while deleting releases, that resulted in releases that should be deleted are not deleted.
The recent addition of the DAG support(`needs`) and the fixes on it broke the delete-on-sync functionality. And there were two more bugs. One is that it was not correctly running `helm delete` when needed and the another is that it was failing when `--selector` is specified and the releases to delete by sync found, but nothing actually got deleted. This fixes all of them.
Fixes#941
This ports the fix for `helfmile apply` to `sync`, so that specifying `--selector` doesn't break `helmfile sync`.
Also make `helmfile template` DAG-aware, so that the manifests are rendered in the order of dependency.
Ref #919
Adds the Dockerfile for Helm v3(Dockerfile.helm3) that contains both Helmfile and Helm v3(currently 3.0.0-rc.2).
Changes the CircleCI workflow to include a job to build the v3 image on every PR. The release job is enhanced to build and push the image with a tag prefixed with `helm3-`. So the resulting image should be available via e.g. `quay.io/roboll/helmfile:helm3-v0.90.5`.
`helmfile delete` is designed to be idempotent. That is, it is safe to be run when there is nothing to delete. A change in helm v3 broke that behavior. This enhances Helmfile to be able to behave the same for helm v2 and v3.
Until now `helmfile delete` with helm v3 had been failing when `--namespace` is provided. Helmfile was missing the fact that in helm v3 namespace must be passed instead of tiller-namespace to scope releases to be deleted.
```
err: release "b-1" failed: helm3rc2 exited with status 1:
Error: uninstall: Release not loaded: b-1: release: not found
in ./helmfile.yaml: release "b-1" failed: helm3rc2 exited with status 1:
Error: uninstall: Release not loaded: b-1: release: not found
```
This overrides the default helm command, if provided, as soon as possible.
This way it is already used in `visitStates`.
I ran into an issue using `HELM3` and `--helm-binary` together with helm-secrets.
I previously used tillerless, which i could now remove. This however caused `DecryptSecret` to fail, as it would still use the helm2 binary; because it runs before the first `helm.SetHelmBinary` call.
While helm-secrets is not fully helm3 compatible yet, its uses within helmfile are, and i was able to just install it as a helm3 plugin.
* fix: Fix `needs` to work for upgrades and when selectors are provided
Fixes#919
* Add test framework for `helmfile apply`
* Various enhancements and fixes to the DAG support
- Make the order of upgrades/deletes more deterministic for testability
- Fix the test framework so that we can validate log outputs and errors
- Add more test cases for `helmfile apply`, along with bug fixes.
- Make sure it fails with an intuitive error when you have non-existent releases referenced from witin "needs"
The fragment variant of the URIs e.g. `foo: ref+awsssm://path/prefix#key` was not working. This change fixes that by bumping `vals` which is the library providing the SSM support, along with other dependencies.
Introduces DAG-aware installation/deletion ordering to Helmfile.
`needs` controls the order of the installation/deletion of the release:
```yaml
relesaes:
- name: somerelease
needs:
- [TILLER_NAMESPACE/][NAMESPACE/]anotherelease
```
All the releases listed under `needs` are installed before(or deleted after) the release itself.
For the following example, `helmfile [sync|apply]` installs releases in this order:
1. logging
2. servicemesh
3. myapp1 and myapp2
```yaml
- name: myapp1
chart: charts/myapp
needs:
- servicemesh
- logging
- name: myapp2
chart: charts/myapp
needs:
- servicemesh
- logging
- name: servicemesh
chart: charts/istio
needs:
- logging
- name: logging
chart: charts/fluentd
```
Note that all the releases in a same group is installed concurrently. That is, myapp1 and myapp2 are installed concurrently.
On `helmdile [delete|destroy]`, deleations happen in the reverse order.
That is, `myapp1` and `myapp2` are deleted first, then `servicemesh`, and finally `logging`.
Resolves#715
- `replace` for thrift had to be removed to avoid the error `go: github.com/apache/thrift@v0.12.0 used for two different module paths (git.apache.org/thrift.git and github.com/apache/thrift)`
- Other dependencies are updated by running `go get -u github.com/variantdev/vals`
In trying to use the base quay.io/roboll/helmfile image to deploy a chart that depended on an external repository, I ran into this error:
```
Adding repo bitnami https://charts.bitnami.com/bitnami
in ./helmfile.yaml: in .helmfiles[0]: in dirac/helmfile.yaml: helm exited with status 1:
Error: Couldn't load repositories file (/root/.helm/repository/repositories.yaml).
You might need to run `helm init` (or `helm init --client-only` if tiller is already installed)
```
The minimum to work around that error would be approximately:
```
mkdir /root/.helm/repository && echo apiVersion: v1 > /root/.helm/repository/repositories.yaml
```
Though there might be other things that I'm unwittingly depending on in the base `helm init`, it should be more friendly if we just run `helm init --client-only` while building.
At the moment, if you have a helmfile.yaml like so:
```
releases:
- name: metrics-server
namespace: kube-system
chart: stable/metrics-server
```
If you try to run `helmfile deps`, you will get a 0 exit code and no log
output at whatsoever, signaling that there weren't any problems, but no
lock file will get created.
For example:
```
root@316073d4a104:/# helmfile deps
root@316073d4a104:/#
```
This behavior doesn't appear to be documented and is unintuitive to the
user.
This change adds a warning output for this same use case:
```
root@316073d4a104:/# helmfile deps
There are no repositories defined in your helmfile.yaml.
This means helmfile cannot update your dependencies or create a lock file.
See https://github.com/roboll/helmfile/issues/878 for more information.
root@316073d4a104:/#
```
Fixes#878
This adds clarity in docs by:
- Changing references to the supported file extension
- Previously, using `values.tmpl` in helmfile.yaml would throw errors. `values.gotmpl` gives expected output
Set `HELMFILE_HELM3=1` and run `helmfile` like `HELMFILE_HELM3=1 helmfile ...`.
When `HELMFILE_HELM3` is set, `test`, `template`, `delete`, `destroy` behave differently so that it works with Helm 3.
Note that `helmfile diff` doesn't work as `helm-diff` called under the hood doesn't support Helm v3 yet.
Ref #668
Closes#444 and #782
This is the final PR to fully cache and parallelize helm secret decryption. It threads the shared helmexec.Interface into the StateCreator and HelmState structs to be used during environment secret decryption. This should effectively cache secrets for the duration of a helmfile run, regardless of where they are first decrypted.
* Add kubectl and jq to Dockerfile
Signed-off-by: Jake Hill <jake@naphta.uk>
* Update kubectl to use checksum verification, and pin to a specific version
Signed-off-by: Jake Hill <jake@naphta.uk>
Resolves#792
Related to #782 and #444
- Allows concurrent decryption of different secrets files
- Caches decrypted secrets by original file path and returns decrypted results from memory
- Secrets being run through an instance of helmexec will be cached and run as fast as possible concurrently
NB: This particular PR doesn't make _all_ calls to secrets cached and concurrent. Environment Secrets in particular seem to not be evaluated with a ScatterGather(), and doesn't use the same helmexec instance as other parts of the code, so it doesn't take advantage of these changes. Some reworking of the plumbing there would be needed.
Use with the helm-x support(#673)
This enhances config syntax to accept adopt: [NS/KIND/RESOURCE_NAME] at the release level so that helmfile calls helm-x to transparently import existing resources at the installation time.
Resolves#84
This adds the possibility to use a "=" in the parameters passed to --state-values-set
previous comportment:
`helmfile --state-values-set 'test="abcde==fg",test2="abcde"'` => outputs 'test="abcde",test2="abcde"'
new comportment:
`helmfile --state-values-set 'test="abcde==fg",test2="abcde"'` => outputs 'test="abcde==fg",test2="abcde"'
Resolves#689
This adds a new yaml entry for the hook definition to allow the users to specifcy if they want to show the `command` logs or not.
here is an example.
```
releases:
- name: myapp
chart: mychart
# *snip*
hooks:
- events: ["cleanup"]
showlogs: true
command: "kubectl"
args: ["get", "ingress"]
```
this will display the following output:
```
hook[cleanup] logs | NAME HOSTS ADDRESS PORTS AGE
hook[cleanup] logs | catalog-gateway tdc.foo 80 2d6h
hook[cleanup] logs | dataset foobar.barr.foo.xxxxxxx.com 80 2d6h
hook[cleanup] logs | rating fooba.barr.foo.xxxxxxx.com 80 2d6h
hook[cleanup] logs | sharing foobar.barr.foo.xxxxxxx.com 80 2d6h
hook[cleanup] logs | tpsvc-iam-dev foo.barr.foo.xxxxxxx.com 80 2d6h
hook[cleanup] logs | tpsvc-iam-front bar.barr.foo.xxxxxxx.com 80 2d6h
```
The root cause of this bug was due to that `--kube-context` and `kubeContext` had been treated specifically in code. So on the way I have made it consistent with other per-release settings - by adding `kubeContext` for each release and treating `helmDefaults.kubeContext` as just the default value for per-release setting.
Fixes#674
This enhances helmfile so that it can:
- Treat K8s manifests directories and Kustomize projects as charts
- Add adhoc chart dependencies on sync/diff/template without forking or modifying chart(s) (#649)
- Add adhoc patches(JSON Patch or Strategic Merge Patch supported) to be applied to the K8s resources before sync/diff/template, without forking or modifyin chart(s) (#650)
The usage is as outlined in https://github.com/mumoshu/helm-x/tree/master/examples/helmfile.
Add any or all of `dependencies:`, `jsonPatches:` and `strategicMergePatches:` so that it adds additional flags to `helm` calls that is only supported by `helm x`.
```yaml
releases:
- name: kustomize
chart: ../kustomize
- name: manifests
chart: ../manifests
- name: foo
chart: incubator/raw
dependencies:
- alias: bar
chart: incubator/raw
values:
- values.yaml
- bar:
enabled: true
resources:
- apiVersion: v1
kind: Pod
metadata:
name: bar
spec:
containers:
- command:
- sleep
- 1000
image: alpine:3.9.4
imagePullPolicy: IfNotPresent
name: bar
jsonPatches:
- target:
version: v1
kind: Pod
name: foo
patch:
- op: replace
path: /spec/containers/0/command
value:
- sleep
- "123"
strategicMergePatches:
- apiVersion: v1
kind: Pod
metadata:
name: bar
spec:
containers:
- name: bar
command:
- sleep
- "234"
```
You can alternatively provide `source: path/to/patch.yaml` for `jsonPatches` and `strategicMergePatches` items to externalize it. Add `.gotmpl` suffix like you would do for values files for templating.
When running `helmfile` you must point `--helm-binary` to the `helm-x` binary like below:
```
$ helmfile --helm-binary ~/.helm/plugins/helm-x/bin/helm-x --log-level debug apply
```
after installing the [helm-x](https://github.com/mumoshu/helm-x) plugin.
The integration should ideally be automatic. That is, it shouldn't force you to set `--helm-binary`. But I had no other way to not bloat helmfile's codebase to just add this experimental feature.
Resolves#649Resolves#650
Probably since #647 helmfile has been unable to merge nested maps in environment values if they were loaded from files. This fixes it.
The relevant test is also enhanced so that no further regression like this happens.
Fixes#677
Extends the remote-helmfile feature to also work when loading the first state file.
This should be useful for people who wants to give helmfile a try without ever opening `$EDITOR`.
* fix: persist original file path when using bases
Prior to this change, the resulting lock file was called `<bases[0]>.lock`,
instead of `<filename>.lock`.
This change ensures the final, merged state has the correct `.FilePath`.
* test: Assert proper FilePath in layered HelmState
helm-secrets uses the `HELM_SECRETS_DEC_SUFFIX` env var to define the name of the output file
we should have the same logic in helmfile, to come up with the same filename
It only affects people using the `HELM_SECRETS_DEC_SUFFIX` env var
Use-case: if you want to run multiple `helmfile` commands in parallel, without conflicts. in this case, you need to decrypt secrets with different suffixes.
Resolves#435 (Git as chart repository)
Resolves#220 (S3 as chart repository)
Resolves#436 (About bundling helm plugins)
A lot of thanks to @aslafy-z for authoring the awesome helm-git plugin and contributing it to the community!
This change enhances helmfile to accept terraform-module-like URLs in nested state files a.k.a sub-helmfiles.
```yaml
helmfiles:
- # Terraform-module-like URL for importing a remote directory and use a file in it as a nested-state file
# The nested-state file is locally checked-out along with the remote directory containing it.
# Therefore all the local paths in the file are resolved relative to the file
path: git::https://github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.40.0
```
The URL isn't equivalent to terraform module sources. The difference is that we use `@` to distinguish between (1) the path to the repository and directory containing the state file and (2) the path to the state file being loaded. This distinction provides us enough fleibiity to instruct helmfile to check-out necessary and sufficient directory to make the state file works.
Under the hood, it uses [hashicorp/go-getter](https://github.com/hashicorp/go-getter), that is used for [terraform module sources](https://www.terraform.io/docs/modules/sources.html) as well.
Only the git provider without authentication like git-credentials helper is tested. But theoretically any go-getter providers should work. Please feel free to test the provider of your choice and contribute documentation or instruction to use it :)
Resolves#347
This adds `values` to state files as proposed in #640.
```yaml
values:
- key1: val1
- defaults.yaml
environments:
default:
- values:
- environments/default.yaml
production:
- values:
- environments/production.yaml
```
`{{ .Valuese.key1 }}` evaluates to `val1` if and only if it is not overrode via the production or the default env, or command-line args.
Resolves#640
Seems like we are affected by https://github.com/golang/go/issues/24963. That is, even though we internally use the template option `missingkey=zero`, in some cases it still prints `<no value>` instead of zero values, which has been confusing the state yaml parsing.
This fixes the issue by naively replacing all the remaining occurrences of `<no value>` in the rendered text, while printing debug logs to ease debugging in the future when there is unexpected side-effects introduced by this native method.
Fixes#553
The addition of `--set k1=v1,k2=v2` and `--values file1 --values file2` was originally planned in #361.
But it turned out we already had `--values` for existing helmfile commands like `sync`. Duplicated flags doesn't work, obviously.
So this actually add `--state-values-set k1=v1,k2=v2` and `--set-values-file file1 --set-values-file file2`.
They are called "state" values according to the discussion we had at #640Resolves#361
* feat: helmfile as a go library
This removes almost all the dependencies from the helmfile core logic to urfave/cli. `main.go` is now a thin wrapper around the core logic implemented in `pkg/app`.
I will start by making as much as possible code in `main.go` independent from urfave/cli, and this illustrates how we can do that by introducing a Config interface that delegates any config value fetch to urfave/cli. We will implement an cobra/pflag impl of the interface when helmfile finally migrates to cobra.
We added envvals overrides in the state file via #622 two days ago:
```
helmfiles:
- name: sub.helmfile.yaml
environment:
values:
- mykey: myvalue
```
This change removes the `environment` level in the above cofig, so that it looks like:
```
helmfiles:
- name: sub.helmfile.yaml
values:
- mykey: myvalue
``
This is an inevitable breaking change towards #361. But I wanted to break it earlier so that less folks are affected.`
Ref https://github.com/roboll/helmfile/issues/361#issuecomment-497530819
* feat: specify env values from the parent to the nested state
Adds the `helmfiles[].environment.values` that accepts a mix of file pathes and inline dictes:
```yaml
helmfiles:
- path: path/to/nested/helmfile.yaml
environment:
values:
- key1: val1
- values.yaml
```
The values files are loaded in the context of the parent state file. For example, in case the above state file is located at `/path/to/helmfile.yaml`,
`values.yaml` is located at `/path/to/values.yaml` instead of `/path/to/nested/values.yaml`.
Resolves#523
* fix: multiple "bases" declarations yields duplicate releases
Fixes#615
* fix regression in double-rendering with env value overrides
The latest commit broke any state files like the below to NOT pass env value overrides at all:
```
helmfiles:
- path: nested/state.yaml
environment:
values:
- overrides.yaml
```
This fixes the issue.
```yaml
environments:
default:
missingFileHandler: Warn
values:
- path/to/values.yaml
secrets:
- path/to/secrets.yaml
```
`missingFileHandler` set to `Warn`, `Info`, or `Debug` results in helmfile NOT stop when `path/to/values.yaml` or `path/to/secrets.yaml` is missing.
Resolves#548
While implementing the above feature, I also found a bug that has been causing #559. This also fixes that.
To verify it is actually fixed, create an example helmfile.yaml that looks like the below, and run `helmfile diff`:
```
$ cat helmfile.yaml
environments:
default:
secrets:
- env-secrets.yaml
releases:
- name: myapp
chart: nginx
namespace: default
secrets: [secrets.yaml] # Notice this file does not exist
values:
- ingress:
enabled: true
$ helmfile diff
could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read helmfile.yaml.part.0: environment values file matching "env-secrets.yaml" does not exist
in ./helmfile.yaml: failed to read helmfile.yaml: environment values file matching "env-secrets.yaml" does not exist
```
Fixes#559
- Change exit code from 2 to 3 when helmfiles have no releases that match a selector
- Introduces new flag `--allow-no-matching-release` to exit 0 when no releases match a selector.
Resolves: #597
This enhances helmfile's internal environment values files loader to expand glob patterns (#606)
Fixes the existing bug that helmfile was unable to load environment values file from absolute path (#549)
Resolves#606Fixes#549
`postsync` events are triggered after each release is applied to the cluster in `helmfile sync` or `helmfile apply`.
This should be a best hook to notify only after each sync failed or succeeded. This can be used for running operations like patching K8s resources managed by helm, but that should be the last-resort. Maybe you should fork/update the chart, or submit a feature request to add `replicated/ship` integration to `helmfile` in that case :)
Resolves#599
In order to maintain predictable deployments, as developer I want to generate and use "lock files" for all chart versions retrieved from a helmfile.
This change solves it by (1)enhancing `helmfile deps` to generate a lock file containing all the direct chart dependencies of each helmfile state file and
(2)making other helmfile sub-commands reads the lock file and merge the locked version numbers to the helmfile state file being processed.
The lock file is named after the helmfile state file being locked, so that you can have multiple set of the helmfile state file and the lock file pairs in a directory.
When `helmfile deps` are not explicitly run before commands like `sync`, all the helmfile behavior should remain as before.
Let's say you have `helmfile.1.yaml`:
```
repositories:
- name: stable
url: https://kubernetes-charts.storage.googleapis.com
releases:
- name: envoy
chart: stable/envoy
- name: envoy2
chart: stable/envoy
```
`helmfile deps` generates `helmfile.1.lock` that looks like:
```
dependencies:
- name: envoy
repository: https://kubernetes-charts.storage.googleapis.com
version: 1.5.0
digest: sha256:e43b05c8528ea8ef1560f4980a519719ad2a634658abde0a98daefdb83a104e9
generated: 2019-05-14T16:45:37.78205+09:00
```
Under the hood, `helmfile deps` creates a temporary local helm chart with a dummy `Chart.yaml` and `requirements.yaml` deduced from the `helmfile.yaml` content, then runs `helm dependency update` to produce od update the corresponding `requirements.lock` file.
`helmfile` then renames it to match the name of the targeted helmfile state file and moves it, so that it becomes adjacent to each `helmfile.yaml`.
Other `helmfile` commands like `sync`, `diiff`, `apply`, `lint` read chart version numbers from the lock file.
Resolves#483
feat: helmfile.yaml layering enhancements
The current [Layering](https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md#layering) system didn't work as documented, as it relies on helmfile to template each "part" of your helmfile.yaml THEN merge them one by one.
The reality was that helmfile template all the parts of your helmfile.yaml at once, and then merge those YAML documents. In https://github.com/roboll/helmfile/issues/388#issuecomment-436186278, @sruon was making a GREAT point that we may need to change helmfile to render templates earlier - that is to evaluate a template per each helmfile.yaml part separated by `---`. Sorry I missed my expertise to follow your great idea last year @sruon 😭
Anyways, this, in combination with the wrong documentation, has made so many people confused. To finally overcome this situation, here's a fairly large PR that introduces the 2 enhancements:
- `bases:` for easier layering without go template expressions, especially `{{ readFunc "path/to/file" }}`s. This is the first commit of this PR.
- `helmfile.yaml` is splited by the separator `---` at first. Each part is then rendered as a go template(double-render applies as before). Finally, All the results are merged in the order of occurence. I assume this as an enhanced version of @sruon's work. This is the second commit of this PR.
Resolves#388Resolve#584Resolves#585 (`HELMFILE_EXPERIMENTA=true -f helmfile.yaml helmfile` disables the whole-file templating, treating the helmfile.yaml as a regular YAML file as the file ext. denotes. Use `helmfile.yaml.gotmpl` or `helmfile.gotmpl` to enable)
Fixes#568 (Use `bases` or `readFile` rather than not importing implicitly with `helmfile.d`
* Various U/X improvements for `helmfile apply`
This improves the U/X of `helmfile apply`, by allowing you to selectively apply sub-helmfiles.
When you have two or more sub-helmfiles processed, typing `n` to cancel the first doesn't automatically stop the whole helmfile execution.
Instead, it proceeds by diffing the next sub-helmfile, and asks you to apply it, which should be what the user would expect.
To support this workflow, I have suppressed useless exec logs, correct exit status when diff exists in sub-helmfiles but not in the parent helmfile, and made the final error message emitted by helmfile better.
More concretely, this moves more output from `helm` to STDERR and the `debug` log-level.
The overall output from `helmfile` should be a bit more cleaner especially for `apply`, `sync`, `diff` and perhaps other `helmfile` sub-commands, too.
For example, when one of release failed, `helmfile`'s final error message now includes the error message from the failed `helm` execution, like seen in the last line:
```
List of updated releases :
RELEASE CHART VERSION
envoy stable/envoy 1.5.0
List of releases in error :
RELEASE
envoy2
in ./helmfile.yaml: in .helmfiles[0]: in /Users/c-ykuoka/helmfile/helmfile.1.yaml: failed processing release envoy2: helm exited with status 1:
Error: UPGRADE FAILED: "envoy2" has no deployed releases
```
This way you can better understand what caused helmfile to finally fail.
`helmfile` has been streaminig a lot of stdout and stderr contents from the `helm` commands regardless of the helmfile's log-level. It has been suppressed by default and moved to the `debug` log-level.
You will see that it helps you focus on what was the cause of a failure.
While working on the above, I found an another bug that made `--detailed-exitcode` useless in some case.
That is, `helmfile diff --detailed-exitcode`, when any diff existed only in sub-helmfiles, has been returning an exit code of `1`. It should return `2` when any release had diff and no release had an error, regardless of the target is a sub-helmfile or a parent helmfile. Why? Because that's what `--detailed-exitcode` meant for!
After this PR gets merged, `helmfile diff --detailed-exitcode` propery return exit code `2` in such cases.
Fixes#543Resolves#540
Fixes#344 by allowing explicit selectors to be specified for composed helmfiles using the following structure
```yaml
helmfiles:
- path: helmfile.d/a*.yaml
selectors:
- name=prometheus
- name!=zipkin
- helmfile.d/b*.yaml
- path: helmfile.d/c*.yaml
selectors: {}
```
2 modes here :
* legacy mode when no the env var HELMFILE_EXPERIMENTAL is not set to true
* no selector : inherit from the command line.
* selector: is specified then it is used (an emty means no inheritance from command line and take everything).
* experimental when the env var HELMFILE_EXPERIMENTAL=true
* no selector : nothing is inherited from the command line so use all releases.
* selector: is specified then it is used (an emty means no inheritance from command line and take everything).
* feat(report): display summary of upgraded, deleted and error releases
* feat(#502): adds dep target in makefile
* feat(#502): removes vendor and fixes pristine in makefile
Since tillerless support we unintentionally broke this, and there isn't a real fix to this.
We must accept a limitation that helmfile needs a tiller installed on your cluster just for decrypting environment secrets.
Fixes#550
Since #526, `helmfile apply` have been really able to detect deletion of the last release only, and `sync` has been unable to mark releases with `installed: false` for removal.
Fixes#554
Actually, 4 helm commands including "list", "diff", "status" and "delete" were not taking tiller-rerelated flags into account in helmfile. This fixes all these commands.
Fixes#534
* feat: `helmfile destroy` deletes and purges releases
This adds `helmfile destroy` that is basically `helmfile delete --purge`.
I've also tweaked the behavior of `delete` and `destroy` for releases with `installed: false`, so that it becomes consistent with other helmfile commands.
It now delete releases only when `installed: true` AND the release is already installed.
**Why an another command?**
Because it's easy to remember, and it also makes it easier to iterate on your helmfile.
We've been using `helmfile delete` from the beginning of helmfile,
and several months have been passed since we've added `--purge` to it.
We noticed that we always prefer to use `--purge` so that we can quickly iterate on helmfile by
e.g. `helmfile delete --purge && helmfile sync`. But making `--purge` default makes the `delete` command inconsistent with the helm's `delete`.
`destroy`, on the other hand, doesn't have such problem, and is still easy to remember for terraform users.
Resolves#511
* Update docs about `helmfile delete` and `helmfile destroy`
This basically allows to define both `releases` and `helmfiles` within a helmfile.yaml, so that you can start using sub-helmfiles easily, by extracting only reused releases.
Resolves#445
`get` is now able to take one more optional argument, that is used as the default value when the value for the key does not exist.
Resolves#465Fixes#427Fixes#357
Ref #460
This removes `release: "your_release_name" not found` errors seen for releases with `installed: false` when running `helmfile sync` and `helmfile apply`.
The problem was that helmfile had been running `helm status` to detect releases to be deleted. helmfile now use `helm list ^YOUR_RELEASE_NAME$` to detect if the release is currently installed or not, which emits no error-like logs on against uninstalled releases.
Fixes#507Fixes#507
Options specified in releases (e.g. `recreatePods`) should override the respective options set in `helmDefaults`. Currently, `helmDefaults` takes precedence.
In the below example, `--force` should not be passed as an additional deployment argument:
```
helmDefaults:
force: true
releases:
- name: example
namespace: example
chart: some/repo
version: ~1.24.1
force: false
```
Fixes#492
* Improve code organization
To make sure it is still readable after upcoming changes to helmfile
* feat: `helmfile deps` to update dependencies of all the local charts
Resolves#450
* feat: helmfile updates repos and build deps by default
But not update deps. Use `helmfile deps` to update deps, and provide `--skip-deps` to skip updating repos and builds deps in sync/diff/apply/template
Resolves#415
* Improve integration test coverage
`helmfile test --concurency N` to set a concurrency number.
It is automatically ceiled at the number of releases just to reduce wasting computing resources.
Also, I've refactored the scatter-gather logic scattered across the code-base.
Resolves#433
This feature is supposed to help advanced use-cases like Conventional Directory Structure explained in several issues like #428.
Newly added configuration keys `templates`, `missingFileHandler`, and the ability to defer executing template expressions in `values`, `secrets`, `namespace`, and `chart` of releases allows you to abstract away repetitions into a reusable template:
```yaml
templates:
default: &default
missingFileHandler: Warn
namespace: "{{`{{ .Release.Name }}`}}"
chart: stable/{{`{{ .Release.Name }}`}}
values:
- config/{{`{{ .Release.Name }}`}}/values.yaml
- config/{{`{{ .Release.Name }}`}}/{{`{{ .Environment.Name }}`}}.yaml
secrets:
- config/{{`{{ .Release.Name }}`}}/secrets.yaml
- config/{{`{{ .Release.Name }}`}}/{{`{{ .Environment.Name }}`}}-secrets.yaml
releases:
- name: envoy
<<: *default
```
See the updated documentation for more details.
Resolves#428
fixes https://github.com/roboll/helmfile/issues/412
the `apply` command first runs the `diff` command, which triggers the execution of the `cleanup` hooks at the end of the `diff`.
In go 1.11, the code formatter seems to have changed its formatting rule a bit, that resulted in `make fmt` producing changes introduced in this commit/pr.
The integration test was failing due to that it was still using `--auto-approve` flag which was removed. The documentation was still refering to `--auto-approve` and had no explanation about the new `--interactive` flag. This fixes all these issues.
This a follow up for #368
See https://github.com/roboll/helmfile/pull/374#issuecomment-425291468 for more context.
Perhaps in latest commits I unexpectedly changed it to require double Ctrl-C to actually interrupt. This fixes it so that one Ctrl-C exists helmfile.
On the way to the fix, I changed the exit-code to 128 + SIGNAL(2 for SIGINT, 15 for SIGTERM) according to common *nix commands.
Prevents helmfile from consuming unnecessarily much time in running `helm repo update` over and over.
helmfile now marks which repository was updated, and skip second and further `helm repo update` when all the `repositories` found in a helmfile.yaml was marked as already updated.
Let's say you had two helmfiles, the first one with repositories `foo` and `bar`, an the second one with only `bar`. `helmfile repos` will run `helm update repo` for the first helmfile, marking `foo` and `bar` as already updated. The second helmfile.yaml contains only `bar`, which is marked as already updated. So helmfile won't run `helm repo update` for the second.
This applies to all the helmfile command that results in `helm repo update`, like `repos`, `sync`, `diff` and so on.
Resolves#335
This allows using the environment values defined in the environments: section of helmfile.yaml to be used from other sections of the file.
This works by having two template renderers, the first-pass and the second-pass renderer.
The first-pass render renders a helmfile.yaml template with replacing template functions has side-effects with noop. So, use only funcs that don't have side-effects to compose your environment values.
Then the second-pass renderer renders the same helmfile.yaml template, but with the environment values loaded by the first-pass renderer.
The implementation uses a buffer instead of re-reading the file twice.
Resolves#297
This enables `helmfile lint` and `helmfile template` commands to fetch and untar all the required charts concurrently. The concurrency is configurable via the `--concurrency` flag, that defaults to `0`.
Ref #292
helmfile as of today ensures that all the targeted helmfile.yaml to have the specified environment defined in it.
That is, `helmfile --environment prod -f helmfile.d/ sync` fails if any helmfile under `helmfile.d/` is missing the `production` environment.
This changes the validation logic, so that helmfile fails only when all the helmfiles miss the environment.
Resolves#279
Enhance the `diff` functionality to be able to return affected releases that has any changes,
so that the succeeding `sync` can be run against only the affected releases.
This provides us extra idempotency.
Resolves#277
This removes --concurrenty flag from `helmfile template` and `helmfile lint`. YAGNI.
Also separates out the downloadCharts into a function so it can be used by lint too.
Fixes#288
`helmfile template` runs `helm template` over releases within the helmfiles, and provide you a stream of generated yaml documents of Kubernetes resources via stdout.
Resolves#283
Adds `--suppress-secrets` to `helmfile apply` and `helmfile diff`, so that the diff command omits the contents of secrets from its output. This is a security feature that should always be turned on for CI/CD use-cases.
With `--suppress-secrets`, the output when there is any change looks like:
```
Comparing bar stable/grafana
default, baz-grafana, Secret (v1) has changed:
+ Changes suppressed on sensitive content of type Secret
```
Resolves#269
This command syncs releases only if there is any difference between the desired and the current state. It asks for an confirmation by default. Provide `--auto-approve` flag after the `apply` command to skip it.
Resolves#205
`helmfile lint` works with relative chart reference (#252)
The tempalte function `readFile` accepts the path relative to helmfile.yaml
Resolves#246Fixes#252
Before opening a bug report, please search for the behaviour in the existing issues.
Go ahead to https://github.com/helmfile/helmfile/search?q=&type= and fill in the error message you received from Helmfile or the keywords you think appropriate.
- type:markdown
id:thankyou
attributes:
value:|
Thank you for taking your time to file a bug report! To confirm it's a bug, we need some information to reproduce it.
- type:input
id:os
attributes:
label:Operating system
description:"Which operating system do you use? Please provide the version as well."
placeholder:"ex. Ubuntu 20.04.4 LTS"
validations:
required:true
- type:input
id:helmfile
attributes:
label:Helmfile Version
description:"Please provide the version number of Helmfile you used. If it isn't the latest, please upgrade first."
placeholder:"Run `helmfile version` to print it."
validations:
required:true
- type:input
id:helm
attributes:
label:Helm Version
description:"Please provide the version number of Helm you used. If it isn't the latest, please upgrade first."
placeholder:"Run `helm version` to print it."
validations:
required:true
- type:textarea
id:bug-description
attributes:
label:Bug description
description:What happened?
validations:
required:true
- type:textarea
id:helmfile-yaml
attributes:
label:Example helmfile.yaml
description:"Please provide an example helmfile.yaml that can be used to reproduce the issue locally."
validations:
required:true
- type:textarea
id:helmfile-error
attributes:
label:"Error message you've seen (if any)"
description:"Please provide the error message emitted by Helmfile."
validations:
required:true
- type:input
id:repo
attributes:
label:Steps to reproduce
description:Please provide the URL to a GitHub repository that contains a helmfile.yaml, other companion files, and a README.md with the steps to reproduce the bug.
validations:
required:true
- type:input
id:helmfile_working_ver
attributes:
label:Working Helmfile Version
description:"Please provide the highest version number of Helmfile that doesn't result in the above problem."
placeholder:"Run `helmfile version` to print it."
validations:
required:true
- type:input
id:discussion
attributes:
label:Relevant discussion
description:Please provide the URL to a relevant GitHub Discussion.
Helmfile is a tool for deploying Helm charts that manages Kubernetes deployments as code. It provides templating, environment management, and GitOps workflows for Helm chart deployments.
Helmfile is a declarative tool. In Helmfile, all elements of the desired state for deployments must be included in the `helmfile.yaml` config file and any associated files. Only operational matters can be provided dynamically, via command-line flags and environment variables.
# Standard build (takes 2-3 minutes due to many dependencies)
make build
# Alternative direct build
go build -o helmfile .
# Build with test tools (required for integration tests, ~1 minute)
make build-test-tools # Creates diff-yamls and downloads dyff
# Cross-platform builds
make cross
```
**Build Timing:** First build downloads 200+ Go packages and takes 2-3 minutes. Subsequent builds are faster due to module cache. Test tools build is faster (~1 minute).
### Validation Pipeline
Run in this exact order to match CI requirements:
```bash
# 1. Code formatting and linting
make check # Run go vet (required - always works)
# Note: make fmt requires gci tool (go install github.com/daixiang0/gci@latest)
# 2. Unit tests (fast, ~30 seconds)
go test -v ./pkg/... -race -p=1
# 3. Integration tests (requires Kubernetes - see Environment Setup)
make integration # Takes 5-10 minutes, needs minikube/k8s cluster
# 4. E2E tests (optional, needs expect package)
sudo apt-get install expect # On Ubuntu/Debian
bash test/e2e/helmfile-init/init_linux.sh
```
### Linting Configuration
Uses golangci-lint with configuration in `.golangci.yaml`. Install via:
- **Integration tests require K8s:** Will fail without cluster access
- **Test isolation:** Use `-p=1` flag to avoid race conditions
- **Minikube timing:** May need to wait for cluster ready state
- **Plugin dependencies:** Tests need helm-diff and helm-secrets plugins
### Runtime Requirements
- **Helm dependency:** Always required at runtime, not just build time (available in CI)
- **kubectl access:** Most operations need valid kubeconfig
- **Plugin management:**`helmfile init` must be run after installation
### Common Error Patterns
```bash
# Missing Helm
"helm: command not found" → Install Helm first
# Plugin missing
"Error: plugin 'diff' not found" → Run helmfile init
# K8s access
"connection refused" → Check kubectl cluster-info
# Permission errors
"permission denied" → Check kubeconfig and cluster access
# Missing tools
"gci: No such file or directory" → go install github.com/daixiang0/gci@latest
```
## Working with the Codebase
### Making Changes
- **Follow Helmfile design**: Helmfile is a declarative deployment tool. Anything that is part of the desired state of the deployments needs to be managed by Helmfile configs. Only operational knowledge that affects "how" to apply the desired state needs to be runtime options, like command-like flags and environment variables.
- **Small, focused changes:** Each PR should address single concern
- **Test coverage:** Add unit tests for new pkg/ functionality
- **Integration tests:** Update test-cases/ for new CLI features
- **Documentation:** Update docs/ for user-facing changes
### Key Packages to Understand
- **pkg/app:** Main business logic, start here for feature changes
- **pkg/state:** Helmfile parsing and release orchestration
- **cmd/:** CLI interface changes and new subcommands
- **pkg/helmexec:** Helm integration and command execution
### Architecture Patterns
- **Dependency injection:** App uses interfaces for testability
- **State management:** Immutable state objects, functional transforms
- **Error handling:** Custom error types with exit codes
- **Plugin system:** Extensible via Helm plugins and Go templates
---
**Trust these instructions:** This information is validated against the current codebase. Only search for additional details if these instructions are incomplete or found to be incorrect for your specific task.
[](https://github.com/helmfile/helmfile/pkgs/container/helmfile)
[](https://github.com/helmfile/helmfile/pkgs/container/helmfile)
[](https://quay.io/repository/roboll/helmfile)
</div>
## status
English | [简体中文](./README-zh_CN.md)
Even though Helmfile is used in production environments [across multiple organizations](USERS.md), it is still in its early stage of development, hence versioned 0.x.
Helmfile complies to Semantic Versioning 2.0.0 in which v0.x means that there could be backward-incompatible changes for every release.
Note that we will try our best to document any backward incompatibility.
## about
## About
Helmfile is a declarative spec for deploying helm charts. It lets you...
@ -20,134 +34,61 @@ Helmfile is a declarative spec for deploying helm charts. It lets you...
* Apply CI/CD to configuration changes.
* Periodically sync to avoid skew in environments.
To avoid upgrades for each iteration of `helm`, the `helmfile` executable delegates to `helm` - as a result, `helm` must be installed.
To avoid upgrades for each iteration of `helm`, the `helmfile` executable delegates to `helm` - as a result, the following must be installed
**CAUTION**: This documentation is for the development version of helmfile. If you are looking for the documentation for any of releases, please switch to the corresponding branch like [v0.12.0](https://github.com/roboll/helmfile/tree/v0.12.0).
**Declarative**: Write, version-control, apply the desired state file for visibility and reproducibility.
The default helmfile is `helmfile.yaml`:
**Modules**: Modularize common patterns of your infrastructure, distribute it via Git, S3, etc. to be reused across the entire company (See [#648](https://github.com/roboll/helmfile/pull/648))
```yaml
repositories:
- name: roboll
url: http://roboll.io/charts
certFile: optional_client_cert
keyFile: optional_client_key
username: optional_username
password: optional_password
**Versatility**: Manage your cluster consisting of charts, [kustomizations](https://github.com/kubernetes-sigs/kustomize), and directories of Kubernetes resources, turning everything to Helm releases (See [#673](https://github.com/roboll/helmfile/pull/673))
**Patch**: JSON/Strategic-Merge Patch Kubernetes resources before `helm-install`ing, without forking upstream charts (See [#673](https://github.com/roboll/helmfile/pull/673))
#default values to set for args along with dedicated keys that can be set by contributers, cli args take precedence overe these
helmDefaults:
tillerNamespace: tiller-namespace #dedicated default key for tiller-namespace
kubeContext: kube-context #dedicated default key for kube-context
args:
- "--recreate-pods"
- "--timeout=600"
- "--force"
## Status
releases:
# Published chart example
- name: vault # name of this release
namespace: vault # target namespace
labels: # Arbitrary key value pairs for filtering releases
foo: bar
chart: roboll/vault-secret-manager # the chart being installed to create this release, referenced by `repository/chart` syntax
version: ~1.24.1 # the semver of the chart. range constraint is supported
values:
# value files passed via --values
- vault.yaml
# inline values, passed via a temporary values file and --values
- address: https://vault.example.com
db:
username: {{ requiredEnv "DB_USERNAME" }}
# value taken from environment variable. Quotes are necessary. Will throw an error if the environment variable is not set. $DB_PASSWORD needs to be set in the calling environment ex: export DB_PASSWORD='password1'
password: {{ requiredEnv "DB_PASSWORD" }}
proxy:
# Interpolate environment variable with a fixed string
# single value loaded from a local file, translates to --set-file foo.config=path/to/file
- name: foo.config
file: path/to/file
# set a single array value in an array, translates to --set bar[0]={1,2}
- name: bar[0]
values:
- 1
- 2
# will attempt to decrypt it using helm-secrets plugin
secrets:
- vault_secret.yaml
# wait for k8s resources via --wait. Defaults to `false`
wait: true
May 2025 Update
# Local chart example
- name: grafana # name of this release
namespace: another # target namespace
chart: ../my-charts/grafana # the chart being installed to create this release, referenced by relative path to local chart
values:
- "../../my-values/grafana/values.yaml" # Values file (relative path to manifest)
- ./values/{{ requiredEnv "PLATFORM_ENV" }}/config.yaml # Values file taken from path with environment variable. $PLATFORM_ENV must be set in the calling environment.
wait: true
* Helmfile v1.0 and v1.1 has been released. We recommend upgrading directly to v1.1 if you are still using v0.x.
* If you haven't already upgraded, please go over this v1 proposal [here](docs/proposals/towards-1.0.md) to see a small list of breaking changes.
```
## Installation
## Templating
**1: Binary Installation**
Helmfile uses [Go templates](https://godoc.org/text/template) for templating your helmfile.yaml. While go ships several built-in functions, we have added all of the functions in the [Sprig library](https://godoc.org/github.com/Masterminds/sprig).
download one of [releases](https://github.com/helmfile/helmfile/releases)
We also added one special template function: `requiredEnv`.
The `requiredEnv` function allows you to declare a particular environment variable as required for template rendering.
If the environment variable is unset or empty, the template rendering will fail with an error message.
**2: Package Manager**
## Using environment variables
* Archlinux: install via `pacman -S helmfile`
* openSUSE: install via `zypper in helmfile` assuming you are on Tumbleweed; if you are on Leap you must add the [kubic](https://download.opensuse.org/repositories/devel:/kubic/) repo for your distribution version once before that command, e.g. `zypper ar https://download.opensuse.org/repositories/devel:/kubic/openSUSE_Leap_\$releasever kubic`
* Windows (using [scoop](https://scoop.sh/)): `scoop install helmfile`
Environment variables can be used in most places for templating the helmfile. Currently this is supported for `name`, `namespace`, `value` (in set), `values` and `url` (in repositories).
**3: Container**
Examples:
For more details, see [run as a container](https://helmfile.readthedocs.io/en/latest/#running-as-a-container)
The `helmfile sync` sub-command sync your cluster state as described in your `helmfile`. The default helmfile is `helmfile.yaml`, but any yaml file can be passed by specifying a `--file path/to/your/yaml/file` flag.
Welcome to contribute together to make helmfile better: [contributing doc](https://helmfile.readthedocs.io/en/latest/contributing/)
Under the covers, Helmfile executes `helm upgrade --install` for each `release` declared in the manifest, by optionally decrypting [secrets](#secrets) to be consumed as helm chart values. It also updates specified chart repositories and updates the
dependencies of any referenced local charts.
## Attribution
For Helm 2.9+ you can use a username and password to authenticate to a remote repository.
We use:
### diff
* [semtag](https://github.com/pnikosis/semtag) for automated semver tagging.
I greatly appreciate the author(pnikosis)'s effort on creating it and their
kindness to share it!
The `helmfile diff` sub-command executes the [helm-diff](https://github.com/databus23/helm-diff) plugin across all of
the charts/releases defined in the manifest.
## Users
To supply the diff functionality Helmfile needs the [helm-diff](https://github.com/databus23/helm-diff) plugin v2.9.0+1 or greater installed. For Helm 2.3+
you should be able to simply execute `helm plugin install https://github.com/databus23/helm-diff`. For more details
please look at their [documentation](https://github.com/databus23/helm-diff#helm-diff-plugin).
Helmfile has been used by many users in production:
### delete
* [gitlab.com](https://gitlab.com)
* [reddit.com](https://reddit.com)
* [Jenkins](https://jenkins.io)
* ...
The `helmfile delete` sub-command deletes all the releases defined in the manfiests
For more users, please see: [Users](https://helmfile.readthedocs.io/en/latest/users/)
Note that `delete` doesn't purge releases. So `helmfile delete && helmfile sync` results in sync failed due to that releases names are not deleted but preserved for future references. If you really want to remove releases for reuse, add `--purge` flag to run it like `helmfile delete --purge`.
The `secrets` parameter in a `helmfile.yaml` causes the [helm-secrets](https://github.com/futuresimple/helm-secrets) plugin to be executed to decrypt the file.
## Star History
To supply the secret functionality Helmfile needs the `helm secrets` plugin installed. For Helm 2.3+
you should be able to simply execute `helm plugin install https://github.com/futuresimple/helm-secrets
`.
### test
The `helmfile test` sub-command runs a `helm test` against specified releases in the manifest, default to all
Use `--cleanup` to delete pods upon completion.
### lint
The `helmfile lint` sub-command runs a `helm lint` across all of the charts/releases defined in the manifest. Non local charts will be fetched into a temporary folder which will be deleted once the task is completed.
## Paths Overview
Using manifest files in conjunction with command line argument can be a bit confusing.
A few rules to clear up this ambiguity:
- Absolute paths are always resolved as absolute paths
- Relative paths referenced *in* the helmfile manifest itself are relative to that manifest
- Relative paths referenced on the command line are relative to the current working directory the user is in
For additional context, take a look at [paths examples](PATHS.md)
## Labels Overview
A selector can be used to only target a subset of releases when running helmfile. This is useful for large helmfiles with releases that are logically grouped together.
Labels are simple key value pairs that are an optional field of the release spec. When selecting by label, the search can be inverted. `tier!=backend` would match all releases that do NOT have the `tier: backend` label. `tier=fronted` would only match releases with the `tier: frontend` label.
Multiple labels can be specified using `,` as a separator. A release must match all selectors in order to be selected for the final helm command.
The `selector` parameter can be specified multiple times. Each parameter is resolved independently so a release that matches any parameter will be used.
`--selector tier=frontend --selector tier=backend` will select all the charts
## Using env files
helmfile itself doesn't have an ability to load env files. But you can write some bash script to achieve the goal:
```console
set -a; . .env; set +a; helmfile sync
```
Please see #203 for more context.
## Running helmfile without an Internet connection
Once you download all required charts into your machine, you can run `helmfile charts` to deploy your apps.
It basically run only `helm upgrade --install` with your already-downloaded charts, hence no Internet connection is required.
See #155 for more information on this topic.
## Examples
For more examples, see the [examples/README.md](https://github.com/roboll/helmfile/blob/master/examples/README.md) or the [`helmfile.d`](https://github.com/cloudposse/helmfiles/tree/master/helmfile.d) distribution of helmfiles by [Cloud Posse](https://github.com/cloudposse/).
[](https://star-history.com/#helmfile/helmfile&Date)
This project is maintained by a small team of four and therefore lacks the resource to provide security fixes in a very timely manner.
That said, even though we are very passionate about making Helmfile rock solid security wise, all issues are handled on the best effort basis.
If you have important business(es) that relies on this project, please consider sponsoring the maintainers, so that they can commit more on providing such service.
> *Note* that we don't currently have project-wide sponsorship enabled as we don't know how to share the amount of sponsorships with fairness.
> Please sponsor individuals instead! Thanks for your understanding.
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| 0.144.0 | :white_check_mark: |
| <0.144.0|:x:|
## Reporting a Vulnerability
To report a security issue, please email helmfile-security@googlegroups.com with a description of the issue, the steps you took to create the issue, affected versions, and, if known, mitigations for the issue.
A maintainer will try to respond within 5 working days. If the issue is confirmed as a vulnerability, a Security Advisory will be opened. This project currently tries to follow a 90 day disclosure timeline.
The following table is compiled from public information or from survey responses. If you are using Helmfile in your
company or organization, we would like to invite you to [participate in our user survey](https://docs.google.com/forms/d/e/1FAIpQLSfrF5kR9nXsr5yWqNZGgN-KquM1mBTuYNn5lRmEVXEvhq0VIQ/viewform)
to help us extend this list.
Who's actually using Helmfile? The following table is compiled from public information, survey response and PRs. It's
likely this is only a small selection of actual users, but it's a start.
If you are using Helmfile in your company or organization, we would like to invite you to create a PR to add your
information to this file.
<!-- TABLE_START -->
| Organization | Workload | More Info | Location |
|---|---|---|---|
| [reddit.com](https://www.reddit.com/) | production | [GitHub issue](https://github.com/roboll/helmfile/issues/96#issue-312012249) | San Francisco, CA, US |
| Organization | Workload | More Info | Location | Added |
|---|---|---|---|---|
| [reddit.com](https://www.reddit.com/) | production | [GitHub issue](https://github.com/roboll/helmfile/issues/96#issue-312012249), [Talk](https://www.slideshare.net/GregoryTaylor11/helm-at-reddit-from-local-dev-staging-to-production) | San Francisco, CA | April 2018 |
| [gitlab.com](https://gitlab.com/) | production | [Gitlab Project](https://gitlab.com/gitlab-com/gl-infra/k8s-workloads/gitlab-com/-/tree/master) | * | August 2022 |
| [ricardo.ch](https://www.ricardo.ch/) | production | We're deploying our complete application platform using Helmfile. | Zug, Switzerland | April 2018 |
| [Hellofresh](https://engineering.hellofresh.com/) | production | We've been using helmfile in production since April 2018 for deploying all our infrastructure applications | Berlin, Germany | April 2018 |
| [Cherre](https://cherre.com/) | production | We have no public posts about using Helmfile, but we have been using it for a long time now | New York, NY | October 2018 |
| [Sight Machine](https://sightmachine.com/) | production | We don't have anything publicly posted about it, but have been using it for quite a while in production. | San Francisco, CA and Ann Arbor, MI | December 2018 |
| [TailorMed](https://tailormed.co/) | production | We're deploying all of our platform and all K8s resources using Helmfile. The power of Helmfile are more then amazing. Thank you! | Tel-Aviv, Israel | September 2020 |
| [VSHN – The DevOps Company](https://vshn.ch) | production | | Zurich, Switzerland | March 2019 |
| [Vlocity](https://vlocity.com/) | proof-of-concept | | Melbourne, Australia | March 2019 |
| [transit](https://transit.app/) | production | [Blog post](https://medium.com/@naseem_60378/helmfile-its-like-a-helm-for-your-helm-74a908581599). | Montreal, Canada | March 2019 |
| [uniqkey](https://uniqkey.eu/) | production | [Wiki Page](https://ocd-scm.github.io/ocd-meta/) | Copenhagen, Denmark | April 2019 |
| [bitsofinfo](https://github.com/bitsofinfo/helmfile-deploy) | production | Used with [helmfile-deploy](https://github.com/bitsofinfo/helmfile-deploy) to manage releases for dozens of apps | USA | July 2019 |
| [kloeckner-i](https://www.kloeckner-i.com/) | production | We are deploying our standard tools via helmfile to all our clusters. | Berlin, Germany | September 2019 |
| [American Express](https://www.americanexpress.com) | proof-of-concept | Orchestration of both internal cluster workloads and local developer environments. | London, GB | January 2020 |
| [Sportradar](https://www.sportradar.com) | production | Since mid-2019, we've been deploying our core infrastructure and several application stacks with Helmfile. | St. Gallen, Switzerland | March 2020 |
| [PedidosYa](https://www.pedidosya.com) | production | | Montevideo, Uruguay | June 2020 |
| [Jenkins](https://jenkins.io) | production | [jenkins-infra/kubernetes-management](https://github.com/jenkins-infra/kubernetes-management) | * | July 2020 |
| [SettleMint](https://settlemint.com) | production | The SettleMint platform allows enterprises to spin up k8s clusters and deploy production grade blockchain networks and additional services. Helmfile is in charge of deploying these networks and services on demand out of the self service management ui. | Belgium, Singapore, UAE, India | October 2020 |
| [AutoTrader (UK)](https://www.autotrader.co.uk) | production | We've used Helmfile for 2+ years to deploy 400+ services | UK | October 2020 |
| [Trend Micro](https://www.trendmicro.com) | production | We manage 9 k8s clusters in a mono git repository with Helmfile | Taipei, Taiwan | December 2019 |
| [William Hill](https://www.williamhill.com) | production | We have standardised on Helmfile for managing both our application services and our Kubernetes platform components. | International | March 2021 |
| [The Hyve](https://www.thehyve.nl) | production | We've been using Helmfile since 2019 to deploy [Radar-base applications](https://github.com/RADAR-base/RADAR-Kubernetes). [Blog Post](https://www.thehyve.nl/articles/kubernetes-added-to-radar-base) | Utrecht, Netherlands | April 2021 |
| [IKEA US](https://www.ikea.com) | production | We started using Helmfile this year to shrink the boilerplate involved in deploying or microservices to cloud Kubernetes clusters. | Utrecht, Netherlands | April 2021 |
| [ShareGate](https://www.sharegate.com) | production | Used to deploy and sync our kubernetes workloads in Azure, using Azure Pipelines and custom extensions to [install](https://marketplace.visualstudio.com/items?itemName=GSoft.HelmfileInstaller) and [execute](https://marketplace.visualstudio.com/items?itemName=GSoft.HelmfileRunner) the cli. | Montreal, Canada | June 2021 |
| [subshell](https://subshell.com) | production | We're using helmfile since 2021 to deploy all our Kubernetes workloads into our clusters. We love helmfile for its simplicity and power. Thank you! | Hamburg, Germany | August 2022 |
| [Norddeutscher Rundfunk](https://www.ndr.de) | production | Using Helmfile since 2020 to deploy workloads to several similar clusters (dev, qa, prod, test, etc.) for sites tagesschau.de and sportschau.de. Thank you so much for your awesome work! | Hamburg, Germany | August 2022 |
| [Dealhub](https://dealhub.io/) | production | Helmfile was an essential part of our k8s migration. Keep up the good work! | Holon, Israel | January 2023 |
| [BlueLabs](https://bluelabs.eu/) | production | Helmfile is the cornerstone of our lightweight, auditable and centralized GKE deployments. | Europe | February 2021 |
| [Zhihu](https://www.zhihu.com/) | production | helmfile is an important tool for the deployment of our basic components, which can achieve standardization and auditability. |China, Beijing | December 2023 |
| [Tudock](https://tudock.de) | production | We have no public post about it, but we recently started using Helmfile to deploy developer applications and have been very happy with the results! | Hamburg, Germany | March 2024 |
| [Incentive.me](https://incentive.me/) | production | We use helmfile as the main tool for deploying our Kubernetes workloads. | Rio de Janeiro, Brazil | November 2021 |
| [RightCapital](https://www.rightcapital.com/) | production | We use helmfile as the main tool for deploying our Kubernetes workloads. | Shelton, CT, USA | May 2019 |
<!-- TABLE_END -->
## Contact
Please fill out [the Google form](https://docs.google.com/forms/d/e/1FAIpQLSfrF5kR9nXsr5yWqNZGgN-KquM1mBTuYNn5lRmEVXEvhq0VIQ/viewform) to add your organization instead of opening a PR.
You can also [reach the maintainer on Twitter](https://twitter.com/DerCed).
f.StringArrayVar(&applyOptions.Set,"set",nil,"additional values to be merged into the helm command --set flag")
f.StringArrayVar(&applyOptions.Values,"values",nil,"additional value files to be merged into the helm command --values flag")
f.IntVar(&applyOptions.Concurrency,"concurrency",0,"maximum number of concurrent helm processes to run, 0 is unlimited")
f.BoolVar(&applyOptions.Validate,"validate",false,"validate your manifests against the Kubernetes cluster you are currently pointing at. Note that this requires access to a Kubernetes cluster to obtain information necessary for validating, like the list of available API versions")
f.IntVar(&applyOptions.Context,"context",0,"output NUM lines of context around changes")
f.StringVar(&applyOptions.Output,"output","","output format for diff plugin")
f.BoolVar(&applyOptions.DetailedExitcode,"detailed-exitcode",false,"return a non-zero exit code 2 instead of 0 when there were changes detected AND the changes are synced successfully")
f.BoolVar(&applyOptions.StripTrailingCR,"strip-trailing-cr",false,"strip trailing carriage return on input")
f.StringVar(&applyOptions.DiffArgs,"diff-args","",`pass args to helm helm-diff`)
f.StringVar(&applyOptions.SyncArgs,"sync-args","",`pass args to helm upgrade`)
f.StringVar(&globalCfg.GlobalOptions.Args,"args","","pass args to helm exec")
f.BoolVar(&applyOptions.SkipCleanup,"skip-cleanup",false,"Stop cleaning up temporary values generated by helmfile and helm-secrets. Useful for debugging. Don't use in production for security")
f.BoolVar(&applyOptions.SkipCRDs,"skip-crds",false,"if set, no CRDs will be installed on sync. By default, CRDs are installed if not already present")
f.BoolVar(&applyOptions.SkipNeeds,"skip-needs",true,`do not automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided. Defaults to true when --include-needs or --include-transitive-needs is not provided`)
f.BoolVar(&applyOptions.IncludeNeeds,"include-needs",false,`automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided`)
f.BoolVar(&applyOptions.IncludeTransitiveNeeds,"include-transitive-needs",false,`like --include-needs, but also includes transitive needs (needs of needs). Does nothing when --selector/-l flag is not provided. Overrides exclusions of other selectors and conditions.`)
f.BoolVar(&applyOptions.SkipDiffOnInstall,"skip-diff-on-install",false,"Skips running helm-diff on releases being newly installed on this apply. Useful when the release manifests are too huge to be reviewed, or it's too time-consuming to diff at all")
f.BoolVar(&applyOptions.IncludeTests,"include-tests",false,"enable the diffing of the helm test hooks")
f.StringArrayVar(&applyOptions.Suppress,"suppress",nil,"suppress specified Kubernetes objects in the diff output. Can be provided multiple times. For example: --suppress KeycloakClient --suppress VaultSecret")
f.BoolVar(&applyOptions.SuppressSecrets,"suppress-secrets",false,"suppress secrets in the diff output. highly recommended to specify on CI/CD use-cases")
f.BoolVar(&applyOptions.ShowSecrets,"show-secrets",false,"do not redact secret values in the diff output. should be used for debug purpose only")
f.BoolVar(&applyOptions.NoHooks,"no-hooks",false,"do not diff changes made by hooks.")
f.BoolVar(&applyOptions.HideNotes,"hide-notes",false,"add --hide-notes flag to helm")
f.BoolVar(&applyOptions.TakeOwnership,"take-ownership",false,"add --take-ownership flag to helm")
f.BoolVar(&applyOptions.SyncReleaseLabels,"sync-release-labels",false,"sync release labels to the target release")
f.BoolVar(&applyOptions.SuppressDiff,"suppress-diff",false,"suppress diff in the output. Usable in new installs")
f.StringVar(&applyOptions.PostRenderer,"post-renderer","",`pass --post-renderer to "helm template" or "helm upgrade --install"`)
f.StringArrayVar(&applyOptions.PostRendererArgs,"post-renderer-args",nil,`pass --post-renderer-args to "helm template" or "helm upgrade --install"`)
f.BoolVar(&applyOptions.SkipSchemaValidation,"skip-schema-validation",false,`pass --skip-schema-validation to "helm template" or "helm upgrade --install"`)
f.StringVar(&applyOptions.Cascade,"cascade","","pass cascade to helm exec, default: background")
f.StringArrayVar(&applyOptions.SuppressOutputLineRegex,"suppress-output-line-regex",nil,"a list of regex patterns to suppress output lines from the diff output")
f.StringVar(&diffOptions.DiffArgs,"diff-args","",`pass args to helm helm-diff`)
f.StringVar(&globalCfg.GlobalOptions.Args,"args","","pass args to helm diff")
f.StringArrayVar(&diffOptions.Set,"set",nil,"additional values to be merged into the helm command --set flag")
f.StringArrayVar(&diffOptions.Values,"values",nil,"additional value files to be merged into the helm command --values flag")
f.IntVar(&diffOptions.Concurrency,"concurrency",0,"maximum number of concurrent helm processes to run, 0 is unlimited")
f.BoolVar(&diffOptions.Validate,"validate",false,"validate your manifests against the Kubernetes cluster you are currently pointing at. Note that this requires access to a Kubernetes cluster to obtain information necessary for validating, like the diff of available API versions")
f.BoolVar(&diffOptions.SkipNeeds,"skip-needs",true,`do not automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided. Defaults to true when --include-needs or --include-transitive-needs is not provided`)
f.BoolVar(&diffOptions.IncludeTests,"include-tests",false,"enable the diffing of the helm test hooks")
f.BoolVar(&diffOptions.IncludeNeeds,"include-needs",false,`automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided`)
f.BoolVar(&diffOptions.IncludeTransitiveNeeds,"include-transitive-needs",false,`like --include-needs, but also includes transitive needs (needs of needs). Does nothing when --selector/-l flag is not provided. Overrides exclusions of other selectors and conditions.`)
f.BoolVar(&diffOptions.SkipDiffOnInstall,"skip-diff-on-install",false,"Skips running helm-diff on releases being newly installed on this apply. Useful when the release manifests are too huge to be reviewed, or it's too time-consuming to diff at all")
f.BoolVar(&diffOptions.ShowSecrets,"show-secrets",false,"do not redact secret values in the output. should be used for debug purpose only")
f.BoolVar(&diffOptions.NoHooks,"no-hooks",false,"do not diff changes made by hooks.")
f.BoolVar(&diffOptions.DetailedExitcode,"detailed-exitcode",false,"return a detailed exit code")
f.BoolVar(&diffOptions.StripTrailingCR,"strip-trailing-cr",false,"strip trailing carriage return on input")
f.IntVar(&diffOptions.Context,"context",0,"output NUM lines of context around changes")
f.StringVar(&diffOptions.Output,"output","","output format for diff plugin")
f.BoolVar(&diffOptions.SuppressSecrets,"suppress-secrets",false,"suppress secrets in the output. highly recommended to specify on CI/CD use-cases")
f.StringArrayVar(&diffOptions.Suppress,"suppress",nil,"suppress specified Kubernetes objects in the output. Can be provided multiple times. For example: --suppress KeycloakClient --suppress VaultSecret")
f.BoolVar(&diffOptions.TakeOwnership,"take-ownership",false,"add --take-ownership flag to helm")
f.StringVar(&diffOptions.PostRenderer,"post-renderer","",`pass --post-renderer to "helm template" or "helm upgrade --install"`)
f.StringArrayVar(&diffOptions.PostRendererArgs,"post-renderer-args",nil,`pass --post-renderer-args to "helm template" or "helm upgrade --install"`)
f.StringArrayVar(&diffOptions.SuppressOutputLineRegex,"suppress-output-line-regex",nil,"a list of regex patterns to suppress output lines from the diff output")
f.IntVar(&fetchOptions.Concurrency,"concurrency",0,"maximum number of concurrent helm processes to run, 0 is unlimited")
f.StringVar(&fetchOptions.OutputDir,"output-dir","","directory to store charts (default: temporary directory which is deleted when the command terminates)")
f.StringVar(&fetchOptions.OutputDirTemplate,"output-dir-template",state.DefaultFetchOutputDirTemplate,"go text template for generating the output directory")
f.IntVar(&lintOptions.Concurrency,"concurrency",0,"maximum number of concurrent helm processes to run, 0 is unlimited")
f.StringVar(&globalCfg.GlobalOptions.Args,"args","","pass args to helm exec")
f.StringArrayVar(&lintOptions.Set,"set",nil,"additional values to be merged into the helm command --set flag")
f.StringArrayVar(&lintOptions.Values,"values",nil,"additional value files to be merged into the helm command --values flag")
f.BoolVar(&lintOptions.SkipNeeds,"skip-needs",true,`do not automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided. Defaults to true when --include-needs or --include-transitive-needs is not provided`)
f.BoolVar(&lintOptions.IncludeNeeds,"include-needs",false,`automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided`)
f.BoolVar(&lintOptions.IncludeTransitiveNeeds,"include-transitive-needs",false,`like --include-needs, but also includes transitive needs (needs of needs). Does nothing when --selector/-l flag is not provided. Overrides exclusions of other selectors and conditions.`)
fs.StringVarP(&globalOptions.HelmBinary,"helm-binary","b",app.DefaultHelmBinary,"Path to the helm binary")
fs.StringVarP(&globalOptions.KustomizeBinary,"kustomize-binary","k",app.DefaultKustomizeBinary,"Path to the kustomize binary")
fs.StringVarP(&globalOptions.File,"file","f","","load config from file or directory. defaults to \"`helmfile.yaml`\" or \"helmfile.yaml.gotmpl\" or \"helmfile.d\" (means \"helmfile.d/*.yaml\" or \"helmfile.d/*.yaml.gotmpl\") in this preference. Specify - to load the config from the standard input.")
fs.StringVarP(&globalOptions.Environment,"environment","e","",`specify the environment name. Overrides "HELMFILE_ENVIRONMENT" OS environment variable when specified. defaults to "default"`)
fs.StringArrayVar(&globalOptions.StateValuesSet,"state-values-set",nil,"set state values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2). Used to override .Values within the helmfile template (not values template).")
fs.StringArrayVar(&globalOptions.StateValuesSetString,"state-values-set-string",nil,"set state STRING values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2). Used to override .Values within the helmfile template (not values template).")
fs.StringArrayVar(&globalOptions.StateValuesFile,"state-values-file",nil,"specify state values in a YAML file. Used to override .Values within the helmfile template (not values template).")
fs.BoolVar(&globalOptions.SkipDeps,"skip-deps",false,`skip running "helm repo update" and "helm dependency build"`)
fs.BoolVar(&globalOptions.StripArgsValuesOnExitError,"strip-args-values-on-exit-error",true,`Strip the potential secret values of the helm command args contained in a helmfile error message`)
fs.BoolVar(&globalOptions.DisableForceUpdate,"disable-force-update",false,`do not force helm repos to update when executing "helm repo add"`)
fs.BoolVarP(&globalOptions.Quiet,"quiet","q",false,"Silence output. Equivalent to log-level warn")
fs.StringVar(&globalOptions.Kubeconfig,"kubeconfig","","Use a particular kubeconfig file")
fs.StringVar(&globalOptions.KubeContext,"kube-context","","Set kubectl context. Uses current context by default")
fs.BoolVar(&globalOptions.Debug,"debug",false,"Enable verbose output for Helm and set log-level to debug, this disables --quiet/-q effect")
fs.BoolVar(&globalOptions.Color,"color",false,"Output with color")
fs.BoolVar(&globalOptions.NoColor,"no-color",false,"Output without color")
fs.StringVarP(&globalOptions.Namespace,"namespace","n","","Set namespace. Uses the namespace set in the context by default, and is available in templates as {{ .Namespace }}")
fs.StringVarP(&globalOptions.Chart,"chart","c","","Set chart. Uses the chart set in release by default, and is available in template as {{ .Chart }}")
fs.BoolVar(&globalOptions.AllowNoMatchingRelease,"allow-no-matching-release",false,`Do not exit with an error code if the provided selector has no matching releases.`)
Short:"It prints a table with 3 columns, GROUP, RELEASE, and DEPENDENCIES. GROUP is the unsigned, monotonically increasing integer starting from 1. All the releases with the same GROUP are deployed concurrently. Everything in GROUP 2 starts being deployed only after everything in GROUP 1 got successfully deployed. RELEASE is the release that belongs to the GROUP. DEPENDENCIES is the list of releases that the RELEASE depends on. It should always be empty for releases in GROUP 1. DEPENDENCIES for a release in GROUP 2 should have some or all dependencies appeared in GROUP 1. It can be \"some\" because Helmfile simplifies the DAGs of releases into a DAG of groups, so that Helmfile always produce a single DAG for everything written in helmfile.yaml, even when there are technically two or more independent DAGs of releases in it.",
f.StringVar(&globalCfg.GlobalOptions.Args,"args","","pass args to helm sync")
f.StringVar(&syncOptions.SyncArgs,"sync-args","","pass args to helm upgrade")
f.StringArrayVar(&syncOptions.Set,"set",nil,"additional values to be merged into the helm command --set flag")
f.StringArrayVar(&syncOptions.Values,"values",nil,"additional value files to be merged into the helm command --values flag")
f.IntVar(&syncOptions.Concurrency,"concurrency",0,"maximum number of concurrent helm processes to run, 0 is unlimited")
f.BoolVar(&syncOptions.Validate,"validate",false,"validate your manifests against the Kubernetes cluster you are currently pointing at. Note that this requires access to a Kubernetes cluster to obtain information necessary for validating, like the sync of available API versions")
f.BoolVar(&syncOptions.SkipNeeds,"skip-needs",true,`do not automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided. Defaults to true when --include-needs or --include-transitive-needs is not provided`)
f.BoolVar(&syncOptions.SkipCRDs,"skip-crds",false,"if set, no CRDs will be installed on sync. By default, CRDs are installed if not already present")
f.BoolVar(&syncOptions.IncludeNeeds,"include-needs",false,`automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided`)
f.BoolVar(&syncOptions.IncludeTransitiveNeeds,"include-transitive-needs",false,`like --include-needs, but also includes transitive needs (needs of needs). Does nothing when --selector/-l flag is not provided. Overrides exclusions of other selectors and conditions.`)
f.BoolVar(&syncOptions.HideNotes,"hide-notes",false,"add --hide-notes flag to helm")
f.BoolVar(&syncOptions.TakeOwnership,"take-ownership",false,`add --take-ownership flag to helm`)
f.BoolVar(&syncOptions.SyncReleaseLabels,"sync-release-labels",false,"sync release labels to the target release")
f.IntVar(&syncOptions.Timeout,"timeout",0,`Override helmDefaults.timeout setting "helm upgrade --install --timeout" (default 0, which means no timeout)`)
f.StringVar(&syncOptions.PostRenderer,"post-renderer","",`pass --post-renderer to "helm template" or "helm upgrade --install"`)
f.StringArrayVar(&syncOptions.PostRendererArgs,"post-renderer-args",nil,`pass --post-renderer-args to "helm template" or "helm upgrade --install"`)
f.BoolVar(&syncOptions.SkipSchemaValidation,"skip-schema-validation",false,`pass --skip-schema-validation to "helm template" or "helm upgrade --install"`)
f.StringVar(&syncOptions.Cascade,"cascade","","pass cascade to helm exec, default: background")
f.StringVar(&globalCfg.GlobalOptions.Args,"args","","pass args to helm template")
f.StringArrayVar(&templateOptions.Set,"set",nil,"additional values to be merged into the helm command --set flag")
f.StringArrayVar(&templateOptions.Values,"values",nil,"additional value files to be merged into the helm command --values flag")
f.StringVar(&templateOptions.OutputDir,"output-dir","","output directory to pass to helm template (helm template --output-dir)")
f.StringVar(&templateOptions.OutputDirTemplate,"output-dir-template","","go text template for generating the output directory. Default: {{ .OutputDir }}/{{ .State.BaseName }}-{{ .State.AbsPathSHA1 }}-{{ .Release.Name}}")
f.IntVar(&templateOptions.Concurrency,"concurrency",0,"maximum number of concurrent helm processes to run, 0 is unlimited")
f.BoolVar(&templateOptions.Validate,"validate",false,"validate your manifests against the Kubernetes cluster you are currently pointing at. Note that this requires access to a Kubernetes cluster to obtain information necessary for validating, like the template of available API versions")
f.BoolVar(&templateOptions.IncludeCRDs,"include-crds",false,"include CRDs in the templated output")
f.BoolVar(&templateOptions.SkipTests,"skip-tests",false,"skip tests from templated output")
f.BoolVar(&templateOptions.SkipNeeds,"skip-needs",true,`do not automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided. Defaults to true when --include-needs or --include-transitive-needs is not provided`)
f.BoolVar(&templateOptions.IncludeNeeds,"include-needs",false,`automatically include releases from the target release's "needs" when --selector/-l flag is provided. Does nothing when --selector/-l flag is not provided`)
f.BoolVar(&templateOptions.IncludeTransitiveNeeds,"include-transitive-needs",false,`like --include-needs, but also includes transitive needs (needs of needs). Does nothing when --selector/-l flag is not provided. Overrides exclusions of other selectors and conditions.`)
f.BoolVar(&templateOptions.SkipCleanup,"skip-cleanup",false,"Stop cleaning up temporary values generated by helmfile and helm-secrets. Useful for debugging. Don't use in production for security")
f.BoolVar(&templateOptions.NoHooks,"no-hooks",false,"do not template files made by hooks.")
f.StringVar(&templateOptions.PostRenderer,"post-renderer","",`pass --post-renderer to "helm template" or "helm upgrade --install"`)
f.StringArrayVar(&templateOptions.PostRendererArgs,"post-renderer-args",nil,`pass --post-renderer-args to "helm template" or "helm upgrade --install"`)
f.BoolVar(&templateOptions.SkipSchemaValidation,"skip-schema-validation",false,`pass skip-schema-validation to "helm template" or "helm upgrade --install"`)
f.StringVar(&templateOptions.KubeVersion,"kube-version","",`pass --kube-version to "helm template". Overrides kubeVersion in helmfile.yaml`)
f.StringArrayVar(&templateOptions.ShowOnly,"show-only",nil,`pass --show-only to "helm template"`)
f.IntVar(&writeValuesOptions.Concurrency,"concurrency",0,"maximum number of concurrent helm processes to run, 0 is unlimited")
f.StringArrayVar(&writeValuesOptions.Set,"set",nil,"additional values to be merged into the helm command --set flag")
f.StringArrayVar(&writeValuesOptions.Values,"values",nil,"additional value files to be merged into the helm command --values flag")
f.StringVar(&writeValuesOptions.OutputFileTemplate,"output-file-template","","go text template for generating the output file. Default: {{ .State.BaseName }}-{{ .State.AbsPathSHA1 }}/{{ .Release.Name}}.yaml")
At this point, Helmfile can generate a complete kustomization from the base kustomization you specified in `releases[].chart` of your helmfile.yaml and `values.yaml`,
which can be included in the temporary chart.
After all, Helmfile just installs the temporary chart like standard charts, which allows you to manage everything with Helmfile regardless of each app is declared using a Helm chart or a kustomization.
Please also see [test/advanced/helmfile.yaml](https://github.com/helmfile/helmfile/tree/master/test/advanced/helmfile.yaml) for an example of kustomization support and more.
### Adhoc Kustomization of Helm charts
With Helmfile's integration with Kustomize, not only deploying Kustomization as a Helm chart, you can kustomize charts before installation.
.. Hint:: The following fields can also specify files, in the same manner as
the `values` field.
Currently, Helmfile allows you to set the following fields for kustomizing the chart:
You can add/update any Kubernetes resource field rendered from a Helm chart by specifying `releases[].strategicMergePatches`:
```
repositories:
- name: incubator
url: https://charts.helm.sh/incubator
releases:
- name: raw1
chart: incubator/raw
values:
- resources:
- apiVersion: v1
kind: ConfigMap
metadata:
name: raw1
namespace: default
data:
foo: FOO
strategicMergePatches:
- apiVersion: v1
kind: ConfigMap
metadata:
name: raw1
namespace: default
data:
bar: BAR
```
Running `helmfile template` on the above example results in a ConfigMap called `raw` whose `data` is:
```yaml
foo: FOO
bar: BAR
```
Please note that the second `data` field `bar` is coming from the strategic-merge patch defined in the above helmfile.yaml.
There's also `releases[].jsonPatches` that works similarly to `strategicMergePatches` but has additional capability to remove fields.
Please also see [test/advanced/helmfile.yaml](https://github.com/helmfile/helmfile/tree/master/test/advanced/helmfile.yaml) for an example of patching support and more.
#### `transformers`
You can set `transformers` to apply [Kustomize's transformers](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/configureBuiltinPlugin.md#configuring-the-builtin-plugins-instead).
Each item can be a path to a YAML or Go template file, or an embedded transformer declaration as a YAML hash.
It's often used to add common labels and annotations to your resources.
In the below example, we add common annotations and labels every resource rendered from the `aws-load-balancer-controller` chart:
```yaml
releases:
- name: "aws-load-balancer-controller"
namespace: "kube-system"
forceNamespace: "kube-system"
chart: "center/aws/aws-load-balancer-controller"
transformers:
- apiVersion: builtin
kind: AnnotationsTransformer
metadata:
name: notImportantHere
annotations:
area: 51
greeting: take me to your leader
fieldSpecs:
- path: metadata/annotations
create: true
- apiVersion: builtin
kind: LabelTransformer
metadata:
name: notImportantHere
labels:
foo: bar
fieldSpecs:
- path: metadata/labels
create: true
```
As explained earlier, `transformers` can be not only a list of embedded transformers, but also YAML or Go template files, or a mix of those three kinds.
```yaml
transformers:
# Embedded transformer
- apiVersion: builtin
kind: AnnotationsTransformer
metadata:
name: notImportantHere
annotations:
area: 51
greeting: take me to your leader
fieldSpecs:
- path: metadata/annotations
create: true
# YAML file
- path/to/transformer.yaml
# Go template
# The same set of template parameters as release values files templates is available.
- path/to/transformer.yaml.gotmpl
```
Please see https://github.com/kubernetes-sigs/kustomize/blob/master/examples/configureBuiltinPlugin.md#configuring-the-builtin-plugins-instead for more information on how to declare transformers.
### Adding dependencies without forking the chart
With Helmfile, you can add chart dependencies to a Helm chart without forking it.
An example `helmfile.yaml` that adds a `stable/envoy` dependency to the release `foo` looks like the below:
```yaml
repositories:
- name: stable
url: https://charts.helm.sh/stable
releases:
- name: foo
chart: ./path/to/foo
dependencies:
- chart: stable/envoy
version: 1.5
```
When Helmfile encounters `releases[].dependencies`, it creates another temporary chart from `./path/to/foo` and adds the following `dependencies` to the `Chart.yaml`, so that you don't need to fork the chart.
```yaml
dependencies:
- name: envoy
repo: https://charts.helm.sh/stable
condition: envoy.enabled
```
A Helm chart can have two or more dependencies for the same chart with different `alias`es. To give your dependency an `alias`, define it like you would do in a standard `Chart.yaml`:
```yaml
repositories:
- name: stable
url: https://charts.helm.sh/stable
releases:
- name: foo
chart: ./path/to/foo
dependencies:
- chart: stable/envoy
version: 1.5
alias: bar
- chart: stable/envoy
version: 1.5
alias: baz
```
which will tweaks the temporary chart's `Chart.yaml` to have:
```yaml
dependencies:
- alias: bar
name: envoy
repo: https://charts.helm.sh/stable
condition: bar.enabled
- alias: baz
name: envoy
repo: https://charts.helm.sh/stable
condition: baz.enabled
```
Please see #649 for more context around this feature.
After the support for adhoc dependency to local chart (#1765),
you can even write local file paths relative to `helmfile.yaml` in `chart`:
```yaml
releases:
- name: foo
chart: ./path/to/foo
dependencies:
- chart: ./path/to/bar
```
Internally, Helmfile creates another temporary chart from the local chart `./path/to/foo`, and modifies the chart's `Chart.yaml` dependencies to look like:
```yaml
dependencies:
- alias: bar
name: bar
repo: file:///abs/path/to/bar
condition: bar.enabled
```
Please read https://github.com/roboll/helmfile/issues/1762#issuecomment-816341251 for more details.
#### OCI chart dependencies
With Helmfile version v0.146.0 or later, you can add OCI chart to chart dependencies.
An example `helmfile.yaml` that adds a OCI chart dependency to the release `foo` looks like the below:
```yaml
releases:
- name: foo
chart: ./path/to/foo
dependencies:
- chart: oci://my-oci-registry/helm-repo/envoy
version: 1.5
```
### Lockfile per environment
In some cases it can be handy for CI/CD pipelines to be able to roll out updates gradually for environments, such as staging and production while using the same
set of charts. This can be achieved by using `lockFilePath` in combination with environments, such as:
The following functions are all from the [go-cty](https://pkg.go.dev/github.com/zclconf/go-cty/cty/function/stdlib#pkg-functions), [go-cty-yaml](https://pkg.go.dev/github.com/zclconf/go-cty-yaml) and [hcl](https://pkg.go.dev/github.com/hashicorp/hcl/v2@v2.20.1/ext/tryfunc#section-readme) libraries
#### abs
`abs` returns the absolute value
```
abs(number)
```
```
abs(-1)
# 1
abs(2)
# 2
```
#### can
`can` evaluates an expression and returns a boolean if a result can be produced without any error
```
can(expr)
```
```
map = {
myvar = "myvar"
}
can1 = can(hv.map.myVar)
# true
can2 = can(hv.map.notMyVar)
# false
```
#### ceil
`ceil` returns the ceiling value of a given number
```
ceil(number)
```
```
ceil(1)
# 1
ceil(1.1)
# 2
```
#### chomp
`chomp` removes newline characters at the end of a string.
```
chomp(string)
```
```
chomp("myVar\n")
# myVar
```
#### coalesce
`coalesce` returns the first of the given arguments that is not null. If all arguments are null, an error is produced.
All arguments must be of the same type apart from some cases
```
coalesce(any...)
```
```
coalesce(null, 2)
# 2
coalesce(null, "value")
# value
```
Use the three dots notation `...` to expand a list
```
coalesce([null, "value"]...)
# value
```
#### coalescelist
`coalescelist` takes any number of list arguments and returns the first one that isn't empty.
```
coalescelist(list)
```
```
coalescelist([], ["value"])
# ["value"]
```
Use the three dots notation `...` when using list of lists
```
coalescelist([[], ["val1", "val2"]]...)
# ["val1", "val2"]
```
#### compact
`compact` returns a new list with any empty string elements removed.
```
compact(list)
```
```
compact(["", "val1", "val2"])
# ["val1", "val2"]
```
#### concat
`concat` takes one or more sequences (lists or tuples) and returns the single sequence that results from concatenating them together in order.
```
concat(list, list...)
```
```
concat(["val1"], ["val2", "val3"])
# ["val1", "val2", "val3"]
```
#### contains
`contains` returns a boolean if a list contains a given value
```
contains(list, value)
```
```
contains(["val1", "val2"], "val2")
# true
```
#### csvdecode
`csvdecode` decodes a CSV-formatted string into a list of maps
```
csvdecode(string)
```
```
csvdecode("col1,col2\nv1,v2\nv3,v4")
###
[
{
"col1" = "v1"
"col2" = "v2"
},
{
"col1" = "v3"
"col2" = "v4"
}
]
```
#### distinct
`distinct` returns a new list from another by removing all duplicates
```
distinct(list)
```
```
distinct(["v1","v1","v2"])
["v1", "v2"]
```
#### element
`element` returns a single element from a given list at the given index. If index is greater than the length of the list then it is wrapped modulo the list length
```
element(list, index)
```
```
element(["val1","val2"], 1)
# val2
```
#### chunklist
`chunklist` splits a single list into fixed-size chunks, returning a list of lists.
```
chunklist(list, size)
```
```
chunklist(["a","b"], 1)
# [["a"], ["b"]]
```
#### flatten
`flatten` takes a list and replaces any elements that are lists with a flattened sequence of the list contents.
```
flatten(list)
```
```
flatten([["a"], ["a","b"], ["c"]])
# ["a","a","b","c"]
```
#### floor
`floor` returns the closest whole number lesser than or equal to the given value.
```
floor(number)
```
```
floor(1)
# 1
floor(0.7)
# 0
```
#### format
`format` produces a string representation of zero or more values using a format string similar to the "printf" function in C.
`formatlist` does the same as `format` but for a list of strings
```
formatlist(formatString, values...)
```
```
formatlist("%s", ["Hello", "World"])
###
[
"Hello",
"World"
]
formatlist("%s %s", "hello", ["World", "You"])
###
[
"hello World",
"hello You",
]
```
#### indent
`indent` adds a given number of spaces to the beginnings of all but the first line in a given multi-line string.
```
indent(number, string)
```
```
indent(4, "hello,\nWorld\n!")
###
hello
World
!
```
#### int
`int` removes the fractional component of the given number returning an integer representing the whole number component, rounding towards zero.
```
int(number)
```
```
int(6.2)
# 6
```
#### join
`join` concatenates together the string elements of one or more lists with a given separator.
```
join(listOfStrings, separator)
```
```
join(" ", ["hello", "world"])
# hello world
```
#### jsondecode
`jsondecode` parses the given JSON string and, if it is valid, returns the value it represents.
```
jsonencode(string)
```
Example :
```
jsonencode({"hello"="world"})
# {"hello": "world"}
```
#### jsonencode
`jsonencode` returns a JSON serialization of the given value.
```
jsondecode(string)
```
Example :
```
jsondecode("{\"hello\": \"world\"}")
# { hello = "world" }
```
#### keys
`keys` takes a map and returns a sorted list of the map keys.
```
keys(map)
```
```
keys({val1=1, val2=2, val3=3})
# ["val1","val2","val3"]
```
#### length
`length` returns the number of elements in the given __collection__.
See `strlen` for strings
```
length(list)
```
```
length([1,2,3])
# 3
```
#### log
`log` returns returns the logarithm of a given number in a given base.
```
log(number, base)
```
```
log(1, 10)
# 0
```
#### lookup
`lookup` performs a dynamic lookup into a map. There are three required arguments, inputMap and key, plus a defaultValue, which is a value to return if the given key is not found in the inputMap.
```
lookup(inputMap, key, defaultValue)
```
```
map = { "luke" = "skywalker"}
lookup(hv.maptest, "luke", "none")
# skywalker
lookup(hv.maptest, "leia", "none")
# none
```
#### lower
`lower` is a Function that converts a given string to lowercase.
```
lower(string)
```
```
lower("HELLO world")
# hello world
```
#### max
`max` returns the maximum number from the given numbers.
```
max(numbers)
```
```
max(1,128,70)
# 128
```
#### merge
`merge` takes an arbitrary number of maps and returns a single map that contains a merged set of elements from all of the maps.
```
merge(maps)
```
```
merge({a="1"}, {a=[1,2], c="world"}, {d=40})
# { a = [1,2], c = "world", d = 40}
```
#### min
`min` returns the minimum number from the given numbers.
```
min(numbers)
```
```
min(1,128,70)
# 1
```
#### parseint
`parseint` parses a string argument and returns an integer of the specified base.
```
parseint(string, base)
```
```
parseint("190", 10)
# 190
parseint("11001", 2)
# 25
```
#### pow
`pow` returns the logarithm of a given number in a given base.
```
pow(number, power)
```
```
pow(1, 10)
# 1
pow(3, 12)
# 531441
```
#### range
`range` creates a list of numbers by starting from the given starting value, then adding the given step value until the result is greater than or equal to the given stopping value. Each intermediate result becomes an element in the resulting list.
```
range(startingNumber, stoppingNumber, stepNumber)
```
```
range(1, 10, 3)
# [1, 4, 7]
```
#### regex
`regex` is a function that extracts one or more substrings from a given string by applying a regular expression pattern, describing the first match.
The return type depends on the composition of the capture groups (if any) in the pattern:
If there are no capture groups at all, the result is a single string representing the entire matched pattern.
If all of the capture groups are named, the result is an object whose keys are the named groups and whose values are their sub-matches, or null if a particular sub-group was inside another group that didn't match.
If none of the capture groups are named, the result is a tuple whose elements are the sub-groups in order and whose values are their sub-matches, or null if a particular sub-group was inside another group that didn't match.
It is invalid to use both named and un-named capture groups together in the same pattern.
If the pattern doesn't match, this function returns an error. To test for a match, call `regexall` and check if the length of the result is greater than zero.
```
regex(pattern, string)
```
```
regex("[0-9]+", "v1.2.3")
# 1
```
#### regexall
`regexall` is similar to Regex but it finds all of the non-overlapping matches in the given string and returns a list of them.
The result type is always a list, whose element type is deduced from the pattern in the same way as the return type for Regex is decided.
If the pattern doesn't match at all, this function returns an empty list.
```
regexall(pattern, string)
```
```
regexall("[0-9]+", "v1.2.3")
# [1 2 3]
```
#### setintersection
`setintersection` returns a new set containing the elements that exist in all of the given sets, which must have element types that can all be converted to some common type using the standard type unification rules. If conversion is not possible, an error is returned.
`setsubtract` returns a new set containing the elements from the first set that are not present in the second set. The sets must have element types that can both be converted to some common type using the standard type unification rules. If conversion is not possible, an error is returned.
```
setsubtract(sets...)
```
```
setsubtract(["a", "b", "c"], ["a", "b"])
###
["c"]
```
#### setunion
`setunion` returns a new set containing all of the elements from the given sets, which must have element types that can all be converted to some common type using the standard type unification rules. If conversion is not possible, an error is returned.
```
setunion(sets...)
```
```
setunion(["a", "b"], ["b", "c"], ["a", "d"])
###
["a", "b", "c", "d"]
```
#### signum
`signum` determines the sign of a number, returning a number between -1 and 1 to represent the sign.
```
signum(number)
```
```
signum(-182)
# -1
```
#### slice
`slice` extracts some consecutive elements from within a list.
`sort` re-orders the elements of a given list of strings so that they are in ascending lexicographical order.
```
sort(list)
```
```
sort(["1", "h", "r", "p", "word"])
# ["1", "h", "p", "r", "word"]
```
#### split
`split` divides a given string by a given separator, returning a list of strings containing the characters between the separator sequences.
```
split(separatorString, string)
```
```
split(".", "host.domain")
# ["host", "domain"]
```
#### strlen
`strlen` is a Function that returns the length of the given string in characters.
```
strlen(string)
```
```
strlen("yes")
# 3
```
#### strrev
`strrev` is a Function that reverses the order of the characters in the given string.
```
strrev(string)
```
```
strrev("yes")
# "sey"
```
#### substr
`substr` is a Function that extracts a sequence of characters from another string and creates a new string.
```
substr(string, offsetNumber, length)
```
```
substr("host.domain", 0, 4)
# "host"
```
#### timeadd
`timeadd` adds a duration to a timestamp, returning a new timestamp.
Only units "inferior" or equal to `h` are supported.
The duration can be negative.
```
substr(timestamp, duration)
```
```
timeadd("2024-01-01T00:00:00Z", "-2600h10m")
# 2023-09-14T15:50:00Z
```
#### trim
`trim` removes the specified characters from the start and end of the given string.
```
trim(string, string)
```
```
trim("Can you do that ? Yes ?", "?")
# "Can you do that ? Yes"
```
#### trimprefix
`trimprefix` removes the specified prefix from the start of the given string.
```
trimprefix(stringToTrim, trimmingString)
```
```
trimprefix("please, do it", "please, ")
# "do it"
```
#### trimspace
`trimspace` removes any space characters from the start and end of the given string.
```
trimspace(string)
```
```
trimspace(" Hello World ")
# "Hello World"
```
#### trimsuffix
`trimsuffix` removes the specified suffix from the end of the given string.
```
trimsuffix(stringToTrim, trimmingString)
```
```
trimsuffix("Hello World", " World")
# "Hello"
```
#### try
`try` is a variadic function that tries to evaluate all of is arguments in sequence until one succeeds, in which case it returns that result, or returns an error if none of them succeed.
```
try(expressions...)
```
```
values {
map = {
hello = "you"
world = "us"
}
try(hv.map.do_not_exist, hv.map.world)
}
# "us"
```
#### upper
`upper` is a Function that converts a given string to uppercase.
```
upper(string)
```
```
upper("up")
# "UP"
```
#### values
`values` returns a list of the map values, in the order of the sorted keys. This function only works on flat maps.
```
values(map)
```
```
values({"a" = 1,"b" = 2})
# [1, 2]
```
#### yamldecode
`yamldecode` parses the given JSON string and, if it is valid, returns the value it represents.
```
yamldecode(string)
```
```
yamldecode("hello: world\narray: [1, 2, 3]")
###
{
array = [1, 2, 3]
hello = "world"
}
```
#### yamlencode
`yamlencode` returns a JSON serialization of the given value.
```
yamlencode({array = [1, 2, 3], hello = "world"})
```
```
yamlencode({array = [1, 2, 3], hello = "world"})
###
"array":
- 1
- 2
- 3
"hello": "world"
```
#### zipmap
`zipmap` constructs a map from a list of keys and a corresponding list of values.
Using manifest files in conjunction with command line argument can be a bit confusing.
# Paths Overview
A few rules to clear up this ambiguity:
Using manifest files in conjunction with command line argument can be a bit confusing.
A few rules to clear up this ambiguity:
- Absolute paths are always resolved as absolute paths
- Relative paths referenced *in* the helmfile manifest itself are relative to that manifest
- Relative paths referenced on the command line are relative to the current working directory the user is in
- Relative paths referenced from within the helmfile loaded from the standard input using `helmfile -f -` are relative to the current working directory
### Examples
There are several examples that we can go through in the `/examples` folder which demonstrate this.
There are several examples that we can go through in the [`/examples`](../examples/) folder which demonstrate this.
**Local Execution**
This is an example of a Helmfile manifest referencing a local value directly.
This is an example of a Helmfile manifest referencing a local value directly.
**Relative Paths in Helmfile w/ --values overrides**
This is an example of a Helmfile manifest using relative paths for values including an additional `--values` from the command line.
This is an example of a Helmfile manifest using relative paths for values including an additional `--values` from the command line.
NOTE: The `--values` is resolved relative to the CWD of the terminal *not* the Helmfile manifest. You can see this with the `replicas` being adjusted to 3 now for the deployment.
I'd like to make 3 breaking changes to Helmfile and mark it as 1.0, so that we can better communicate with the current and future Helmfile users about our stance on maintaining Helmfile.
Note that every breaking change should have an easy alternative way to achieve the same goal achieved using the removed functionality.
## Backward compatibility
v1 is backward-compatible with v0.x, except for the following breaking changes.
Each breaking change has an easy alternative way to achieve the same goal achieved using the removed functionality.
We also provide the alternative way in the latest v0.x release before v1.0. That way you can start using the alternative way today and be ready for v1.0. Note that in v0.x, some of those alternative ways are enabled only when `HELMFILE_V1MODE=true` is set.
> Context:
>
> Even though Helmfile had been used in production environments [across multiple organizations](USERS.md), it had been considered to be in its early stage of development, hence versioned 0.x.
>
> Helmfile complies to Semantic Versioning 2.0.0 in which v0.x means that there could be backward-incompatible changes for every release. However, Helmfile has been very conservative about breaking changes, and we had no breaking change for a year or so before start thinking about v1.
>
> That said, you can expect Helmfile v1 to be backward-compatible as much as it was in v0.x.
## The changes in 1.0
1. [Forbid the use of `environments` and `releases` within a single helmfile.yaml.gotmpl part](#forbid-the-use-of-environments-and-releases-within-a-single-helmfileyamlgotmpl-part)
2. [Force `.gotmpl` file extension for `helmfile.yaml` in case you want helmfile to render it as a go template before yaml parsing.](#force-gotmpl-file-extension-for-helmfileyaml-in-case-you-want-helmfile-to-render-it-as-a-go-template-before-yaml-parsing)
3. [Remove the `--args` flag from the `helmfile` command](#remove-the---args-flag-from-the-helmfile-command)
4. [Remove `HELMFILE_SKIP_INSECURE_TEMPLATE_FUNCTIONS` in favor of `HELMFILE_DISABLE_INSECURE_FEATURES`](#remove-helmfile_skip_insecure_template_functions-in-favor-of-helmfile_disable_insecure_features)
5. [The long deprecated `charts.yaml` has been finally removed](#the-long-deprecated-chartsyaml-has-been-finally-removed)
7. [Remove charts and delete sub-commands](#remove-charts-and-delete-subcommands)
### Forbid the use of `environments` and `releases` within a single helmfile.yaml.gotmpl part
- Helmfile currently relies on a hack called "double rendering" which no one understands correctly (I suppose) to solve the chicken-and-egg problem of rendering the helmfile template(which requires helmfile to parse `environments` as yaml first) and parsing the rendered helmfile as yaml(which requires helmfile to render the template first).
- By forcing (or print a big warning) the user to do separate helmfile parts for `environments` and `releases`, it's very unlikely Helmfile needs double-rendering at all.
- After this change, every helmfile.yaml written this way:
environments:
default:
values:
- foo: bar
releases:
- name: myapp
chart: charts/myapp
values:
- {{ .Values | toYaml | nindent 4 }}
must be rewritten like:
environments:
default:
values:
- foo: bar
---
releases:
- name: myapp
chart: charts/myapp
values:
- {{ .Values | toYaml | nindent 4 }}
It might not be a deal breaker as you already had to separate helmfile parts when you need to generate `environments` dynamically:
environments:
default:
values:
- foo: bar
---
environments:
default:
values:
- {{ .Values | toYaml | nindent 6}}}
- bar: baz
---
releases:
- name: myapp
chart: charts/myapp
values:
- {{ .Values | toYaml | nindent 4 }}
If you're already using any helmfile.yaml files that are written in the first style, do start using `---` today! It will probably reveal and fix unintended template evaluations. If you start using `---` today, you won't need to do anything after Helmfile 1.0.
### Force `.gotmpl` file extension for `helmfile.yaml` in case you want helmfile to render it as a go template before yaml parsing.
- As the primary maintainer of the project, I'm tired of explaining why Helmfile renders go template expressions embedded in a yaml comment. [The latest example of it](https://github.com/helmfile/helmfile/issues/127).
- When we first introduced helmfile the ability to render helmfile.yaml as a go template, I did propose it to require `.gotmpl` file extension to enable the feature. But none of active helmfile users and contributors at that time agreed with it (although I didn't have a very strong opinion on the matter either), so we enabled it without the extension. I consider it as a tech debt now.
### Remove the `--args` flag from the `helmfile` command
- It has been provided as-is, and never intended to work reliably because it's fundamentally flawed because we have no way to specify which args to be passed to which `helm` commands being called by which `helmfile` command.
- However, I periodically see a new user finds it's not working and reporting it as a bug([the most recent example](https://github.com/roboll/helmfile/issues/2034#issuecomment-1147059088)). It's not a fault of the user. It's our fault that left this broken feature.
- Every use-case previsouly covered (by any chance) with `--args` should be covered in new `helmfile.yaml` fields or flags.
### Remove `HELMFILE_SKIP_INSECURE_TEMPLATE_FUNCTIONS` in favor of `HELMFILE_DISABLE_INSECURE_FEATURES`
- This option didn't make much sense in practice. Generally, you'd want to disable all the insecure features altogether to make your deployment secure, or not disable any features. Disabling all is already possible via `HELMFILE_DISABLE_INSECURE_FEATURES `. In addition, `HELMFILE_SKIP_INSECURE_TEMPLATE_FUNCTIONS` literally made every insecure template function to silently skipped without any error or warning, which made debugging unnecessarily hard when the user accidentally used an insecure function.
- See https://github.com/helmfile/helmfile/pull/564 for more context.
### The long deprecated `charts.yaml` has been finally removed
Helmfile used to load `helmfile.yaml` or `charts.yaml` when you omitted the `-f` flag. `charts.yaml` has been deprecated for a long time but never been removed. We take v1 as a chance to finally remove it.
### List experimental features
We have some experimental features that are not stable yet. We should list them in a list and mark them as experimental.
In Helmfile v1.x, all features should be backward-compatible within v1.x as we follow semver. We can't fix features in a backward incompatible way by default. To do so, we need a list of experimental features and say "anything in the experimentals can be modified backward-incompatible ways", and include features that are consireded experimental into the list beforehand.
Why now?
In Helmfile v0.x, all features considered experimental as we follow semver. However, we "ended up" preserving backward-compatibility within v0 and between v0 and v1 "by chance". This doesn't mean anything
introduced in v0 is stable. For example, we might have some features implemented in a very later stage of v0 that are not stable yet. We should mark them as experimental, or we can't fix them in a backward-incompatible way in v1.x. That's why we need to list experimental features now.
### remove-charts-and-delete-subcommands
Now we remove `helmfile charts` and `helmfile delete` subcommands. you can use `helmfile destroy` and `helmfile sync` instead.
## After 1.0
We won't add any backward-incompatible changes while in 1.x, as long as it's inevitable to fix unseen important bug(s).
We also quit saying [Helmfile is in its early days](https://github.com/helmfile/helmfile#status) in your README as... it's just untrue today.
Assume you have two or more teams, each work for a different internal or external service, like:
- Product 1
- Product 2
- Observability
The simplest `helmfile.yaml` that declares the whole cluster that is composed of the three services would look like the below:
```yaml
releases:
- name: product1-api
chart: product1-charts/api
# snip
- name: product1-web
chart: product1-charts/web
# snip
- name: product2-api
chart: saas-charts/api
# snip
- name: product2-web
chart: product2-charts/web
# snip
- name: observability-prometheus-operator
chart: stable/prometheus-operator
# snip
- name: observability-process-exporter
chart: stable/prometheus-operator
# snip
```
This works, but what if you wanted to a separate cluster per service to achieve a smaller blast radius?
Let's start by creating a `helmfile.yaml` for each service.
`product1/helmfile.yaml`:
```yaml
releases:
- name: product1-api
chart: product1-charts/api
# snip
- name: product1-web
chart: product1-charts/web
# snip
- name: observability-prometheus-operator
chart: stable/prometheus-operator
# snip
- name: observability-process-exporter
chart: stable/prometheus-operator
# snip
```
`product2/helmfile.yaml`:
```yaml
releases:
- name: product2-api
chart: product2-charts/api
# snip
- name: product2-web
chart: product2-charts/web
# snip
- name: observability-prometheus-operator
chart: stable/prometheus-operator
# snip
- name: observability-process-exporter
chart: stable/prometheus-operator
# snip
```
You will (of course!) notice this isn't DRY.
To remove the duplication of observability stack between the two helmfiles, create a "sub-helmfile" for the observability stack.
`observability/helmfile.yaml`:
```yaml
- name: observability-prometheus-operator
chart: stable/prometheus-operator
# snip
- name: observability-process-exporter
chart: stable/prometheus-operator
# snip
```
As you might have imagined, the observability helmfile can be reused from the two product helmfiles by declaring `helmfiles`.
`product1/helmfile.yaml`:
```yaml
helmfiles:
- ../observability/helmfile.yaml
releases:
- name: product1-api
chart: product1-charts/api
# snip
- name: product1-web
chart: product1-charts/web
# snip
```
`product2/helmfile.yaml`:
```yaml
helmfiles:
- ../observability/helmfile.yaml
releases:
- name: product2-api
chart: product2-charts/api
# snip
- name: product2-web
chart: product2-charts/web
# snip
```
## Using sub-helmfile as a template
You can go even further by generalizing the product related releases as a pair of `api` and `web`:
`shared/helmfile.yaml`:
```yaml
releases:
- name: product{{ env "PRODUCT_ID" }}-api
chart: product{{ env "PRODUCT_ID" }}-charts/api
# snip
- name: product{{ env "PRODUCT_ID" }}-web
chart: product{{ env "PRODUCT_ID" }}-charts/web
# snip
```
Then you only need one single product helmfile
`product/helmfile.yaml`:
```yaml
helmfiles:
- ../observability/helmfile.yaml
- ../shared/helmfile.yaml
```
Now that we use the environment variable `PRODUCT_ID` to as the parameters of release names, you need to set it before running `helmfile`, so that it produces the differently named releases per product:
The `env` function allows you to declare a particular environment variable as an optional for template rendering.
If the environment variable is unset or empty, the template rendering will continue with an empty string as a value.
```yaml
{{ $envValue := env "envName" }}
```
#### `requiredEnv`
The `requiredEnv` function allows you to declare a particular environment variable as required for template rendering.
If the environment variable is unset or empty, the template rendering will fail with an error message.
```yaml
{{ $envValue := requiredEnv "envName" }}
```
> If the environment variable value starts with '/' (forward slash) and [Git for Windows](https://git-scm.com/download/win) is used, you must set `MSYS_NO_PATHCONV=1` to preserve values as-is, or the environment variable value will be prefixed with the `C:\Program Files\Git`. [reference](https://github.com/git-for-windows/build-extra/blob/main/ReleaseNotes.md#known-issues)
#### `exec`
The `exec` function allows you to run a command, returning the stdout of the command. When the command fails, the template rendering will fail with an error message.
The `envExec` function allows you to run a command with environment variables declared on-the-fly in addition to existing environment variables, returning the stdout of the command. When the command fails, the template rendering will fail with an error message.
The `isFile` function allows you to check if a file exists. On failure, the template rendering will fail with an error message.
```yaml
{{ if isFile "./myfile" }}
```
#### `isDir`
The `isDir` function allows you to check if a directory exists. On failure, the template rendering will fail with an error message.
```yaml
{{ if isDir "./mydirectory" }}
```
#### `readFile`
The `readFile` function allows you to read a file and return its content as the function output. On failure, the template rendering will fail with an error message.
```yaml
{{ $fileContent := readFile "./myfile" }}
```
#### `readDir`
The `readDir` function returns a list of the relative paths to the files contained within the directory. (No folders included. Use `readDirEntries` if you need folders too)
```yaml
{{ range $index,$item := readDir "./testdata/tmpl/sample_folder/" }}
{{- $itemSplit := splitList "/" $item -}}
{{- if contains "\\" $item -}}
{{- $itemSplit = splitList "\\" $item -}}
{{- end -}}
{{- $itemValue := $itemSplit | last -}}
{{- $itemValue -}}
{{- end -}}
```
#### `readDirEntries`
The `readDirEntries` function returns a list of [DirEntry](https://pkg.go.dev/os#DirEntry) contained within the directory
```yaml
{{ range $index,$item := readDirEntries "./testdata/tmpl/sample_folder/" }}
{{- if $item.IsDir -}}
{{- $item.Name -}}
{{- end -}}
{{- end -}}
```
#### `toYaml`
The `toYaml` function allows you to convert a value to YAML string. When has failed, the template rendering will fail with an error message.
```yaml
{{ $yaml := $value | toYaml }}
```
#### `fromYaml`
The `fromYaml` function allows you to convert a YAML string to a value. When has failed, the template rendering will fail with an error message.
```yaml
{{ $value := $yamlString | fromYaml }}
```
#### `setValueAtPath`
The `setValueAtPath` function allows you to set a value at a path. When has failed, the template rendering will fail with an error message.
The `get` function allows you to get a value at a path. you can set a default value when the path is not found. When has failed, the template rendering will fail with an error message.
```yaml
{{ $Getvalue := $value | get "path.key" "defaultValue" }}
```
#### `getOrNil`
The `getOrNil` function allows you to get a value at a path. it will return nil when the value of path is not found. When has failed, the template rendering will fail with an error message.
The `required` function returns the second argument as-is only if it is not empty. If empty, the template rendering will fail with an error message containing the first argument.
```yaml
{{ $requiredValue := $value | required "value not set" }}
```
#### `fetchSecretValue`
The `fetchSecretValue` function parses the argument as a [vals](https://github.com/helmfile/vals) ref URL, retrieves and returns the remote secret value referred by the URL. In case it failed to access the remote secret backend for whatever reason or the URL was invalid, the template rendering will fail with an error message.
The `expandSecretRefs` function takes an object as the argument and expands every [vals](https://github.com/helmfile/vals) secret reference URL embedded in the object's values. See ["Remote Secrets" page in our documentation](./remote-secrets.md) for more information.
The 'include' function allows including and rendering nested templates. The function returns the created template or an error if any occurred. It will load functions from `_*.tpl` files in the directory where the helmfile.yaml is located.
For nested helmfile.yaml files, it will load `_*.tpl` files in the directory where each nested helmfile.yaml is located. example: [include](https://github.com/helmfile/helmfile/tree/main/test/integration/test-cases/include-template-func/input)
waitTemplate: '{{`{{ eq .Release.Labels.tag "safe" | not }}`}}'
# ...
- `set` block values:
# ...
setTemplate:
- name: '{{`{{ .Release.Name }}`}}'
values: '{{`{{ .Release.Namespace }}`}}'
# ...
- `values` and `secrets` file paths:
# ...
valuesTemplate:
- config/{{`{{ .Release.Name }}`}}/values.yaml
secrets:
- config/{{`{{ .Release.Name }}`}}/secrets.yaml
# ...
- inline `values` map:
# ...
valuesTemplate:
- image:
tag: '{{`{{ .Release.Labels.tag }}`}}'
# ...
Previously, we've been using YAML anchors for release template inheritance.
It turned out not work well when you wanted to nest templates for complex use cases and/or you want a fine control over which fields to inherit or not.
Thus we added a new way for inheritance, which uses the `inherit` field we introduced above.
See [issue helmfile/helmfile#435](https://github.com/helmfile/helmfile/issues/435#issuecomment-1362177510) for more context.
You might also find [issue roboll/helmfile#428](https://github.com/roboll/helmfile/issues/428) useful for more context on how we originally designed the release template and what it's supposed to solve.
## Layering Release Values
Please note, that it is not possible to layer `values` sections. If `values` is defined in the release and in the release template, only the `values` defined in the release will be considered. The same applies to `secrets` and `set`.
## Layering State Files
> See **Layering State Template Files** if you're layering templates.
You may occasionally end up with many helmfiles that shares common parts like which repositories to use, and which release to be bundled by default.
Use Layering to extract the common parts into a dedicated *library helmfile*s, so that each helmfile becomes DRY.
Let's assume that your `helmfile.yaml` looks like:
At run time, `bases` in your `helmfile.yaml` are evaluated to produce:
```yaml
---
# environments.yaml
environments:
development:
production:
---
# helmfile.yaml
releases:
- name: myapp
chart: mychart
- name: metricbeat
chart: stable/metricbeat
```
Finally the resulting YAML documents are merged in the order of occurrence,
so that your `helmfile.yaml` becomes:
```yaml
environments:
development:
production:
releases:
- name: metricbeat
chart: stable/metricbeat
- name: myapp
chart: mychart
```
Great!
Now, repeat the above steps for each your `helmfile.yaml`, so that all your helmfiles becomes DRY.
Please also see [the discussion in the issue 388](https://github.com/roboll/helmfile/issues/388#issuecomment-491710348) for more advanced layering examples.
## Merging Arrays in Layers
Helmfile doesn't merge arrays across layers. That is, the below example doesn't work as you might have expected:
```yaml
releases:
- name: metricbeat
chart: stable/metricbeat
---
releases:
- name: myapp
chart: mychart
```
Helmfile overrides the `releases` array with the latest layer so the resulting state file will be:
```yaml
releases:
# metricbeat release disappeared! but that's how helmfile works
- name: myapp
chart: mychart
```
A work-around is to treat the state file as a go template and use `readFile` template function to import the common part of your state file as a plain text:
`common.yaml`:
```yaml
templates:
metricbeat: &metricbeat
name: metricbeat
chart: stable/metricbeat
```
`helmfile.yaml`:
```yaml
{{ readFile "common.yaml" }}
releases:
- <<:*metricbeat
- name: myapp
chart: mychart
```
## Layering State Template Files
Do you need to make your state file even more DRY?
Turned out layering state files wasn't enough for you?
Helmfile supports an advanced feature that allows you to compose state "template" files to generate the final state to be processed.
In the following example `helmfile.yaml.gotmpl`, each `---` separated part of the file is a go template.
`helmfile.yaml.gotmpl`:
```yaml
# Part 1: Reused Environment Values
bases:
- myenv.yaml
---
# Part 2: Reused Defaults
bases:
- mydefaults.yaml.gotmpl
---
# Part 3: Dynamic Releases
releases:
- name: test1
chart: mychart-{{ .Values.myname }}
values:
- replicaCount: 1
image:
repository: "nginx"
tag: "latest"
```
Suppose the `myenv.yaml` and `test.env.yaml` loaded in the first part looks like:
`myenv.yaml`:
```yaml
environments:
test:
values:
- test.env.yaml
```
`test.env.yaml`:
```yaml
kubeContext: test
wait: false
cvOnly: false
myname: "dog"
```
Where the gotmpl file loaded in the second part looks like:
`mydefaults.yaml.gotmpl`:
```yaml
helmDefaults:
kubeContext: {{ .Values.kubeContext }}
verify: false
{{ if .Values.wait }}
wait: true
{{ else }}
wait: false
{{ end }}
timeout: 600
recreatePods: false
force: true
```
Each go template is rendered in the context where `.Values` is inherited from the previous part.
So in `mydefaults.yaml.gotmpl`, both `.Values.kubeContext` and `.Values.wait` are valid as they do exist in the environment values inherited from the previous part(=the first part) of your `helmfile.yaml.gotmpl`, and therefore the template is rendered to:
```yaml
helmDefaults:
kubeContext: test
verify: false
wait: false
timeout: 600
recreatePods: false
force: true
```
Similarly, the third part of the top-level `helmfile.yaml.gotmpl`, `.Values.myname` is valid as it is included in the environment values inherited from the previous parts:
```yaml
# Part 3: Dynamic Releases
releases:
- name: test1
chart: mychart-{{ .Values.myname }}
values:
replicaCount: 1
image:
repository: "nginx"
tag: "latest"
```
hence rendered to:
```yaml
# Part 3: Dynamic Releases
releases:
- name: test1
chart: mychart-dog
values:
replicaCount: 1
image:
repository: "nginx"
tag: "latest"
```
## Re-using environment state in sub-helmfiles
Do you want to decouple the environment state loading from the sub-helmfiles and load it only once?
This example shows how to do this:
```yaml
environments:
stage:
values:
- env/stage.yaml
prod:
values:
- env/prod.yaml
---
helmfiles:
- path: releases/myrelease/helmfile.yaml
values:
- {{ toYaml .Values | nindent 4 }}
# pass the current state values to the sub-helmfile
# add other values to use overlay logic here
```
and `releases/myrelease/helmfile.yaml` is as DRY as
```yaml
releases:
- name: mychart-{{ .Values.myrelease.myname }}
installed: {{ .Values | get "myrelease.enabled" false }}
chart: mychart
version: {{ .Values.myrelease.version }}
labels:
chart: mychart
values:
- values.yaml.gotmpl
# templated values would also inherit the values passed from upstream
This example is intended to show how you can use Helmfile with ArgoCD in a private way. The example deployment demonstrates how to deploy ArgoCD with the helmfile plugin, and how to use helmfile within ArgoCD to connect to various private external systems.
If you intend to follow this example, please pay close attention to the caveats. This chart will not work if you deploy it as-is, you will need to customise many parts of it to match the infrastructure of your organisation.
## What is the point of this chart?
There are many organisations out there which would like to widen Kubernetes adoption but struggle with the requirement to keep all data confidential and follow best practices. For example, one best practice is to to implement a CI/CD process for all deployments including Helm charts. As part of this process we may want to keep secrets in private Vault secret storage, use Helm charts from a private chart museum, customised Helm values from a private Git repository and docker images from a private docker repository. ArgoCD alone cannot do all of these things at once, but we can achieve this if we configure it with Helmfile.
This example is intended to show how you COULD use ArgoCD together with Helmfile to implement a fully private CI/CD system. Your deployment may be a litte bit different, or very different to this example. It's intended only as an example.
The helm chart in this repo can be deployed with Helmfile, if you follow the prerequisites. In order for the deployment to work correctly, you will need to follow the prerequisites and update all of the `values.yaml` file entries with a `# TODO` comment.
What are the interactions between the different systems?
* ArgoCD will run on Kubernetes. We must use a custom build of ArgoCD as the official build doesn't ship with the runtime binaries which we will need to execute Helmfile and interact with Vault.
* ArgoCD interacts with AzureAD for SSO. ArgoCD has to be configured as an enterprise app in Azure AD for authorization and authentication by an administrator. We put the client ID and client secrets into Vault for safe keeping.
* ArgoCD interacts with Vault in order to pull secrets for customising Helm deployments. Sometimes we need to inject secrets into the values.yaml file of a Helm deployment. ArgoCD will use an AppRole for authentication into Vault, and this is done once during the deployment. The ArgoCD Vault credentials are saved in Vault for safe keeping.
* ArgoCD may need to interact with a private Git server. This is so that ArgoCD can access the `helmfile.yaml` files before executing Helmfile. Additional configuration files (like `conf.d` and other `values.yaml`) files can be kept in here.
* ArgoCD will authenticate to a private chart museum. This is so that we can keep the private helm charts private.
# Prerequisites
##Docker image
* Build the docker image from the `argocd.dockerfile` file in this folder.
* Upload the image to your private docker registry and make a note of the registry URL and tag.
* Customise the `values.yaml` file in this folder.
Replace `argo-cd.global.image.repository` for the docker registry URL of your private registry.
Replace `argo-cd.global.image.tag` for the tag which you used to upload your private docker image.
## Configure Single Sign-On for ArgoCD
* Log into AzureAD as an administrator.
* Create a new App Registration in your tenant for ArgoCD.
* Create a new Client ID and client secret.
* Save the client ID and client secret into your vault server.
- Save the client ID and client secret into Vault. Suggested keys: `internal/argocd/auth#azure_oidc_client_id` and `internal/argocd/auth#azure_oidc_client_secret`.
* Update `values.yaml` for your deployment
- Update `argo-cd.server.config.oidc.config` - replace `SOME_AZURE_AD_TENANT` for your actual plaintext AzureAD tenant ID.
- Update `argo-cd.server.config.oidc.config` - replace `SOME_AZURE_AD_UUID` for your actual plaintext AzureAD client app ID.
- Verify that the Vault path specified in `argo-cd.config.secret.extra.oidc.azure.clientSecret` is the correct path in Vault.
* Customise the ingress objects in the source to match what your expected external URL will be. Update `values.yaml` to have the correct values for the hostname you plan to use.
- `argo-cd.config.server.(url|hostname)`
```yaml
url: 'https://my.argocd.deployment.org' # TODO
hostname: my.argocd.deployment.org #TODO
```
## Configure authentication from ArgoCD into Vault (or read next section on how to disable it)
* Read the documentation from Hashicorp on AppRoles. Provision a new AppRole for ArgoCD with policies.
- In the example the authentication path in Vault is assumed to be `auth/approle/login`. If that's not the case then update line 123 in `values.yaml`.
* Store the AppRole credentials for ArgoCD in Vault. We will use Helmfile to pull them from here when we launch the chart.
- Suggested paths: `internal/vault/argocd#role_id` and `internal/vault/argocd#secret_id`
* Verify that the Vault paths in `values.yaml` are correct
- `argo-cd.config.secret.extra.vault_role_id`
- `argo-cd.config.secret.extra.vault_secret_id`
* Replace `ROLE_ID` and `SECRET_ID` in `values.yaml` (line 124) with the actual plaintext values (values key `argocd.server.config.configManagementPlugins`). This is not 100% secure but was the only way that I could manage to get the rest of the deployment working. I advise you add `values.yaml` to `.gitignore` after doing this step.
## Disable Vault (optional)
If you are not going to use Vault, then you need to update the values to get your deployment to use Helmfile.
* If you are not going to use Vault, then remove the line in `values.yaml` that starts with `export VAULT_TOKEN=...`.
##Configure authentication from ArgoCD into private Git repository
* Create a new API user in your Git system (username + password auth). ArgoCD will use these credentials to pull `values.yaml` and `helmfile.yaml` for your project.
* Save the username and password into Vault
- Recommended keys: `internal/git/users/argocd#username` and `internal/git/users/argocd#password`
* Verify that the Vault paths in `values.yaml` are correct
- `argo-cd.config.secret.extra.git_username`
- `argo-cd.config.secret.extra.git_password`
* Update the settings in `values.yaml` for repositories - remove the example Git URLS and replace with actual Git URLs for your project.
## Configure authentication from ArgoCD to your private Docker registry and Chartmuseum
* This guide is written with Harbor in mind, which is both a docker registry and chart museum. If you have separate systems in place you might need to configure these things separately.
* Create a new set of credentials for Harbor and save them into Vault.
- If everything worked correctly, then ArgoCD should have deployed using your customised docker image into your cluster.
* If the ingress deployed correctly then we should be able to access the UI: https://my.argocd.deployment.org
* Grab the admin password from `argocd-initial-admin-secret secret` and use it to log into the UI.
```bash
echo "<password>" | base64 -d
```
* Verify that SSO works by signing on with AzureAD.
* Verify that Helmfile appears in the list of configured plugins in the UI.
# Configure ArgoCD apps for deployment with Helmfile
* Log into ArgoCD in the UI and configure a new app.
* Make sure that it's using the Helmfile plugin.
* For the source, select one of the Git repos which you configured previously.
* Make sure that your Helm repo (including `helmfile.yaml` is present in the Git repository).
* ArgoCD will pull the project from your private Git server and read the `helmfile.yaml` file.
* ArgoCD will also execute helmfile directly in the container using the files checked out from your project.
* If you configured ArgoCD with logins for Vault and private Chartmuseums, these can be used in ArgoCD by Helmfile.
* You can use the diff feature and resource views in ArgoCD when using Helmfile. Unfortunately you can no longer use the `values.yaml` editor in the UI. You can change `values.yaml` for deployments by creating `values.yaml` files and committing them to the folders in your project Git repo. You can also customise them using `!vault` tags.
If you would like an example, this ArgoCD + helmfile deployment can itself be deployed by ArgoCD + helmfile (you may need to drop the pods after you do deploy this way). Any other projects - just structure them in the same way as this example.
Be careful when reconfiguring a project which was previously deploy by ArgoCD with Helmfile. There are some differences around the way in which annotations are used.
labels:# Arbitrary key value pairs for filtering releases
env:prod
chart:"."# the chart being installed to create this release, referenced by `repository/chart` syntax
version:3.2.2# the semver of the chart. range constraint is supported
missingFileHandler:Warn# set to either "Error" or "Warn". "Error" instructs helmfile to fail when unable to find a values or secrets file. When "Warn", it prints the file and continues.
# will attempt to decrypt secrets using helm-secrets plugin
return[]string{},fmt.Errorf("specified state file %s is not found",specifiedPath)
}
}else{
vardefaultFilestring
iffileExistsAt(DefaultHelmfile){
defaultFile=DefaultHelmfile
}elseiffileExistsAt(DeprecatedHelmfile){
log.Printf(
"warn: %s is being loaded: %s is deprecated in favor of %s. See https://github.com/roboll/helmfile/issues/25 for more information",
DeprecatedHelmfile,
DeprecatedHelmfile,
DefaultHelmfile,
)
defaultFile=DeprecatedHelmfile
}
ifdirectoryExistsAt(DefaultHelmfileDirectory){
ifdefaultFile!=""{
return[]string{},fmt.Errorf("configuration conlict error: you can have either %s or %s, but not both",defaultFile,DefaultHelmfileDirectory)
}
helmfileDir=DefaultHelmfileDirectory
}elseifdefaultFile!=""{
return[]string{defaultFile},nil
}else{
return[]string{},fmt.Errorf("no state file found. It must be named %s/*.yaml, %s, or %s, or otherwise specified with the --file flag",DefaultHelmfileDirectory,DefaultHelmfile,DeprecatedHelmfile)
t.Run("fail on unselected need by default",func(t*testing.T){
check(t,testcase{
selectors:[]string{"app=test"},
error:`in ./helmfile.yaml: release "default/default/external-secrets" depends on "default/kube-system/kubernetes-external-secrets" which does not match the selectors. Please add a selector like "--selector name=kubernetes-external-secrets", or indicate whether to skip (--skip-needs) or include (--include-needs) these dependencies`,
t.Fatalf("unable to update lint log snapshot: %v",err)
}
}
assert.Equal(t,wantLog,gotLog)
}
t.Run("fail on unselected need by default",func(t*testing.T){
check(t,testcase{
selectors:[]string{"app=test"},
error:`in ./helmfile.yaml: release "default/default/external-secrets" depends on "default/kube-system/kubernetes-external-secrets" which does not match the selectors. Please add a selector like "--selector name=kubernetes-external-secrets", or indicate whether to skip (--skip-needs) or include (--include-needs) these dependencies`,