Fixing releases being included which do not match the environment
requested, which is a regression introduced by #234. The issue remains
when Helmfile state values are supplied, which is not a regression and
will be addressed separately.
Partial resolution for #271
Signed-off-by: David Ackroyd <dackroyd@fairfaxmedia.com.au>
Signed-off-by: David Ackroyd <dackroyd@fairfaxmedia.com.au>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Make a few helmfile sub-commands to consistently support needs-related flags
* helmfile-diff adds support for --include-transitive-needs
* helmfile-template adds support for --skip-needs
* helmfile-lint adds support for --skip-needs, --include-needs, and --include-transitive-needs
Ref https://github.com/roboll/helmfile/issues/2055
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix a few helmfile-lint needs related bugs and add tests
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Is include-transitive-needs realy working as intended? 🤔
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Confirm that it does fail on unselected need by default
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Add missing testdata
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Test helmfile-template for include/skip needs support
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix a few terms
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Add more tests to better know the current helmfile-diff behavior around needs
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix failing tests
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix helmfile-diff to consistently handle skip/include-needs
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Extract testhelper.RequireLog for reusing
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix all bugs and test cases for TestDiff and TestDiff_2
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix TestDiff_2
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix TestDiff
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix TestDiffWithNeeds
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Unify behavior on including disabled releases as needs for lint and template
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* Fix bug that --include-transitive-needs does not imply include-needs
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* introduce DISABLE_INSECURE_FEATURES to disable insecure executions
Signed-off-by: Quan TRAN <account@itscaro.me>
* disable remote sources when DISABLE_INSECURE_FEATURES is set to "true"
Signed-off-by: Quan TRAN <account@itscaro.me>
* refactor envvar package
Signed-off-by: Quan TRAN <account@itscaro.me>
* (test) fix test fixtures
Signed-off-by: Quan TRAN <account@itscaro.me>
* use absolute path to avoid unit test failure
Signed-off-by: Quan TRAN <account@itscaro.me>
* Fix conflicts
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
Since helm-diff has added an ability to auto-detect the term to decide if it should output with color or not, helmfile had been defaulted to no-color.
This resoloves that, by adding a term-detection logic that is same as helm-diff.
As a part of this work, I have also implemented a new global flag `--color`, which is used for forcing color without relying on the term-detection logic implemented in helmfile or explicitly setting the HELM_DIFF_COLOR envvar. I hope it is useful for folks.
Ref https://github.com/roboll/helmfile/issues/2043
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
When chartify is involved due to the use of `forceNamespace`, `strategicMergePatches`, `jsonPatches`, and so on, We had been internally mutating the Release.Chart with the path to the local temporary directory that contains the modified version of the chart.
This resulted in us unintentionally making `helmfile deps` to remove entries for the chart being modified out of helmfile.lock file, which resulted in issues like #2110.
To be clear, although the original issue is reported to occur for `strategicMergePatches`, I believe that it occurered also for any remote charts using `jsonPatches` and `forceNamespace` too.
I also believe this has been the issue since our introduction of chartify (maybe a year or so ago??), and I guess why it took so much time to be found and reported is that not so many people with chartify in combination with `helmfile deps` 🤔
Lastly, this changes chart names surfaced in the various log output from Helmfile, from temporary chart paths to the chart name/path declared in the helmfile.yaml. I think this is generally a good change, no fear of being a breaking change. But if anyone has any concern about that, please feel free to comment/report/etc.
Ref https://github.com/roboll/helmfile/issues/2110
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Use the value of the `condition` field instead of the `installed` field of a release in the `enabled` column of helmfile list.
The value of the `installed` field is shown in a new `installed` column.
Fixes#1920
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
Parses a new field in repositories named `skipTLSVerify` and if set to `true`, it appends `--insecure-skip-tls-verify` in `helm repo add` command.
This should be useful with internal self-signed repos, mitm proxies etc.
Resolves#1871
* Do fail on a possible typo in `needs` entries
Helmfile kindly fails with a friendly error when you made a typo in a `needs` entry, i.e. a `needs` entry included a reference to a release that is not defined in the helmfile config.
Example Output:
```
in ./helmfile.needs.yaml: release(s) "app" depend(s) on an undefined release "infrastructure/cert-manager2". Perhaps you made a typo in `needs` or forgot defining a release named "cert-manager2" with appropriate `namespace` and `kubeContext`?
```
This prevents issues like #1959
* Fix regression in helmfile-diff (This may break when you had two or more duplicated releases that are intended to be de-duplicated before DAG calculation using selectors
* Fix regression when you used selector to deduplicate releases before DAG calculation
* Comments
* Fix regressions in helmfile-apply and helmfile-sync
* Fix regression in duplicate release detection
Currently it's not possible to use `.Environment` values in `*.gomtpl` files. The documentation states the opposite:
https://github.com/roboll/helmfile#environment (2nd paragraph).
The problem is already described in #1090.
This PR fixes this bug.
Fixes#1090
Co-authored-by: Peter Aichinger <petera@topdesk.com>
* Guard collectNeeds-method against infinite recursion.
* Also check for namespace and kubecontext when collecting needs.
Co-authored-by: Peter Aichinger <petera@topdesk.com>
Apparently we needed to pass `--validate` on helm-template run by chartify when the targeted chart contains Capabilities.APIVersions in a chart template. Otherwise, you can never make such chart work with chartify, as at apply time helm template expressions that involved Capabilities.APIVersions are already nowhere.
This adds the ability to include the --pass-credentials flag to the helm add repo command by:
- Adding repo.passCredentials to the helmfile yaml
- Changing state, helmexec, and app to include RepositorySpec.PassCredentials
Resolves#1898
Co-authored-by: almed4 <alexandre.meddin@ingka.ikea.com>
#1772 broke `--selector` with `needs` in many ways.
The two biggest problems I've encountered were:
- duplicate releases even if you've provided a proper `selector` to deduplicate
- sync/deletion ordering broken when you have `needs`
For the first issue, we had to update `getSelectedReleases` function to also calculate the "selected releases and releases needed by the selected releases", and use that to calculate the DAG. That should have been done in #1772.
The latter started happening after I've fixed the first issue. The source of the issue was that `needs` turned out to be ambiguous in a few cases.
Previously, `needs: ["foo/bar"]` had two meanings. One for "needs release bar in kubecontext foo", another for "needs release bar in namespace foo".
Moreover, `needs: ["foo/bar/baz"]` had three meanings.
- `needs release baz in tiller namespace foo and namespace baz`
- `needs release baz in namespace bar in kubecontext foo`
- `needs release baz in tiller namespace bar in kubecontext foo`.
Especially, the first meaning doesn't make sense at all. Helm 2 solely use tillerNamespace for namespacing the release and Helm 3 uses namespace for that.
This fix sorts all the bugs and issues I've found so far around that, by changing the meanings of the above two examples as follows:
- `foo/bar` means `namespace=foo,name=bar` for Helm 3 and `tillerNamespace=foo,name=bar` for Helm 2
- `needs release bar in kubecontext foo` is now `foo//bar`. Notice the extra `/` between `foo` and `bar`.
- `foo/bar/baz` means `kubecontext=foo,namespace=bar,name=baz` for Helm 3 and `kubecontext=foo,tillerNamespace=bar,name=baz` in Helm 2
Fixes#1818
* tests: fix vagrant test run
* feat: added an option to specify the different diff output format
* renamed diff-output to output
* renamed diff-output to output
Co-authored-by: Andrey Tuzhilin <andrey@zelf.co>
* Add test for helmfile-destroy with selector
I was curious if destroy with a selector is working at all while reading https://github.com/roboll/helmfile/issues/1750#issuecomment-823521572. So I added this test for verification. From the test result, it is working, though.
* Add one more test case for destroying disabled but installed release
* Add --{include,skip}-needs to helmfile-sync and helmfile-apply
* Add --include-needs to helmfile-template
* Add TODO related to #1018
* Add a few new test files to cover new functionalities
* Update apply test to incorporate the change that the destroy and sync steps target affected releases only
Adds `--chart` flag for overriding the selected release's chart ad-hoc-ly like `helmfile --chart $CHART template`.
This is handy when e.g. you want to have an ArgoCD application per each release in your helmfile.yaml, while also providing the ability to customize the release's chart without touching helmfile.yaml.
See https://github.com/roboll/helmfile/issues/1690#issuecomment-812321354 for more context.
Closes#1690
This would allow cli flag `--kube-context` to override value in helmDefaults allowing to use different values in local development and CI context.
Co-authored-by: Andrey Tuzhilin <andrey@3adigital.ru>
This improves helmfile-apply with two things:
- Some users had timing-out issues or annoyed by huge output from helm-diff run as part of helmfile-apply on first install. `--skip-diff-on-install` skips running helm-diff for releases being newly installed, so that you can avoid those issues.
- Some users had difficultly or found it not straight-forward to install CRDs and custom resources from separate charts in one helmfile-apply (#1353). The new helmfile.yaml release field `disableValidationOnInstall: true` adds `--disable-validation` to helm-diff only for releases being newly released, which should mostly resolve the issue.
Resolves#1353
This deprecates the old `--retain-values` which was not working as intended.
Also see #1570 - there's now `--skip-cleanup` for `helmfile-template`, too.
This allows you to use helmfile-template output as a GitOps source, when the template output contains CRDs and you use Helm 3.
Helm 3 by default removes CRDs from the template output. If you want to git-commit helmfile-template containing CRDs for GitOps and you use Helm 3 for templating, the only way is provide this newly added `--include-crds` flag.
I had mistakenly inverted the necessary flag to turn off the legacy processing on selectors that should be used only for helmfile commands that does not support DAGs/needs.
Fixes#1552
Adds a basic support for Helm repositories hosted on Azure Container Registry (not OCI but classic ones). Add a new field to RepositorySpec to state that is externally managed and runs the `az-cli` command instead of the helm one to manage the repository.
* Parse and process helm version using github.com/Masterminds/semver/v3.
* Add --force-update only when Helm version >= 3.3.2, < 3.3.4.
See: https://github.com/helm/helm/pull/8777.
* Add test cases.
* Bump sprig to v3.1.0
test for mergeOverwrite
* Let mergo not (accidentally) try to merge unexported fields
This is also a good chance separate `HelmState` with the config loaded from YAML, which I had been wanting to do for a long time.
Co-authored-by: Johannes Alkjær <johannes.alkjaer@wunderman.com>
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
This adds `comonLabels` option to helmfile by:
- Adding `CommonLabels` to HelmState
- Changing `markExcludedReleases` and `ListReleases` functions to merge common labels into release labels
Resolves#1266
This reverts a part of #1383 so that repository updates are done in the pre-0.125.0 way, which tries to update any repositories only once regardless of they are referenced by selected releases or not.
Ref #1404
In #1172, we accidentally changed the meaning of prepare hook that is intended to be called BEFORE the pathExists check. It broke the scenario where one used a prepare hook for generating the local chart dynamically. This fixes Helmfile not to fetch local chart generated by prepare hook.
In addition to that, this patch results in the following fixes:
- Fix an issue that `helmfile template` without `--skip-deps` fails while trying to run `helm dep build` on `helm fetch`ed chart, when the remote chart has outdated dependencies in the Chart.lock file. It should be up to the chart maintainer to update Chart.lock and the user should not be blocked due to that. So, after this patch `helm dep build` is run only on the local chart, not on fetched remote chart.
- Skip fetching chart on `helmfile template` when using Helm v3. `helm template` in helm v3 does support rendering remote charts so we do not need to fetch beforehand.
Fixes#1328
May relate to #1341
Changes:
- Prevent Helmfile from unnecessarily running `helm repo add` and `helm repo up` against repositories for unused repositories(repositories of releases filtered out by selector)
- Fixes#1330
This, in combination with #1172, allows you to use `go-getter`-supported URL for K8s manifests on `chart`, so that Helmfile automatically fetches it and then turning it into a temporary local chart, which is then installed by Helmfile as similar as standard Helm charts.
An example usecase of this is to install cert-manager CRDs which is distributed separately from the chart:
```
releases:
- name: cert-manager-crds
chart: git::http://github.com/jetstack/cert-manager.git@deploy/crds?ref=v0.15.2
```
I'm adding this based on discussion with @lukasmrtvy. He was trying to install cert-manager and prometheus-opreator with Helmfile, and this combined with #1373 should do the job. Thanks for the input!
This is useful for e.g. removing state file names and their hash values out of output dirs so that it can be used easily in a gitops setup. For example, `--output-dir-template mybasedir/{{.Release.Name}}` produces `mybasedir/RELEASE/CHART/templates/*.yaml` for each release in your helmfile.yaml.
Changes:
* Add global hooks
* Add top level hooks field to yaml spec
* Add functions for global prepare and cleanup events
* Call global prepare and cleanup events in withPreparedCharts function
* Update README
* Add helmfileCommand variable to withPreparedCharts
Pass the information on what helmfileCommand has been run down from the
top level functions through withReposAndPreparedCharts and withPreparedCharts.
This is the GA version of the helm-x integration #673 developed last year.
You get all the following benefits without an extra helm plugin:
- Ability to add ad-hoc chart dependencies/aliases, without forking the chart (Fixes#876 )
- Ability to patch resulting K8s resources before installing the helm chart
- Ability to install a kustomization as a chart (Requires `kustomize` binary to be available in `$PATH`
- Ability to install a directory of K8s manifests as a chart
- etc.
Fixes https://github.com/roboll/helmfile/issues/1142
desired_state_file_loader.go
- Will now normalize the content before splitting it to parts
context:
Me & and a fellow dev have tried to figure out why helmfile didn't fill in certain values on his machine;
turns out, he'd mistakenly checked out our project w/ CRLF line endings, which had caused part splitting to not work (as it's hard coded to look for '\n').
The following was acted on as a single part, causing values from the bases not to be available in the next yaml part:
```
bases:\r\n
- base.yaml\r\n
---\r\n
releases:
- name: external-secrets-crd
... some templated yaml ...
```
I've thought about regex-ing it out instead of replace-all, but benchmarks had shown that a plain replace is faster.
I've also considered splitting by "\n---" instead of "\n---", but that would break if the dashes were to continue with some other text.
Resolves#1232
CHANGES:
* Disable repo update when installed is false
When install is false, we do not need to update the repositories and get
the chart.
Signed-off-by: Guillaume Perrin <guillaume28.perrin@gmail.com>
- createNamespace is a new attribute that can be added to helmDefaults
or an individual release to enforce the creation of a release namespace
during sync if the namespace does not exist. This leverages helm's
(3.2+) --create-namespace flag for the install/upgrade command. If
running helm < 3.2, the createNamespace attribute has no effect.
Resolves#891Resolves#1140
* feat(tmpl): added fetchSecretValue template function
This adds a tmpl `fetchSecretValue` and `expandSecretRefs` function by:
- Adding:
- `expandSecretRefs` function in tmpl package that uses vals
package to fetch secrets
- `fetchSecretValue` function in tmpl package like below but for
single string value
- gomock for tests purpose
- Changing:
- move init of vals package to function (so the same instance can be used for template values and rendering the whole template)
* doc(secret): added doc how to use new tmpl methods
Added example usage of `fetchSecretValue` and `expandSecretRefs`
* New output flag for list command
Support output as json
Add new formatters file to handle extrac formatting to its own concern
New config interface to support list command specification
* Fix usage message
* Add error handling for formatters
This commit proposes a potential solution for
https://github.com/roboll/helmfile/issues/1201
The gist is that, if prestate rendering fails, for any reason,
we do not populate the .Values in the second pass renderer.
I think that what have been expected in this case is to populate the
.Values irregardless.
pkg/app/two_pass_renderer.go
- Migrated to use finalEnv.GetMergedValues()
pkg/environment/environment.go
- Introduced GetMergedValues, which merges the environment's defaults
and current values, and then casts the keys to string;
This was previously defined in HelmState.Values() - however, as this
method is only concerned with the environment, I think it's more
appropriate for it to sit here.
pkg/state/state_exec_tmpl.go
- Extracted out HelmState.Values() to environment.go, see above
`--retain-values-files` prevents temporary values files that were passed to Helm commands run by Helmfile for debugging purpose.
With that, you can manually rerun helm commands that were logged when `--log-level=debug` is enabled.
Resolves ##1117
* Add option to suppress diff on apply
Add --supress-diff option on apply. Usable for fresh installs when a
lot of output is produces by diff.
Resolves#458
* fix tests for suppress-diff
Arkaitz Jimenez
Yusuke Kuoka
David Ackroyd
yxxhero
Sergii Iefremov
yxxhero
Quan TRAN
Adam Gardner
Eng Zer Jun
Quan TRAN
austin ce
Grégory SANCHEZ
Anton Bretting
Sören Jentzsch
drivelikebrazil
Christoph Petrausch
Mårten Svantesson
Babis K
Jonas Rutishauser
pjotre86
Alex Meddin
dudicoco
katsew
katsew
astorath
Yujun Zhang
Nenad Strainovic
Jason Witkowski
Thomas Loubiou
Chris Mellard
Wi1dcard
Javier Palacios
Johannes Alkjær
Maksym Lushpenko
Max Audron
Kevin J. Qiu
Ilya Vassyutovich
Danny Shemesh
Guillaume Perrin
Victor Noël
nejtr0n
Craig Dunford
Marcin Kaciuba
chenrui
Rene Hernandez
RaymondKYLiu
Eric Bailey
Emil
Christoph Hösler
a-hat