* feat: make environment context available
This feature adds the "{{.Environment.KubeContext}}" variable.
Discussion #829
Signed-off-by: sewieland <sebastian.wieland@iav.de>
* chore: fix tests which compare logging outputs
This commit adds an addtional space wherever needed to the expected log outputs due to the added "KubeContext" in the environment struct.
Discussion #829
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
* docs: added documentation for `Environment.KubeContext`
Discussion #829
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
* test: make sure the `Environment.KubeContext` is mapped out correctly
Discussion #829
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
---------
Signed-off-by: sewieland <sebastian.wieland@iav.de>
Signed-off-by: Sebastian Wieland <wieland.s@mailbox.org>
Co-authored-by: sewieland <sebastian.wieland@iav.de>
Ref https://github.com/helmfile/helmfile/discussions/593
* implemented --reset-values flag overriding .helmDefaults.reuseValues=true
* tests for --reset-values flag
* fixed pkg/app tests
* added tests for both reset & reuse flags given
* added appendValuesControlMode method for --reuse/reset-values flag processing
* updated code comments & docs
Signed-off-by: Karol Ossowski <k@koralsky.pl>
This should fix#435 for Helmfile v0.x releases since the next v0.150.0.
We introduce a new envvar to opt-in to the new YAML library, so that you can give it a shot before upgrading your Helmfile to v1. The same envvar can be used to opt-out of the new YAML library after you upgrade to Helmfile v1, giving you a more flexible migration story.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Previously a values template rendering failure resulted in a cryptic error message like:
```
g release templates in "input.yaml": failed executing templates in release "input.yaml"."foo1": failed executing template expressions in release "foo1".values[0] = "[98 97 115 101 76 97 98 101 108 58 32 34 123 123 32 103 101 116 32 46 82 101 108 101 97 115 101 46 76 97 98 101 108 115 32 92 34 98 97 115 101 92 34 32 92 34 78 79 84 95 73 78 72 69 82 73 84 69 68 92 34 32 125 125 34 10]": template: stringTemplate:1: unexpected "\\" in operand
```
The issue was that the values template at the value position of the YAML dict was printed as a formatted byte array. We now print it as a standard string so that it is easily readable.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* fix env value lost in environment values
Signed-off-by: yxxhero <aiopsclub@163.com>
* add more test
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
This is a successor to #596. We need a smooth migration path from `gopkg.in/yaml.v2`, and this pull request moves it forward with `goccy/go-yaml` instead of `gopkg.in/yaml.v3`. Merging this unblocks users stuck in Helmfile v0.146.x or earlier due to #435, so that they can upgrade to 0.147.x or greater without updating their helmfile configs.
We previously tried to upgrade to `yaml.v3` (https://github.com/helmfile/helmfile/issues/394) in Helmfile v0.x, presuming it won't break anything. Apparently, it broke use-cases where you want to layer release's `values` field over three or more release templates and releases (#435).
We then tried to bring back `yaml.v2` for Helmfile v0.x and keep `yaml.v3` for the upcoming Helmfile v1. However, it failed due to incompatibility in the Unmarshaller interface between `yaml.v2` and `yaml.v3` (https://github.com/helmfile/helmfile/pull/596).
`goccy/go-yaml` is, from my observation, a well-maintained alternative to `yaml.v2`. One of its premises is that it enables us to swap the implementation from `gopkg.in/yaml.v2` to `goccy/go-yaml` just by replacing the import directive. It seems to use the same `Unmarshaller` interface as yaml.v2 too.
Once this PR gets merged, I'd like to follow-up with adding a new build-time variable and an envvar to set the proper default for the yaml parser Helmfile uses and the ability to switch the parser at runtime. All in all, the next Helmfile release, v0.150.0 will get reverted to use `gopkg.in/yaml.v2` by default which resolves#435.
New users who started using Helmfile since any of v0.148.0, v0.148.1, and v0.149.0 might be already relying on the new behavior, They might need to specify a new envvar to enable `goccy/go-yaml`.
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: yxxhero <aiopsclub@163.com>
All the dependencies get correctly installed when dealing with remote
charts.
If there's a local chart that depends on remote dependencies then those
don't get automatically installed. See #526. They end up with this
error:
```
Error: no cached repository for helm-manager-b6cf96b91af4f01317d185adfbe32610179e5246214be9646a52cb0b86032272 found. (try 'helm repo update'): open /root/.cache/helm/repository/helm-manager-b6cf96b91af4f01317d185adfbe32610179e5246214be9646a52cb0b86032272-index.yaml: no such file or directory
```
One workaround for that would be to add the repositories from the local
charts. Something like this:
```
cd local-chart/ && helm dependency list $dir 2> /dev/null | tail +2 | head -n -1 | awk '{ print "helm repo add " $1 " " $3 }' | while read cmd; do $cmd; done
```
This however is not trivial to parse and implement.
An easier fix which I did here is just to not allow doing
`--skip-refresh` for local repositories.
Fixes#526
Signed-off-by: Indrek Juhkam <indrek@urgas.eu>
Signed-off-by: Indrek Juhkam <indrek@urgas.eu>
Signed-off-by: yxxhero <aiopsclub@163.com>
1. only implement post-renderer flags this patch
2. As mumoshu advise, add helmfile flags `--post-render` and add the
postRenderer config in helmDefaults and release. the priority is
helmfile flags > release > helmDefaults.
3. fix the test case in state_test.go and some other tests.
Signed-off-by: guofutan <guofutan@tencent.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Allow configuring the lockfile in the state. This makes it possible for
example maintain a lock per environment.
Signed-off-by: Lassi Pölönen <lassi.polonen@iki.fi>
Signed-off-by: Lassi Pölönen <lassi.polonen@iki.fi>
I had been unhappy with the fact that our go-test output had a lot of debug log messages which obfuscated test results.
I'm finally removeing all those by directing the test log output to io.Discard.
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Most shells do not support hyphens in environment variables.
However, there are cases where you may want to include hyphens in the repository name.
Therefore, I have included a process in `gatherOCIUsernamePassword` to replace hyphens with underbar.
Signed-off-by: mugioka <okamugi0722@gmail.com>
* feat: show live output from the Helm binary
Signed-off-by: Rodrigo Fior Kuntzer <rodrigo@miro.com>
* fixup! Merge branch 'main' into enable-live-output
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
* add interactive in sync & remove --interactive in global options
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix unittest
Signed-off-by: yxxhero <aiopsclub@163.com>
* same behave as apply when in interactive
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
If remote file isn't present in repo we will add it anyway to the files list and check `if len(files) == 0 {` never will be true. That leads to missing section with `MissingFileHandler`.
That fix check that cloned file actually exists. In that case if we add a link to non-existing remote file `MissingFileHandler` will be called as expected.
Signed-off-by: Vladimir Kuznichenkov <kuzaxak.tech@gmail.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* introduce DISABLE_INSECURE_FEATURES to disable insecure executions
Signed-off-by: Quan TRAN <account@itscaro.me>
* disable remote sources when DISABLE_INSECURE_FEATURES is set to "true"
Signed-off-by: Quan TRAN <account@itscaro.me>
* refactor envvar package
Signed-off-by: Quan TRAN <account@itscaro.me>
* (test) fix test fixtures
Signed-off-by: Quan TRAN <account@itscaro.me>
* use absolute path to avoid unit test failure
Signed-off-by: Quan TRAN <account@itscaro.me>
* Fix conflicts
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
* Adds feature to fetch environment values from remote
The releases and environment section allow for values files on the local
disk.
This enhancement allows for referencing remote (go-getter) files to be
fetched, cached and referenced.
In addition when fetching a remote git source with a ssh key the ssh key
will not be part of the caching folder name. This avoids two problems:
1. Don't leak sensitive information in the name of the caching folder
2. Base64 encoded SSH keys are very long. On some file systems the max
lenght of the directory name is hit when using the full base64
information in the path name.
The sshkey informations are reducted. Because of this fixed string
there is a change of colloding cache names. The likelihood of this
collision is very low. The git repo and git reference need to be the
same, but the sshkey can change. This will result in the same source to
be checkout out and referenced.
Signed-off-by: Lüchinger Dominic <dev@snowgarden.ch>
* Update pkg/state/storage.go
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
Since helm-diff has added an ability to auto-detect the term to decide if it should output with color or not, helmfile had been defaulted to no-color.
This resoloves that, by adding a term-detection logic that is same as helm-diff.
As a part of this work, I have also implemented a new global flag `--color`, which is used for forcing color without relying on the term-detection logic implemented in helmfile or explicitly setting the HELM_DIFF_COLOR envvar. I hope it is useful for folks.
Ref https://github.com/roboll/helmfile/issues/2043
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
When chartify is involved due to the use of `forceNamespace`, `strategicMergePatches`, `jsonPatches`, and so on, We had been internally mutating the Release.Chart with the path to the local temporary directory that contains the modified version of the chart.
This resulted in us unintentionally making `helmfile deps` to remove entries for the chart being modified out of helmfile.lock file, which resulted in issues like #2110.
To be clear, although the original issue is reported to occur for `strategicMergePatches`, I believe that it occurered also for any remote charts using `jsonPatches` and `forceNamespace` too.
I also believe this has been the issue since our introduction of chartify (maybe a year or so ago??), and I guess why it took so much time to be found and reported is that not so many people with chartify in combination with `helmfile deps` 🤔
Lastly, this changes chart names surfaced in the various log output from Helmfile, from temporary chart paths to the chart name/path declared in the helmfile.yaml. I think this is generally a good change, no fear of being a breaking change. But if anyone has any concern about that, please feel free to comment/report/etc.
Ref https://github.com/roboll/helmfile/issues/2110
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
This adds support for `kube-version` and `api-versions` to be available to `chartify` so that it works even if your release requires `chartify` due to that you use features like `forceNamespace`, `jsonPatches`, `strategicMergePatches`, and so on.
This also enhances `ReleaseSpec` which corresponds to each item of `releases[]` in your `helmfile.yaml` to also accept `kubeVersion` and `apiVersions`, in addition to the top-level `kubeVersion` and `apiVersions` we have today.
The top-level ones works as the default values for release-specific ones. If you have been using the top-level ones, keep using it. It is backward-compatible. If you want to specify it per release, because, for example, your releases are deployed across clusters(in case you differentiate `kubeContext` fields), try the new fields added to the release spec.
Resolves#1864
It turned out that Helmfile has never had support for release template on `needs`.
This adds that, along with the new end-to-end test suite to verify helmfile template output with snapshot testing involving a real `helmfile build` command.
Ref #2098
Use the value of the `condition` field instead of the `installed` field of a release in the `enabled` column of helmfile list.
The value of the `installed` field is shown in a new `installed` column.
Fixes#1920
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
Parses a new field in repositories named `skipTLSVerify` and if set to `true`, it appends `--insecure-skip-tls-verify` in `helm repo add` command.
This should be useful with internal self-signed repos, mitm proxies etc.
Resolves#1871
* Do fail on a possible typo in `needs` entries
Helmfile kindly fails with a friendly error when you made a typo in a `needs` entry, i.e. a `needs` entry included a reference to a release that is not defined in the helmfile config.
Example Output:
```
in ./helmfile.needs.yaml: release(s) "app" depend(s) on an undefined release "infrastructure/cert-manager2". Perhaps you made a typo in `needs` or forgot defining a release named "cert-manager2" with appropriate `namespace` and `kubeContext`?
```
This prevents issues like #1959
* Fix regression in helmfile-diff (This may break when you had two or more duplicated releases that are intended to be de-duplicated before DAG calculation using selectors
* Fix regression when you used selector to deduplicate releases before DAG calculation
* Comments
* Fix regressions in helmfile-apply and helmfile-sync
* Fix regression in duplicate release detection
Currently it's not possible to use `.Environment` values in `*.gomtpl` files. The documentation states the opposite:
https://github.com/roboll/helmfile#environment (2nd paragraph).
The problem is already described in #1090.
This PR fixes this bug.
Fixes#1090
Co-authored-by: Peter Aichinger <petera@topdesk.com>
Apparently we needed to pass `--validate` on helm-template run by chartify when the targeted chart contains Capabilities.APIVersions in a chart template. Otherwise, you can never make such chart work with chartify, as at apply time helm template expressions that involved Capabilities.APIVersions are already nowhere.
This adds the ability to include the --pass-credentials flag to the helm add repo command by:
- Adding repo.passCredentials to the helmfile yaml
- Changing state, helmexec, and app to include RepositorySpec.PassCredentials
Resolves#1898
Co-authored-by: almed4 <alexandre.meddin@ingka.ikea.com>
#1772 broke `--selector` with `needs` in many ways.
The two biggest problems I've encountered were:
- duplicate releases even if you've provided a proper `selector` to deduplicate
- sync/deletion ordering broken when you have `needs`
For the first issue, we had to update `getSelectedReleases` function to also calculate the "selected releases and releases needed by the selected releases", and use that to calculate the DAG. That should have been done in #1772.
The latter started happening after I've fixed the first issue. The source of the issue was that `needs` turned out to be ambiguous in a few cases.
Previously, `needs: ["foo/bar"]` had two meanings. One for "needs release bar in kubecontext foo", another for "needs release bar in namespace foo".
Moreover, `needs: ["foo/bar/baz"]` had three meanings.
- `needs release baz in tiller namespace foo and namespace baz`
- `needs release baz in namespace bar in kubecontext foo`
- `needs release baz in tiller namespace bar in kubecontext foo`.
Especially, the first meaning doesn't make sense at all. Helm 2 solely use tillerNamespace for namespacing the release and Helm 3 uses namespace for that.
This fix sorts all the bugs and issues I've found so far around that, by changing the meanings of the above two examples as follows:
- `foo/bar` means `namespace=foo,name=bar` for Helm 3 and `tillerNamespace=foo,name=bar` for Helm 2
- `needs release bar in kubecontext foo` is now `foo//bar`. Notice the extra `/` between `foo` and `bar`.
- `foo/bar/baz` means `kubecontext=foo,namespace=bar,name=baz` for Helm 3 and `kubecontext=foo,tillerNamespace=bar,name=baz` in Helm 2
Fixes#1818
yxxhero
xiaomudk
Yusuke Kuoka
Dejan Benedik
SeWieland
William Lahti
Vincent Stchu
Soh Takada
AlexAxenti
Quan TRAN
Mitsuo Heijo
koralsky
Indrek Juhkam
guofutan
koralsky
yxxhero
Lassi Pölönen
Helge Eichelberg
Kevin Leturc
Felipe Santos
mugioka
Anton Bretting
Arpan
Jean-Yves CAMIER
Rodrigo Fior Kuntzer
Viktor Oreshkin
Arkaitz Jimenez
Vladimir Kuznichenkov
Sergii Iefremov
Eng Zer Jun
Sergio Morales
Dominic
austin ce
Quan TRAN
Sören Jentzsch
drivelikebrazil
Christoph Petrausch
Samo Jelovšek
Mårten Svantesson
Babis K
pjotre86
Aditya Menon
James Christian
Alex Meddin
Radon Rosborough
katsew
astorath
katsew