Probably since #647 helmfile has been unable to merge nested maps in environment values if they were loaded from files. This fixes it.
The relevant test is also enhanced so that no further regression like this happens.
Fixes#677
Extends the remote-helmfile feature to also work when loading the first state file.
This should be useful for people who wants to give helmfile a try without ever opening `$EDITOR`.
* fix: persist original file path when using bases
Prior to this change, the resulting lock file was called `<bases[0]>.lock`,
instead of `<filename>.lock`.
This change ensures the final, merged state has the correct `.FilePath`.
* test: Assert proper FilePath in layered HelmState
helm-secrets uses the `HELM_SECRETS_DEC_SUFFIX` env var to define the name of the output file
we should have the same logic in helmfile, to come up with the same filename
It only affects people using the `HELM_SECRETS_DEC_SUFFIX` env var
Use-case: if you want to run multiple `helmfile` commands in parallel, without conflicts. in this case, you need to decrypt secrets with different suffixes.
Resolves#435 (Git as chart repository)
Resolves#220 (S3 as chart repository)
Resolves#436 (About bundling helm plugins)
A lot of thanks to @aslafy-z for authoring the awesome helm-git plugin and contributing it to the community!
This change enhances helmfile to accept terraform-module-like URLs in nested state files a.k.a sub-helmfiles.
```yaml
helmfiles:
- # Terraform-module-like URL for importing a remote directory and use a file in it as a nested-state file
# The nested-state file is locally checked-out along with the remote directory containing it.
# Therefore all the local paths in the file are resolved relative to the file
path: git::https://github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.40.0
```
The URL isn't equivalent to terraform module sources. The difference is that we use `@` to distinguish between (1) the path to the repository and directory containing the state file and (2) the path to the state file being loaded. This distinction provides us enough fleibiity to instruct helmfile to check-out necessary and sufficient directory to make the state file works.
Under the hood, it uses [hashicorp/go-getter](https://github.com/hashicorp/go-getter), that is used for [terraform module sources](https://www.terraform.io/docs/modules/sources.html) as well.
Only the git provider without authentication like git-credentials helper is tested. But theoretically any go-getter providers should work. Please feel free to test the provider of your choice and contribute documentation or instruction to use it :)
Resolves#347
This adds `values` to state files as proposed in #640.
```yaml
values:
- key1: val1
- defaults.yaml
environments:
default:
- values:
- environments/default.yaml
production:
- values:
- environments/production.yaml
```
`{{ .Valuese.key1 }}` evaluates to `val1` if and only if it is not overrode via the production or the default env, or command-line args.
Resolves#640
Seems like we are affected by https://github.com/golang/go/issues/24963. That is, even though we internally use the template option `missingkey=zero`, in some cases it still prints `<no value>` instead of zero values, which has been confusing the state yaml parsing.
This fixes the issue by naively replacing all the remaining occurrences of `<no value>` in the rendered text, while printing debug logs to ease debugging in the future when there is unexpected side-effects introduced by this native method.
Fixes#553
The addition of `--set k1=v1,k2=v2` and `--values file1 --values file2` was originally planned in #361.
But it turned out we already had `--values` for existing helmfile commands like `sync`. Duplicated flags doesn't work, obviously.
So this actually add `--state-values-set k1=v1,k2=v2` and `--set-values-file file1 --set-values-file file2`.
They are called "state" values according to the discussion we had at #640Resolves#361
* feat: helmfile as a go library
This removes almost all the dependencies from the helmfile core logic to urfave/cli. `main.go` is now a thin wrapper around the core logic implemented in `pkg/app`.
I will start by making as much as possible code in `main.go` independent from urfave/cli, and this illustrates how we can do that by introducing a Config interface that delegates any config value fetch to urfave/cli. We will implement an cobra/pflag impl of the interface when helmfile finally migrates to cobra.
We added envvals overrides in the state file via #622 two days ago:
```
helmfiles:
- name: sub.helmfile.yaml
environment:
values:
- mykey: myvalue
```
This change removes the `environment` level in the above cofig, so that it looks like:
```
helmfiles:
- name: sub.helmfile.yaml
values:
- mykey: myvalue
``
This is an inevitable breaking change towards #361. But I wanted to break it earlier so that less folks are affected.`
Ref https://github.com/roboll/helmfile/issues/361#issuecomment-497530819
* feat: specify env values from the parent to the nested state
Adds the `helmfiles[].environment.values` that accepts a mix of file pathes and inline dictes:
```yaml
helmfiles:
- path: path/to/nested/helmfile.yaml
environment:
values:
- key1: val1
- values.yaml
```
The values files are loaded in the context of the parent state file. For example, in case the above state file is located at `/path/to/helmfile.yaml`,
`values.yaml` is located at `/path/to/values.yaml` instead of `/path/to/nested/values.yaml`.
Resolves#523
* fix: multiple "bases" declarations yields duplicate releases
Fixes#615
* fix regression in double-rendering with env value overrides
The latest commit broke any state files like the below to NOT pass env value overrides at all:
```
helmfiles:
- path: nested/state.yaml
environment:
values:
- overrides.yaml
```
This fixes the issue.
```yaml
environments:
default:
missingFileHandler: Warn
values:
- path/to/values.yaml
secrets:
- path/to/secrets.yaml
```
`missingFileHandler` set to `Warn`, `Info`, or `Debug` results in helmfile NOT stop when `path/to/values.yaml` or `path/to/secrets.yaml` is missing.
Resolves#548
While implementing the above feature, I also found a bug that has been causing #559. This also fixes that.
To verify it is actually fixed, create an example helmfile.yaml that looks like the below, and run `helmfile diff`:
```
$ cat helmfile.yaml
environments:
default:
secrets:
- env-secrets.yaml
releases:
- name: myapp
chart: nginx
namespace: default
secrets: [secrets.yaml] # Notice this file does not exist
values:
- ingress:
enabled: true
$ helmfile diff
could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read helmfile.yaml.part.0: environment values file matching "env-secrets.yaml" does not exist
in ./helmfile.yaml: failed to read helmfile.yaml: environment values file matching "env-secrets.yaml" does not exist
```
Fixes#559
- Change exit code from 2 to 3 when helmfiles have no releases that match a selector
- Introduces new flag `--allow-no-matching-release` to exit 0 when no releases match a selector.
Resolves: #597
This enhances helmfile's internal environment values files loader to expand glob patterns (#606)
Fixes the existing bug that helmfile was unable to load environment values file from absolute path (#549)
Resolves#606Fixes#549
`postsync` events are triggered after each release is applied to the cluster in `helmfile sync` or `helmfile apply`.
This should be a best hook to notify only after each sync failed or succeeded. This can be used for running operations like patching K8s resources managed by helm, but that should be the last-resort. Maybe you should fork/update the chart, or submit a feature request to add `replicated/ship` integration to `helmfile` in that case :)
Resolves#599
In order to maintain predictable deployments, as developer I want to generate and use "lock files" for all chart versions retrieved from a helmfile.
This change solves it by (1)enhancing `helmfile deps` to generate a lock file containing all the direct chart dependencies of each helmfile state file and
(2)making other helmfile sub-commands reads the lock file and merge the locked version numbers to the helmfile state file being processed.
The lock file is named after the helmfile state file being locked, so that you can have multiple set of the helmfile state file and the lock file pairs in a directory.
When `helmfile deps` are not explicitly run before commands like `sync`, all the helmfile behavior should remain as before.
Let's say you have `helmfile.1.yaml`:
```
repositories:
- name: stable
url: https://kubernetes-charts.storage.googleapis.com
releases:
- name: envoy
chart: stable/envoy
- name: envoy2
chart: stable/envoy
```
`helmfile deps` generates `helmfile.1.lock` that looks like:
```
dependencies:
- name: envoy
repository: https://kubernetes-charts.storage.googleapis.com
version: 1.5.0
digest: sha256:e43b05c8528ea8ef1560f4980a519719ad2a634658abde0a98daefdb83a104e9
generated: 2019-05-14T16:45:37.78205+09:00
```
Under the hood, `helmfile deps` creates a temporary local helm chart with a dummy `Chart.yaml` and `requirements.yaml` deduced from the `helmfile.yaml` content, then runs `helm dependency update` to produce od update the corresponding `requirements.lock` file.
`helmfile` then renames it to match the name of the targeted helmfile state file and moves it, so that it becomes adjacent to each `helmfile.yaml`.
Other `helmfile` commands like `sync`, `diiff`, `apply`, `lint` read chart version numbers from the lock file.
Resolves#483
feat: helmfile.yaml layering enhancements
The current [Layering](https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md#layering) system didn't work as documented, as it relies on helmfile to template each "part" of your helmfile.yaml THEN merge them one by one.
The reality was that helmfile template all the parts of your helmfile.yaml at once, and then merge those YAML documents. In https://github.com/roboll/helmfile/issues/388#issuecomment-436186278, @sruon was making a GREAT point that we may need to change helmfile to render templates earlier - that is to evaluate a template per each helmfile.yaml part separated by `---`. Sorry I missed my expertise to follow your great idea last year @sruon 😭
Anyways, this, in combination with the wrong documentation, has made so many people confused. To finally overcome this situation, here's a fairly large PR that introduces the 2 enhancements:
- `bases:` for easier layering without go template expressions, especially `{{ readFunc "path/to/file" }}`s. This is the first commit of this PR.
- `helmfile.yaml` is splited by the separator `---` at first. Each part is then rendered as a go template(double-render applies as before). Finally, All the results are merged in the order of occurence. I assume this as an enhanced version of @sruon's work. This is the second commit of this PR.
Resolves#388Resolve#584Resolves#585 (`HELMFILE_EXPERIMENTA=true -f helmfile.yaml helmfile` disables the whole-file templating, treating the helmfile.yaml as a regular YAML file as the file ext. denotes. Use `helmfile.yaml.gotmpl` or `helmfile.gotmpl` to enable)
Fixes#568 (Use `bases` or `readFile` rather than not importing implicitly with `helmfile.d`
* Various U/X improvements for `helmfile apply`
This improves the U/X of `helmfile apply`, by allowing you to selectively apply sub-helmfiles.
When you have two or more sub-helmfiles processed, typing `n` to cancel the first doesn't automatically stop the whole helmfile execution.
Instead, it proceeds by diffing the next sub-helmfile, and asks you to apply it, which should be what the user would expect.
To support this workflow, I have suppressed useless exec logs, correct exit status when diff exists in sub-helmfiles but not in the parent helmfile, and made the final error message emitted by helmfile better.
More concretely, this moves more output from `helm` to STDERR and the `debug` log-level.
The overall output from `helmfile` should be a bit more cleaner especially for `apply`, `sync`, `diff` and perhaps other `helmfile` sub-commands, too.
For example, when one of release failed, `helmfile`'s final error message now includes the error message from the failed `helm` execution, like seen in the last line:
```
List of updated releases :
RELEASE CHART VERSION
envoy stable/envoy 1.5.0
List of releases in error :
RELEASE
envoy2
in ./helmfile.yaml: in .helmfiles[0]: in /Users/c-ykuoka/helmfile/helmfile.1.yaml: failed processing release envoy2: helm exited with status 1:
Error: UPGRADE FAILED: "envoy2" has no deployed releases
```
This way you can better understand what caused helmfile to finally fail.
`helmfile` has been streaminig a lot of stdout and stderr contents from the `helm` commands regardless of the helmfile's log-level. It has been suppressed by default and moved to the `debug` log-level.
You will see that it helps you focus on what was the cause of a failure.
While working on the above, I found an another bug that made `--detailed-exitcode` useless in some case.
That is, `helmfile diff --detailed-exitcode`, when any diff existed only in sub-helmfiles, has been returning an exit code of `1`. It should return `2` when any release had diff and no release had an error, regardless of the target is a sub-helmfile or a parent helmfile. Why? Because that's what `--detailed-exitcode` meant for!
After this PR gets merged, `helmfile diff --detailed-exitcode` propery return exit code `2` in such cases.
Fixes#543Resolves#540
KUOKA Yusuke
sgandon
Eric Bailey
Kevin J. Qiu
Vincent Behar
刘相轩
jonathan cowling msm
bitsofinfo
Raj Perera
Shane Starcher