feat: helmfile.yaml layering enhancements
The current [Layering](https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md#layering) system didn't work as documented, as it relies on helmfile to template each "part" of your helmfile.yaml THEN merge them one by one.
The reality was that helmfile template all the parts of your helmfile.yaml at once, and then merge those YAML documents. In https://github.com/roboll/helmfile/issues/388#issuecomment-436186278, @sruon was making a GREAT point that we may need to change helmfile to render templates earlier - that is to evaluate a template per each helmfile.yaml part separated by `---`. Sorry I missed my expertise to follow your great idea last year @sruon 😭
Anyways, this, in combination with the wrong documentation, has made so many people confused. To finally overcome this situation, here's a fairly large PR that introduces the 2 enhancements:
- `bases:` for easier layering without go template expressions, especially `{{ readFunc "path/to/file" }}`s. This is the first commit of this PR.
- `helmfile.yaml` is splited by the separator `---` at first. Each part is then rendered as a go template(double-render applies as before). Finally, All the results are merged in the order of occurence. I assume this as an enhanced version of @sruon's work. This is the second commit of this PR.
Resolves#388Resolve#584Resolves#585 (`HELMFILE_EXPERIMENTA=true -f helmfile.yaml helmfile` disables the whole-file templating, treating the helmfile.yaml as a regular YAML file as the file ext. denotes. Use `helmfile.yaml.gotmpl` or `helmfile.gotmpl` to enable)
Fixes#568 (Use `bases` or `readFile` rather than not importing implicitly with `helmfile.d`
* Various U/X improvements for `helmfile apply`
This improves the U/X of `helmfile apply`, by allowing you to selectively apply sub-helmfiles.
When you have two or more sub-helmfiles processed, typing `n` to cancel the first doesn't automatically stop the whole helmfile execution.
Instead, it proceeds by diffing the next sub-helmfile, and asks you to apply it, which should be what the user would expect.
To support this workflow, I have suppressed useless exec logs, correct exit status when diff exists in sub-helmfiles but not in the parent helmfile, and made the final error message emitted by helmfile better.
More concretely, this moves more output from `helm` to STDERR and the `debug` log-level.
The overall output from `helmfile` should be a bit more cleaner especially for `apply`, `sync`, `diff` and perhaps other `helmfile` sub-commands, too.
For example, when one of release failed, `helmfile`'s final error message now includes the error message from the failed `helm` execution, like seen in the last line:
```
List of updated releases :
RELEASE CHART VERSION
envoy stable/envoy 1.5.0
List of releases in error :
RELEASE
envoy2
in ./helmfile.yaml: in .helmfiles[0]: in /Users/c-ykuoka/helmfile/helmfile.1.yaml: failed processing release envoy2: helm exited with status 1:
Error: UPGRADE FAILED: "envoy2" has no deployed releases
```
This way you can better understand what caused helmfile to finally fail.
`helmfile` has been streaminig a lot of stdout and stderr contents from the `helm` commands regardless of the helmfile's log-level. It has been suppressed by default and moved to the `debug` log-level.
You will see that it helps you focus on what was the cause of a failure.
While working on the above, I found an another bug that made `--detailed-exitcode` useless in some case.
That is, `helmfile diff --detailed-exitcode`, when any diff existed only in sub-helmfiles, has been returning an exit code of `1`. It should return `2` when any release had diff and no release had an error, regardless of the target is a sub-helmfile or a parent helmfile. Why? Because that's what `--detailed-exitcode` meant for!
After this PR gets merged, `helmfile diff --detailed-exitcode` propery return exit code `2` in such cases.
Fixes#543Resolves#540
Fixes#344 by allowing explicit selectors to be specified for composed helmfiles using the following structure
```yaml
helmfiles:
- path: helmfile.d/a*.yaml
selectors:
- name=prometheus
- name!=zipkin
- helmfile.d/b*.yaml
- path: helmfile.d/c*.yaml
selectors: {}
```
2 modes here :
* legacy mode when no the env var HELMFILE_EXPERIMENTAL is not set to true
* no selector : inherit from the command line.
* selector: is specified then it is used (an emty means no inheritance from command line and take everything).
* experimental when the env var HELMFILE_EXPERIMENTAL=true
* no selector : nothing is inherited from the command line so use all releases.
* selector: is specified then it is used (an emty means no inheritance from command line and take everything).
* feat(report): display summary of upgraded, deleted and error releases
* feat(#502): adds dep target in makefile
* feat(#502): removes vendor and fixes pristine in makefile
Since tillerless support we unintentionally broke this, and there isn't a real fix to this.
We must accept a limitation that helmfile needs a tiller installed on your cluster just for decrypting environment secrets.
Fixes#550
Since #526, `helmfile apply` have been really able to detect deletion of the last release only, and `sync` has been unable to mark releases with `installed: false` for removal.
Fixes#554
Actually, 4 helm commands including "list", "diff", "status" and "delete" were not taking tiller-rerelated flags into account in helmfile. This fixes all these commands.
Fixes#534
* feat: `helmfile destroy` deletes and purges releases
This adds `helmfile destroy` that is basically `helmfile delete --purge`.
I've also tweaked the behavior of `delete` and `destroy` for releases with `installed: false`, so that it becomes consistent with other helmfile commands.
It now delete releases only when `installed: true` AND the release is already installed.
**Why an another command?**
Because it's easy to remember, and it also makes it easier to iterate on your helmfile.
We've been using `helmfile delete` from the beginning of helmfile,
and several months have been passed since we've added `--purge` to it.
We noticed that we always prefer to use `--purge` so that we can quickly iterate on helmfile by
e.g. `helmfile delete --purge && helmfile sync`. But making `--purge` default makes the `delete` command inconsistent with the helm's `delete`.
`destroy`, on the other hand, doesn't have such problem, and is still easy to remember for terraform users.
Resolves#511
* Update docs about `helmfile delete` and `helmfile destroy`
This basically allows to define both `releases` and `helmfiles` within a helmfile.yaml, so that you can start using sub-helmfiles easily, by extracting only reused releases.
Resolves#445
`get` is now able to take one more optional argument, that is used as the default value when the value for the key does not exist.
Resolves#465Fixes#427Fixes#357
Ref #460
This removes `release: "your_release_name" not found` errors seen for releases with `installed: false` when running `helmfile sync` and `helmfile apply`.
The problem was that helmfile had been running `helm status` to detect releases to be deleted. helmfile now use `helm list ^YOUR_RELEASE_NAME$` to detect if the release is currently installed or not, which emits no error-like logs on against uninstalled releases.
Fixes#507Fixes#507
Options specified in releases (e.g. `recreatePods`) should override the respective options set in `helmDefaults`. Currently, `helmDefaults` takes precedence.
In the below example, `--force` should not be passed as an additional deployment argument:
```
helmDefaults:
force: true
releases:
- name: example
namespace: example
chart: some/repo
version: ~1.24.1
force: false
```
Fixes#492
KUOKA Yusuke
Raj Perera
prakharrr-sl
sgandon
Andrey Afoninsky
Aaron Gershman
Facundo Guerrero
Patrick Valsecchi
Erik Osterman
Mike Eves
Dan O'Brien