Closes#444 and #782
This is the final PR to fully cache and parallelize helm secret decryption. It threads the shared helmexec.Interface into the StateCreator and HelmState structs to be used during environment secret decryption. This should effectively cache secrets for the duration of a helmfile run, regardless of where they are first decrypted.
* Add kubectl and jq to Dockerfile
Signed-off-by: Jake Hill <jake@naphta.uk>
* Update kubectl to use checksum verification, and pin to a specific version
Signed-off-by: Jake Hill <jake@naphta.uk>
Resolves#792
Related to #782 and #444
- Allows concurrent decryption of different secrets files
- Caches decrypted secrets by original file path and returns decrypted results from memory
- Secrets being run through an instance of helmexec will be cached and run as fast as possible concurrently
NB: This particular PR doesn't make _all_ calls to secrets cached and concurrent. Environment Secrets in particular seem to not be evaluated with a ScatterGather(), and doesn't use the same helmexec instance as other parts of the code, so it doesn't take advantage of these changes. Some reworking of the plumbing there would be needed.
Use with the helm-x support(#673)
This enhances config syntax to accept adopt: [NS/KIND/RESOURCE_NAME] at the release level so that helmfile calls helm-x to transparently import existing resources at the installation time.
Resolves#84
This adds the possibility to use a "=" in the parameters passed to --state-values-set
previous comportment:
`helmfile --state-values-set 'test="abcde==fg",test2="abcde"'` => outputs 'test="abcde",test2="abcde"'
new comportment:
`helmfile --state-values-set 'test="abcde==fg",test2="abcde"'` => outputs 'test="abcde==fg",test2="abcde"'
Resolves#689
This adds a new yaml entry for the hook definition to allow the users to specifcy if they want to show the `command` logs or not.
here is an example.
```
releases:
- name: myapp
chart: mychart
# *snip*
hooks:
- events: ["cleanup"]
showlogs: true
command: "kubectl"
args: ["get", "ingress"]
```
this will display the following output:
```
hook[cleanup] logs | NAME HOSTS ADDRESS PORTS AGE
hook[cleanup] logs | catalog-gateway tdc.foo 80 2d6h
hook[cleanup] logs | dataset foobar.barr.foo.xxxxxxx.com 80 2d6h
hook[cleanup] logs | rating fooba.barr.foo.xxxxxxx.com 80 2d6h
hook[cleanup] logs | sharing foobar.barr.foo.xxxxxxx.com 80 2d6h
hook[cleanup] logs | tpsvc-iam-dev foo.barr.foo.xxxxxxx.com 80 2d6h
hook[cleanup] logs | tpsvc-iam-front bar.barr.foo.xxxxxxx.com 80 2d6h
```
The root cause of this bug was due to that `--kube-context` and `kubeContext` had been treated specifically in code. So on the way I have made it consistent with other per-release settings - by adding `kubeContext` for each release and treating `helmDefaults.kubeContext` as just the default value for per-release setting.
Fixes#674
This enhances helmfile so that it can:
- Treat K8s manifests directories and Kustomize projects as charts
- Add adhoc chart dependencies on sync/diff/template without forking or modifying chart(s) (#649)
- Add adhoc patches(JSON Patch or Strategic Merge Patch supported) to be applied to the K8s resources before sync/diff/template, without forking or modifyin chart(s) (#650)
The usage is as outlined in https://github.com/mumoshu/helm-x/tree/master/examples/helmfile.
Add any or all of `dependencies:`, `jsonPatches:` and `strategicMergePatches:` so that it adds additional flags to `helm` calls that is only supported by `helm x`.
```yaml
releases:
- name: kustomize
chart: ../kustomize
- name: manifests
chart: ../manifests
- name: foo
chart: incubator/raw
dependencies:
- alias: bar
chart: incubator/raw
values:
- values.yaml
- bar:
enabled: true
resources:
- apiVersion: v1
kind: Pod
metadata:
name: bar
spec:
containers:
- command:
- sleep
- 1000
image: alpine:3.9.4
imagePullPolicy: IfNotPresent
name: bar
jsonPatches:
- target:
version: v1
kind: Pod
name: foo
patch:
- op: replace
path: /spec/containers/0/command
value:
- sleep
- "123"
strategicMergePatches:
- apiVersion: v1
kind: Pod
metadata:
name: bar
spec:
containers:
- name: bar
command:
- sleep
- "234"
```
You can alternatively provide `source: path/to/patch.yaml` for `jsonPatches` and `strategicMergePatches` items to externalize it. Add `.gotmpl` suffix like you would do for values files for templating.
When running `helmfile` you must point `--helm-binary` to the `helm-x` binary like below:
```
$ helmfile --helm-binary ~/.helm/plugins/helm-x/bin/helm-x --log-level debug apply
```
after installing the [helm-x](https://github.com/mumoshu/helm-x) plugin.
The integration should ideally be automatic. That is, it shouldn't force you to set `--helm-binary`. But I had no other way to not bloat helmfile's codebase to just add this experimental feature.
Resolves#649Resolves#650
Probably since #647 helmfile has been unable to merge nested maps in environment values if they were loaded from files. This fixes it.
The relevant test is also enhanced so that no further regression like this happens.
Fixes#677
Extends the remote-helmfile feature to also work when loading the first state file.
This should be useful for people who wants to give helmfile a try without ever opening `$EDITOR`.
* fix: persist original file path when using bases
Prior to this change, the resulting lock file was called `<bases[0]>.lock`,
instead of `<filename>.lock`.
This change ensures the final, merged state has the correct `.FilePath`.
* test: Assert proper FilePath in layered HelmState
helm-secrets uses the `HELM_SECRETS_DEC_SUFFIX` env var to define the name of the output file
we should have the same logic in helmfile, to come up with the same filename
It only affects people using the `HELM_SECRETS_DEC_SUFFIX` env var
Use-case: if you want to run multiple `helmfile` commands in parallel, without conflicts. in this case, you need to decrypt secrets with different suffixes.
Resolves#435 (Git as chart repository)
Resolves#220 (S3 as chart repository)
Resolves#436 (About bundling helm plugins)
A lot of thanks to @aslafy-z for authoring the awesome helm-git plugin and contributing it to the community!
This change enhances helmfile to accept terraform-module-like URLs in nested state files a.k.a sub-helmfiles.
```yaml
helmfiles:
- # Terraform-module-like URL for importing a remote directory and use a file in it as a nested-state file
# The nested-state file is locally checked-out along with the remote directory containing it.
# Therefore all the local paths in the file are resolved relative to the file
path: git::https://github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.40.0
```
The URL isn't equivalent to terraform module sources. The difference is that we use `@` to distinguish between (1) the path to the repository and directory containing the state file and (2) the path to the state file being loaded. This distinction provides us enough fleibiity to instruct helmfile to check-out necessary and sufficient directory to make the state file works.
Under the hood, it uses [hashicorp/go-getter](https://github.com/hashicorp/go-getter), that is used for [terraform module sources](https://www.terraform.io/docs/modules/sources.html) as well.
Only the git provider without authentication like git-credentials helper is tested. But theoretically any go-getter providers should work. Please feel free to test the provider of your choice and contribute documentation or instruction to use it :)
Resolves#347
Yusuke Kuoka
Travis Groth
Jake Hill
OlivierB
a-hat
bitsofinfo
Marc Sutter
Naseem
Clément Loiselet
Max Williams
Erik Forsberg
sgandon
刘相轩
Cedric Meury
Eric Bailey
Kevin J. Qiu
Vincent Behar