Commit Graph

100 Commits

Author SHA1 Message Date
Yusuke Kuoka a136c46c06 fix the bug that `helmfile template` is unable to render anything when `--namespace` is specified 2019-11-07 21:23:36 +09:00
Yusuke Kuoka e2e4e8440d fix the bug that resulted in `helmfile sync` not delete releases with `--namespace` 2019-11-07 20:48:55 +09:00
Yusuke Kuoka 77082cef58 fix regression that `--namespace` breaks delete/destroy and possibly sync/apply as well
The problem was that `--namespace NS` had been not taken into account while deleting releases, that resulted in releases that should be deleted are not deleted.
2019-11-07 19:53:27 +09:00
Yusuke Kuoka 8d7c79a323 fix: release not found on uninstall through sync/apply
The recent addition of the DAG support(`needs`) and the fixes on it broke the delete-on-sync functionality. And there were two more bugs. One is that it was not correctly running `helm delete` when needed and the another is that it was failing when `--selector` is specified and the releases to delete by sync found, but nothing actually got deleted. This fixes all of them.

Fixes #941
2019-11-07 10:09:54 +09:00
KUOKA Yusuke ed7a6d9051
Port the `needs` fix for `helmfile apply` to `sync`, and make `template` DAG-aware (#940)
This ports the fix for `helfmile apply` to `sync`, so that specifying `--selector` doesn't break `helmfile sync`.

Also make `helmfile template` DAG-aware, so that the manifests are rendered in the order of dependency.

Ref #919
2019-11-06 23:16:57 +09:00
Yusuke Kuoka a0c902d6d1 Remove unnecessary code from pkg/app.Run 2019-11-06 22:33:05 +09:00
Yusuke Kuoka ebad0ad20f Remove unnecessary code 2019-11-06 22:30:08 +09:00
Yusuke Kuoka 4751613f39 Port the fix for `needs` to `helmfile delete` and `destroy`
This ports the fix for `helfmile apply` to `delete` and `destroy`, so that specifying `--selector` does not break those commands anymore.

Ref #919
2019-11-06 22:27:53 +09:00
KUOKA Yusuke f41fe86452
fix: Fix --selector to not break `needs` (#934) 2019-11-06 17:05:25 +09:00
Yusuke Kuoka fbbd7630e7 v3 compatibility: Fix `helmfile delete` to not fail when there was no release to be deleted
`helmfile delete` is designed to be idempotent. That is, it is safe to be run when there is nothing to delete. A change in helm v3 broke that behavior. This enhances Helmfile to be able to behave the same for helm v2 and v3.
2019-11-02 15:52:07 +09:00
Yusuke Kuoka 30751e94f7 Fix `helmfile --namespace ns delete` to not fail consistently
Until now `helmfile delete` with helm v3 had been failing when `--namespace` is provided. Helmfile was missing the fact that in helm v3 namespace must be passed instead of tiller-namespace to scope releases to be deleted.

```
err: release "b-1" failed: helm3rc2 exited with status 1:
  Error: uninstall: Release not loaded: b-1: release: not found
in ./helmfile.yaml: release "b-1" failed: helm3rc2 exited with status 1:
  Error: uninstall: Release not loaded: b-1: release: not found
```
2019-11-02 15:52:07 +09:00
Jan Möller 6955d07ea8 fix: Set HelmBinary in execer constructor (#928)
This overrides the default helm command, if provided, as soon as possible.
This way it is already used in `visitStates`.


I ran into an issue using `HELM3` and `--helm-binary` together with helm-secrets. 
I previously used tillerless, which i could now remove. This however caused `DecryptSecret` to fail, as it would still use the helm2 binary; because it runs before the first `helm.SetHelmBinary` call.

While helm-secrets is not fully helm3 compatible yet, its uses within helmfile are, and i was able to just install it as a helm3 plugin.
2019-11-02 14:15:33 +09:00
KUOKA Yusuke 3f02b86640
fix: Fix `needs` to work for upgrades and when selectors are provided (#922)
* fix: Fix `needs` to work for upgrades and when selectors are provided

Fixes #919

* Add test framework for `helmfile apply`

* Various enhancements and fixes to the DAG support

- Make the order of upgrades/deletes more deterministic for testability
- Fix the test framework so that we can validate log outputs and errors
- Add more test cases for `helmfile apply`, along with bug fixes.
- Make sure it fails with an intuitive error when you have non-existent releases referenced from witin "needs"
2019-11-02 14:04:16 +09:00
Tom Duffield 464e6bc782 Run vals against values files that contain the syntax (#920)
If we specify ref+ syntax in a values file, run vals against that file.

Signed-off-by: Tom Duffield <tom@chef.io>
2019-10-31 08:48:30 +09:00
Yusuke Kuoka fc0a7b68a8 fix unexpectedly high concurrency since the dag support 2019-10-28 13:25:25 +09:00
Yusuke Kuoka 88caa7514f fix potential race in sync 2019-10-28 13:24:27 +09:00
KUOKA Yusuke 7666e95690
feat: Add `needs: [NS/NAME]` for controlling installation/deletion order declaratively (#914)
Introduces DAG-aware installation/deletion ordering to Helmfile.

`needs` controls the order of the installation/deletion of the release:

```yaml
relesaes:
- name: somerelease
  needs:
  - [TILLER_NAMESPACE/][NAMESPACE/]anotherelease
```

All the releases listed under `needs` are installed before(or deleted after) the release itself.

For the following example, `helmfile [sync|apply]` installs releases in this order:

1. logging
2. servicemesh
3. myapp1 and myapp2

```yaml
  - name: myapp1
    chart: charts/myapp
    needs:
    - servicemesh
    - logging
  - name: myapp2
    chart: charts/myapp
    needs:
    - servicemesh
    - logging
  - name: servicemesh
    chart: charts/istio
    needs:
    - logging
  - name: logging
    chart: charts/fluentd
```

Note that all the releases in a same group is installed concurrently. That is, myapp1 and myapp2 are installed concurrently.

On `helmdile [delete|destroy]`, deleations happen in the reverse order.

That is, `myapp1` and `myapp2` are deleted first, then `servicemesh`, and finally `logging`.

Resolves #715
2019-10-28 12:57:25 +09:00
Kyrylo Lebediev 4680010c60 Add integration with Hashicorp Vault, AWS SSM, SecretsManager (#906)
* feat: Add integration with Hashicorp Vault, AWS SSM, SecretsManager

Fields which are rendered: Release.Values, Release.SetValues.Value, Release.SetValues.Values

Example:
```
values:
- foo: ref+vault://mykv/foo?address=http://127.0.0.1:8200#/mykey
set:
- name: xyz
  values:
  - ref+vault://mykv/foo?address=http://127.0.0.1:8200#/mykey3
```

Resolves #881

* feat: Update integration with variantdev/vals

New ref+.\* secret formats are used:
6565695a03 (suported-backends)

Resolves #881
2019-10-25 22:55:26 +09:00
Jarryd Tilbrook 4294a5de06 feat: Enhance postsync event to include `.Error` (#882)
* Refactor postsync event to include success flag

* Change success boolean to error

Fixes #826
2019-10-17 08:27:12 +09:00
KUOKA Yusuke fdca10678b
fix(helm3): `Error: unknown flag: --purge` on deletion via `helmfile sync` (#890)
Fixes #888
2019-10-10 16:54:15 +09:00
Aaron Batilo 921f69bae7 Warn users when no repositories are defined (#879)
At the moment, if you have a helmfile.yaml like so:
```
releases:
  - name: metrics-server
    namespace: kube-system
    chart: stable/metrics-server
```

If you try to run `helmfile deps`, you will get a 0 exit code and no log
output at whatsoever, signaling that there weren't any problems, but no
lock file will get created.

For example:
```
root@316073d4a104:/# helmfile deps
root@316073d4a104:/#
```
This behavior doesn't appear to be documented and is unintuitive to the
user.

This change adds a warning output for this same use case:
```
root@316073d4a104:/# helmfile deps
There are no repositories defined in your helmfile.yaml.
This means helmfile cannot update your dependencies or create a lock file.
See https://github.com/roboll/helmfile/issues/878 for more information.
root@316073d4a104:/#
```

Fixes #878
2019-09-29 16:11:06 +09:00
KUOKA Yusuke 2e98e907b0
fix: invalid duration passed to helm 3 upgrade (#864)
Fixes #863
2019-09-19 08:30:29 +09:00
Mike Splain b762ab0b78 Fix delete/destroy (#859) 2019-09-17 15:04:04 +09:00
Theo Meneau 216c228c0b feat: `helm repo add --ca-file` via repositories definition (#856)
Resolves #855
2019-09-14 06:23:54 +02:00
KUOKA Yusuke ef63a05513
fix(helm3): delete/destroy/apply/sync unable to detect releases to be deleted (#857)
Fixes #853
2019-09-14 06:23:22 +02:00
art kon 06b0c99a0b Fix recursion for helmfiles pulled from git (#854) 2019-09-14 05:49:59 +02:00
eddycharly fd0133e10a Update documentation and tests for .Values (#839)
Resolves #816
2019-09-14 05:35:16 +02:00
KUOKA Yusuke 9d851cda3b
feat: `--skip-repos` for `helmfile deps` (#851)
Resolves #661
2019-09-12 19:33:18 +09:00
KUOKA Yusuke f79db2ec8d
feat(diff,apply,lint,sync,template): `--set k=v` for setting adhoc chart values (#850)
Resolves #840
2019-09-12 19:24:43 +09:00
KUOKA Yusuke 94a6fcfb9f
feat(diff,apply): --context=N for limiting diff context (#849)
Resolves #787
2019-09-12 18:36:13 +09:00
KUOKA Yusuke fb2041555e
feat(diff,apply): --no-color for removing color from output (#848)
Resolves #788
2019-09-12 18:32:30 +09:00
刘相轩 cbf5b8b1e7 Fix helm2 lock file does not get updated (#847)
Ref: https://github.com/helm/helm/issues/2731
2019-09-12 17:58:16 +09:00
Shane Starcher 5488198d6d fix: allow empty pattern matching and move on (#827)
Ref #778
2019-09-11 12:47:00 +09:00
KUOKA Yusuke 4bbd09ccb2
fix(cmd/deps): make `helmfile deps` to work w/ helm 3.0.0-beta.3 (#842)
Ref https://github.com/roboll/helmfile/issues/668#issuecomment-529054165
2019-09-07 22:06:29 +09:00
KUOKA Yusuke 4e4f1bee59
feat: Experimental Helm v3 mode (#841)
Set `HELMFILE_HELM3=1` and run `helmfile` like `HELMFILE_HELM3=1 helmfile ...`.

When `HELMFILE_HELM3` is set, `test`, `template`, `delete`, `destroy` behave differently so that it works with Helm 3.

Note that `helmfile diff` doesn't work as `helm-diff` called under the hood doesn't support Helm v3 yet.

Ref #668
2019-09-07 09:40:44 +09:00
KUOKA Yusuke 4bc99337b2
Make advanced templating backward-compatible (#833)
This is a follow-up for #823
Ref https://github.com/roboll/helmfile/pull/823#discussion_r319712283
2019-08-31 22:37:46 +09:00
Shane Starcher 2a6bd24e3c move context outside to limit duplicate repo updates (#828) 2019-08-31 14:32:48 +09:00
astorath 11d0abba6e feat: Advanced Templating (#823)
1. Added `helmfile build` command to print final state
Motivation: useful for debugging purposes and some CI scenarios

Ref #780 

2. Template interpolation is now recursive (you can cross-reference release fields) like:
```yaml
templates:
  release:
    name: {{`app-{{ .Release.Namespace }}`}}
    namespace: {{`{{ .Release.Labels.ns }}`}}
    labels:
      ns: dev
```
3. Experimental: Added some boolean release fields interpolation in templates:
```yaml
templates:
  release:
    name: {{`app-{{ .Release.Namespace }}`}}
    namespace: dev
    installedTemplate: {{`{{ eq .Release.Namespace "dev" }}`}}
```

Resolves #818

4. Added more template interpolations: Labels, SetValues
5. Added template interpolation for inline Values
6. Added `helmfile list` command to print target releases in simple tabular form
7. Added release names in some `helm` output messages, e.g.: `Comparing release=%v, chart=%v`
2019-08-31 14:31:31 +09:00
Yusuke Kuoka cd5d906afb fix: clean up invalid remote state file cache
Fixes #815
2019-08-24 09:47:49 +09:00
Travis Groth a584aeab2e Share helmexec from State Creation (#804)
Closes #444 and #782 

This is the final PR to fully cache and parallelize helm secret decryption.  It threads the shared helmexec.Interface into the StateCreator and HelmState structs to be used during environment secret decryption.  This should effectively cache secrets for the duration of a helmfile run, regardless of where they are first decrypted.
2019-08-15 09:27:55 +09:00
Travis Groth 765bfe6cfd Handle environment secrets concurrently
Ref #782
2019-08-13 09:23:29 +09:00
Travis Groth 63a337ee57 Base concurrency on items parameter (#798)
Fixes #793
2019-08-08 10:20:05 +09:00
Travis Groth 6baad71b1f Cache secrets and concurrent decryption (#790)
Related to #782 and #444 

- Allows concurrent decryption of different secrets files
- Caches decrypted secrets by original file path and returns decrypted results from memory
- Secrets being run through an instance of helmexec will be cached and run as fast as possible concurrently

NB: This particular PR doesn't make _all_ calls to secrets cached and concurrent.  Environment Secrets in particular seem to not be evaluated with a ScatterGather(), and doesn't use the same helmexec instance as other parts of the code, so it doesn't take advantage of these changes.  Some reworking of the plumbing there would be needed.
2019-08-07 23:00:19 +09:00
OlivierB bce2f4728b fix: pass namespace to helm template command (#771)
Resolves #770
2019-07-30 10:44:42 +09:00
a-hat e5038fb04f print yaml content on error in fromYaml (#765) 2019-07-30 10:43:30 +09:00
KUOKA Yusuke eda961edc2
fix: --state-values-set not setting more than first child in nested paths (#774)
Fixes #773
2019-07-27 22:57:54 +09:00
KUOKA Yusuke b2a6231dcf
fix: absolute chart path (#753)
Resolves #743
2019-07-12 22:37:54 +09:00
Marc Sutter 4166b418c9 feat: Adopt existing K8s resources as Helm release on install (#746)
Use with the helm-x support(#673)

This enhances config syntax to accept adopt: [NS/KIND/RESOURCE_NAME] at the release level so that helmfile calls helm-x to transparently import existing resources at the installation time.

Resolves #84
2019-07-11 09:13:34 +09:00
OlivierB 2f9f52033c feat: add --output-dir on template command (#693)
It generates templates in a subdirectory named "stateFileName-stateFileHash-releaseName"
2019-07-11 09:07:46 +09:00
KUOKA Yusuke 63b5040ec4
fix: `fatal error: concurrent map read and map write` on concurrency > 1 (#742)
This is a revised version of 41e44f74a3, which doesn't make CI angry and a more straight-forward(yet smelling) implementation

Fixes #737
2019-07-05 09:20:16 +09:00