Commit Graph

63 Commits

Author SHA1 Message Date
KUOKA Yusuke 1e260e4a5e
Fix and enhancement to repository update (#1383)
Changes:

- Prevent Helmfile from unnecessarily running `helm repo add` and `helm repo up` against repositories for unused repositories(repositories of releases filtered out by selector)
- Fixes #1330
2020-07-28 10:17:43 +09:00
KUOKA Yusuke 1e956ae8a5
Fix list failure when patches are used (#1371)
Fixes #1368
2020-07-22 22:33:45 +09:00
KUOKA Yusuke df6489ab43
feat: `helmfile template --output-dir-template` for customizing output dirs (#1357)
This is useful for e.g. removing state file names and their hash values out of output dirs so that it can be used easily in a gitops setup. For example, `--output-dir-template mybasedir/{{.Release.Name}}` produces `mybasedir/RELEASE/CHART/templates/*.yaml` for each release in your helmfile.yaml.
2020-07-16 09:02:26 +09:00
KUOKA Yusuke 3a2a460fe7
Do cleanup decrypted env secrets files (#1304)
* Do cleanup decrypted env secrets files

Resolves #503
2020-06-16 08:59:48 +09:00
Max Audron f16d96bc8f
Add global hooks (#1301)
Changes:

* Add global hooks
* Add top level hooks field to yaml spec
* Add functions for global prepare and cleanup events
* Call global prepare and cleanup events in withPreparedCharts function
* Update README
* Add helmfileCommand variable to withPreparedCharts
  Pass the information on what helmfileCommand has been run down from the
  top level functions through withReposAndPreparedCharts and withPreparedCharts.
2020-06-11 10:05:38 +09:00
Kevin J. Qiu 0f86cc9b87
Add the ability to load a remote environment values file (#1296)
Enables the user to specify a remote path for an environment values file, e.g.,

```yaml
environments:
  cluster-azure-us-west:
    values:
      - git::https://git.company.org/helmfiles/global/azure.yaml?ref=master
      - git::https://git.company.org/helmfiles/global/us-west.yaml?ref=master
  cluster-gcp-europe-west:
    values:
      - git::https://git.company.org/helmfiles/global/gcp.yaml?ref=master
      - git::https://git.company.org/helmfiles/global/europe-west.yaml?ref=master

releases:
  - ...
```

This is particularly useful when you co-locate helmfiles within your project repo but want to reuse the definitions in a global repo.
2020-06-11 10:04:01 +09:00
Ilya Vassyutovich 28e13c3a18
typo: Fix message for detected diff (#1284) 2020-06-06 09:31:25 +09:00
KUOKA Yusuke eff2a7bf84
Fix repo update timing (#1287)
Fixes #1283
2020-05-30 18:01:14 +09:00
KUOKA Yusuke 16288dfa7d
feat: GA of Kustomize and K8s manifests support (#1172)
This is the GA version of the helm-x integration #673 developed last year.

You get all the following benefits without an extra helm plugin:

- Ability to add ad-hoc chart dependencies/aliases, without forking the chart (Fixes #876 )
- Ability to patch resulting K8s resources before installing the helm chart
- Ability to install a kustomization as a chart (Requires `kustomize` binary to be available in `$PATH`
- Ability to install a directory of K8s manifests as a chart
- etc.
2020-05-27 11:42:43 +09:00
Guillaume Perrin 6d5f8c71cb
Allow skipping chart-related operations when `installed: false` (#1233)
Resolves #1232

CHANGES:

* Disable repo update when installed is false

When install is false, we do not need to update the repositories and get
the chart.

Signed-off-by: Guillaume Perrin <guillaume28.perrin@gmail.com>
2020-05-09 14:03:56 +09:00
Victor Noël 923bd54db0
Use namespace for release unicity with helm3 (#1213) (#1235)
This is for #1213 so that the releases are considered scoped to their namespace and not to tiller namespace.

Fixes #1213
2020-05-02 09:35:51 +09:00
nejtr0n a0e30c5193
Correct ordering between releases and sub-helmfiles on destroy/delete (#1234)
Fixes #1192

Co-authored-by: Alexey Shishkin <a.shishkin@directheroes.com>
2020-05-01 09:06:13 +09:00
Marcin Kaciuba b1190508b2
feat(template): added secret template function (#1221)
* feat(tmpl): added fetchSecretValue template function

This adds a tmpl `fetchSecretValue` and `expandSecretRefs` function by:
- Adding:
    - `expandSecretRefs` function in tmpl package that uses vals
    package to fetch secrets
    - `fetchSecretValue` function in tmpl package like below but for
    single string value
    - gomock for tests purpose
- Changing:
    - move init of vals package to function (so the same instance can be used for template values and rendering the whole template)

* doc(secret): added doc how to use new tmpl methods

Added example usage of `fetchSecretValue` and `expandSecretRefs`
2020-04-25 21:10:02 +09:00
Rene Hernandez e0a793b7c5
New output flag for list command (#1215)
* New output flag for list command

Support output as json
Add new formatters file to handle extrac formatting to its own concern
New config interface to support list command specification

* Fix usage message

* Add error handling for formatters
2020-04-18 21:11:12 +09:00
KUOKA Yusuke 870cc03c70
feat: `helmfile diff --detailed-exitcode` should also detect deletions (#1186)
Resolves #499
Resolves #1072
2020-04-10 08:22:33 +09:00
RaymondKYLiu 71635caace
feat: add option `--include-tests` for diff and apply command (#1179)
Co-authored-by: Raymond Liu (RD-TW) <raymond_liu@trend.com.tw>
2020-04-05 17:43:54 +09:00
KUOKA Yusuke 9d7d2de6f5
Fix misleading `helmfile diff` output (#1174)
Fixes #749
2020-04-04 17:39:20 +09:00
KUOKA Yusuke 04c963d050
feat: Complete `helmfile diff` output when diff exists (#1173)
Fixes #874
2020-04-04 16:24:01 +09:00
KUOKA Yusuke 35e5454994
Change the `helmfile list` header "INSTALLED" -> "ENABLED" (#1161)
Resolves #1076
2020-03-29 19:11:58 +09:00
KUOKA Yusuke 69feadc360
feat: `helmBinary` in helmfile.yaml (#1160)
* feat: `helmBinary` in helmfile.yaml

Resolves #1083

* Add regression test for `helmfile destroy`
2020-03-29 17:51:07 +09:00
KUOKA Yusuke bf22502a2f
feat: add helmfile template --validate (#1135)
This adds the ability for helmfile to call `helm template --validate` introduced in helm 3.

fixes #1105
2020-03-05 08:07:02 +09:00
KUOKA Yusuke af44965949
feat: `helmfile --log-level=debug apply --retain-values-files` (#1127)
`--retain-values-files` prevents temporary values files that were passed to Helm commands run by Helmfile for debugging purpose.

With that, you can manually rerun helm commands that were logged when `--log-level=debug` is enabled.

Resolves ##1117
2020-02-28 19:39:01 +09:00
KUOKA Yusuke 0186254e79
feat: --detailed-exitcode for `helmfile apply` (#1120)
Resolves #1113
2020-02-26 21:09:05 +09:00
KUOKA Yusuke f1bdb65777
fix: delete/destroy ordering within directory (#1119)
Fixes #979
2020-02-25 09:46:01 +09:00
Emil 05add478c1
Add option to suppress diff on apply (#1092)
* Add option to suppress diff on apply

Add --supress-diff option on apply. Usable for fresh installs when a
lot of output is produces by diff.

Resolves #458

* fix tests for suppress-diff
2020-02-05 21:29:55 +09:00
Christoph Hösler 9fab6851f0
fix test command for helm 3 (#1080) 2020-02-03 08:41:38 +09:00
KUOKA Yusuke 468b9b659d
Fix random "expansion errors" in large values contained in `values` (#974)
Those are not actually random but would have looked like so. We use an external go pkg `variantdev/vals` to expand urls like `ref+vault://foo/bar` contained in release values into their respective secret values.

There was a bug in `vals` that it tries to expand unintended types of strings which resulted in confusing errors like reported in #973.

`vals` fixed the issue in ba4c7a2987. This commit upgrades `vals` to accomodate that.

Fixes #973
2019-11-17 09:37:45 +09:00
KUOKA Yusuke f466800e1a
Fix regression since 0.90.0 that Helmfile becomes too slow when there are many releases (#964)
* Fix regression since 0.90.0 that Helmfile becomes too slow when there are many releases

Fixes #959

* Ensure that the up-to-date helm-diff is installed and used in integration tests
2019-11-14 20:57:04 +09:00
Yusuke Kuoka a136c46c06 fix the bug that `helmfile template` is unable to render anything when `--namespace` is specified 2019-11-07 21:23:36 +09:00
Yusuke Kuoka e2e4e8440d fix the bug that resulted in `helmfile sync` not delete releases with `--namespace` 2019-11-07 20:48:55 +09:00
Yusuke Kuoka 77082cef58 fix regression that `--namespace` breaks delete/destroy and possibly sync/apply as well
The problem was that `--namespace NS` had been not taken into account while deleting releases, that resulted in releases that should be deleted are not deleted.
2019-11-07 19:53:27 +09:00
Yusuke Kuoka 8d7c79a323 fix: release not found on uninstall through sync/apply
The recent addition of the DAG support(`needs`) and the fixes on it broke the delete-on-sync functionality. And there were two more bugs. One is that it was not correctly running `helm delete` when needed and the another is that it was failing when `--selector` is specified and the releases to delete by sync found, but nothing actually got deleted. This fixes all of them.

Fixes #941
2019-11-07 10:09:54 +09:00
KUOKA Yusuke ed7a6d9051
Port the `needs` fix for `helmfile apply` to `sync`, and make `template` DAG-aware (#940)
This ports the fix for `helfmile apply` to `sync`, so that specifying `--selector` doesn't break `helmfile sync`.

Also make `helmfile template` DAG-aware, so that the manifests are rendered in the order of dependency.

Ref #919
2019-11-06 23:16:57 +09:00
Yusuke Kuoka ebad0ad20f Remove unnecessary code 2019-11-06 22:30:08 +09:00
Yusuke Kuoka 4751613f39 Port the fix for `needs` to `helmfile delete` and `destroy`
This ports the fix for `helfmile apply` to `delete` and `destroy`, so that specifying `--selector` does not break those commands anymore.

Ref #919
2019-11-06 22:27:53 +09:00
KUOKA Yusuke f41fe86452
fix: Fix --selector to not break `needs` (#934) 2019-11-06 17:05:25 +09:00
Jan Möller 6955d07ea8 fix: Set HelmBinary in execer constructor (#928)
This overrides the default helm command, if provided, as soon as possible.
This way it is already used in `visitStates`.


I ran into an issue using `HELM3` and `--helm-binary` together with helm-secrets. 
I previously used tillerless, which i could now remove. This however caused `DecryptSecret` to fail, as it would still use the helm2 binary; because it runs before the first `helm.SetHelmBinary` call.

While helm-secrets is not fully helm3 compatible yet, its uses within helmfile are, and i was able to just install it as a helm3 plugin.
2019-11-02 14:15:33 +09:00
KUOKA Yusuke 3f02b86640
fix: Fix `needs` to work for upgrades and when selectors are provided (#922)
* fix: Fix `needs` to work for upgrades and when selectors are provided

Fixes #919

* Add test framework for `helmfile apply`

* Various enhancements and fixes to the DAG support

- Make the order of upgrades/deletes more deterministic for testability
- Fix the test framework so that we can validate log outputs and errors
- Add more test cases for `helmfile apply`, along with bug fixes.
- Make sure it fails with an intuitive error when you have non-existent releases referenced from witin "needs"
2019-11-02 14:04:16 +09:00
Kyrylo Lebediev 4680010c60 Add integration with Hashicorp Vault, AWS SSM, SecretsManager (#906)
* feat: Add integration with Hashicorp Vault, AWS SSM, SecretsManager

Fields which are rendered: Release.Values, Release.SetValues.Value, Release.SetValues.Values

Example:
```
values:
- foo: ref+vault://mykv/foo?address=http://127.0.0.1:8200#/mykey
set:
- name: xyz
  values:
  - ref+vault://mykv/foo?address=http://127.0.0.1:8200#/mykey3
```

Resolves #881

* feat: Update integration with variantdev/vals

New ref+.\* secret formats are used:
6565695a03 (suported-backends)

Resolves #881
2019-10-25 22:55:26 +09:00
KUOKA Yusuke f79db2ec8d
feat(diff,apply,lint,sync,template): `--set k=v` for setting adhoc chart values (#850)
Resolves #840
2019-09-12 19:24:43 +09:00
Shane Starcher 2a6bd24e3c move context outside to limit duplicate repo updates (#828) 2019-08-31 14:32:48 +09:00
astorath 11d0abba6e feat: Advanced Templating (#823)
1. Added `helmfile build` command to print final state
Motivation: useful for debugging purposes and some CI scenarios

Ref #780 

2. Template interpolation is now recursive (you can cross-reference release fields) like:
```yaml
templates:
  release:
    name: {{`app-{{ .Release.Namespace }}`}}
    namespace: {{`{{ .Release.Labels.ns }}`}}
    labels:
      ns: dev
```
3. Experimental: Added some boolean release fields interpolation in templates:
```yaml
templates:
  release:
    name: {{`app-{{ .Release.Namespace }}`}}
    namespace: dev
    installedTemplate: {{`{{ eq .Release.Namespace "dev" }}`}}
```

Resolves #818

4. Added more template interpolations: Labels, SetValues
5. Added template interpolation for inline Values
6. Added `helmfile list` command to print target releases in simple tabular form
7. Added release names in some `helm` output messages, e.g.: `Comparing release=%v, chart=%v`
2019-08-31 14:31:31 +09:00
Travis Groth a584aeab2e Share helmexec from State Creation (#804)
Closes #444 and #782 

This is the final PR to fully cache and parallelize helm secret decryption.  It threads the shared helmexec.Interface into the StateCreator and HelmState structs to be used during environment secret decryption.  This should effectively cache secrets for the duration of a helmfile run, regardless of where they are first decrypted.
2019-08-15 09:27:55 +09:00
OlivierB 2f9f52033c feat: add --output-dir on template command (#693)
It generates templates in a subdirectory named "stateFileName-stateFileHash-releaseName"
2019-07-11 09:07:46 +09:00
KUOKA Yusuke 65d404b276
feat: `helmfile -f <go-getter url>` (#670)
Extends the remote-helmfile feature to also work when loading the first state file.
This should be useful for people who wants to give helmfile a try without ever opening `$EDITOR`.
2019-06-11 14:28:50 +09:00
KUOKA Yusuke 820abbc06d
feat: remote state files (#648)
This change enhances helmfile to accept terraform-module-like URLs in nested state files a.k.a sub-helmfiles.

```yaml
helmfiles:
- # Terraform-module-like URL for importing a remote directory and use a file in it as a nested-state file
  # The nested-state file is locally checked-out along with the remote directory containing it.
  # Therefore all the local paths in the file are resolved relative to the file
  path: git::https://github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.40.0
```

The URL isn't equivalent to terraform module sources. The difference is that we use `@` to distinguish between (1) the path to the repository and directory containing the state file and (2) the path to the state file being loaded. This distinction provides us enough fleibiity to instruct helmfile to check-out necessary and sufficient directory to make the state file works.

Under the hood, it uses [hashicorp/go-getter](https://github.com/hashicorp/go-getter), that is used for [terraform module sources](https://www.terraform.io/docs/modules/sources.html) as well.

Only the git provider without authentication like git-credentials helper is tested. But theoretically any go-getter providers should work. Please feel free to test the provider of your choice and contribute documentation or instruction to use it :)

Resolves #347
2019-06-04 22:59:54 +09:00
KUOKA Yusuke 1d3f5f8a33
feat: override state(former "enviroment") values via command-line args (#644)
The addition of `--set k1=v1,k2=v2` and `--values file1 --values file2` was originally planned in #361.

But it turned out we already had `--values` for existing helmfile commands like `sync`. Duplicated flags doesn't work, obviously.

So this actually add `--state-values-set k1=v1,k2=v2` and `--set-values-file file1 --set-values-file file2`.

They are called "state" values according to the discussion we had at #640

Resolves #361
2019-06-04 11:03:01 +09:00
KUOKA Yusuke e2d6dc4afa
feat: helmfile as a go library (#639)
* feat: helmfile as a go library

This removes almost all the dependencies from the helmfile core logic to urfave/cli. `main.go` is now a thin wrapper around the core logic implemented in `pkg/app`.
2019-06-04 09:12:00 +09:00
KUOKA Yusuke c68fc5bc50
chore: tidy up pkgs (#636)
for readability and towards potentially making helmfile usable as a go library
2019-06-01 13:36:05 +09:00
KUOKA Yusuke 1226ea6d1a
feat: specify env values from the parent to the nested state (#622)
* feat: specify env values from the parent to the nested state

Adds the `helmfiles[].environment.values` that accepts a mix of file pathes and inline dictes:

```yaml
helmfiles:
- path: path/to/nested/helmfile.yaml
  environment:
    values:
    - key1: val1
    - values.yaml
```

The values files are loaded in the context of the parent state file. For example, in case the above state file is located at `/path/to/helmfile.yaml`,
`values.yaml` is located at `/path/to/values.yaml` instead of `/path/to/nested/values.yaml`.

Resolves #523

* fix: multiple "bases" declarations yields duplicate releases

Fixes #615

* fix regression in double-rendering with env value overrides

The latest commit broke any state files like the below to NOT pass env value overrides at all:

```
helmfiles:
- path: nested/state.yaml
  environment:
    values:
    - overrides.yaml
```

This fixes the issue.
2019-05-29 19:08:51 +09:00