Commit Graph

87 Commits

Author SHA1 Message Date
Yusuke Kuoka dbc40f68b8
Make release template to work on `needs` (#2099)
It turned out that Helmfile has never had support for release template on `needs`.
This adds that, along with the new end-to-end test suite to verify helmfile template output with snapshot testing involving a real `helmfile build` command.

Ref #2098
2022-03-10 18:42:55 +09:00
Graeme Gillies 8bdf377be0
Bump sprig library to v3.2.1 (#1880)
Fixes #1646

This will allow us to use the new `fromJson` and `mustFromJson` functions
among others.

Co-authored-by: Graeme Gillies <ggillies@gitlab.com>
2022-01-10 17:04:08 +09:00
Simon Caron 578c559640
Bumped go-getter to v1.5.9 (#1982)
Co-authored-by: Simon Caron <simon.caron@protonmail.com>
2022-01-07 09:00:29 +09:00
Ian Muge 554cbc19f5
upgrade vals allowing for the use of the AzureRM terraform backend (#2029) 2022-01-07 08:57:33 +09:00
Yusuke Kuoka 9efb7afb47
Do fail on a possible typo in `needs` entries (#2026)
* Do fail on a possible typo in `needs` entries

Helmfile kindly fails with a friendly error when you made a typo in a `needs` entry, i.e. a `needs` entry included a reference to a release that is not defined in the helmfile config.

Example Output:

```
in ./helmfile.needs.yaml: release(s) "app" depend(s) on an undefined release "infrastructure/cert-manager2". Perhaps you made a typo in `needs` or forgot defining a release named "cert-manager2" with appropriate `namespace` and `kubeContext`?
```

This prevents issues like #1959

* Fix regression in helmfile-diff (This may break when you had two or more duplicated releases that are intended to be de-duplicated before DAG calculation using selectors

* Fix regression when you used selector to deduplicate releases before DAG calculation

* Comments

* Fix regressions in helmfile-apply and helmfile-sync

* Fix regression in duplicate release detection
2021-12-18 17:44:55 +09:00
Yusuke Kuoka f57b510d01
Fix json patches and strategic patches to do work when there was only one resource in the targeted chart/directory (#1962)
Ref https://github.com/roboll/helmfile/issues/1954
2021-09-14 14:48:20 +09:00
Yusuke Kuoka e1cabc82e3
Add support for --validate on chartify (#1960)
Apparently we needed to pass `--validate` on helm-template run by chartify when the targeted chart contains Capabilities.APIVersions in a chart template. Otherwise, you can never make such chart work with chartify, as at apply time helm template expressions that involved Capabilities.APIVersions are already nowhere.
2021-09-14 04:48:21 +09:00
Yusuke Kuoka c623730374
Bump golang.org/x/crypto (#1877)
Resolves #1876
2021-06-10 09:16:11 +09:00
Yusuke Kuoka 72e71608b1
Fix chartify regression of missing chart dependencies (#1869)
* Fix chartify regression of missing chart dependencies

Fixes https://github.com/roboll/helmfile/issues/1867

* Add integration test cases for issues #1857 and #1867
2021-06-08 13:06:26 +09:00
Yusuke Kuoka 1a109f7400
Fix chartify inability to disable chart dependency (#1861)
Fixes #1857
2021-05-24 08:31:09 +09:00
Yusuke Kuoka cb7fa6bf63
Bump vals to 0.14.0 (#1832)
This release includes new `ref+tfstates3://` and `ref+tfstategs://` backends for reading terraform states stored in s3 and google cloud storage, and features a new aws-sdk-go that supports AWS SSO, and the fix to allow referencing a local tfstate file by the absolute path.

See https://github.com/variantdev/vals/releases/tag/v0.14.0 for all the changes.
2021-05-08 17:33:37 +09:00
Yusuke Kuoka 204f78c8ff
Fix --set and --set-file flags are not passed to chartify (#1803)
Fixes #1681
Ref 286ed08a27
2021-04-24 22:27:53 +09:00
Yusuke Kuoka 59b91a45a0
Fix adhoc dependencies not working with remote chart (#1802)
Fixes #1680

Ref 8ff30374cb
2021-04-24 21:11:07 +09:00
Yusuke Kuoka ce6a621414
Bump chartify to 0.8.3 to fix it not to break when two or more releases sharing the same chart and the namespace, but kubeContext (#1799)
Fixes #1695
2021-04-24 19:03:13 +09:00
Yusuke Kuoka 28ade19419
Bump chartify to 0.8.2 (#1793)
* Bump chartify to 0.8.2

This version fixes charitfy not to fail when you used the combination of (1)helm 3 and (2)strategicMergePatches/jsonPatches/transformers etc that triggers chartify on (3)a chart that contains CRDs.

See https://github.com/roboll/helmfile/issues/1778#issuecomment-824451990 for details of the issue.

The chartify-side of this fix is 55b23f9e9d

Fixes #1778
2021-04-23 09:53:52 +09:00
Yusuke Kuoka 30e0356ad6
Bump chartify to 0.8.1 (#1787)
This version fixes that chartify not to fail when the target chart misses `templates` directory.
2021-04-23 09:16:59 +09:00
Yusuke Kuoka 5d43b30a7c
Add --{include,skip}-needs to various helmfile commands (#1772)
* Add --{include,skip}-needs to helmfile-sync and helmfile-apply

* Add --include-needs to helmfile-template

* Add TODO related to #1018

* Add a few new test files to cover new functionalities

* Update apply test to incorporate the change that the destroy and sync steps target affected releases only
2021-04-20 23:06:51 +09:00
Yusuke Kuoka 81b04fcb8b Add support for adhoc dependency to local chart
Helmfile has been providing a feature called "adhoc chart dependency" that basially enabled you to add Chart.yaml `dependencies` entry adhocly without forking or modifying the chart.

It was missing the support for using a local chart as the adhoc dependency. This patch adds that.

Usage:

`releases[].dependencies[].chart` is enhanced to accept the fs path to the local chart:

```
releases:
- name: foo
  chart: ./path/to/foo
  dependencies:
  - chart: ./path/to/bar
```

Resolves #1762
2021-04-10 16:09:27 +09:00
Yusuke Kuoka 2ff06a8abd
Fix chartify to take --include-crds into account (#1761)
Fixes #1760
2021-04-08 10:06:34 +09:00
Yusuke Kuoka d703e17239
Fix chartify not to fail on a chart with dependencies (#1759)
Manually verified to work by using the following example from @damoon (https://github.com/roboll/helmfile/issues/1655#issuecomment-801122431):

```
repositories:
  - name: prometheus-community
    url: https://prometheus-community.github.io/helm-charts
releases:
- name: metrics
  namespace: monitoring
  chart: prometheus-community/kube-prometheus-stack
  version: 13.13.0
  jsonPatches:
    - target:
        version: v1
        kind: Service
        name: metrics-kube-prometheus-st-kube-etcd
        namespace: kube-system
      patch:
        - op: replace
          path: /spec/ports/0/port
          value: "2381"
        - op: replace
          path: /spec/ports/0/targetPort
          value: "2381"
```

Before this fix:
```
$ helmfile template
(snip)
COMBINED OUTPUT:
  Error: found in Chart.yaml, but missing in charts/ directory: kube-state-metrics, prometheus-node-exporter, grafana
```

After this fix:

```
$ helmfile template | grep 2381
(snip)
Removing /tmp/chartify718204429/monitoring-metrics-f5479c8b8/kube-prometheus-stack/strategicmergepatches
Removing /tmp/chartify718204429/monitoring-metrics-f5479c8b8/kube-prometheus-stack/kustomization.yaml
Templating release=metrics, chart=/tmp/chartify718204429/monitoring-metrics-f5479c8b8/kube-prometheus-stack
    port: "2381"
    targetPort: "2381"
```

Fixes #1655

Ref 07c3054d66
2021-04-08 07:03:18 +09:00
Yusuke Kuoka 3bfcc1ea42
Add support for kustomize v4 (#1758)
We use kustomize in two places. One for turning kustomize into a chart, and another for patching manifests and the chart.

This fixes the former to work with kustomize v4.

Fixes #1688
2021-04-08 06:15:16 +09:00
Yusuke Kuoka f614e8b1bf
feat: Helmfile renders *.yaml.gotmpl in a K8s manifests/kustomization directory (#1745)
Related to #494

This feature is mostly a built-in alternative to the `incubator/raw` chart without external dependency and has
access to helmfile's own template functions and template data.

The expected use-case of this feature is to add arbitrary K8s resources to your deployment.

Unlike the original issue raised in #494 this doesn't enable you to add arbitary resources to a release. That's another story. But this would be a good foundation for that, too.
2021-04-06 14:22:34 +09:00
astorath aafe74a3e1
Fix/secrets ordering (#1726)
Fixes issue introduced in #790: the order of secrets merged is not defined now, leading to unpredictable results in `helmfile apply`.
2021-03-23 18:00:57 +09:00
Yusuke Kuoka 4e1ecb5890
Bump variantdev/vals to 0.13.0 (#1666)
Ref https://github.com/variantdev/vals/pull/42
2021-02-03 09:00:46 +09:00
Chris Mellard 8212b630ff
fix: fixed up OCI repos so they support paths and URLs that differ from chart names contained there (#1661) 2021-01-28 19:11:12 +09:00
Yusuke Kuoka f7a1d09be7
Bump variantdev/vals to 0.12.0 (#1644)
For Azure KeyVault support for the fix for AWS SSM provider.
2021-01-12 10:20:57 +09:00
Yusuke Kuoka 9b64d65d94
feat: Stabilize helmfile-diff output (#1622)
This finishes the work started at #1619 by introducing the following changes:

- You can now set `HELMFILE_TEMPDIR` and `CHARTIFY_TEMPDIR` as the directory to persist temporary values files rendered by helmfile and temporary charts generated by chartify. Note that chartify is used internally when you use kustomize integration or raw K8s manifests as a chart.
- Helmfile uses the hash sum of the release config and the values file content in the file name of the temporary values file.

  In combination with `HELMFILE_TEMPDIR` and `CHARTIFY_TEMPDIR`, this enables helmfile to use the stable file names for temporary values files, which contribute to stabilize log messages like `Comparing release=appset, chart=PATH/TO/TEMPORARY/CHART`, where the `PATH/TO/TEMPORARY/CHART` had been randomized due to formerly random helmfile tempdir and temporary values file names, and random chartify tempdir.

You can try this feature with a script like:

```
tempdir=$(mktemp -d -t helmfile)

HELMFILE_TEMPDIR=${tempdir} CHARTIFY_TEMPDIR=${tempdir} helmfile diff

rm -rf ${tempdir}
```
2020-12-15 10:24:27 +09:00
Yoann Ciabaud 54eb73b423
Use ghodss/yaml for yaml marshaling & unmarshaling in template (#1556)
Fixes #1555

Co-authored-by: Yoann Ciabaud <yoann@linxo.com>
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
2020-12-13 11:03:20 +09:00
Yusuke Kuoka b910591e1d
Add support for Kustomize Transformers (#1592)
Please see the updated `advanced-features.md` for more details.

This is often used for adding common labels and annotations to any resources rendered from a Helm chart.
2020-11-19 09:33:05 +09:00
Yusuke Kuoka f6bf885fb7
Fix `helmfile template --include-crds` not to break with `chartify` (#1583)
* Fix `helmfile template --include-crds` not to break with `chartify`

This bumps variantdev/chartify to 0.4.9 so that we can incorporate fed8bb2953

* Fix integration test
2020-11-12 09:26:43 +09:00
Yusuke Kuoka 709913b5ea
Bump dependencies (#1573)
Bumps dependencies mainly to fix possible go mod error that is fixed via 0edd534322, but also bumps all the other dependencies because they looked very outdated :)
2020-11-09 08:50:00 +09:00
Yusuke Kuoka 0663831dd5
Disable dependency update while running helm-x/chartify in more cases (#1548)
`helm dep up` is now skipped while running helm-x/chartify when the chart/directory is obtained by running go-getter, or `skipDeps` is configured using a command-line flag, helmDefaults, or release configuration.

Resolves #1547
2020-10-25 11:25:59 +09:00
Yusuke Kuoka 080080f57f
Bump chartify to 0.4.4 (#1540)
To incorporate the fix for unconventional chart templates like seen in the Datadog chart 5443ca1a1d
2020-10-15 10:17:37 +09:00
Yusuke Kuoka 563fce4adf
Bump vals (#1537)
To incorporate unexpected SSM secret exposure issue when the parameter version is specified.
See https://github.com/variantdev/vals/pull/37
2020-10-14 09:56:17 +09:00
Yusuke Kuoka 34acf14d7c
Bump chartify to v0.4.3 (#1528)
This fixes the bug that generated Chart.yaml misses the `apiVersion` field which resulted in `helm lint` always failing on the generated chart.

Fixes #1527
2020-10-12 09:23:58 +09:00
Wi1dcard 5d8eba9b29
Append --force-update for specific helm versions. (#1494)
* Parse and process helm version using github.com/Masterminds/semver/v3.

* Add --force-update only when Helm version >= 3.3.2, < 3.3.4.

See: https://github.com/helm/helm/pull/8777.

* Add test cases.
2020-10-12 09:20:55 +09:00
Yusuke Kuoka 5dd65e8d9a
Bump vals (#1475)
Ref https://github.com/variantdev/vals/issues/34
2020-09-14 20:37:51 +09:00
Johannes Alkjær 9d2c0d4285
Bump sprig to v3.1.0 and mergo 3.11 (#1456)
* Bump sprig to v3.1.0
test for mergeOverwrite

* Let mergo not (accidentally) try to merge unexported fields

This is also a good chance separate `HelmState` with the config loaded from YAML, which I had been wanting to do for a long time.

Co-authored-by: Johannes Alkjær <johannes.alkjaer@wunderman.com>
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
2020-09-04 09:58:54 +09:00
Yusuke Kuoka efd26f288a
Bump sprig to v3 (#1452)
Resolves #1294
Resolves #1447
2020-09-01 19:10:29 +09:00
Yusuke Kuoka 94e01b79e0
Bump chartify to 0.4.2 (#1437)
To fix the issue that adhoc json patches were not working on kustomize/raw manifests.

Note that regular kustomize project was working. In other words, this only affetcts `chart: path/to/dir` combined with `jsonPatches: ...` when the `path/to/dir` points to a kustomize project or a local directory containing raw K8s manifests.

Ref https://github.com/roboll/helmfile/issues/1434#issuecomment-682247709
2020-08-28 21:13:04 +09:00
KUOKA Yusuke a9aa7af572
Fix regression installing raw K8s manifests (#1417)
Fixes #1413
2020-08-14 19:52:00 +09:00
KUOKA Yusuke ab1c118022
Bump chartify to v0.4.0 (#1409)
* Bump chartify to v0.4.0

Should fix #1338

* Bump minimum Helm 3 version to v3.1.0 for `helm template --include-crds`
2020-08-12 09:52:51 +09:00
KUOKA Yusuke 1671b8ba7d
Bump variantdev/vals for better Variant KV v2 support (#1367)
Ref https://github.com/variantdev/vals/pull/33
2020-07-21 07:17:40 +09:00
KUOKA Yusuke 608ba19c3c
Bump chartify to 0.3.11 (#1366)
Fixes #1365
2020-07-21 04:56:27 +09:00
KUOKA Yusuke 34f6d4d568
Bump chartify to 0.3.10 (#1363)
Fixes #1360
2020-07-19 20:08:18 +09:00
KUOKA Yusuke 68c51d8c16
Bump variantdev/vals to 0.9.3 (#1361) 2020-07-18 22:28:37 +09:00
KUOKA Yusuke 4bbb1699a3
Bump variantdev/chartify to 0.3.8 (#1359)
Fixes #1335
2020-07-17 09:41:05 +09:00
KUOKA Yusuke cd427d9714
Bump variantdev/vals to 0.7.1 (#1332)
To incorporate the tfstate provider fix

Ref https://github.com/roboll/helmfile/issues/505#issuecomment-653848769
2020-07-06 08:30:19 +09:00
KUOKA Yusuke 8de9b70b33
Bump variantdev/vals to 0.6.0 (#1311)
- Add `version` parameter for Vault provider
- Add `profile` parameter for AWS SSM/SecretsManager provider
= Add `version` parameter for AWS SSM Parameter Store provider
- Add support for app-role authentication when using Vault provider
2020-06-16 09:07:04 +09:00
ento face92536c
Add integration test for Kustomize inetgration (#1288)
Summary of changes:

* Output any error from Mkdir in `helmfile template`

* Add failing test for .Release.Name interpolation

* Add golden files for testing

* Parse resources with kustomize to compare them structure by structure

* Decode resources into plain maps

The RNode type from kustomize uses yaml.Node under the hood,
which carries extra information like line numbers, which
become noisy when comparing with deep.Equal.
2020-06-16 09:06:52 +09:00