Compare commits

...
48 Commits
Author SHA1 Message Date
Wi1dcard ad5fba56e0 Bump Helm to v3.5.0. (#1656) 2021-01-29 09:06:29 +09:00
Caleb Collins-Parks 0ad62b504a docs: update helm command name (#1635)
helm delete is now helm uninstall. See https://helm.sh/docs/helm/helm_delete/
2021-01-29 09:05:55 +09:00
Cameron Braid 8487970a2e fix: dont sent RegistryLogin password via args (#1662) 2021-01-29 09:05:03 +09:00
Chris Mellard 8212b630ff fix: fixed up OCI repos so they support paths and URLs that differ from chart names contained there (#1661) 2021-01-28 19:11:12 +09:00
Chris Mellard 2a71640095 feat: added in oci repository flag and added helm methods to pull and export charts (#1629) 2021-01-28 09:02:00 +09:00
Shunsuke Suzuki 33880dab77 docs: fix typo (#1645)
* docs: fix typo

* docs: syntax highlight
2021-01-12 10:23:21 +09:00
Sven Schliesing 87fcd83275 Fix typo "verion" (#1641) 2021-01-12 10:22:35 +09:00
jjangga0214 600721ca09 docs(readme): fix typo (#1633) 2021-01-12 10:22:04 +09:00
Yusuke Kuoka f7a1d09be7 Bump variantdev/vals to 0.12.0 (#1644)
For Azure KeyVault support for the fix for AWS SSM provider.
2021-01-12 10:20:57 +09:00
Yusuke Kuoka e5d982ed33 Fix "No valid platforms to build for" build error on release
Follow-up for #1612
May be related to https://github.com/mitchellh/gox/issues/92
2021-01-01 21:23:04 +09:00
Yusuke Kuoka c815d0192a Bump helm versions in Dockerfile to deal with stable and incubator repositories change 2021-01-01 12:16:41 +09:00
Yusuke Kuoka 9569d73e2f Bump helm versions in CI to deal with stable and incubator repositories change 2021-01-01 10:44:46 +09:00
Damiano AlbaniandDamiano Albani dd5bc46ead Include kustomize binary in helm3-based Docker image (#1626)
Co-authored-by: Damiano Albani <damiano.albani@geodan.nl>
2020-12-28 09:28:59 +09:00
Gastón Acosta Ramas fc3567ddda Adding documentation for PR #6819 (#1623)
Adding documentation for this feature: https://github.com/helm/helm/pull/6819
not being aware of this, make turn helm2 to 3 migrations very complicated.
2020-12-28 09:24:38 +09:00
Yusuke Kuoka 9b64d65d94 feat: Stabilize helmfile-diff output (#1622)
This finishes the work started at #1619 by introducing the following changes:

- You can now set `HELMFILE_TEMPDIR` and `CHARTIFY_TEMPDIR` as the directory to persist temporary values files rendered by helmfile and temporary charts generated by chartify. Note that chartify is used internally when you use kustomize integration or raw K8s manifests as a chart.
- Helmfile uses the hash sum of the release config and the values file content in the file name of the temporary values file.

  In combination with `HELMFILE_TEMPDIR` and `CHARTIFY_TEMPDIR`, this enables helmfile to use the stable file names for temporary values files, which contribute to stabilize log messages like `Comparing release=appset, chart=PATH/TO/TEMPORARY/CHART`, where the `PATH/TO/TEMPORARY/CHART` had been randomized due to formerly random helmfile tempdir and temporary values file names, and random chartify tempdir.

You can try this feature with a script like:

```
tempdir=$(mktemp -d -t helmfile)

HELMFILE_TEMPDIR=${tempdir} CHARTIFY_TEMPDIR=${tempdir} helmfile diff

rm -rf ${tempdir}
```
2020-12-15 10:24:27 +09:00
54eb73b423 Use ghodss/yaml for yaml marshaling & unmarshaling in template (#1556)
Fixes #1555

Co-authored-by: Yoann Ciabaud <yoann@linxo.com>
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
2020-12-13 11:03:20 +09:00
jayme-github 4c0987a618 Don't append --tiller-namespace flag when running helm3 (#1594) 2020-12-13 10:21:12 +09:00
Chris Mellard 0637973e40 fix: allow helmfiles to be corrected marshalled to YAML so they can be manipulated by other programs (#1604) 2020-12-13 10:08:03 +09:00
Felipe Santos 3690bde94c Add documentation on how to use the docker image (#1607)
* Add documentation on how to use the docker image

* Remove default workdir and add it to snippet
2020-12-13 10:00:36 +09:00
Life ChristianandLife Lofranco b92ff2baff Fix selector syntax example (#1614)
Co-authored-by: Life Lofranco <life@dealgrocer.com>
2020-12-13 09:58:24 +09:00
Chris Mellard afe82f6453 chore: suppress empty values on serializing several helmfile values (#1616) 2020-12-13 09:57:35 +09:00
Yusuke Kuoka 1c7b872476 Stabilize helmfile-diff output (#1619)
`helmfile-diff` sorts multiple and concurrent helm-diff outputs and stabilizes writes to stdout.

It's required to use the stdout from helmfile-diff to detect if there was another change(s) between 2 points in time.

For example, terraform-provider-helmfile runs a helmfile-diff on `terraform plan` and another on `terraform apply`. `terraform`, by design, fails when helmfile-diff outputs were not equivalent. Stabilized helmfile-diff output rescues that.
2020-12-11 09:51:26 +09:00
James Strachan 28e7ebb4a4 fix: support arm64 images (#1612) 2020-12-11 09:44:07 +09:00
Yusuke Kuoka 1ef9b29f6d Improve handling of releases being newly installed by helmfile-apply (#1618)
This improves helmfile-apply with two things:

- Some users had timing-out issues or annoyed by huge output from helm-diff run as part of helmfile-apply on first install. `--skip-diff-on-install` skips running helm-diff for releases being newly installed, so that you can avoid those issues.
- Some users had difficultly or found it not straight-forward to install CRDs and custom resources from separate charts in one helmfile-apply (#1353). The new helmfile.yaml release field `disableValidationOnInstall: true` adds `--disable-validation` to helm-diff only for releases being newly released, which should mostly resolve the issue.

Resolves #1353
2020-12-11 09:09:35 +09:00
Yusuke Kuoka b910591e1d Add support for Kustomize Transformers (#1592)
Please see the updated `advanced-features.md` for more details.

This is often used for adding common labels and annotations to any resources rendered from a Helm chart.
2020-11-19 09:33:05 +09:00
Krzysztof Pawlik d807510dd7 Enable shell completion (#1559)
* feat: enable EnableBashCompletion to get --generate-bash-completion command line flag

* feat: add scripts for bash and zsh

* feat: document shell completion
2020-11-19 09:32:33 +09:00
Wi1dcard 4e485219d7 Fix the logic of helmfile deps and add tests. (#1588) 2020-11-19 09:29:59 +09:00
Wi1dcard b0e042988c Fix incorrect help text of some CLI options. (#1587) 2020-11-19 07:20:08 +09:00
manfred-h f877de7778 Fix openSUSE repo URL. (#1584) 2020-11-19 07:19:00 +09:00
Baryczka 655d5934a2 Mention about helmBinary in readme (#1590) 2020-11-19 07:17:55 +09:00
Yusuke Kuoka f6bf885fb7 Fix helmfile template --include-crds not to break with chartify (#1583)
* Fix `helmfile template --include-crds` not to break with `chartify`

This bumps variantdev/chartify to 0.4.9 so that we can incorporate https://github.com/variantdev/chartify/commit/fed8bb29533904b1af3044a266220652231afe87

* Fix integration test
2020-11-12 09:26:43 +09:00
James Strachan 2857382006 fix: support arm binaries (#1580)
fixes #1433
2020-11-12 08:58:54 +09:00
manfred-h 972667f115 Mention openSUSE packages for helmfile (#1581)
* Mention openSUSE.

* Add example how to add a repo.
2020-11-12 08:57:26 +09:00
Brian Choy 3c88145329 Update readme stable/incubator URLs (#1582)
Small update to the readme repository examples since Google is no longer officially supporting hosting the stable/incubator helm charts. The official helm docs are using these repos now, eg. https://helm.sh/docs/intro/quickstart/.
2020-11-12 08:56:34 +09:00
Yusuke Kuoka 709913b5ea Bump dependencies (#1573)
Bumps dependencies mainly to fix possible go mod error that is fixed via https://github.com/variantdev/vals/commit/0edd5343228b96e1ca5f3599f923fee0ff1fd0cb, but also bumps all the other dependencies because they looked very outdated :)
2020-11-09 08:50:00 +09:00
Yusuke Kuoka 88884b68dc feat: helmfile template --skip-cleanup (#1570)
* feat: `helmfile template --skip-cleanup`

Resolves #1517
2020-11-06 09:23:49 +09:00
Yusuke Kuoka 5a15b65b94 feat: Add helmfile apply --skip-cleanup (#1571)
This deprecates the old `--retain-values` which was not working as intended.

Also see #1570 - there's now `--skip-cleanup` for `helmfile-template`, too.
2020-11-05 10:18:30 +09:00
Yusuke Kuoka 3899680672 feat: Add helmfile test --logs (#1569)
When `--logs` is provided, Helmfile runs `helm test --logs` so that it can stream test logs

Ref #1541
2020-11-05 10:17:18 +09:00
Yusuke Kuoka 6b86408500 feat: Add helmfile template --include-crds (#1568)
This allows you to use helmfile-template output as a GitOps source, when the template output contains CRDs and you use Helm 3.

Helm 3 by default removes CRDs from the template output. If you want to git-commit helmfile-template containing CRDs for GitOps and you use Helm 3 for templating, the only way is provide this newly added `--include-crds` flag.
2020-11-05 10:13:25 +09:00
Yusuke Kuoka bdbaa00628 Fix redundant helm-dep-build and prepare hook regression (#1566)
Fixes #1511
Fixes #1497
2020-11-05 09:43:42 +09:00
Yusuke Kuoka 87e86dcfd2 Fix helmfile --selector x=y template with needs (#1564)
I had mistakenly inverted the necessary flag to turn off the legacy processing on selectors that should be used only for helmfile commands that does not support DAGs/needs.

Fixes #1552
2020-11-04 09:29:23 +09:00
Christian Rebischke 995b201e47 mention Arch Linux support (#1549) 2020-10-26 08:59:05 +09:00
Yusuke Kuoka 0663831dd5 Disable dependency update while running helm-x/chartify in more cases (#1548)
`helm dep up` is now skipped while running helm-x/chartify when the chart/directory is obtained by running go-getter, or `skipDeps` is configured using a command-line flag, helmDefaults, or release configuration.

Resolves #1547
2020-10-25 11:25:59 +09:00
isindir afb2653452 fix: printing error to stdout together with templated resources (#1550)
Fixes #1551
2020-10-25 09:14:07 +09:00
isindir 9ec4a85821 Minor README improvement (#1543)
Fixes #1544
2020-10-19 08:48:39 +09:00
Wi1dcard 4d868794fd Improve capitalization and formats of README.md. (#1545) 2020-10-19 08:47:19 +09:00
Jake Hill 1aca099457 feat: Upgrade to kubectl version supported by latest stable release and upgrade helm3 to 3.3.4 (#1498)
Changes:

* Bump helm to v3.3.4.
* Bump kubectl to v1.18.9
* Update Dockerfile to be a little more verbose to help understand which command in a chain failed.

Signed-off-by: Jake Hill <jake@naphta.uk>
2020-10-19 08:45:44 +09:00
Wi1dcard f5d098876a Retain the option helm repo add --force-update when Helm is v3.3.4 or above. (#1546)
See: https://github.com/roboll/helmfile/pull/1542#issuecomment-710082201
2020-10-19 08:44:09 +09:00
42 changed files with 2036 additions and 824 deletions
+2 -2
View File
@@ -95,7 +95,7 @@ jobs:
- run:
name: Install helm
environment:
HELM_VERSION: v2.16.12
HELM_VERSION: v2.17.0
command: |
HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
curl -Lo ${HELM_FILENAME} "https://kubernetes-helm.storage.googleapis.com/${HELM_FILENAME}"
@@ -153,7 +153,7 @@ jobs:
- run:
name: Install helm
environment:
HELM_VERSION: v3.3.3
HELM_VERSION: v3.4.2
command: |
HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
curl -Lo ${HELM_FILENAME} "https://get.helm.sh/${HELM_FILENAME}"
+5
View File
@@ -0,0 +1,5 @@
root = true
[*.md]
insert_final_newline = true
trim_trailing_whitespace = true
+12 -8
View File
@@ -11,11 +11,12 @@ FROM alpine:3.11
RUN apk add --no-cache ca-certificates git bash curl jq
ARG HELM_VERSION=v2.16.12
ARG HELM_VERSION="v2.17.0"
ARG HELM_LOCATION="https://kubernetes-helm.storage.googleapis.com"
ARG HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
ARG HELM_SHA256="756ab375314329b66b452c0f9d569f74b0760141670217c07b79890ad314c214"
RUN wget ${HELM_LOCATION}/${HELM_FILENAME} && \
ARG HELM_SHA256="f3bec3c7c55f6a9eb9e6586b8c503f370af92fe987fcbf741f37707606d70296"
RUN set -x && \
wget ${HELM_LOCATION}/${HELM_FILENAME} && \
echo Verifying ${HELM_FILENAME}... && \
sha256sum ${HELM_FILENAME} | grep -q "${HELM_SHA256}" && \
echo Extracting ${HELM_FILENAME}... && \
@@ -26,14 +27,17 @@ RUN wget ${HELM_LOCATION}/${HELM_FILENAME} && \
# using the install documentation found at https://kubernetes.io/docs/tasks/tools/install-kubectl/
# for now but in a future version of alpine (in the testing version at the time of writing)
# we should be able to install using apk add.
ENV KUBECTL_VERSION="v1.14.5"
ENV KUBECTL_SHA256="26681319de56820a8467c9407e9203d5b15fb010ffc75ac5b99c9945ad0bd28c"
RUN curl --retry 3 --retry-connrefused -LO "https://storage.googleapis.com/kubernetes-release/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl" && \
# the sha256 sum can be found at https://storage.googleapis.com/kubernetes-release/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl.sha256
# maybe a good idea to automate in the future?
ENV KUBECTL_VERSION="v1.18.9"
ENV KUBECTL_SHA256="6a68756a2d3d04b4d0f52b00de6493ba2c1fcb28b32f3e4a0e99b3d9f6c4e8ed"
RUN set -x & \
curl --retry 5 --retry-connrefused -LO "https://storage.googleapis.com/kubernetes-release/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl" && \
sha256sum kubectl | grep ${KUBECTL_SHA256} && \
chmod +x kubectl && \
mv kubectl /usr/local/bin/kubectl
RUN ["helm", "init", "--client-only"]
RUN ["helm", "init", "--client-only", "--stable-repo-url", "https://charts.helm.sh/stable"]
RUN helm plugin install https://github.com/databus23/helm-diff && \
helm plugin install https://github.com/futuresimple/helm-secrets && \
helm plugin install https://github.com/hypnoglow/helm-s3.git && \
@@ -42,4 +46,4 @@ RUN helm plugin install https://github.com/databus23/helm-diff && \
COPY --from=builder /workspace/helmfile/dist/helmfile_linux_amd64 /usr/local/bin/helmfile
CMD ["/usr/local/bin/helmfile"]
CMD ["/usr/local/bin/helmfile", "--help"]
+19 -6
View File
@@ -11,11 +11,12 @@ FROM alpine:3.11
RUN apk add --no-cache ca-certificates git bash curl jq
ARG HELM_VERSION=v3.3.3
ARG HELM_VERSION="v3.5.0"
ARG HELM_LOCATION="https://get.helm.sh"
ARG HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
ARG HELM_SHA256="246d58b6b353e63ae8627415a7340089015e3eb542ff7b5ce124b0b1409369cc"
RUN wget ${HELM_LOCATION}/${HELM_FILENAME} && \
ARG HELM_SHA256="3fff0354d5fba4c73ebd5db59a59db72f8a5bbe1117a0b355b0c2983e98db95b"
RUN set -x && \
wget ${HELM_LOCATION}/${HELM_FILENAME} && \
echo Verifying ${HELM_FILENAME}... && \
sha256sum ${HELM_FILENAME} | grep -q "${HELM_SHA256}" && \
echo Extracting ${HELM_FILENAME}... && \
@@ -25,13 +26,25 @@ RUN wget ${HELM_LOCATION}/${HELM_FILENAME} && \
# using the install documentation found at https://kubernetes.io/docs/tasks/tools/install-kubectl/
# for now but in a future version of alpine (in the testing version at the time of writing)
# we should be able to install using apk add.
ENV KUBECTL_VERSION="v1.14.5"
ENV KUBECTL_SHA256="26681319de56820a8467c9407e9203d5b15fb010ffc75ac5b99c9945ad0bd28c"
RUN curl --retry 3 --retry-connrefused -LO "https://storage.googleapis.com/kubernetes-release/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl" && \
# the sha256 sum can be found at https://storage.googleapis.com/kubernetes-release/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl.sha256
# maybe a good idea to automate in the future?
ENV KUBECTL_VERSION="v1.18.9"
ENV KUBECTL_SHA256="6a68756a2d3d04b4d0f52b00de6493ba2c1fcb28b32f3e4a0e99b3d9f6c4e8ed"
RUN set -x && \
curl --retry 5 --retry-connrefused -LO "https://storage.googleapis.com/kubernetes-release/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl" && \
sha256sum kubectl | grep ${KUBECTL_SHA256} && \
chmod +x kubectl && \
mv kubectl /usr/local/bin/kubectl
ENV KUSTOMIZE_VERSION="v3.8.8"
ENV KUSTOMIZE_SHA256="175938206f23956ec18dac3da0816ea5b5b485a8493a839da278faac82e3c303"
RUN set -x && \
curl --retry 5 --retry-connrefused -LO https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize/${KUSTOMIZE_VERSION}/kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz && \
sha256sum kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz | grep ${KUSTOMIZE_SHA256} && \
tar zxf kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz && \
rm kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz && \
mv kustomize /usr/local/bin/kustomize
RUN helm plugin install https://github.com/databus23/helm-diff --version v3.1.3 && \
helm plugin install https://github.com/futuresimple/helm-secrets && \
helm plugin install https://github.com/hypnoglow/helm-s3.git && \
+1 -1
View File
@@ -30,7 +30,7 @@ integration:
.PHONY: integration
cross:
env CGO_ENABLED=0 gox -os '!openbsd !freebsd !netbsd' -arch '!arm !mips !mipsle !mips64 !mips64le !s390x' -output "dist/{{.Dir}}_{{.OS}}_{{.Arch}}" -ldflags '-X github.com/roboll/helmfile/pkg/app/version.Version=${TAG}' ${TARGETS}
env CGO_ENABLED=0 gox -os 'windows darwin linux' -arch '386 amd64 arm64' -osarch '!darwin/arm64' -output "dist/{{.Dir}}_{{.OS}}_{{.Arch}}" -ldflags '-X github.com/roboll/helmfile/pkg/app/version.Version=${TAG}' ${TARGETS}
.PHONY: cross
static-linux:
+12 -10
View File
@@ -1,25 +1,27 @@
# Paths Overivew
Using manifest files in conjunction with command line argument can be a bit confusing.
# Paths Overivew
A few rules to clear up this ambiguity:
Using manifest files in conjunction with command line argument can be a bit confusing.
A few rules to clear up this ambiguity:
- Absolute paths are always resolved as absolute paths
- Relative paths referenced *in* the helmfile manifest itself are relative to that manifest
- Relative paths referenced on the command line are relative to the current working directory the user is in
### Examples
There are several examples that we can go through in the `/examples` folder which demonstrate this.
There are several examples that we can go through in the [`/examples`](examples) folder which demonstrate this.
**Local Execution**
This is an example of a Helmfile manifest referencing a local value directly.
This is an example of a Helmfile manifest referencing a local value directly.
Indirect:
```
helmfile -f examples/deployments/local/charts.yaml sync
```
Direct:
Direct:
```
cd examples/deployments/local/
helmfile sync
@@ -27,14 +29,14 @@ helmfile sync
**Relative Paths in Helmfile**
This is an example of a Helmfile manifest using relative paths for values.
This is an example of a Helmfile manifest using relative paths for values.
Indirect:
```
helmfile -f examples/deployments/dev/charts.yaml sync
```
Direct:
Direct:
```
cd examples/deployments/dev/
helmfile sync
@@ -42,7 +44,7 @@ helmfile sync
**Relative Paths in Helmfile w/ --values overrides**
This is an example of a Helmfile manifest using relative paths for values including an additional `--values` from the command line.
This is an example of a Helmfile manifest using relative paths for values including an additional `--values` from the command line.
NOTE: The `--values` is resolved relative to the CWD of the terminal *not* the Helmfile manifest. You can see this with the `replicas` being adjusted to 3 now for the deployment.
@@ -51,7 +53,7 @@ Indirect:
helmfile -f examples/deployments/dev/charts.yaml sync --values values/replica-values.yaml
```
Direct:
Direct:
```
cd examples/deployments/dev/
helmfile sync --values ../../values/replica-values.yaml
+129 -41
View File
@@ -47,20 +47,21 @@ The default name for a helmfile is `helmfile.yaml`:
repositories:
# To use official "stable" charts a.k.a https://github.com/helm/charts/tree/master/stable
- name: stable
url: https://kubernetes-charts.storage.googleapis.com
url: https://charts.helm.sh/stable
# To use official "incubator" charts a.k.a https://github.com/helm/charts/tree/master/incubator
- name: incubator
url: https://kubernetes-charts-incubator.storage.googleapis.com
url: https://charts.helm.sh/incubator
# helm-git powered repository: You can treat any Git repository as a charts repository
- name: polaris
url: git+https://github.com/reactiveops/polaris@deploy/helm?ref=master
# Advanced configuration: You can setup basic or tls auth
# Advanced configuration: You can setup basic or tls auth and optionally enable helm OCI integration
- name: roboll
url: http://roboll.io/charts
certFile: optional_client_cert
keyFile: optional_client_key
username: optional_username
password: optional_password
oci: true
# Advanced configuration: You can use a ca bundle to use an https repo
# with a self-signed certificate
- name: insecure
@@ -69,6 +70,9 @@ repositories:
# context: kube-context # this directive is deprecated, please consider using helmDefaults.kubeContext
# Path to alternative helm binary (--helm-binary)
helmBinary: path/to/helm3
# Default values to set for args along with dedicated keys that can be set by contributors, cli args take precedence over these.
# In other words, unset values results in no flags passed to helm.
# See the helm usage (helm SUBCOMMAND -h) for more info on default values when those flags aren't provided.
@@ -102,6 +106,11 @@ helmDefaults:
historyMax: 10
# when using helm 3.2+, automatically create release namespaces if they do not exist (default true)
createNamespace: true
# if used with charts museum allows to pull unstable charts for deployment, for example: if 1.2.3 and 1.2.4-dev versions exist and set to true, 1.2.4-dev will be pulled (default false)
devel: true
# When set to `true`, skips running `helm dep up` and `helm dep build` on this release's chart.
# Useful when the chart is broken, like seen in https://github.com/roboll/helmfile/issues/1547
skipDeps: false
# these labels will be applied to all releases in a Helmfile. Useful in templating if you have a helmfile per environment or customer and don't want to copy the same label to each release
commonLabels:
@@ -189,8 +198,19 @@ releases:
# See https://github.com/roboll/helmfile/issues/642
# (default "", which means the standard kubeconfig, either ~/kubeconfig or the file pointed by $KUBECONFIG environment variable)
kubeContext: kube-context
# passes --disable-validation to helm 3 diff plugin, this requires diff plugin >= 3.1.2
# It may be helpful to deploy charts with helm api v1 CRDS
# https://github.com/roboll/helmfile/pull/1373
disableValidation: false
# passes --disable-openapi-validation to helm 3 diff plugin, this requires diff plugin >= 3.1.2
# It may be helpful to deploy charts with helm api v1 CRDS
# https://github.com/roboll/helmfile/pull/1373
disableOpenApiValidation: false
# limit the maximum number of revisions saved per release. Use 0 for no limit (default 10)
historyMax: 10
# When set to `true`, skips running `helm dep up` and `helm dep build` on this release's chart.
# Useful when the chart is broken, like seen in https://github.com/roboll/helmfile/issues/1547
skipDeps: false
# Local chart example
- name: grafana # name of this release
@@ -351,15 +371,44 @@ releases:
If you wish to treat your enviroment variables as strings always, even if they are boolean or numeric values you can use `{{ env "ENV_NAME" | quote }}` or `"{{ env "ENV_NAME" }}"`. These approaches also work with `requiredEnv`.
## installation
## Installation
- download one of [releases](https://github.com/roboll/helmfile/releases) or
- run as a [container](https://quay.io/roboll/helmfile) or
- install from [AUR](https://aur.archlinux.org/packages/kubernetes-helmfile-bin/) for Archlinux or
- [run as a container](#running-as-a-container) or
- Archlinux: install via `pacman -S helmfile` or from [AUR](https://aur.archlinux.org/packages/kubernetes-helmfile-bin/) or
- openSUSE: install via `zypper in helmfile` assuming you are on Tumbleweed; if you are on Leap you must add the [kubic](https://download.opensuse.org/repositories/devel:/kubic/) repo for your distribution version once before that command, e.g. `zypper ar https://download.opensuse.org/repositories/devel:/kubic/openSUSE_Leap_\$releasever kubic`, or
- Windows (using [scoop](https://scoop.sh/)): `scoop install helmfile`
- macOS (using [homebrew](https://brew.sh/)): `brew install helmfile`
## getting started
### Running as a container
The [Helmfile Docker images are available in Quay](https://quay.io/roboll/helmfile). There is no `latest` tag, since the `0.x` versions can contain breaking changes, so make sure you pick the right tag. Example using `helmfile 0.135.0`:
```sh-session
# helm 2
$ docker run --rm --net=host -v "${HOME}/.kube:/root/.kube" -v "${HOME}/.helm:/root/.helm" -v "${PWD}:/wd" --workdir /wd quay.io/roboll/helmfile:v0.135.0 helmfile sync
# helm 3
$ docker run --rm --net=host -v "${HOME}/.kube:/root/.kube" -v "${HOME}/.config/helm:/root/.config/helm" -v "${PWD}:/wd" --workdir /wd quay.io/roboll/helmfile:helm3-v0.135.0 helmfile sync
```
You can also use shims to make calling the binaries easier:
```sh-session
# helm 2
$ printf '%s\n' '#!/bin/sh' 'docker run --rm --net=host -v "${HOME}/.kube:/root/.kube" -v "${HOME}/.helm:/root/.helm" -v "${PWD}:/wd" --workdir /wd quay.io/roboll/helmfile:v0.135.0 helmfile "$@"' |
tee helmfile
$ chmod +x helmfile
$ ./helmfile sync
# helm 3
$ printf '%s\n' '#!/bin/sh' 'docker run --rm --net=host -v "${HOME}/.kube:/root/.kube" -v "${HOME}/.config/helm:/root/.config/helm" -v "${PWD}:/wd" --workdir /wd quay.io/roboll/helmfile:helm3-v0.135.0 helmfile "$@"' |
tee helmfile
$ chmod +x helmfile
$ ./helmfile sync
```
## Getting Started
Let's start with a simple `helmfile` and gradually improve it to fit your use-case!
@@ -389,7 +438,7 @@ Iterate on the `helmfile.yaml` by referencing:
- [CLI reference](#cli-reference).
- [Helmfile Best Practices Guide](https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md)
## cli reference
## CLI Reference
```
NAME:
@@ -412,7 +461,7 @@ COMMANDS:
apply apply all resources from state file only when there are changes
status retrieve status of releases in state file
delete DEPRECATED: delete releases from state file (helm delete)
destroy deletes and then purges releases
destroy uninstalls and then purges releases
test test releases from state file (helm test)
build output compiled helmfile state(s) as YAML
list list releases defined in state file
@@ -482,11 +531,11 @@ An expected use-case of `apply` is to schedule it to run periodically, so that y
### destroy
The `helmfile destroy` sub-command deletes and purges all the releases defined in the manifests.
The `helmfile destroy` sub-command uninstalls and purges all the releases defined in the manifests.
`helmfile --interactive destroy` instructs Helmfile to request your confirmation before actually deleting releases.
`destroy` basically runs `helm delete --purge` on all the targeted releases. If you don't want purging, use `helmfile delete` instead.
`destroy` basically runs `helm uninstall --purge` on all the targeted releases. If you don't want purging, use `helmfile delete` instead.
### delete (DEPRECATED)
@@ -496,7 +545,6 @@ The `helmfile delete` sub-command deletes all the releases defined in the manife
Note that `delete` doesn't purge releases. So `helmfile delete && helmfile sync` results in sync failed due to that releases names are not deleted but preserved for future references. If you really want to remove releases for reuse, add `--purge` flag to run it like `helmfile delete --purge`.
### secrets
The `secrets` parameter in a `helmfile.yaml` causes the [helm-secrets](https://github.com/futuresimple/helm-secrets) plugin to be executed to decrypt the file.
@@ -516,6 +564,7 @@ Use `--cleanup` to delete pods upon completion.
The `helmfile lint` sub-command runs a `helm lint` across all of the charts/releases defined in the manifest. Non local charts will be fetched into a temporary folder which will be deleted once the task is completed.
## Paths Overview
Using manifest files in conjunction with command line argument can be a bit confusing.
A few rules to clear up this ambiguity:
@@ -524,9 +573,10 @@ A few rules to clear up this ambiguity:
- Relative paths referenced *in* the Helmfile manifest itself are relative to that manifest
- Relative paths referenced on the command line are relative to the current working directory the user is in
For additional context, take a look at [paths examples](PATHS.md)
For additional context, take a look at [paths examples](PATHS.md).
## Labels Overview
A selector can be used to only target a subset of releases when running Helmfile. This is useful for large helmfiles with releases that are logically grouped together.
Labels are simple key value pairs that are an optional field of the release spec. When selecting by label, the search can be inverted. `tier!=backend` would match all releases that do NOT have the `tier: backend` label. `tier=fronted` would only match releases with the `tier: frontend` label.
@@ -535,7 +585,7 @@ Multiple labels can be specified using `,` as a separator. A release must match
The `selector` parameter can be specified multiple times. Each parameter is resolved independently so a release that matches any parameter will be used.
`--selector tier=frontend --selector tier=backend` will select all the charts
`--selector tier=frontend --selector tier=backend` will select all the charts.
In addition to user supplied labels, the name, the namespace, and the chart are available to be used as selectors. The chart will just be the chart name excluding the repository (Example `stable/filebeat` would be selected using `--selector chart=filebeat`).
@@ -544,7 +594,7 @@ For instance, you install a number of charts on every customer but need to provi
templates/common.yaml:
```
```yaml
templates:
nginx: &nginx
name: nginx
@@ -563,7 +613,7 @@ templates:
helmfile.yaml:
```
```yaml
{{ readFile "templates/common.yaml" }}
commonLabels:
@@ -731,7 +781,7 @@ domain: {{ .Values | get "domain" "dev.example.com" }}
```
`helmfile sync` installs `myapp` with the value `domain=dev.example.com`,
whereas `helmfile --environment production sync` installs the app with the value `domain=production.example.com`.
whereas `helmfile --environment production sync` installs the app with the value `domain=prod.example.com`.
For even more flexibility, you can now use values declared in the `environments:` section in other parts of your helmfiles:
@@ -984,14 +1034,14 @@ that is accessible by running a command:
A usual usage of `exec` would look like this:
```
```yaml
mysetting: |
{{ exec "./mycmd" (list "arg1" "arg2" "--flag1") | indent 2 }}
```
Or even with a pipeline:
```
```yaml
mysetting: |
{{ yourinput | exec "./mycmd-consume-stdin" (list "arg1" "arg2") | indent 2 }}
```
@@ -1037,7 +1087,7 @@ This is the counterpart to `prepare`, as any release on which `prepare` has been
The following is an example hook that just prints the contextual information provided to hook:
```
```yaml
releases:
- name: myapp
chart: mychart
@@ -1072,10 +1122,11 @@ For templating, imagine that you created a hook that generates a helm chart on-t
It will allow you to write your helm releases with any language you like, while still leveraging goodies provided by helm.
### Global Hooks
In contrast to the per release hooks mentioned above these are run only once at the very beginning and end of the execution of a helmfile command and only the `prepare` and `cleanup` hooks are available respectively.
They use the same syntax as per release hooks, but at the top level of your helmfile:
``` yaml
```yaml
hooks:
- events: ["prepare", "cleanup"]
showlogs: true
@@ -1146,9 +1197,9 @@ We also have dedicated documentation on the following topics which might interes
Or join our friendly slack community in the [`#helmfile`](https://slack.sweetops.com) channel to ask questions and get help. Check out our [slack archive](https://archive.sweetops.com/helmfile/) for good examples of how others are using it.
## Using env files
## Using .env files
Helmfile itself doesn't have an ability to load env files. But you can write some bash script to achieve the goal:
Helmfile itself doesn't have an ability to load .env files. But you can write some bash script to achieve the goal:
```console
set -a; . .env; set +a; helmfile sync
@@ -1156,7 +1207,7 @@ set -a; . .env; set +a; helmfile sync
Please see #203 for more context.
## Running helmfile interactively
## Running Helmfile interactively
`helmfile --interactive [apply|destroy]` requests confirmation from you before actually modifying your cluster.
@@ -1170,28 +1221,17 @@ Once you download all required charts into your machine, you can run `helmfile c
It basically run only `helm upgrade --install` with your already-downloaded charts, hence no Internet connection is required.
See #155 for more information on this topic.
## Experimental features
## Experimental Features
Some experimental features may be available for testing in perspective of being (or not) included in a future release.
Those features are set using the environment variable `HELMFILE_EXPERIMENTAL`. Here is the current experimental feature :
* `explicit-selector-inheritance` : remove today implicit cli selectors inheritance for composed helmfiles, see [composition selector](#selectors)
If you want to enable all experimental features set the env var to `HELMFILE_EXPERIMENTAL=true`
## Azure ACR integration
## `bash` and `zsh` completion
Azure offers helm repository [support for Azure Container Registry](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-helm-repos) as a preview feature.
To use this you must first `az login` and then `az acr helm repo add -n <MyRegistry>`. This will extract a token for the given ACR and configure `helm` to use it, e.g. `helm repo update` should work straight away.
To use `helmfile` with ACR, on the other hand, you must either include a username/password in the repository definition for the ACR in your `helmfile.yaml` or use the `--skip-deps` switch, e.g. `helmfile template --skip-deps`.
An ACR repository definition in `helmfile.yaml` looks like this:
```
repositories:
- name: <MyRegistry>
url: https://<MyRegistry>.azurecr.io/helm/v1/repo
```
Copy `autocomplete/helmfile_bash_autocomplete` or `autocomplete/helmfile_zsh_autocomplete` (depending on your shell of choice) to directory where you keep other shell completion scripts to make sure it is sourced.
## Examples
@@ -1202,6 +1242,7 @@ For more examples, see the [examples/README.md](https://github.com/roboll/helmfi
- [renovate](https://github.com/renovatebot/renovate) automates chart version updates. See [this PR for more information](https://github.com/renovatebot/renovate/pull/5257).
- For updating container image tags and git tags embedded within helmfile.yaml and values, you can use [renovate's regexManager](https://docs.renovatebot.com/modules/manager/regex/). Please see [this comment in the renovate repository](https://github.com/renovatebot/renovate/issues/6130#issuecomment-624061289) for more information.
- [ArgoCD Integration](#argocd-integration)
- [Azure ACR Integration](#azure-acr-integration)
### ArgoCD Integration
@@ -1235,7 +1276,7 @@ git push origin $BRANCH
> Note that `$(pwd)` is necessary when `hemlfile.yaml` has one or more sub-helmfiles in nested directories,
> because setting a relative file path in `--output-dir` or `--output-dir-template` results in each sub-helmfile render
> to the directory relative to the specified path.
> to the directory relative to the specified path.
so that they can be deployed by Argo CD as usual.
@@ -1252,7 +1293,54 @@ Recommendations:
- If you don't directly push it to the main Git branch and instead go through a pull-request, do lint rendered manifests on your CI, so that you can catch easy mistakes earlier/before ArgoCD finally deploys it
- See [this ArgoCD issue](https://github.com/argoproj/argo-cd/issues/2143#issuecomment-570478329) for why you may want this, and see [this helmfile issue](https://github.com/roboll/helmfile/pull/1357) for how `--output-dir-template` works.
# Attribution
### Azure ACR Integration
Azure offers helm repository [support for Azure Container Registry](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-helm-repos) as a preview feature.
To use this you must first `az login` and then `az acr helm repo add -n <MyRegistry>`. This will extract a token for the given ACR and configure `helm` to use it, e.g. `helm repo update` should work straight away.
To use `helmfile` with ACR, on the other hand, you must either include a username/password in the repository definition for the ACR in your `helmfile.yaml` or use the `--skip-deps` switch, e.g. `helmfile template --skip-deps`.
An ACR repository definition in `helmfile.yaml` looks like this:
```yaml
repositories:
- name: <MyRegistry>
url: https://<MyRegistry>.azurecr.io/helm/v1/repo
```
## OCI Registries
In order to use OCI chart registries firstly they must be marked in the repository list as OCI enabled, e.g.
```yaml
repositories:
- name: myOCIRegistry
url: myregistry.azurecr.io
oci: true
```
It is important not to include a scheme for the URL as helm requires that these are not present for OCI registries
Secondly the credentials for the OCI registry can either be specified within `helmfile.yaml` similar to
```yaml
repositories:
- name: myOCIRegistry
url: myregistry.azurecr.io
oci: true
username: spongebob
password: squarepants
```
or for CI scenarios these can be sourced from the environment with the format `<registryName>_USERNAME` and `<registryName_PASSWORD>`, e.g.
```shell
export MYOCIREGISTRY_USERNAME=spongebob
export MYOCIREGISTRY_PASSWORD=squarepants
```
## Attribution
We use:
+18
View File
@@ -0,0 +1,18 @@
#! /bin/bash
_helmfile_bash_autocomplete() {
if [[ "${COMP_WORDS[0]}" != "source" ]]; then
local cur opts base
COMPREPLY=()
cur="${COMP_WORDS[COMP_CWORD]}"
if [[ "$cur" == "-"* ]]; then
opts=$( ${COMP_WORDS[@]:0:$COMP_CWORD} ${cur} --generate-bash-completion )
else
opts=$( ${COMP_WORDS[@]:0:$COMP_CWORD} --generate-bash-completion )
fi
COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
return 0
fi
}
complete -o bashdefault -o default -o nospace -F _helmfile_bash_autocomplete helmfile
+23
View File
@@ -0,0 +1,23 @@
#compdef helmfile
_helmfile_zsh_autocomplete() {
local -a opts
local cur
cur=${words[-1]}
if [[ "$cur" == "-"* ]]; then
opts=("${(@f)$(_CLI_ZSH_AUTOCOMPLETE_HACK=1 ${words[@]:0:#words[@]-1} ${cur} --generate-bash-completion)}")
else
opts=("${(@f)$(_CLI_ZSH_AUTOCOMPLETE_HACK=1 ${words[@]:0:#words[@]-1} --generate-bash-completion)}")
fi
if [[ "${opts[1]}" != "" ]]; then
_describe 'values' opts
else
_files
fi
return
}
compdef _helmfile_zsh_autocomplete helmfile
+77 -3
View File
@@ -1,6 +1,8 @@
## Advanced Features
- [Import Configuration Parameters into Helmfile](#import-configuration-parameters-into-helmfile)
- [Deploy Kustomization with Helmfile](#deploy-kustomizations-with-helmfile)
- [Adhoc Kustomization of Helm Charts](#adhoc-kustomization-of-helm-charts)
### Import Configuration Parameters into Helmfile
@@ -35,7 +37,7 @@ $ somehow_generate_chart_yaml ${TMPCHART}/Chart.yaml
$ TMPKUSTOMIZATION=/tmp/sometmpdir2
$ somehow_generate_temp_kustomization_yaml ${TMPKUSTOMIZATION}/kustomization.yaml
$ kustomize build ${TMPKUSTOMIZATION}/kustomization.yaml > ${TMPCHART}/templates/all.yaml
$ kustomize build ${TMPKUSTOMIZATION}/kustomization.yaml > ${TMPCHART}/templates/all.yaml
```
Let's say you have a `helmfile.yaml` that looks like the below:
@@ -57,7 +59,7 @@ bases:
Followed by the below steps:
- Running `kustomize edit set image $IMAGE` for every `$IMAGE` generated from your values.yaml
- Running `kustomize edit set image $IMAGE` for every `$IMAGE` generated from your values.yaml
- Running `kustomize edit set nameprefix $NAMEPREFIX` with the nameprefix specified in your values.yaml
- Running `kustomize edit set namesuffix $NAMESUFFIX` with the namesuffix specified in your values.yaml
- Running `kustomize edit set namespace $NS` with the namespace specified in your values.yaml
@@ -92,7 +94,17 @@ After all, Helmfile just installs the temporary chart like standard charts, whic
Please also see [test/advanced/helmfile.yaml](https://github.com/roboll/helmfile/tree/master/test/advanced/helmfile.yaml) for an example of kustomization support and more.
## Adhoc Customization of Helm charts
### Adhoc Kustomization of Helm charts
With Helmfile's integration with Helmfile, not only deploying Kustomization as a Helm chart, you can kustomize charts before installation.
Currently, Helmfile allows you to set the following fields for kustomizing the chart:
- [`releases[].strategicMergePatches`](#strategicmergepatches)
- `releases[].jsonPatches`
- [`releases[].transformers`](#transformers)
#### `strategicMergePatches`
You can add/update any Kubernetes resource field rendered from a Helm chart by specifying `releases[].strategicMergePatches`:
@@ -135,3 +147,65 @@ Please note that the second `data` field `bar` is coming from the strategic-merg
There's also `releases[].jsonPatches` that works similarly to `strategicMergePatches` but has additional capability to remove fields.
Please also see [test/advanced/helmfile.yaml](https://github.com/roboll/helmfile/tree/master/test/advanced/helmfile.yaml) for an example of patching support and more.
#### `transformers`
You can set `transformers` to apply [Kustomize's transformers](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/configureBuiltinPlugin.md#configuring-the-builtin-plugins-instead).
Each item can be a path to a YAML or go template file, or an embedded transformer declaration as a YAML hash.
It's often used to add common labels and annotations to your resources.
In the below example. we add common annotations and labels every resource rendered from the `aws-load-balancer-controller` chart:
```yaml
releases:
- name: "aws-load-balancer-controller"
namespace: "kube-system"
forceNamespace: "kube-system"
chart: "center/aws/aws-load-balancer-controller"
transformers:
- apiVersion: builtin
kind: AnnotationsTransformer
metadata:
name: notImportantHere
annotations:
area: 51
greeting: take me to your leader
fieldSpecs:
- path: metadata/annotations
create: true
- apiVersion: builtin
kind: LabelTransformer
metadata:
name: notImportantHere
labels:
foo: bar
fieldSpecs:
- path: metadata/labels
create: true
```
As explained earlier, `transformers` can be not only a list of embedded transformers, but also YAML or go template files, or a mix of those three kinds.
```yaml
transformers:
# Embedded transformer
- apiVersion: builtin
kind: AnnotationsTransformer
metadata:
name: notImportantHere
annotations:
area: 51
greeting: take me to your leader
fieldSpecs:
- path: metadata/annotations
create: true
# YAML file
- path/to/transformer.yaml
# Go template
# The same set of template parameters as release values files templates is available.
- path/to/transformer.yaml.gotmpl
```
Please see https://github.com/kubernetes-sigs/kustomize/blob/master/examples/configureBuiltinPlugin.md#configuring-the-builtin-plugins-instead for more information on how to declare transformers.
+2 -2
View File
@@ -135,13 +135,13 @@ Please note, that it is not possible to layer `values` sections. If `values` is
> See **Layering State Template Files** if you're layering templates.
You may occasionally end up with many helmfiles that shares common parts like which repositories to use, and whichi release to be bundled by default.
You may occasionally end up with many helmfiles that shares common parts like which repositories to use, and which release to be bundled by default.
Use Layering to extract the common parts into a dedicated *library helmfile*s, so that each helmfile becomes DRY.
Let's assume that your `helmfile.yaml` looks like:
```
```yaml
bases:
- environments.yaml
+1 -1
View File
@@ -26,7 +26,7 @@ You would then start a database migration job by executing:
```console
# Start a database migration for the prod environment
$ RAILS_ENV=prod helmfile sync --selector job=dbmigrator
$ RAILS_ENV=prod helmfile --selector job=dbmigrator sync
# Tail log until you are satisfied
$ kubectl logs -l job=dbmigrator
+19 -17
View File
@@ -7,30 +7,32 @@ require (
github.com/Masterminds/semver/v3 v3.1.0
github.com/Masterminds/sprig/v3 v3.1.0
github.com/aryann/difflib v0.0.0-20170710044230-e206f873d14a
github.com/go-test/deep v1.0.3
github.com/golang/mock v1.4.3
github.com/google/go-cmp v0.4.0
github.com/gosuri/uitable v0.0.3
github.com/hashicorp/go-getter v1.3.0
github.com/davecgh/go-spew v1.1.1
github.com/frankban/quicktest v1.11.2 // indirect
github.com/ghodss/yaml v1.0.1-0.20190212211648-25d852aebe32
github.com/go-test/deep v1.0.7
github.com/golang/mock v1.4.4
github.com/google/go-cmp v0.5.2
github.com/gosuri/uitable v0.0.4
github.com/hashicorp/go-getter v1.5.0
github.com/hashicorp/go-retryablehttp v0.6.3 // indirect
github.com/hashicorp/go-version v1.2.0
github.com/hashicorp/go-version v1.2.1
github.com/howeyc/gopass v0.0.0-20190910152052-7cb4b85ec19c // indirect
github.com/imdario/mergo v0.3.11
github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect
github.com/pierrec/lz4 v2.3.0+incompatible // indirect
github.com/r3labs/diff v0.0.0-20190801153147-a71de73c46ad
github.com/spf13/cobra v0.0.3
github.com/r3labs/diff v1.1.0
github.com/spf13/cobra v1.1.1
github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939
github.com/urfave/cli v1.20.0
github.com/variantdev/chartify v0.4.4
github.com/urfave/cli v1.22.5
github.com/variantdev/chartify v0.6.0
github.com/variantdev/dag v0.0.0-20191028002400-bb0b3c785363
github.com/variantdev/vals v0.10.3
go.uber.org/multierr v1.1.0
go.uber.org/zap v1.9.1
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a
github.com/variantdev/vals v0.12.0
go.uber.org/multierr v1.6.0
go.uber.org/zap v1.16.0
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9
gopkg.in/square/go-jose.v2 v2.4.0 // indirect
gopkg.in/yaml.v2 v2.3.0
gotest.tools v2.2.0+incompatible
gotest.tools/v3 v3.0.3-0.20200410202438-4e4a41b7851a
k8s.io/apimachinery v0.0.0-20190409092423-760d1845f48b
gotest.tools/v3 v3.0.3
k8s.io/apimachinery v0.19.3
)
+311 -514
View File
File diff suppressed because it is too large Load Diff
+47 -10
View File
@@ -42,6 +42,7 @@ func main() {
cliApp.Name = "helmfile"
cliApp.Usage = ""
cliApp.Version = version.Version
cliApp.EnableBashCompletion = true
cliApp.Flags = []cli.Flag{
cli.StringFlag{
Name: "helm-binary, b",
@@ -54,7 +55,7 @@ func main() {
},
cli.StringFlag{
Name: "environment, e",
Usage: "specify the environment name. defaults to `default`",
Usage: `specify the environment name. defaults to "default"`,
},
cli.StringSliceFlag{
Name: "state-values-set",
@@ -109,7 +110,7 @@ func main() {
cliApp.Commands = []cli.Command{
{
Name: "deps",
Usage: "update charts based on the contents of requirements.yaml",
Usage: "update charts based on their requirements",
Flags: []cli.Flag{
cli.StringFlag{
Name: "args",
@@ -118,7 +119,7 @@ func main() {
},
cli.BoolFlag{
Name: "skip-repos",
Usage: "skip running `helm repo update` before running `helm dependency build`",
Usage: `skip running "helm repo update" before running "helm dependency build"`,
},
},
Action: action(func(run *app.App, c configImpl) error {
@@ -185,7 +186,7 @@ func main() {
},
cli.BoolFlag{
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
Usage: `skip running "helm repo update" and "helm dependency build"`,
},
cli.BoolFlag{
Name: "detailed-exitcode",
@@ -248,9 +249,17 @@ func main() {
Name: "validate",
Usage: "validate your manifests against the Kubernetes cluster you are currently pointing at",
},
cli.BoolFlag{
Name: "include-crds",
Usage: "include CRDs in the templated output",
},
cli.BoolFlag{
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
Usage: `skip running "helm repo update" and "helm dependency build"`,
},
cli.BoolFlag{
Name: "skip-cleanup",
Usage: "Stop cleaning up temporary values generated by helmfile and helm-secrets. Useful for debugging. Don't use in production for security",
},
},
Action: action(func(run *app.App, c configImpl) error {
@@ -280,7 +289,7 @@ func main() {
},
cli.BoolFlag{
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
Usage: `skip running "helm repo update" and "helm dependency build"`,
},
},
Action: action(func(run *app.App, c configImpl) error {
@@ -311,7 +320,7 @@ func main() {
},
cli.BoolFlag{
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
Usage: `skip running "helm repo update" and "helm dependency build"`,
},
},
Action: action(func(run *app.App, c configImpl) error {
@@ -342,7 +351,7 @@ func main() {
},
cli.BoolFlag{
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
Usage: `skip running "helm repo update" and "helm dependency build"`,
},
},
Action: action(func(run *app.App, c configImpl) error {
@@ -382,7 +391,15 @@ func main() {
},
cli.BoolFlag{
Name: "retain-values-files",
Usage: "Stop cleaning up values files passed to Helm. Together with --log-level=debug, you can manually rerun helm commands as Helmfile did for debugging purpose",
Usage: "DEPRECATED: Use skip-cleanup instead",
},
cli.BoolFlag{
Name: "skip-cleanup",
Usage: "Stop cleaning up temporary values generated by helmfile and helm-secrets. Useful for debugging. Don't use in production for security",
},
cli.BoolFlag{
Name: "skip-diff-on-install",
Usage: "Skips running helm-diff on releases being newly installed on this apply. Useful when the release manifests are too huge to be reviewed, or it's too time-consuming to diff at all",
},
cli.BoolFlag{
Name: "include-tests",
@@ -398,7 +415,7 @@ func main() {
},
cli.BoolFlag{
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
Usage: `skip running "helm repo update" and "helm dependency build"`,
},
},
Action: action(func(run *app.App, c configImpl) error {
@@ -474,6 +491,10 @@ func main() {
Name: "cleanup",
Usage: "delete test pods upon completion",
},
cli.BoolFlag{
Name: "logs",
Usage: "Dump the logs from test pods (this runs after all tests are complete, but before any cleanup)",
},
cli.StringFlag{
Name: "args",
Value: "",
@@ -658,6 +679,10 @@ func (c configImpl) Cleanup() bool {
return c.c.Bool("cleanup")
}
func (c configImpl) Logs() bool {
return c.c.Bool("logs")
}
func (c configImpl) Timeout() int {
if !c.c.IsSet("timeout") {
return state.EmptyTimeout
@@ -713,10 +738,22 @@ func (c configImpl) Context() int {
return c.c.Int("context")
}
func (c configImpl) SkipCleanup() bool {
return c.c.Bool("skip-cleanup")
}
func (c configImpl) SkipDiffOnInstall() bool {
return c.c.Bool("skip-diff-on-install")
}
func (c configImpl) EmbedValues() bool {
return c.c.Bool("embed-values")
}
func (c configImpl) IncludeCRDs() bool {
return c.c.Bool("include-crds")
}
func (c configImpl) Logger() *zap.SugaredLogger {
return c.c.App.Metadata["logger"].(*zap.SugaredLogger)
}
+47 -17
View File
@@ -105,6 +105,7 @@ func (a *App) Deps(c DepsConfigProvider) error {
return a.ForEachState(func(run *Run) (_ bool, errs []error) {
prepErr := run.withPreparedCharts("deps", state.ChartPrepareOptions{
SkipRepos: c.SkipRepos(),
SkipDeps: true,
SkipResolve: true,
}, func() {
errs = run.Deps(c)
@@ -122,7 +123,10 @@ func (a *App) Repos(c ReposConfigProvider) error {
return a.ForEachState(func(run *Run) (_ bool, errs []error) {
var reposErr error
err := run.withPreparedCharts("repos", state.ChartPrepareOptions{SkipRepos: true}, func() {
err := run.withPreparedCharts("repos", state.ChartPrepareOptions{
SkipRepos: true,
SkipDeps: true,
}, func() {
reposErr = run.Repos(c)
})
@@ -140,7 +144,10 @@ func (a *App) Repos(c ReposConfigProvider) error {
func (a *App) DeprecatedSyncCharts(c DeprecatedChartsConfigProvider) error {
return a.ForEachState(func(run *Run) (_ bool, errs []error) {
err := run.withPreparedCharts("charts", state.ChartPrepareOptions{SkipRepos: true}, func() {
err := run.withPreparedCharts("charts", state.ChartPrepareOptions{
SkipRepos: true,
SkipDeps: true,
}, func() {
errs = run.DeprecatedSyncCharts(c)
})
@@ -166,7 +173,10 @@ func (a *App) Diff(c DiffConfigProvider) error {
var errs []error
prepErr := run.withPreparedCharts("diff", state.ChartPrepareOptions{SkipRepos: c.SkipDeps()}, func() {
prepErr := run.withPreparedCharts("diff", state.ChartPrepareOptions{
SkipRepos: c.SkipDeps(),
SkipDeps: c.SkipDeps(),
}, func() {
msg, matched, affected, errs = a.diff(run, c)
})
@@ -218,12 +228,16 @@ func (a *App) Diff(c DiffConfigProvider) error {
}
func (a *App) Template(c TemplateConfigProvider) error {
opts := []LoadOption{SetRetainValuesFiles(c.SkipCleanup())}
return a.ForEachState(func(run *Run) (ok bool, errs []error) {
// `helm template` in helm v2 does not support local chart.
// So, we set forceDownload=true for helm v2 only
prepErr := run.withPreparedCharts("template", state.ChartPrepareOptions{
ForceDownload: !run.helm.IsHelm3(),
SkipRepos: c.SkipDeps(),
SkipDeps: c.SkipDeps(),
}, func() {
ok, errs = a.template(run, c)
})
@@ -233,7 +247,7 @@ func (a *App) Template(c TemplateConfigProvider) error {
}
return
}, SetFilter(true))
}, opts...)
}
func (a *App) WriteValues(c WriteValuesConfigProvider) error {
@@ -243,6 +257,7 @@ func (a *App) WriteValues(c WriteValuesConfigProvider) error {
prepErr := run.withPreparedCharts("write-values", state.ChartPrepareOptions{
ForceDownload: !run.helm.IsHelm3(),
SkipRepos: c.SkipDeps(),
SkipDeps: c.SkipDeps(),
}, func() {
ok, errs = a.writeValues(run, c)
})
@@ -261,6 +276,7 @@ func (a *App) Lint(c LintConfigProvider) error {
prepErr := run.withPreparedCharts("lint", state.ChartPrepareOptions{
ForceDownload: true,
SkipRepos: c.SkipDeps(),
SkipDeps: c.SkipDeps(),
}, func() {
errs = run.Lint(c)
})
@@ -277,6 +293,7 @@ func (a *App) Sync(c SyncConfigProvider) error {
return a.ForEachState(func(run *Run) (ok bool, errs []error) {
prepErr := run.withPreparedCharts("sync", state.ChartPrepareOptions{
SkipRepos: c.SkipDeps(),
SkipDeps: c.SkipDeps(),
}, func() {
ok, errs = a.sync(run, c)
})
@@ -296,11 +313,12 @@ func (a *App) Apply(c ApplyConfigProvider) error {
var opts []LoadOption
opts = append(opts, SetRetainValuesFiles(c.RetainValuesFiles()))
opts = append(opts, SetRetainValuesFiles(c.RetainValuesFiles() || c.SkipCleanup()))
err := a.ForEachState(func(run *Run) (ok bool, errs []error) {
prepErr := run.withPreparedCharts("apply", state.ChartPrepareOptions{
SkipRepos: c.SkipDeps(),
SkipDeps: c.SkipDeps(),
}, func() {
matched, updated, es := a.apply(run, c)
@@ -335,6 +353,7 @@ func (a *App) Status(c StatusesConfigProvider) error {
return a.ForEachState(func(run *Run) (_ bool, errs []error) {
err := run.withPreparedCharts("status", state.ChartPrepareOptions{
SkipRepos: true,
SkipDeps: true,
}, func() {
errs = run.Status(c)
})
@@ -351,6 +370,7 @@ func (a *App) Delete(c DeleteConfigProvider) error {
return a.ForEachState(func(run *Run) (ok bool, errs []error) {
err := run.withPreparedCharts("delete", state.ChartPrepareOptions{
SkipRepos: true,
SkipDeps: true,
}, func() {
ok, errs = a.delete(run, c.Purge(), c)
})
@@ -367,6 +387,7 @@ func (a *App) Destroy(c DestroyConfigProvider) error {
return a.ForEachState(func(run *Run) (ok bool, errs []error) {
err := run.withPreparedCharts("destroy", state.ChartPrepareOptions{
SkipRepos: true,
SkipDeps: true,
}, func() {
ok, errs = a.delete(run, true, c)
})
@@ -389,6 +410,7 @@ func (a *App) Test(c TestConfigProvider) error {
err := run.withPreparedCharts("test", state.ChartPrepareOptions{
SkipRepos: true,
SkipDeps: true,
}, func() {
errs = a.test(run, c)
})
@@ -405,6 +427,7 @@ func (a *App) PrintState(c StateConfigProvider) error {
return a.ForEachState(func(run *Run) (_ bool, errs []error) {
err := run.withPreparedCharts("build", state.ChartPrepareOptions{
SkipRepos: true,
SkipDeps: true,
}, func() {
if c.EmbedValues() {
for i := range run.state.Releases {
@@ -453,6 +476,7 @@ func (a *App) ListReleases(c ListConfigProvider) error {
err := a.ForEachState(func(run *Run) (_ bool, errs []error) {
err := run.withPreparedCharts("list", state.ChartPrepareOptions{
SkipRepos: true,
SkipDeps: true,
}, func() {
//var releases m
@@ -584,14 +608,15 @@ func (a *App) loadDesiredStateFromYaml(file string, opts ...LoadOpts) (*state.He
}
ld := &desiredStateLoader{
readFile: a.readFile,
deleteFile: a.deleteFile,
fileExists: a.fileExists,
env: a.Env,
namespace: a.Namespace,
logger: a.Logger,
abs: a.abs,
remote: a.remote,
readFile: a.readFile,
deleteFile: a.deleteFile,
fileExists: a.fileExists,
directoryExistsAt: a.directoryExistsAt,
env: a.Env,
namespace: a.Namespace,
logger: a.Logger,
abs: a.abs,
remote: a.remote,
overrideKubeContext: a.OverrideKubeContext,
overrideHelmBinary: a.OverrideHelmBinary,
@@ -1059,9 +1084,11 @@ func (a *App) apply(r *Run, c ApplyConfigProvider) (bool, bool, []error) {
detailedExitCode := true
diffOpts := &state.DiffOpts{
NoColor: c.NoColor(),
Context: c.Context(),
Set: c.Set(),
NoColor: c.NoColor(),
Context: c.Context(),
Set: c.Set(),
SkipCleanup: c.RetainValuesFiles() || c.SkipCleanup(),
SkipDiffOnInstall: c.SkipDiffOnInstall(),
}
infoMsg, releasesToBeUpdated, releasesToBeDeleted, errs := r.diff(false, detailedExitCode, c, diffOpts)
@@ -1150,7 +1177,8 @@ Do you really want to apply?
subst.Releases = rs
syncOpts := state.SyncOpts{
Set: c.Set(),
Set: c.Set(),
SkipCleanup: c.RetainValuesFiles() || c.SkipCleanup(),
}
return subst.SyncReleases(&affectedReleases, helm, c.Values(), c.Concurrency(), &syncOpts)
}))
@@ -1433,7 +1461,9 @@ func (a *App) template(r *Run, c TemplateConfigProvider) (bool, []error) {
opts := &state.TemplateOpts{
Set: c.Set(),
IncludeCRDs: c.IncludeCRDs(),
OutputDirTemplate: c.OutputDirTemplate(),
SkipCleanup: c.SkipCleanup(),
}
return subst.TemplateReleases(helm, c.OutputDir(), c.Values(), args, c.Concurrency(), c.Validate(), opts)
}))
+430 -23
View File
@@ -2237,9 +2237,16 @@ services:
}
type configImpl struct {
set []string
output string
skipDeps bool
selectors []string
set []string
output string
includeCRDs bool
skipCleanup bool
skipDeps bool
}
func (a configImpl) Selectors() []string {
return a.selectors
}
func (c configImpl) Set() []string {
@@ -2258,6 +2265,10 @@ func (c configImpl) Validate() bool {
return true
}
func (c configImpl) SkipCleanup() bool {
return c.skipCleanup
}
func (c configImpl) SkipDeps() bool {
return c.skipDeps
}
@@ -2270,6 +2281,10 @@ func (c configImpl) OutputDirTemplate() string {
return ""
}
func (c configImpl) IncludeCRDs() bool {
return c.includeCRDs
}
func (c configImpl) Concurrency() int {
return 1
}
@@ -2288,6 +2303,7 @@ type applyConfig struct {
retainValuesFiles bool
set []string
validate bool
skipCleanup bool
skipDeps bool
includeTests bool
suppressSecrets bool
@@ -2297,6 +2313,7 @@ type applyConfig struct {
concurrency int
detailedExitcode bool
interactive bool
skipDiffOnInstall bool
logger *zap.SugaredLogger
}
@@ -2316,6 +2333,10 @@ func (a applyConfig) Validate() bool {
return a.validate
}
func (a applyConfig) SkipCleanup() bool {
return a.skipCleanup
}
func (a applyConfig) SkipDeps() bool {
return a.skipDeps
}
@@ -2360,6 +2381,22 @@ func (a applyConfig) RetainValuesFiles() bool {
return a.retainValuesFiles
}
func (a applyConfig) SkipDiffOnInstall() bool {
return a.skipDiffOnInstall
}
type depsConfig struct {
skipRepos bool
}
func (d depsConfig) SkipRepos() bool {
return d.skipRepos
}
func (d depsConfig) Args() string {
return ""
}
// Mocking the command-line runner
type mockRunner struct {
@@ -2367,6 +2404,10 @@ type mockRunner struct {
err error
}
func (mock *mockRunner) ExecuteStdIn(cmd string, args []string, env map[string]string, stdin io.Reader) ([]byte, error) {
return []byte{}, nil
}
func (mock *mockRunner) Execute(cmd string, args []string, env map[string]string) ([]byte, error) {
return []byte{}, nil
}
@@ -2404,6 +2445,14 @@ func (helm *mockHelmExec) TemplateRelease(name, chart string, flags ...string) e
return nil
}
func (helm *mockHelmExec) ChartPull(chart string, flags ...string) error {
return nil
}
func (helm *mockHelmExec) ChartExport(chart string, path string, flags ...string) error {
return nil
}
func (helm *mockHelmExec) UpdateDeps(chart string) error {
return nil
}
@@ -2425,6 +2474,9 @@ func (helm *mockHelmExec) AddRepo(name, repository, cafile, certfile, keyfile, u
func (helm *mockHelmExec) UpdateRepo() error {
return nil
}
func (helm *mockHelmExec) RegistryLogin(name string, username string, password string) error {
return nil
}
func (helm *mockHelmExec) SyncRelease(context helmexec.HelmContext, name, chart string, flags ...string) error {
return nil
}
@@ -2619,18 +2671,19 @@ releases:
func TestApply(t *testing.T) {
testcases := []struct {
name string
loc string
ns string
concurrency int
error string
files map[string]string
selectors []string
lists map[exectest.ListKey]string
diffs map[exectest.DiffKey]error
upgraded []exectest.Release
deleted []exectest.Release
log string
name string
loc string
ns string
concurrency int
skipDiffOnInstall bool
error string
files map[string]string
selectors []string
lists map[exectest.ListKey]string
diffs map[exectest.DiffKey]error
upgraded []exectest.Release
deleted []exectest.Release
log string
}{
//
// complex test cases for smoke testing
@@ -3039,6 +3092,222 @@ baz stable/mychart3
bar stable/mychart2
foo stable/mychart1
`,
},
//
// install with upgrade
//
{
name: "install-with-upgrade-with-validation-control",
loc: location(),
files: map[string]string{
"/path/to/helmfile.yaml": `
releases:
- name: baz
chart: stable/mychart3
disableValidationOnInstall: true
- name: foo
chart: stable/mychart1
disableValidationOnInstall: true
needs:
- bar
- name: bar
chart: stable/mychart2
disableValidation: true
`,
},
diffs: map[exectest.DiffKey]error{
exectest.DiffKey{Name: "baz", Chart: "stable/mychart3", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
exectest.DiffKey{Name: "foo", Chart: "stable/mychart1", Flags: "--disable-validation--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
exectest.DiffKey{Name: "bar", Chart: "stable/mychart2", Flags: "--disable-validation--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
},
lists: map[exectest.ListKey]string{
exectest.ListKey{Filter: "^foo$", Flags: "--kube-contextdefault--deployed--failed--pending"}: ``,
exectest.ListKey{Filter: "^bar$", Flags: "--kube-contextdefault--deployed--failed--pending"}: `NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
bar 4 Fri Nov 1 08:40:07 2019 DEPLOYED mychart2-3.1.0 3.1.0 default
`,
exectest.ListKey{Filter: "^baz$", Flags: "--kube-contextdefault--deployed--failed--pending"}: `NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
baz 4 Fri Nov 1 08:40:07 2019 DEPLOYED mychart3-3.1.0 3.1.0 default
`,
},
upgraded: []exectest.Release{
{Name: "baz", Flags: []string{"--kube-context", "default"}},
{Name: "bar", Flags: []string{"--kube-context", "default"}},
{Name: "foo", Flags: []string{"--kube-context", "default"}},
},
deleted: []exectest.Release{},
concurrency: 1,
log: `processing file "helmfile.yaml" in directory "."
first-pass rendering starting for "helmfile.yaml.part.0": inherited=&{default map[] map[]}, overrode=<nil>
first-pass uses: &{default map[] map[]}
first-pass rendering output of "helmfile.yaml.part.0":
0:
1: releases:
2: - name: baz
3: chart: stable/mychart3
4: disableValidationOnInstall: true
5: - name: foo
6: chart: stable/mychart1
7: disableValidationOnInstall: true
8: needs:
9: - bar
10: - name: bar
11: chart: stable/mychart2
12: disableValidation: true
13:
first-pass produced: &{default map[] map[]}
first-pass rendering result of "helmfile.yaml.part.0": {default map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "helmfile.yaml.part.0":
0:
1: releases:
2: - name: baz
3: chart: stable/mychart3
4: disableValidationOnInstall: true
5: - name: foo
6: chart: stable/mychart1
7: disableValidationOnInstall: true
8: needs:
9: - bar
10: - name: bar
11: chart: stable/mychart2
12: disableValidation: true
13:
merged environment: &{default map[] map[]}
3 release(s) found in helmfile.yaml
Affected releases are:
bar (stable/mychart2) UPDATED
baz (stable/mychart3) UPDATED
foo (stable/mychart1) UPDATED
processing 2 groups of releases in this order:
GROUP RELEASES
1 baz, bar
2 foo
processing releases in group 1/2: baz, bar
processing releases in group 2/2: foo
getting deployed release version failed:Failed to get the version for:mychart1
UPDATED RELEASES:
NAME CHART VERSION
baz stable/mychart3 3.1.0
bar stable/mychart2 3.1.0
foo stable/mychart1
`,
},
//
// install with upgrade and --skip-diff-on-install
//
{
name: "install-with-upgrade-with-skip-diff-on-install",
loc: location(),
skipDiffOnInstall: true,
files: map[string]string{
"/path/to/helmfile.yaml": `
releases:
- name: baz
chart: stable/mychart3
disableValidationOnInstall: true
- name: foo
chart: stable/mychart1
disableValidationOnInstall: true
needs:
- bar
- name: bar
chart: stable/mychart2
disableValidation: true
`,
},
diffs: map[exectest.DiffKey]error{
exectest.DiffKey{Name: "baz", Chart: "stable/mychart3", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
exectest.DiffKey{Name: "bar", Chart: "stable/mychart2", Flags: "--disable-validation--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
},
lists: map[exectest.ListKey]string{
exectest.ListKey{Filter: "^foo$", Flags: "--kube-contextdefault--deployed--failed--pending"}: ``,
exectest.ListKey{Filter: "^bar$", Flags: "--kube-contextdefault--deployed--failed--pending"}: `NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
bar 4 Fri Nov 1 08:40:07 2019 DEPLOYED mychart2-3.1.0 3.1.0 default
`,
exectest.ListKey{Filter: "^baz$", Flags: "--kube-contextdefault--deployed--failed--pending"}: `NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
baz 4 Fri Nov 1 08:40:07 2019 DEPLOYED mychart3-3.1.0 3.1.0 default
`,
},
upgraded: []exectest.Release{
{Name: "baz", Flags: []string{"--kube-context", "default"}},
{Name: "bar", Flags: []string{"--kube-context", "default"}},
{Name: "foo", Flags: []string{"--kube-context", "default"}},
},
deleted: []exectest.Release{},
concurrency: 1,
log: `processing file "helmfile.yaml" in directory "."
first-pass rendering starting for "helmfile.yaml.part.0": inherited=&{default map[] map[]}, overrode=<nil>
first-pass uses: &{default map[] map[]}
first-pass rendering output of "helmfile.yaml.part.0":
0:
1: releases:
2: - name: baz
3: chart: stable/mychart3
4: disableValidationOnInstall: true
5: - name: foo
6: chart: stable/mychart1
7: disableValidationOnInstall: true
8: needs:
9: - bar
10: - name: bar
11: chart: stable/mychart2
12: disableValidation: true
13:
first-pass produced: &{default map[] map[]}
first-pass rendering result of "helmfile.yaml.part.0": {default map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "helmfile.yaml.part.0":
0:
1: releases:
2: - name: baz
3: chart: stable/mychart3
4: disableValidationOnInstall: true
5: - name: foo
6: chart: stable/mychart1
7: disableValidationOnInstall: true
8: needs:
9: - bar
10: - name: bar
11: chart: stable/mychart2
12: disableValidation: true
13:
merged environment: &{default map[] map[]}
3 release(s) found in helmfile.yaml
Affected releases are:
bar (stable/mychart2) UPDATED
baz (stable/mychart3) UPDATED
foo (stable/mychart1) UPDATED
processing 2 groups of releases in this order:
GROUP RELEASES
1 baz, bar
2 foo
processing releases in group 1/2: baz, bar
processing releases in group 2/2: foo
getting deployed release version failed:Failed to get the version for:mychart1
UPDATED RELEASES:
NAME CHART VERSION
baz stable/mychart3 3.1.0
bar stable/mychart2 3.1.0
foo stable/mychart1
`,
},
//
@@ -3887,8 +4156,9 @@ err: "foo" depends on nonexistent release "bar"
applyErr := app.Apply(applyConfig{
// if we check log output, concurrency must be 1. otherwise the test becomes non-deterministic.
concurrency: tc.concurrency,
logger: logger,
concurrency: tc.concurrency,
logger: logger,
skipDiffOnInstall: tc.skipDiffOnInstall,
})
if tc.error == "" && applyErr != nil {
t.Fatalf("unexpected error for data defined at %s: %v", tc.loc, applyErr)
@@ -3941,6 +4211,147 @@ err: "foo" depends on nonexistent release "bar"
}
}
func TestDeps(t *testing.T) {
testcases := []struct {
name string
loc string
error string
files map[string]string
log string
charts []string
}{
//
// complex test cases for smoke testing
//
{
name: "smoke",
loc: location(),
files: map[string]string{
"/path/to/helmfile.yaml": `
repositories:
- name: bitnami
url: https://charts.bitnami.com/bitnami/
releases:
- name: example
chart: /path/to/charts/example
`,
"/path/to/charts/example/Chart.yaml": `foo: FOO`,
},
log: `processing file "helmfile.yaml" in directory "."
first-pass rendering starting for "helmfile.yaml.part.0": inherited=&{default map[] map[]}, overrode=<nil>
first-pass uses: &{default map[] map[]}
first-pass rendering output of "helmfile.yaml.part.0":
0:
1: repositories:
2: - name: bitnami
3: url: https://charts.bitnami.com/bitnami/
4: releases:
5: - name: example
6: chart: /path/to/charts/example
7:
first-pass produced: &{default map[] map[]}
first-pass rendering result of "helmfile.yaml.part.0": {default map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "helmfile.yaml.part.0":
0:
1: repositories:
2: - name: bitnami
3: url: https://charts.bitnami.com/bitnami/
4: releases:
5: - name: example
6: chart: /path/to/charts/example
7:
merged environment: &{default map[] map[]}
There are no repositories defined in your helmfile.yaml.
This means helmfile cannot update your dependencies or create a lock file.
See https://github.com/roboll/helmfile/issues/878 for more information.
`,
charts: []string{"/path/to/charts/example"},
},
}
for i := range testcases {
tc := testcases[i]
t.Run(tc.name, func(t *testing.T) {
var helm = &exectest.Helm{
DiffMutex: &sync.Mutex{},
ChartsMutex: &sync.Mutex{},
ReleasesMutex: &sync.Mutex{},
}
bs := &bytes.Buffer{}
func() {
logReader, logWriter := io.Pipe()
logFlushed := &sync.WaitGroup{}
// Ensure all the log is consumed into `bs` by calling `logWriter.Close()` followed by `logFlushed.Wait()`
logFlushed.Add(1)
go func() {
scanner := bufio.NewScanner(logReader)
for scanner.Scan() {
bs.Write(scanner.Bytes())
bs.WriteString("\n")
}
logFlushed.Done()
}()
defer func() {
// This is here to avoid data-trace on bytes buffer `bs` to capture logs
if err := logWriter.Close(); err != nil {
panic(err)
}
logFlushed.Wait()
}()
logger := helmexec.NewLogger(logWriter, "debug")
app := appWithFs(&App{
OverrideHelmBinary: DefaultHelmBinary,
glob: filepath.Glob,
abs: filepath.Abs,
OverrideKubeContext: "default",
Env: "default",
Logger: logger,
helms: map[helmKey]helmexec.Interface{
createHelmKey("helm", "default"): helm,
},
}, tc.files)
depsErr := app.Deps(depsConfig{
skipRepos: false,
})
if tc.error == "" && depsErr != nil {
t.Fatalf("unexpected error for data defined at %s: %v", tc.loc, depsErr)
} else if tc.error != "" && depsErr == nil {
t.Fatalf("expected error did not occur for data defined at %s", tc.loc)
} else if tc.error != "" && depsErr != nil && tc.error != depsErr.Error() {
t.Fatalf("invalid error: expected %q, got %q", tc.error, depsErr.Error())
}
if !reflect.DeepEqual(helm.Charts, tc.charts) {
t.Fatalf("expected charts %v, got %v", helm.Charts, tc.charts)
}
}()
if tc.log != "" {
actual := bs.String()
diff, exists := testhelper.Diff(tc.log, actual, 3)
if exists {
t.Errorf("unexpected log for data defined %s:\nDIFF\n%s\nEOD", tc.loc, diff)
}
}
})
}
}
func captureStdout(f func()) string {
reader, writer, err := os.Pipe()
if err != nil {
@@ -4160,12 +4571,8 @@ releases:
assert.NilError(t, err)
})
expected := "[" +
"{\"name\":\"myrelease1\",\"namespace\":\"\",\"enabled\":false,\"labels\":\"id:myrelease1\"}," +
"{\"name\":\"myrelease2\",\"namespace\":\"\",\"enabled\":true,\"labels\":\"\"}," +
"{\"name\":\"myrelease3\",\"namespace\":\"\",\"enabled\":true,\"labels\":\"\"}," +
"{\"name\":\"myrelease4\",\"namespace\":\"\",\"enabled\":true,\"labels\":\"id:myrelease1\"}" +
"]\n"
expected := `[{"name":"myrelease1","namespace":"","enabled":false,"labels":"id:myrelease1"},{"name":"myrelease2","namespace":"","enabled":true,"labels":""},{"name":"myrelease3","namespace":"","enabled":true,"labels":""},{"name":"myrelease4","namespace":"","enabled":true,"labels":"id:myrelease1"}]
`
assert.Equal(t, expected, out)
}
+5
View File
@@ -51,6 +51,8 @@ type ApplyConfigProvider interface {
Context() int
RetainValuesFiles() bool
SkipCleanup() bool
SkipDiffOnInstall() bool
concurrencyConfig
interactive
@@ -110,6 +112,7 @@ type TestConfigProvider interface {
Timeout() int
Cleanup() bool
Logs() bool
concurrencyConfig
}
@@ -132,7 +135,9 @@ type TemplateConfigProvider interface {
OutputDirTemplate() string
Validate() bool
SkipDeps() bool
SkipCleanup() bool
OutputDir() string
IncludeCRDs() bool
concurrencyConfig
}
+10 -8
View File
@@ -4,6 +4,8 @@ import (
"bytes"
"errors"
"fmt"
"path/filepath"
"github.com/imdario/mergo"
"github.com/roboll/helmfile/pkg/environment"
"github.com/roboll/helmfile/pkg/helmexec"
@@ -11,7 +13,6 @@ import (
"github.com/roboll/helmfile/pkg/state"
"github.com/variantdev/vals"
"go.uber.org/zap"
"path/filepath"
)
const (
@@ -25,12 +26,13 @@ type desiredStateLoader struct {
env string
namespace string
readFile func(string) ([]byte, error)
deleteFile func(string) error
fileExists func(string) (bool, error)
abs func(string) (string, error)
glob func(string) ([]string, error)
getHelm func(*state.HelmState) helmexec.Interface
readFile func(string) ([]byte, error)
deleteFile func(string) error
fileExists func(string) (bool, error)
abs func(string) (string, error)
glob func(string) ([]string, error)
directoryExistsAt func(string) bool
getHelm func(*state.HelmState) helmexec.Interface
remote *remote.Remote
logger *zap.SugaredLogger
@@ -144,7 +146,7 @@ func (ld *desiredStateLoader) loadFileWithOverrides(inheritedEnv, overrodeEnv *e
}
func (a *desiredStateLoader) underlying() *state.StateCreator {
c := state.NewCreator(a.logger, a.readFile, a.fileExists, a.abs, a.glob, a.valsRuntime, a.getHelm, a.overrideHelmBinary, a.remote)
c := state.NewCreator(a.logger, a.readFile, a.fileExists, a.abs, a.glob, a.directoryExistsAt, a.valsRuntime, a.getHelm, a.overrideHelmBinary, a.remote)
c.DeleteFile = a.deleteFile
c.LoadFile = a.loadFile
return c
+12 -1
View File
@@ -22,7 +22,14 @@ func (helm *noCallHelmExec) TemplateRelease(name, chart string, flags ...string)
helm.doPanic()
return nil
}
func (helm *noCallHelmExec) ChartPull(chart string, flags ...string) error {
helm.doPanic()
return nil
}
func (helm *noCallHelmExec) ChartExport(chart string, path string, flags ...string) error {
helm.doPanic()
return nil
}
func (helm *noCallHelmExec) UpdateDeps(chart string) error {
helm.doPanic()
return nil
@@ -49,6 +56,10 @@ func (helm *noCallHelmExec) UpdateRepo() error {
helm.doPanic()
return nil
}
func (helm *noCallHelmExec) RegistryLogin(name string, username string, password string) error {
helm.doPanic()
return nil
}
func (helm *noCallHelmExec) SyncRelease(context helmexec.HelmContext, name, chart string, flags ...string) error {
helm.doPanic()
return nil
+1 -1
View File
@@ -183,7 +183,7 @@ func (a *App) test(r *Run, c TestConfigProvider) []error {
r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
return st.TestReleases(r.helm, cleanup, timeout, concurrency)
return st.TestReleases(r.helm, cleanup, timeout, concurrency, state.Logs(c.Logs()))
}
func (r *Run) Lint(c LintConfigProvider) []error {
+5
View File
@@ -2,6 +2,7 @@ package event
import (
"fmt"
"io"
"os"
"testing"
@@ -16,6 +17,10 @@ var logger = helmexec.NewLogger(os.Stdout, "warn")
type runner struct {
}
func (r *runner) ExecuteStdIn(cmd string, args []string, env map[string]string, stdin io.Reader) ([]byte, error) {
return []byte(""), nil
}
func (r *runner) Execute(cmd string, args []string, env map[string]string) ([]byte, error) {
if cmd == "ng" {
return nil, fmt.Errorf("cmd failed due to invalid cmd: %s", cmd)
+9 -1
View File
@@ -89,6 +89,9 @@ func (helm *Helm) AddRepo(name, repository, cafile, certfile, keyfile, username,
func (helm *Helm) UpdateRepo() error {
return nil
}
func (helm *Helm) RegistryLogin(name string, username string, password string) error {
return nil
}
func (helm *Helm) SyncRelease(context helmexec.HelmContext, name, chart string, flags ...string) error {
if strings.Contains(name, "error") {
return errors.New("error")
@@ -158,7 +161,12 @@ func (helm *Helm) Lint(name, chart string, flags ...string) error {
func (helm *Helm) TemplateRelease(name, chart string, flags ...string) error {
return nil
}
func (helm *Helm) ChartPull(chart string, flags ...string) error {
return nil
}
func (helm *Helm) ChartExport(chart string, path string, flags ...string) error {
return nil
}
func (helm *Helm) IsHelm3() bool {
return false
}
+2
View File
@@ -1,6 +1,7 @@
package helmexec
import (
"io"
"os"
"path/filepath"
)
@@ -10,6 +11,7 @@ type HelmContext struct {
TillerNamespace string
HistoryMax int
WorkerIndex int
Writer io.Writer
}
func (context *HelmContext) GetTillerlessArgs(helm *execer) []string {
+97 -27
View File
@@ -1,6 +1,7 @@
package helmexec
import (
"bytes"
"fmt"
"io"
"io/ioutil"
@@ -29,6 +30,7 @@ type execer struct {
extra []string
decryptedSecretMutex sync.Mutex
decryptedSecrets map[string]*decryptedSecret
writeTempFile func([]byte) (string, error)
}
func NewLogger(writer io.Writer, logLevel string) *zap.SugaredLogger {
@@ -58,7 +60,7 @@ func parseHelmVersion(versionStr string) (semver.Version, error) {
ver, err := semver.NewVersion(versionStr)
if err != nil {
return semver.Version{}, fmt.Errorf("error parsing helm verion '%s'", versionStr)
return semver.Version{}, fmt.Errorf("error parsing helm version '%s'", versionStr)
}
// Support explicit helm3 opt-in via environment variable
@@ -71,13 +73,13 @@ func parseHelmVersion(versionStr string) (semver.Version, error) {
func getHelmVersion(helmBinary string, runner Runner) (semver.Version, error) {
// Autodetect from `helm verison`
bytes, err := runner.Execute(helmBinary, []string{"version", "--client", "--short"}, nil)
// Autodetect from `helm version`
outBytes, err := runner.Execute(helmBinary, []string{"version", "--client", "--short"}, nil)
if err != nil {
return semver.Version{}, fmt.Errorf("error determining helm version: %w", err)
}
return parseHelmVersion(string(bytes))
return parseHelmVersion(string(outBytes))
}
// New for running helm commands
@@ -121,7 +123,7 @@ func (helm *execer) AddRepo(name, repository, cafile, certfile, keyfile, usernam
args = append(args, "repo", "add", name, repository)
// See https://github.com/helm/helm/pull/8777
if cons, err := semver.NewConstraint(">= 3.3.2, < 3.3.4"); err == nil {
if cons, err := semver.NewConstraint(">= 3.3.2"); err == nil {
if cons.Check(&helm.version) {
args = append(args, "--force-update")
}
@@ -156,6 +158,23 @@ func (helm *execer) UpdateRepo() error {
return err
}
func (helm *execer) RegistryLogin(repository string, username string, password string) error {
helm.logger.Info("Logging in to registry")
args := []string{
"registry",
"login",
repository,
"--username",
username,
"--password-stdin",
}
buffer := bytes.Buffer{}
buffer.Write([]byte(fmt.Sprintf("%s\n", password)))
out, err := helm.execStdIn(args, map[string]string{"HELM_EXPERIMENTAL_OCI": "1"}, &buffer)
helm.info(out)
return err
}
func (helm *execer) BuildDeps(name, chart string) error {
helm.logger.Infof("Building dependency release=%v, chart=%v", name, chart)
out, err := helm.exec([]string{"dependency", "build", chart}, map[string]string{})
@@ -182,7 +201,7 @@ func (helm *execer) SyncRelease(context HelmContext, name, chart string, flags .
}
out, err := helm.exec(append(append(preArgs, "upgrade", "--install", "--reset-values", name, chart), flags...), env)
helm.write(out)
helm.write(nil, out)
return err
}
@@ -191,7 +210,7 @@ func (helm *execer) ReleaseStatus(context HelmContext, name string, flags ...str
preArgs := context.GetTillerlessArgs(helm)
env := context.getTillerlessEnv()
out, err := helm.exec(append(append(preArgs, "status", name), flags...), env)
helm.write(out)
helm.write(nil, out)
return err
}
@@ -218,7 +237,7 @@ func (helm *execer) List(context HelmContext, filter string, flags ...string) (s
lines = lines[1:]
out = []byte(strings.Join(lines, "\n"))
}
helm.write(out)
helm.write(nil, out)
return string(out), err
}
@@ -278,16 +297,32 @@ func (helm *execer) DecryptSecret(context HelmContext, name string, flags ...str
defer secret.mutex.RUnlock()
}
tmpFile, err := ioutil.TempFile("", "secret")
if err != nil {
return "", err
tempFile := helm.writeTempFile
if tempFile == nil {
tempFile = func(content []byte) (string, error) {
tmpFile, err := ioutil.TempFile("", "secret")
if err != nil {
return "", err
}
_, err = tmpFile.Write(content)
if err != nil {
return "", err
}
return tmpFile.Name(), nil
}
}
_, err = tmpFile.Write(secret.bytes)
tmpFileName, err := tempFile(secret.bytes)
if err != nil {
return "", err
}
return tmpFile.Name(), err
helm.logger.Debugf("Decrypted %s into %s", absPath, tmpFileName)
return tmpFileName, err
}
func (helm *execer) TemplateRelease(name string, chart string, flags ...string) error {
@@ -300,12 +335,16 @@ func (helm *execer) TemplateRelease(name string, chart string, flags ...string)
}
out, err := helm.exec(append(args, flags...), map[string]string{})
helm.write(out)
helm.write(nil, out)
return err
}
func (helm *execer) DiffRelease(context HelmContext, name, chart string, suppressDiff bool, flags ...string) error {
helm.logger.Infof("Comparing release=%v, chart=%v", name, chart)
if context.Writer != nil {
fmt.Fprintf(context.Writer, "Comparing release=%v, chart=%v\n", name, chart)
} else {
helm.logger.Infof("Comparing release=%v, chart=%v", name, chart)
}
preArgs := context.GetTillerlessArgs(helm)
env := context.getTillerlessEnv()
out, err := helm.exec(append(append(preArgs, "diff", "upgrade", "--reset-values", "--allow-unreleased", name, chart), flags...), env)
@@ -323,13 +362,13 @@ func (helm *execer) DiffRelease(context HelmContext, name, chart string, suppres
case ExitError:
if e.ExitStatus() == 2 {
if !(suppressDiff) {
helm.write(out)
helm.write(context.Writer, out)
}
return err
}
}
} else if !(suppressDiff) {
helm.write(out)
helm.write(context.Writer, out)
}
return err
}
@@ -337,7 +376,7 @@ func (helm *execer) DiffRelease(context HelmContext, name, chart string, suppres
func (helm *execer) Lint(name, chart string, flags ...string) error {
helm.logger.Infof("Linting release=%v, chart=%v", name, chart)
out, err := helm.exec(append([]string{"lint", chart}, flags...), map[string]string{})
helm.write(out)
helm.write(nil, out)
return err
}
@@ -348,12 +387,26 @@ func (helm *execer) Fetch(chart string, flags ...string) error {
return err
}
func (helm *execer) ChartPull(chart string, flags ...string) error {
helm.logger.Infof("Pulling %v", chart)
out, err := helm.exec(append([]string{"chart", "pull", chart}, flags...), map[string]string{"HELM_EXPERIMENTAL_OCI": "1"})
helm.info(out)
return err
}
func (helm *execer) ChartExport(chart string, path string, flags ...string) error {
helm.logger.Infof("Exporting %v", chart)
out, err := helm.exec(append([]string{"chart", "export", chart, "--destination", path}, flags...), map[string]string{"HELM_EXPERIMENTAL_OCI": "1"})
helm.info(out)
return err
}
func (helm *execer) DeleteRelease(context HelmContext, name string, flags ...string) error {
helm.logger.Infof("Deleting %v", name)
preArgs := context.GetTillerlessArgs(helm)
env := context.getTillerlessEnv()
out, err := helm.exec(append(append(preArgs, "delete", name), flags...), env)
helm.write(out)
helm.write(nil, out)
return err
}
@@ -363,7 +416,7 @@ func (helm *execer) TestRelease(context HelmContext, name string, flags ...strin
env := context.getTillerlessEnv()
args := []string{"test", name}
out, err := helm.exec(append(append(preArgs, args...), flags...), env)
helm.write(out)
helm.write(nil, out)
return err
}
@@ -377,17 +430,31 @@ func (helm *execer) exec(args []string, env map[string]string) ([]byte, error) {
}
cmd := fmt.Sprintf("exec: %s %s", helm.helmBinary, strings.Join(cmdargs, " "))
helm.logger.Debug(cmd)
bytes, err := helm.runner.Execute(helm.helmBinary, cmdargs, env)
return bytes, err
outBytes, err := helm.runner.Execute(helm.helmBinary, cmdargs, env)
return outBytes, err
}
func (helm *execer) execStdIn(args []string, env map[string]string, stdin io.Reader) ([]byte, error) {
cmdargs := args
if len(helm.extra) > 0 {
cmdargs = append(cmdargs, helm.extra...)
}
if helm.kubeContext != "" {
cmdargs = append([]string{"--kube-context", helm.kubeContext}, cmdargs...)
}
cmd := fmt.Sprintf("exec: %s %s", helm.helmBinary, strings.Join(cmdargs, " "))
helm.logger.Debug(cmd)
outBytes, err := helm.runner.ExecuteStdIn(helm.helmBinary, cmdargs, env, stdin)
return outBytes, err
}
func (helm *execer) azcli(name string) ([]byte, error) {
cmdargs := append(strings.Split("acr helm repo add --name", " "), name)
cmd := fmt.Sprintf("exec: az %s", strings.Join(cmdargs, " "))
helm.logger.Debug(cmd)
bytes, err := helm.runner.Execute("az", cmdargs, map[string]string{})
helm.logger.Debugf("%s: %s", cmd, bytes)
return bytes, err
outBytes, err := helm.runner.Execute("az", cmdargs, map[string]string{})
helm.logger.Debugf("%s: %s", cmd, outBytes)
return outBytes, err
}
func (helm *execer) info(out []byte) {
@@ -396,9 +463,12 @@ func (helm *execer) info(out []byte) {
}
}
func (helm *execer) write(out []byte) {
func (helm *execer) write(w io.Writer, out []byte) {
if len(out) > 0 {
fmt.Printf("%s\n", out)
if w == nil {
w = os.Stdout
}
fmt.Fprintf(w, "%s\n", out)
}
}
+16 -3
View File
@@ -3,6 +3,8 @@ package helmexec
import (
"bytes"
"fmt"
"github.com/google/go-cmp/cmp"
"io"
"os"
"path"
"path/filepath"
@@ -20,6 +22,10 @@ type mockRunner struct {
err error
}
func (mock *mockRunner) ExecuteStdIn(cmd string, args []string, env map[string]string, stdin io.Reader) ([]byte, error) {
return mock.output, mock.err
}
func (mock *mockRunner) Execute(cmd string, args []string, env map[string]string) ([]byte, error) {
return mock.output, mock.err
}
@@ -257,6 +263,12 @@ func Test_DecryptSecret(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
tmpFilePath := "path/to/temp/file"
helm.writeTempFile = func(content []byte) (string, error) {
return tmpFilePath, nil
}
helm.DecryptSecret(HelmContext{}, "secretName")
cwd, err := filepath.Abs(".")
if err != nil {
@@ -270,9 +282,10 @@ Decrypting secret %s/secretName
exec: helm --kube-context dev secrets dec %s/secretName
Preparing to decrypt secret %s/secretName
Found secret in cache %s/secretName
`, cwd, cwd, cwd, cwd, cwd)
if buffer.String() != expected {
t.Errorf("helmexec.DecryptSecret()\nactual = %v\nexpect = %v", buffer.String(), expected)
Decrypted %s/secretName into path/to/temp/file
`, cwd, cwd, cwd, cwd, cwd, cwd)
if d := cmp.Diff(expected, buffer.String()); d != "" {
t.Errorf("helmexec.DecryptSecret(): want (-), got (+):\n%s", d)
}
}
+3
View File
@@ -14,12 +14,15 @@ type Interface interface {
AddRepo(name, repository, cafile, certfile, keyfile, username, password string, managed string) error
UpdateRepo() error
RegistryLogin(name string, username string, password string) error
BuildDeps(name, chart string) error
UpdateDeps(chart string) error
SyncRelease(context HelmContext, name, chart string, flags ...string) error
DiffRelease(context HelmContext, name, chart string, suppressDiff bool, flags ...string) error
TemplateRelease(name, chart string, flags ...string) error
Fetch(chart string, flags ...string) error
ChartPull(chart string, flags ...string) error
ChartExport(chart string, path string, flags ...string) error
Lint(name, chart string, flags ...string) error
ReleaseStatus(context HelmContext, name string, flags ...string) error
DeleteRelease(context HelmContext, name string, flags ...string) error
+12
View File
@@ -22,6 +22,7 @@ const (
// Runner interface for shell commands
type Runner interface {
Execute(cmd string, args []string, env map[string]string) ([]byte, error)
ExecuteStdIn(cmd string, args []string, env map[string]string, stdin io.Reader) ([]byte, error)
}
// ShellRunner implemention for shell commands
@@ -41,6 +42,17 @@ func (shell ShellRunner) Execute(cmd string, args []string, env map[string]strin
})
}
// Execute a shell command
func (shell ShellRunner) ExecuteStdIn(cmd string, args []string, env map[string]string, stdin io.Reader) ([]byte, error) {
preparedCmd := exec.Command(cmd, args...)
preparedCmd.Dir = shell.Dir
preparedCmd.Env = mergeEnv(os.Environ(), env)
preparedCmd.Stdin = stdin
return Output(preparedCmd, &logWriterGenerator{
log: shell.Logger,
})
}
func Output(c *exec.Cmd, logWriterGenerators ...*logWriterGenerator) ([]byte, error) {
if c.Stdout != nil {
return nil, errors.New("exec: Stdout already set")
+13 -12
View File
@@ -2,19 +2,20 @@ package remote
import (
"fmt"
"os"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/testhelper"
"os"
"testing"
)
func TestRemote_HttpsGitHub(t *testing.T) {
cleanfs := map[string]string{
"path/to/home": "",
"/path/to/home": "",
}
cachefs := map[string]string{
"path/to/home/.helmfile/cache/https_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml": "foo: bar",
"/path/to/home/.helmfile/cache/https_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml": "foo: bar",
}
type testcase struct {
@@ -36,7 +37,7 @@ func TestRemote_HttpsGitHub(t *testing.T) {
hit := true
get := func(wd, src, dst string) error {
if wd != "path/to/home" {
if wd != "/path/to/home" {
return fmt.Errorf("unexpected wd: %s", wd)
}
if src != "git::https://github.com/cloudposse/helmfiles.git?ref=0.40.0" {
@@ -53,7 +54,7 @@ func TestRemote_HttpsGitHub(t *testing.T) {
}
remote := &Remote{
Logger: helmexec.NewLogger(os.Stderr, "debug"),
Home: "path/to/home",
Home: "/path/to/home",
Getter: getter,
ReadFile: testfs.ReadFile,
FileExists: testfs.FileExistsAt,
@@ -72,7 +73,7 @@ func TestRemote_HttpsGitHub(t *testing.T) {
t.Fatalf("unexpected error: %v", err)
}
if file != "path/to/home/.helmfile/cache/https_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml" {
if file != "/path/to/home/.helmfile/cache/https_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml" {
t.Errorf("unexpected file located: %s", file)
}
@@ -88,10 +89,10 @@ func TestRemote_HttpsGitHub(t *testing.T) {
func TestRemote_SShGitHub(t *testing.T) {
cleanfs := map[string]string{
"path/to/home": "",
"/path/to/home": "",
}
cachefs := map[string]string{
"path/to/home/.helmfile/cache/ssh_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml": "foo: bar",
"/path/to/home/.helmfile/cache/ssh_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml": "foo: bar",
}
type testcase struct {
@@ -113,7 +114,7 @@ func TestRemote_SShGitHub(t *testing.T) {
hit := true
get := func(wd, src, dst string) error {
if wd != "path/to/home" {
if wd != "/path/to/home" {
return fmt.Errorf("unexpected wd: %s", wd)
}
if src != "git::ssh://git@github.com/cloudposse/helmfiles.git?ref=0.40.0" {
@@ -130,7 +131,7 @@ func TestRemote_SShGitHub(t *testing.T) {
}
remote := &Remote{
Logger: helmexec.NewLogger(os.Stderr, "debug"),
Home: "path/to/home",
Home: "/path/to/home",
Getter: getter,
ReadFile: testfs.ReadFile,
FileExists: testfs.FileExistsAt,
@@ -143,7 +144,7 @@ func TestRemote_SShGitHub(t *testing.T) {
t.Fatalf("unexpected error: %v", err)
}
if file != "path/to/home/.helmfile/cache/ssh_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml" {
if file != "/path/to/home/.helmfile/cache/ssh_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml" {
t.Errorf("unexpected file located: %s", file)
}
+25 -16
View File
@@ -4,11 +4,12 @@ import (
"bytes"
"errors"
"fmt"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/remote"
"io"
"os"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/remote"
"github.com/imdario/mergo"
"github.com/roboll/helmfile/pkg/environment"
"github.com/roboll/helmfile/pkg/maputil"
@@ -39,12 +40,15 @@ func (e *UndefinedEnvError) Error() string {
}
type StateCreator struct {
logger *zap.SugaredLogger
readFile func(string) ([]byte, error)
fileExists func(string) (bool, error)
abs func(string) (string, error)
glob func(string) ([]string, error)
DeleteFile func(string) error
logger *zap.SugaredLogger
readFile func(string) ([]byte, error)
fileExists func(string) (bool, error)
abs func(string) (string, error)
glob func(string) ([]string, error)
DeleteFile func(string) error
directoryExistsAt func(string) bool
valsRuntime vals.Evaluator
Strict bool
@@ -58,13 +62,16 @@ type StateCreator struct {
remote *remote.Remote
}
func NewCreator(logger *zap.SugaredLogger, readFile func(string) ([]byte, error), fileExists func(string) (bool, error), abs func(string) (string, error), glob func(string) ([]string, error), valsRuntime vals.Evaluator, getHelm func(*HelmState) helmexec.Interface, overrideHelmBinary string, remote *remote.Remote) *StateCreator {
func NewCreator(logger *zap.SugaredLogger, readFile func(string) ([]byte, error), fileExists func(string) (bool, error), abs func(string) (string, error), glob func(string) ([]string, error), directoryExistsAt func(string) bool, valsRuntime vals.Evaluator, getHelm func(*HelmState) helmexec.Interface, overrideHelmBinary string, remote *remote.Remote) *StateCreator {
return &StateCreator{
logger: logger,
readFile: readFile,
fileExists: fileExists,
abs: abs,
glob: glob,
logger: logger,
readFile: readFile,
fileExists: fileExists,
abs: abs,
glob: glob,
directoryExistsAt: directoryExistsAt,
Strict: true,
valsRuntime: valsRuntime,
getHelm: getHelm,
@@ -131,6 +138,7 @@ func (c *StateCreator) Parse(content []byte, baseDir, file string) (*HelmState,
state.removeFile = os.Remove
state.fileExists = c.fileExists
state.glob = c.glob
state.directoryExistsAt = c.directoryExistsAt
state.valsRuntime = c.valsRuntime
return &state, nil
@@ -264,6 +272,7 @@ func (c *StateCreator) loadEnvValues(st *HelmState, name string, failOnMissingEn
func (c *StateCreator) scatterGatherEnvSecretFiles(st *HelmState, envSecretFiles []string, envVals map[string]interface{}, readFile func(string) ([]byte, error)) error {
var errs []error
helm := c.getHelm(st)
inputs := envSecretFiles
inputsSize := len(inputs)
@@ -286,8 +295,8 @@ func (c *StateCreator) scatterGatherEnvSecretFiles(st *HelmState, envSecretFiles
func(id int) {
for path := range secrets {
release := &ReleaseSpec{}
flags := st.appendConnectionFlags([]string{}, release)
decFile, err := c.getHelm(st).DecryptSecret(st.createHelmContext(release, 0), path, flags...)
flags := st.appendConnectionFlags([]string{}, helm, release)
decFile, err := helm.DecryptSecret(st.createHelmContext(release, 0), path, flags...)
if err != nil {
results <- secretResult{nil, err, path}
continue
+5 -4
View File
@@ -1,13 +1,14 @@
package state
import (
"github.com/roboll/helmfile/pkg/remote"
"io/ioutil"
"os"
"path/filepath"
"reflect"
"testing"
"github.com/roboll/helmfile/pkg/remote"
"github.com/roboll/helmfile/pkg/testhelper"
"go.uber.org/zap"
@@ -83,7 +84,7 @@ func (testEnv stateTestEnv) MustLoadState(t *testing.T, file, envName string) *H
}
r := remote.NewRemote(logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
state, err := NewCreator(logger, testFs.ReadFile, testFs.FileExists, testFs.Abs, testFs.Glob, nil, nil, "", r).
state, err := NewCreator(logger, testFs.ReadFile, testFs.FileExists, testFs.Abs, testFs.Glob, testFs.DirectoryExistsAt, nil, nil, "", r).
ParseAndLoad([]byte(yamlContent), filepath.Dir(file), file, envName, true, nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
@@ -148,7 +149,7 @@ releaseNamespace: mynamespace
testFs.Cwd = "/example/path/to"
r := remote.NewRemote(logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
state, err := NewCreator(logger, testFs.ReadFile, testFs.FileExists, testFs.Abs, testFs.Glob, nil, nil, "", r).
state, err := NewCreator(logger, testFs.ReadFile, testFs.FileExists, testFs.Abs, testFs.Glob, testFs.DirectoryExistsAt, nil, nil, "", r).
ParseAndLoad(yamlContent, filepath.Dir(yamlFile), yamlFile, "production", true, nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
@@ -235,7 +236,7 @@ overrideNamespace: myns
testFs.Cwd = "/example/path/to"
r := remote.NewRemote(logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
state, err := NewCreator(logger, testFs.ReadFile, testFs.FileExists, testFs.Abs, testFs.Glob, nil, nil, "", r).
state, err := NewCreator(logger, testFs.ReadFile, testFs.FileExists, testFs.Abs, testFs.Glob, testFs.DirectoryExistsAt, nil, nil, "", r).
ParseAndLoad(yamlContent, filepath.Dir(yamlFile), yamlFile, "production", true, nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
+16
View File
@@ -144,6 +144,22 @@ func (st *HelmState) PrepareChartify(helm helmexec.Interface, release *ReleaseSp
shouldRun = true
}
transformers := release.Transformers
if len(transformers) > 0 {
generatedFiles, err := st.generateTemporaryReleaseValuesFiles(release, transformers, release.MissingFileHandler)
if err != nil {
return nil, clean, err
}
for _, f := range generatedFiles {
chartify.Opts.Transformers = append(chartify.Opts.Transformers, f)
}
filesNeedCleaning = append(filesNeedCleaning, generatedFiles...)
shouldRun = true
}
if release.ForceNamespace != "" {
chartify.Opts.OverrideNamespace = release.ForceNamespace
+353 -62
View File
@@ -70,7 +70,7 @@ type ReleaseSetSpec struct {
// If set to "Error", return an error when a subhelmfile points to a
// non-existent path. The default behavior is to print a warning. Note the
// differing default compared to other MissingFileHandlers.
MissingFileHandler string `yaml:"missingFileHandler"`
MissingFileHandler string `yaml:"missingFileHandler,omitempty"`
}
// HelmState structure for the helmfile
@@ -82,12 +82,12 @@ type HelmState struct {
logger *zap.SugaredLogger
readFile func(string) ([]byte, error)
removeFile func(string) error
fileExists func(string) (bool, error)
glob func(string) ([]string, error)
tempDir func(string, string) (string, error)
readFile func(string) ([]byte, error)
removeFile func(string) error
fileExists func(string) (bool, error)
glob func(string) ([]string, error)
tempDir func(string, string) (string, error)
directoryExistsAt func(string) bool
runner helmexec.Runner
valsRuntime vals.Evaluator
@@ -139,6 +139,10 @@ type HelmSpec struct {
HistoryMax *int `yaml:"historyMax,omitempty"`
// CreateNamespace, when set to true (default), --create-namespace is passed to helm3 on install/upgrade (ignored for helm2)
CreateNamespace *bool `yaml:"createNamespace,omitempty"`
// SkipDeps disables running `helm dependency up` and `helm dependency build` on this release's chart.
// This is relevant only when your release uses a local chart or a directory containing K8s manifests or a Kustomization
// as a Helm chart.
SkipDeps bool `yaml:"skipDeps"`
TLS bool `yaml:"tls"`
TLSCACert string `yaml:"tlsCACert,omitempty"`
@@ -158,6 +162,7 @@ type RepositorySpec struct {
Username string `yaml:"username,omitempty"`
Password string `yaml:"password,omitempty"`
Managed string `yaml:"managed,omitempty"`
OCI bool `yaml:"oci,omitempty"`
}
// ReleaseSpec defines the structure of a helm release
@@ -207,6 +212,10 @@ type ReleaseSpec struct {
// FYI, such diff without `--disable-validation` fails on first install because the K8s cluster doesn't have CRDs registered yet.
DisableValidation *bool `yaml:"disableValidation,omitempty"`
// DisableValidationOnInstall disables the K8s API validation while running helm-diff on the release being newly installed on helmfile-apply.
// It is useful when any release contains custom resources for CRDs that is not yet installed onto the cluster.
DisableValidationOnInstall *bool `yaml:"disableValidationOnInstall,omitempty"`
// MissingFileHandler is set to either "Error" or "Warn". "Error" instructs helmfile to fail when unable to find a values or secrets file. When "Warn", it prints the file and continues.
// The default value for MissingFileHandler is "Error".
MissingFileHandler *string `yaml:"missingFileHandler,omitempty"`
@@ -252,7 +261,14 @@ type ReleaseSpec struct {
Dependencies []Dependency `yaml:"dependencies,omitempty"`
JSONPatches []interface{} `yaml:"jsonPatches,omitempty"`
StrategicMergePatches []interface{} `yaml:"strategicMergePatches,omitempty"`
Adopt []string `yaml:"adopt,omitempty"`
// Transformers is the list of Kustomize transformers
//
// Each item can be a path to a YAML or go template file, or an embedded transformer declaration as a YAML hash.
// It's often used to add common labels and annotations to your resources.
// See https://github.com/kubernetes-sigs/kustomize/blob/master/examples/configureBuiltinPlugin.md#configuring-the-builtin-plugins-instead for more information.
Transformers []interface{} `yaml:"transformers,omitempty"`
Adopt []string `yaml:"adopt,omitempty"`
//version of the chart that has really been installed cause desired version may be fuzzy (~2.0.0)
installedVersion string
@@ -269,7 +285,12 @@ type ReleaseSpec struct {
// This is only needed when you can't FIX your chart to have `namespace: {{ .Namespace }}` AND you're using `helmfile template`.
// In standard use-cases, `Namespace` should be sufficient.
// Use this only when you know what you want to do!
ForceNamespace string `yaml:"forceNamespace"`
ForceNamespace string `yaml:"forceNamespace,omitempty"`
// SkipDeps disables running `helm dependency up` and `helm dependency build` on this release's chart.
// This is relevant only when your release uses a local chart or a directory containing K8s manifests or a Kustomization
// as a Helm chart.
SkipDeps *bool `yaml:"skipDeps,omitempty"`
}
type Release struct {
@@ -316,6 +337,7 @@ func (st *HelmState) ApplyOverrides(spec *ReleaseSpec) {
type RepoUpdater interface {
AddRepo(name, repository, cafile, certfile, keyfile, username, password string, managed string) error
UpdateRepo() error
RegistryLogin(name string, username string, password string) error
}
// getRepositoriesToSync returns the names of repositories to be updated
@@ -355,8 +377,18 @@ func (st *HelmState) SyncRepos(helm RepoUpdater, shouldSkip map[string]bool) ([]
if shouldSkip[repo.Name] {
continue
}
var err error
if repo.OCI {
username, password := gatherOCIUsernamePassword(repo.Name, repo.Username, repo.Password)
if username == "" || password == "" {
return nil, fmt.Errorf("username and password are required fields for logging in to OCI registries with helm")
}
err = helm.RegistryLogin(repo.URL, username, password)
} else {
err = helm.AddRepo(repo.Name, repo.URL, repo.CaFile, repo.CertFile, repo.KeyFile, repo.Username, repo.Password, repo.Managed)
}
if err := helm.AddRepo(repo.Name, repo.URL, repo.CaFile, repo.CertFile, repo.KeyFile, repo.Username, repo.Password, repo.Managed); err != nil {
if err != nil {
return nil, err
}
@@ -366,6 +398,24 @@ func (st *HelmState) SyncRepos(helm RepoUpdater, shouldSkip map[string]bool) ([]
return updated, nil
}
func gatherOCIUsernamePassword(repoName string, username string, password string) (string, string) {
var user, pass string
if username != "" {
user = username
} else if u := os.Getenv(fmt.Sprintf("%s_USERNAME", strings.ToUpper(repoName))); u != "" {
user = u
}
if password != "" {
pass = password
} else if p := os.Getenv(fmt.Sprintf("%s_PASSWORD", strings.ToUpper(repoName))); p != "" {
pass = p
}
return user, pass
}
type syncResult struct {
errors []*ReleaseError
}
@@ -526,7 +576,8 @@ func (st *HelmState) DetectReleasesToBeDeleted(helm helmexec.Interface, releases
}
type SyncOpts struct {
Set []string
Set []string
SkipCleanup bool
}
type SyncOpt interface{ Apply(*SyncOpts) }
@@ -598,7 +649,7 @@ func (st *HelmState) DeleteReleasesForSync(affectedReleases *AffectedReleases, h
} else {
args = []string{"--purge"}
}
deletionFlags := st.appendConnectionFlags(args, release)
deletionFlags := st.appendConnectionFlags(args, helm, release)
m.Lock()
if _, err := st.triggerReleaseEvent("preuninstall", nil, release, "sync"); err != nil {
affectedReleases.Failed = append(affectedReleases.Failed, release)
@@ -660,6 +711,10 @@ func (st *HelmState) SyncReleases(affectedReleases *AffectedReleases, helm helme
preps, prepErrs := st.prepareSyncReleases(helm, additionalValues, workerLimit, opts)
defer func() {
if opts.SkipCleanup {
return
}
for _, p := range preps {
st.removeFiles(p.files)
}
@@ -708,7 +763,7 @@ func (st *HelmState) SyncReleases(affectedReleases *AffectedReleases, helm helme
} else {
args = []string{"--purge"}
}
deletionFlags := st.appendConnectionFlags(args, release)
deletionFlags := st.appendConnectionFlags(args, helm, release)
m.Lock()
if _, err := st.triggerReleaseEvent("preuninstall", nil, release, "sync"); err != nil {
affectedReleases.Failed = append(affectedReleases.Failed, release)
@@ -777,7 +832,7 @@ func (st *HelmState) SyncReleases(affectedReleases *AffectedReleases, helm helme
}
func (st *HelmState) listReleases(context helmexec.HelmContext, helm helmexec.Interface, release *ReleaseSpec) (string, error) {
flags := st.connectionFlags(release)
flags := st.connectionFlags(helm, release)
if helm.IsHelm3() && release.Namespace != "" {
flags = append(flags, "--namespace", release.Namespace)
}
@@ -819,6 +874,7 @@ func releasesNeedCharts(releases []ReleaseSpec) []ReleaseSpec {
type ChartPrepareOptions struct {
ForceDownload bool
SkipRepos bool
SkipDeps bool
SkipResolve bool
}
@@ -831,6 +887,20 @@ type chartPrepareResult struct {
chartFetchedByGoGetter bool
}
func (st *HelmState) GetRepositoryAndNameFromChartName(chartName string) (*RepositorySpec, string) {
chart := strings.Split(chartName, "/")
if len(chart) == 1 {
return nil, chartName
}
repo := chart[0]
for _, r := range st.Repositories {
if r.Name == repo {
return &r, strings.Join(chart[1:], "/")
}
}
return nil, chartName
}
// PrepareCharts creates temporary directories of charts.
//
// Each resulting "chart" can be one of the followings:
@@ -846,7 +916,22 @@ type chartPrepareResult struct {
//
// If exists, it will also patch resources by json patches, strategic-merge patches, and injectors.
func (st *HelmState) PrepareCharts(helm helmexec.Interface, dir string, concurrency int, helmfileCommand string, opts ChartPrepareOptions) (map[string]string, []error) {
releases := releasesNeedCharts(st.Releases)
var selected []ReleaseSpec
if len(st.Selectors) > 0 {
var err error
// This and releasesNeedCharts ensures that we run operations like helm-dep-build and prepare-hook calls only on
// releases that are (1) selected by the selectors and (2) to be installed.
selected, err = st.GetSelectedReleasesWithOverrides()
if err != nil {
return nil, []error{err}
}
} else {
selected = st.Releases
}
releases := releasesNeedCharts(selected)
temp := make(map[string]string, len(releases))
@@ -899,9 +984,16 @@ func (st *HelmState) PrepareCharts(helm helmexec.Interface, dir string, concurre
results <- &chartPrepareResult{err: fmt.Errorf("release %q: %w", release.Name, err)}
return
}
chartFetchedByGoGetter := chartPath != chartName
isOCI, chartPath, err := st.getOCIChart(release, dir, helm)
if err != nil {
results <- &chartPrepareResult{err: fmt.Errorf("release %q: %w", release.Name, err)}
return
}
isLocal := st.directoryExistsAt(normalizeChart(st.basePath, chartName))
chartification, clean, err := st.PrepareChartify(helm, release, chartPath, workerIndex)
defer clean()
if err != nil {
@@ -911,21 +1003,35 @@ func (st *HelmState) PrepareCharts(helm helmexec.Interface, dir string, concurre
var buildDeps bool
skipDepsGlobal := opts.SkipDeps
skipDepsRelease := release.SkipDeps != nil && *release.SkipDeps
skipDepsDefault := release.SkipDeps == nil && st.HelmDefaults.SkipDeps
skipDeps := !isLocal || skipDepsGlobal || skipDepsRelease || skipDepsDefault || !isOCI
if chartification != nil {
c := chartify.New(
chartify.HelmBin(st.DefaultHelmBinary),
chartify.UseHelm3(helm3),
)
out, err := c.Chartify(release.Name, chartPath, chartify.WithChartifyOpts(chartification.Opts))
chartifyOpts := chartification.Opts
if skipDeps {
chartifyOpts.SkipDeps = true
}
out, err := c.Chartify(release.Name, chartPath, chartify.WithChartifyOpts(chartifyOpts))
if err != nil {
results <- &chartPrepareResult{err: err}
return
} else {
// TODO Chartify
chartPath = out
}
} else if pathExists(normalizeChart(st.basePath, chartPath)) {
// Skip `helm dep build` and `helm dep up` altogether when the chart is from remote or the dep is
// explicitly skipped.
buildDeps = !skipDeps
} else if normalizedChart := normalizeChart(st.basePath, chartPath); st.directoryExistsAt(normalizedChart) {
// At this point, we are sure that chartPath is a local directory containing either:
// - A remote chart fetched by go-getter or
// - A local chart
@@ -949,9 +1055,9 @@ func (st *HelmState) PrepareCharts(helm helmexec.Interface, dir string, concurre
// Given that, we always run `helm dep build` on the chart here, but tolerate any error caused by it
// for a remote chart, so that the user can notice/fix the issue in a local chart while
// a broken remote chart won't completely block their job.
chartPath = normalizeChart(st.basePath, chartPath)
chartPath = normalizedChart
buildDeps = !opts.SkipRepos
buildDeps = !skipDeps
} else if !opts.ForceDownload {
// At this point, we are sure that either:
// 1. It is a local chart and we can use it in later process (helm upgrade/template/lint/etc)
@@ -1084,7 +1190,9 @@ func (st *HelmState) runHelmDepBuilds(helm helmexec.Interface, concurrency int,
type TemplateOpts struct {
Set []string
SkipCleanup bool
OutputDirTemplate string
IncludeCRDs bool
}
type TemplateOpt interface{ Apply(*TemplateOpts) }
@@ -1094,7 +1202,9 @@ func (o *TemplateOpts) Apply(opts *TemplateOpts) {
}
// TemplateReleases wrapper for executing helm template on the releases
func (st *HelmState) TemplateReleases(helm helmexec.Interface, outputDir string, additionalValues []string, args []string, workerLimit int, validate bool, opt ...TemplateOpt) []error {
func (st *HelmState) TemplateReleases(helm helmexec.Interface, outputDir string, additionalValues []string, args []string, workerLimit int,
validate bool, opt ...TemplateOpt) []error {
opts := &TemplateOpts{}
for _, o := range opt {
o.Apply(opts)
@@ -1114,6 +1224,10 @@ func (st *HelmState) TemplateReleases(helm helmexec.Interface, outputDir string,
flags, files, err := st.flagsForTemplate(helm, release, 0)
defer func() {
if opts.SkipCleanup {
return
}
st.removeFiles(files)
}()
@@ -1157,6 +1271,10 @@ func (st *HelmState) TemplateReleases(helm helmexec.Interface, outputDir string,
flags = append(flags, "--validate")
}
if opts.IncludeCRDs {
flags = append(flags, "--include-crds")
}
if len(errs) == 0 {
if err := helm.TemplateRelease(release.Name, release.Chart, flags...); err != nil {
errs = append(errs, err)
@@ -1349,14 +1467,17 @@ func (st *HelmState) LintReleases(helm helmexec.Interface, additionalValues []st
}
type diffResult struct {
err *ReleaseError
release *ReleaseSpec
err *ReleaseError
buf *bytes.Buffer
}
type diffPrepareResult struct {
release *ReleaseSpec
flags []string
errors []*ReleaseError
files []string
release *ReleaseSpec
flags []string
errors []*ReleaseError
files []string
upgradeDueToSkippedDiff bool
}
func (st *HelmState) prepareDiffReleases(helm helmexec.Interface, additionalValues []string, concurrency int, detailedExitCode, includeTests, suppressSecrets bool, opt ...DiffOpt) ([]diffPrepareResult, []error) {
@@ -1365,6 +1486,29 @@ func (st *HelmState) prepareDiffReleases(helm helmexec.Interface, additionalValu
o.Apply(opts)
}
mu := &sync.Mutex{}
installedReleases := map[string]bool{}
isInstalled := func(r *ReleaseSpec) bool {
mu.Lock()
defer mu.Unlock()
id := ReleaseToID(r)
if v, ok := installedReleases[id]; ok {
return v
}
v, err := st.isReleaseInstalled(st.createHelmContext(r, 0), helm, *r)
if err != nil {
st.logger.Warnf("confirming if the release is already installed or not: %v", err)
} else {
installedReleases[id] = v
}
return v
}
releases := []*ReleaseSpec{}
for i, _ := range st.Releases {
if !st.Releases[i].Desired() {
@@ -1379,6 +1523,7 @@ func (st *HelmState) prepareDiffReleases(helm helmexec.Interface, additionalValu
numReleases := len(releases)
jobs := make(chan *ReleaseSpec, numReleases)
results := make(chan diffPrepareResult, numReleases)
resultsMap := map[string]diffPrepareResult{}
rs := []diffPrepareResult{}
errs := []error{}
@@ -1400,10 +1545,17 @@ func (st *HelmState) prepareDiffReleases(helm helmexec.Interface, additionalValu
st.ApplyOverrides(release)
if opts.SkipDiffOnInstall && !isInstalled(release) {
results <- diffPrepareResult{release: release, upgradeDueToSkippedDiff: true}
continue
}
disableValidation := release.DisableValidationOnInstall != nil && *release.DisableValidationOnInstall && !isInstalled(release)
// TODO We need a long-term fix for this :)
// See https://github.com/roboll/helmfile/issues/737
mut.Lock()
flags, files, err := st.flagsForDiff(helm, release, workerIndex)
flags, files, err := st.flagsForDiff(helm, release, disableValidation, workerIndex)
mut.Unlock()
if err != nil {
errs = append(errs, err)
@@ -1466,12 +1618,18 @@ func (st *HelmState) prepareDiffReleases(helm helmexec.Interface, additionalValu
errs = append(errs, e)
}
} else if res.release != nil {
rs = append(rs, res)
resultsMap[ReleaseToID(res.release)] = res
}
}
},
)
for _, r := range releases {
if p, ok := resultsMap[ReleaseToID(r)]; ok {
rs = append(rs, p)
}
}
return rs, errs
}
@@ -1500,10 +1658,22 @@ func (st *HelmState) createHelmContext(spec *ReleaseSpec, workerIndex int) helme
}
}
func (st *HelmState) createHelmContextWithWriter(spec *ReleaseSpec, w io.Writer) helmexec.HelmContext {
ctx := st.createHelmContext(spec, 0)
ctx.Writer = w
return ctx
}
type DiffOpts struct {
Context int
NoColor bool
Set []string
SkipCleanup bool
SkipDiffOnInstall bool
}
func (o *DiffOpts) Apply(opts *DiffOpts) {
@@ -1513,7 +1683,13 @@ func (o *DiffOpts) Apply(opts *DiffOpts) {
type DiffOpt interface{ Apply(*DiffOpts) }
// DiffReleases wrapper for executing helm diff on the releases
// It returns releases that had any changes
// It returns releases that had any changes, and errors if any.
//
// This function has responsibility to stabilize the order of writes to stdout from multiple concurrent helm-diff runs.
// It's required to use the stdout from helmfile-diff to detect if there was another change(s) between 2 points in time.
// For example, terraform-provider-helmfile runs a helmfile-diff on `terraform plan` and another on `terraform apply`.
// `terraform`, by design, fails when helmfile-diff outputs were not equivalent.
// Stabilized helmfile-diff output rescues that.
func (st *HelmState) DiffReleases(helm helmexec.Interface, additionalValues []string, workerLimit int, detailedExitCode, includeTests, suppressSecrets, suppressDiff, triggerCleanupEvents bool, opt ...DiffOpt) ([]ReleaseSpec, []error) {
opts := &DiffOpts{}
for _, o := range opt {
@@ -1523,6 +1699,10 @@ func (st *HelmState) DiffReleases(helm helmexec.Interface, additionalValues []st
preps, prepErrs := st.prepareDiffReleases(helm, additionalValues, workerLimit, detailedExitCode, includeTests, suppressSecrets, opts)
defer func() {
if opts.SkipCleanup {
return
}
for _, p := range preps {
st.removeFiles(p.files)
}
@@ -1536,8 +1716,12 @@ func (st *HelmState) DiffReleases(helm helmexec.Interface, additionalValues []st
results := make(chan diffResult, len(preps))
rs := []ReleaseSpec{}
outputs := map[string]*bytes.Buffer{}
errs := []error{}
// The exit code returned by helm-diff when it detected any changes
HelmDiffExitCodeChanged := 2
st.scatterGather(
workerLimit,
len(preps),
@@ -1551,17 +1735,20 @@ func (st *HelmState) DiffReleases(helm helmexec.Interface, additionalValues []st
for prep := range jobQueue {
flags := prep.flags
release := prep.release
if err := helm.DiffRelease(st.createHelmContext(release, workerIndex), release.Name, normalizeChart(st.basePath, release.Chart), suppressDiff, flags...); err != nil {
buf := &bytes.Buffer{}
if prep.upgradeDueToSkippedDiff {
results <- diffResult{release, &ReleaseError{ReleaseSpec: release, err: nil, Code: HelmDiffExitCodeChanged}, buf}
} else if err := helm.DiffRelease(st.createHelmContextWithWriter(release, buf), release.Name, normalizeChart(st.basePath, release.Chart), suppressDiff, flags...); err != nil {
switch e := err.(type) {
case helmexec.ExitError:
// Propagate any non-zero exit status from the external command like `helm` that is failed under the hood
results <- diffResult{&ReleaseError{release, err, e.ExitStatus()}}
results <- diffResult{release, &ReleaseError{release, err, e.ExitStatus()}, buf}
default:
results <- diffResult{&ReleaseError{release, err, 0}}
results <- diffResult{release, &ReleaseError{release, err, 0}, buf}
}
} else {
// diff succeeded, found no changes
results <- diffResult{}
results <- diffResult{release, nil, buf}
}
if triggerCleanupEvents {
@@ -1576,14 +1763,22 @@ func (st *HelmState) DiffReleases(helm helmexec.Interface, additionalValues []st
res := <-results
if res.err != nil {
errs = append(errs, res.err)
if res.err.Code == 2 {
if res.err.Code == HelmDiffExitCodeChanged {
rs = append(rs, *res.err.ReleaseSpec)
}
}
outputs[ReleaseToID(res.release)] = res.buf
}
},
)
for _, p := range preps {
if stdout, ok := outputs[ReleaseToID(p.release)]; ok {
fmt.Print(stdout.String())
}
}
return rs, errs
}
@@ -1599,7 +1794,7 @@ func (st *HelmState) ReleaseStatuses(helm helmexec.Interface, workerLimit int) [
if helm.IsHelm3() && release.Namespace != "" {
flags = append(flags, "--namespace", release.Namespace)
}
flags = st.appendConnectionFlags(flags, &release)
flags = st.appendConnectionFlags(flags, helm, &release)
return helm.ReleaseStatus(st.createHelmContext(&release, workerIndex), release.Name, flags...)
})
@@ -1614,7 +1809,7 @@ func (st *HelmState) DeleteReleases(affectedReleases *AffectedReleases, helm hel
if purge && !helm.IsHelm3() {
flags = append(flags, "--purge")
}
flags = st.appendConnectionFlags(flags, &release)
flags = st.appendConnectionFlags(flags, helm, &release)
if helm.IsHelm3() && release.Namespace != "" {
flags = append(flags, "--namespace", release.Namespace)
}
@@ -1641,8 +1836,26 @@ func (st *HelmState) DeleteReleases(affectedReleases *AffectedReleases, helm hel
})
}
type TestOpts struct {
Logs bool
}
type TestOption func(*TestOpts)
func Logs(v bool) func(*TestOpts) {
return func(o *TestOpts) {
o.Logs = v
}
}
// TestReleases wrapper for executing helm test on the releases
func (st *HelmState) TestReleases(helm helmexec.Interface, cleanup bool, timeout int, concurrency int) []error {
func (st *HelmState) TestReleases(helm helmexec.Interface, cleanup bool, timeout int, concurrency int, options ...TestOption) []error {
var opts TestOpts
for _, o := range options {
o(&opts)
}
return st.scatterGatherReleases(helm, concurrency, func(release ReleaseSpec, workerIndex int) error {
if !release.Desired() {
return nil
@@ -1655,6 +1868,9 @@ func (st *HelmState) TestReleases(helm helmexec.Interface, cleanup bool, timeout
if cleanup && !helm.IsHelm3() {
flags = append(flags, "--cleanup")
}
if opts.Logs {
flags = append(flags, "--logs")
}
if timeout == EmptyTimeout {
flags = append(flags, st.timeoutFlags(helm, &release)...)
@@ -1666,7 +1882,7 @@ func (st *HelmState) TestReleases(helm helmexec.Interface, cleanup bool, timeout
flags = append(flags, "--timeout", duration)
}
flags = st.appendConnectionFlags(flags, &release)
flags = st.appendConnectionFlags(flags, helm, &release)
return helm.TestRelease(st.createHelmContext(&release, workerIndex), release.Name, flags...)
})
@@ -1845,10 +2061,12 @@ func (st *HelmState) UpdateDeps(helm helmexec.Interface) []error {
var errs []error
for _, release := range st.Releases {
if isLocalChart(release.Chart) {
if err := helm.UpdateDeps(normalizeChart(st.basePath, release.Chart)); err != nil {
if st.directoryExistsAt(release.Chart) {
if err := helm.UpdateDeps(release.Chart); err != nil {
errs = append(errs, err)
}
} else {
st.logger.Debugf("skipped updating dependencies for remote chart %s", release.Chart)
}
}
@@ -1869,11 +2087,6 @@ func (st *HelmState) UpdateDeps(helm helmexec.Interface) []error {
return nil
}
func pathExists(chart string) bool {
_, err := os.Stat(chart)
return err == nil
}
func chartNameWithoutRepository(chart string) string {
chartSplit := strings.Split(chart, "/")
return chartSplit[len(chartSplit)-1]
@@ -1905,25 +2118,27 @@ func findChartDirectory(topLevelDir string) (string, error) {
}
// appendConnectionFlags append all the helm command-line flags related to K8s API and Tiller connection including the kubecontext
func (st *HelmState) appendConnectionFlags(flags []string, release *ReleaseSpec) []string {
adds := st.connectionFlags(release)
func (st *HelmState) appendConnectionFlags(flags []string, helm helmexec.Interface, release *ReleaseSpec) []string {
adds := st.connectionFlags(helm, release)
for _, a := range adds {
flags = append(flags, a)
}
return flags
}
func (st *HelmState) connectionFlags(release *ReleaseSpec) []string {
func (st *HelmState) connectionFlags(helm helmexec.Interface, release *ReleaseSpec) []string {
flags := []string{}
tillerless := st.HelmDefaults.Tillerless
if release.Tillerless != nil {
tillerless = *release.Tillerless
}
if !tillerless {
if release.TillerNamespace != "" {
flags = append(flags, "--tiller-namespace", release.TillerNamespace)
} else if st.HelmDefaults.TillerNamespace != "" {
flags = append(flags, "--tiller-namespace", st.HelmDefaults.TillerNamespace)
if !helm.IsHelm3() {
if release.TillerNamespace != "" {
flags = append(flags, "--tiller-namespace", release.TillerNamespace)
} else if st.HelmDefaults.TillerNamespace != "" {
flags = append(flags, "--tiller-namespace", st.HelmDefaults.TillerNamespace)
}
}
if release.TLS != nil && *release.TLS || release.TLS == nil && st.HelmDefaults.TLS {
@@ -2020,7 +2235,7 @@ func (st *HelmState) flagsForUpgrade(helm helmexec.Interface, release *ReleaseSp
flags = append(flags, "--disable-openapi-validation")
}
flags = st.appendConnectionFlags(flags, release)
flags = st.appendConnectionFlags(flags, helm, release)
var err error
flags, err = st.appendHelmXFlags(flags, release)
@@ -2062,7 +2277,7 @@ func (st *HelmState) flagsForTemplate(helm helmexec.Interface, release *ReleaseS
return append(flags, common...), files, nil
}
func (st *HelmState) flagsForDiff(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, []string, error) {
func (st *HelmState) flagsForDiff(helm helmexec.Interface, release *ReleaseSpec, disableValidation bool, workerIndex int) ([]string, []string, error) {
flags := st.chartVersionFlags(release)
disableOpenAPIValidation := false
@@ -2076,7 +2291,6 @@ func (st *HelmState) flagsForDiff(helm helmexec.Interface, release *ReleaseSpec,
flags = append(flags, "--disable-openapi-validation")
}
disableValidation := false
if release.DisableValidation != nil {
disableValidation = *release.DisableValidation
} else if st.HelmDefaults.DisableValidation != nil {
@@ -2087,7 +2301,7 @@ func (st *HelmState) flagsForDiff(helm helmexec.Interface, release *ReleaseSpec,
flags = append(flags, "--disable-validation")
}
flags = st.appendConnectionFlags(flags, release)
flags = st.appendConnectionFlags(flags, helm, release)
var err error
flags, err = st.appendHelmXFlags(flags, release)
@@ -2206,7 +2420,7 @@ func (st *HelmState) ExpandedHelmfiles() ([]SubHelmfileSpec, error) {
if st.MissingFileHandler == "Error" {
return nil, err
}
fmt.Println(err)
st.logger.Warnf("no matches for path: %s", hf.Path)
continue
}
for _, match := range matches {
@@ -2253,7 +2467,7 @@ func (st *HelmState) generateTemporaryReleaseValuesFiles(release *ReleaseSpec, v
return generatedFiles, fmt.Errorf("failed to render values files \"%s\": %v", typedValue, err)
}
valfile, err := ioutil.TempFile("", "values")
valfile, err := createTempValuesFile(release, yamlBytes)
if err != nil {
return generatedFiles, err
}
@@ -2262,19 +2476,24 @@ func (st *HelmState) generateTemporaryReleaseValuesFiles(release *ReleaseSpec, v
if _, err := valfile.Write(yamlBytes); err != nil {
return generatedFiles, fmt.Errorf("failed to write %s: %v", valfile.Name(), err)
}
st.logger.Debugf("successfully generated the value file at %s. produced:\n%s", path, string(yamlBytes))
st.logger.Debugf("Successfully generated the value file at %s. produced:\n%s", path, string(yamlBytes))
generatedFiles = append(generatedFiles, valfile.Name())
case map[interface{}]interface{}, map[string]interface{}:
valfile, err := ioutil.TempFile("", "values")
valfile, err := createTempValuesFile(release, typedValue)
if err != nil {
return generatedFiles, err
}
defer valfile.Close()
encoder := yaml.NewEncoder(valfile)
defer encoder.Close()
if err := encoder.Encode(typedValue); err != nil {
return generatedFiles, err
}
generatedFiles = append(generatedFiles, valfile.Name())
default:
return generatedFiles, fmt.Errorf("unexpected type of value: value=%v, type=%T", typedValue, typedValue)
@@ -2360,7 +2579,7 @@ func (st *HelmState) generateSecretValuesFiles(helm helmexec.Interface, release
}
path := paths[0]
decryptFlags := st.appendConnectionFlags([]string{}, release)
decryptFlags := st.appendConnectionFlags([]string{}, helm, release)
valfile, err := helm.DecryptSecret(st.createHelmContext(release, workerIndex), path, decryptFlags...)
if err != nil {
return nil, err
@@ -2519,6 +2738,23 @@ func escape(value string) string {
return strings.Replace(intermediate, ",", "\\,", -1)
}
//MarshalYAML will ensure we correctly marshal SubHelmfileSpec structure correctly so it can be unmarshalled at some
//future time
func (p SubHelmfileSpec) MarshalYAML() (interface{}, error) {
type SubHelmfileSpecTmp struct {
Path string `yaml:"path,omitempty"`
Selectors []string `yaml:"selectors,omitempty"`
SelectorsInherited bool `yaml:"selectorsInherited,omitempty"`
OverrideValues []interface{} `yaml:"values,omitempty"`
}
return &SubHelmfileSpecTmp{
Path: p.Path,
Selectors: p.Selectors,
SelectorsInherited: p.SelectorsInherited,
OverrideValues: p.Environment.OverrideValues,
}, nil
}
//UnmarshalYAML will unmarshal the helmfile yaml section and fill the SubHelmfileSpec structure
//this is required to keep allowing string scalar for defining helmfile
func (hf *SubHelmfileSpec) UnmarshalYAML(unmarshal func(interface{}) error) error {
@@ -2741,3 +2977,58 @@ func (st *HelmState) Reverse() {
st.Helmfiles[i], st.Helmfiles[j] = st.Helmfiles[j], st.Helmfiles[i]
}
}
func (st *HelmState) getOCIChart(release *ReleaseSpec, tempDir string, helm helmexec.Interface) (bool, string, error) {
isOCI := false
repo, name := st.GetRepositoryAndNameFromChartName(release.Chart)
if repo == nil {
return false, release.Chart, nil
}
if repo.OCI {
isOCI = true
}
if !isOCI {
return isOCI, release.Chart, nil
}
chartVersion := "latest"
if release.Version != "" {
chartVersion = release.Version
}
qualifiedChartName := fmt.Sprintf("%s/%s:%s", repo.URL, name, chartVersion)
err := helm.ChartPull(qualifiedChartName)
if err != nil {
return isOCI, release.Chart, err
}
pathElems := []string{
tempDir,
}
if release.Namespace != "" {
pathElems = append(pathElems, release.Namespace)
}
if release.KubeContext != "" {
pathElems = append(pathElems, release.KubeContext)
}
pathElems = append(pathElems, release.Name, name, chartVersion)
chartPath := path.Join(pathElems...)
err = helm.ChartExport(qualifiedChartName, chartPath)
fullChartPath, err := findChartDirectory(chartPath)
if err != nil {
return isOCI, release.Chart, err
}
chartPath = filepath.Dir(fullChartPath)
return isOCI, chartPath, nil
}
+42 -23
View File
@@ -22,6 +22,7 @@ func injectFs(st *HelmState, fs *testhelper.TestFs) *HelmState {
st.glob = fs.Glob
st.readFile = fs.ReadFile
st.fileExists = fs.FileExists
st.directoryExistsAt = fs.DirectoryExistsAt
return st
}
@@ -786,15 +787,23 @@ func Test_normalizeChart(t *testing.T) {
{
name: "construct local chart path",
args: args{
basePath: "/Users/jane/code/deploy/charts",
basePath: "/src",
chart: "./app",
},
want: "/Users/jane/code/deploy/charts/app",
want: "/src/app",
},
{
name: "construct local chart path, without leading dot",
args: args{
basePath: "/src",
chart: "published",
},
want: "/src/published",
},
{
name: "repo path",
args: args{
basePath: "/Users/jane/code/deploy/charts",
basePath: "/src",
chart: "remote/app",
},
want: "remote/app",
@@ -802,18 +811,26 @@ func Test_normalizeChart(t *testing.T) {
{
name: "chartcenter repo path",
args: args{
basePath: "/Users/jane/code/deploy/charts",
basePath: "/src",
chart: "center/stable/myapp",
},
want: "center/stable/myapp",
},
{
name: "construct local chart path, parent dir",
name: "construct local chart path, sibling dir",
args: args{
basePath: "/Users/jane/code/deploy/charts",
basePath: "/src",
chart: "../app",
},
want: "/Users/jane/code/deploy/app",
want: "/app",
},
{
name: "construct local chart path, parent dir",
args: args{
basePath: "/src",
chart: "./..",
},
want: "/",
},
{
name: "too much parent levels",
@@ -1752,22 +1769,17 @@ generated: 2019-05-16T15:42:45.50486+09:00
}
logger := helmexec.NewLogger(os.Stderr, "debug")
basePath := "/src"
state := &HelmState{
basePath: "/src",
basePath: basePath,
FilePath: "/src/helmfile.yaml",
ReleaseSetSpec: ReleaseSetSpec{
Releases: []ReleaseSpec{
{
Chart: "./..",
Chart: "/example",
},
{
Chart: "../examples",
},
{
Chart: "../../helmfile",
},
{
Chart: "published",
Chart: "./example",
},
{
Chart: "published/deeper",
@@ -1792,25 +1804,32 @@ generated: 2019-05-16T15:42:45.50486+09:00
logger: logger,
}
fs := testhelper.NewTestFs(map[string]string{
"/example/Chart.yaml": `foo: FOO`,
"/src/example/Chart.yaml": `foo: FOO`,
})
fs.Cwd = basePath
state = injectFs(state, fs)
errs := state.UpdateDeps(helm)
want := []string{"/", "/examples", "/helmfile", "/src/published", generatedDir}
want := []string{"/example", "./example", generatedDir}
if !reflect.DeepEqual(helm.Charts, want) {
t.Errorf("HelmState.UpdateDeps() = %v, want %v", helm.Charts, want)
}
if len(errs) != 0 {
t.Errorf("HelmState.UpdateDeps() - no errors, but got %d: %v", len(errs), errs)
t.Errorf("HelmState.UpdateDeps() - unexpected %d errors: %v", len(errs), errs)
}
resolved, err := state.ResolveDeps()
if err != nil {
t.Errorf("unexpected error: %v", err)
t.Errorf("HelmState.ResolveDeps() - unexpected error: %v", err)
}
if resolved.Releases[5].Version != "1.5.0" {
t.Errorf("unexpected version number: expected=1.5.0, got=%s", resolved.Releases[5].Version)
if resolved.Releases[3].Version != "1.5.0" {
t.Errorf("HelmState.ResolveDeps() - unexpected version number: expected=1.5.0, got=%s", resolved.Releases[5].Version)
}
if resolved.Releases[6].Version != "1.4.0" {
t.Errorf("unexpected version number: expected=1.4.0, got=%s", resolved.Releases[6].Version)
if resolved.Releases[4].Version != "1.4.0" {
t.Errorf("HelmState.ResolveDeps() - unexpected version number: expected=1.4.0, got=%s", resolved.Releases[6].Version)
}
}
+90
View File
@@ -0,0 +1,90 @@
package state
import (
"errors"
"fmt"
"hash/fnv"
"io/ioutil"
"os"
"path/filepath"
"strings"
"github.com/davecgh/go-spew/spew"
)
func createTempValuesFile(release *ReleaseSpec, data interface{}) (*os.File, error) {
p, err := tempValuesFilePath(release, data)
if err != nil {
return nil, err
}
f, err := os.Create(*p)
if err != nil {
return nil, err
}
return f, nil
}
func tempValuesFilePath(release *ReleaseSpec, data interface{}) (*string, error) {
id, err := generateValuesID(release, data)
if err != nil {
panic(err)
}
workDir := os.Getenv("HELMFILE_TEMPDIR")
if workDir == "" {
workDir, err = ioutil.TempDir(os.TempDir(), "helmfile")
if err != nil {
panic(err)
}
}
d := filepath.Join(workDir, id)
info, err := os.Stat(d)
if err != nil && !errors.Is(err, os.ErrNotExist) {
panic(err)
} else if info == nil {
}
return &d, nil
}
func generateValuesID(release *ReleaseSpec, data interface{}) (string, error) {
var id []string
if release.Namespace != "" {
id = append(id, release.Namespace)
}
id = append(id, release.Name, "values")
hash, err := HashObject([]interface{}{release, data})
if err != nil {
return "", err
}
id = append(id, hash)
return strings.Join(id, "-"), nil
}
func HashObject(obj interface{}) (string, error) {
hash := fnv.New32a()
hash.Reset()
printer := spew.ConfigState{
Indent: " ",
SortKeys: true,
DisableMethods: true,
SpewKeys: true,
}
printer.Fprintf(hash, "%#v", obj)
sum := fmt.Sprint(hash.Sum32())
return SafeEncodeString(sum), nil
}
+33
View File
@@ -0,0 +1,33 @@
/*
Copyright 2015 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// See k8s.io/apimachinery/pkg/util/rand/rand.go
package state
const (
// We omit vowels from the set of available characters to reduce the chances
// of "bad words" being formed.
alphanums = "bcdfghjklmnpqrstvwxz2456789"
)
// SafeEncodeString encodes s using the same characters as rand.String. This reduces the chances of bad words and
// ensures that strings generated from hash functions appear consistent throughout the API.
func SafeEncodeString(s string) string {
r := make([]byte, len(s))
for i, b := range []rune(s) {
r[i] = alphanums[(int(b) % len(alphanums))]
}
return string(r)
}
+80
View File
@@ -0,0 +1,80 @@
package state
import (
"github.com/google/go-cmp/cmp"
"testing"
)
func TestGenerateID(t *testing.T) {
type testcase struct {
subject string
release ReleaseSpec
data interface{}
want string
}
ids := map[string]int{}
run := func(tc testcase) {
t.Helper()
t.Run(tc.subject, func(t *testing.T) {
t.Helper()
got, err := generateValuesID(&tc.release, tc.data)
if err != nil {
t.Fatalf("uenxpected error: %v", err)
}
if d := cmp.Diff(tc.want, got); d != "" {
t.Fatalf("unexpected result: want (-), got (+):\n%s", d)
}
ids[got]++
})
}
run(testcase{
subject: "baseline",
release: ReleaseSpec{Name: "foo", Chart: "incubator/raw"},
want: "foo-values-67b55dc69b",
})
run(testcase{
subject: "different bytes content",
release: ReleaseSpec{Name: "foo", Chart: "incubator/raw"},
data: []byte(`{"k":"v"}`),
want: "foo-values-5988bf4947",
})
run(testcase{
subject: "different map content",
release: ReleaseSpec{Name: "foo", Chart: "incubator/raw"},
data: map[string]interface{}{"k": "v"},
want: "foo-values-5d6fb4db97",
})
run(testcase{
subject: "different chart",
release: ReleaseSpec{Name: "foo", Chart: "stable/envoy"},
want: "foo-values-58db655b79",
})
run(testcase{
subject: "different name",
release: ReleaseSpec{Name: "bar", Chart: "incubator/raw"},
want: "bar-values-797d6df4dc",
})
run(testcase{
subject: "specific ns",
release: ReleaseSpec{Name: "foo", Chart: "incubator/raw", Namespace: "myns"},
want: "myns-foo-values-5f867c6d49",
})
for id, n := range ids {
if n > 1 {
t.Fatalf("too many occurences of %s: %d", id, n)
}
}
}
+5 -5
View File
@@ -39,7 +39,7 @@ func NewTestFs(files map[string]string) *TestFs {
func (f *TestFs) FileExistsAt(path string) bool {
var ok bool
if strings.Contains(path, "/") {
if strings.HasPrefix(path, "/") {
_, ok = f.files[path]
} else {
_, ok = f.files[filepath.Join(f.Cwd, path)]
@@ -53,7 +53,7 @@ func (f *TestFs) FileExists(path string) (bool, error) {
func (f *TestFs) DirectoryExistsAt(path string) bool {
var ok bool
if strings.Contains(path, "/") {
if strings.HasPrefix(path, "/") {
_, ok = f.dirs[path]
} else {
_, ok = f.dirs[filepath.Join(f.Cwd, path)]
@@ -64,7 +64,7 @@ func (f *TestFs) DirectoryExistsAt(path string) bool {
func (f *TestFs) ReadFile(filename string) ([]byte, error) {
var str string
var ok bool
if filename[0] == '/' {
if strings.HasPrefix(filename, "/") {
str, ok = f.files[filename]
} else {
str, ok = f.files[filepath.Join(f.Cwd, filename)]
@@ -90,7 +90,7 @@ func (f *TestFs) FileReaderCalls() int {
func (f *TestFs) Glob(relPattern string) ([]string, error) {
var pattern string
if relPattern[0] == '/' {
if strings.HasPrefix(relPattern, "/") {
pattern = relPattern
} else {
pattern = filepath.Join(f.Cwd, relPattern)
@@ -116,7 +116,7 @@ func (f *TestFs) Glob(relPattern string) ([]string, error) {
func (f *TestFs) Abs(path string) (string, error) {
var p string
if path[0] == '/' {
if strings.HasPrefix(path, "/") {
p = path
} else {
p = filepath.Join(f.Cwd, path)
+1 -1
View File
@@ -2,9 +2,9 @@ package tmpl
import (
"fmt"
"github.com/ghodss/yaml"
"github.com/roboll/helmfile/pkg/helmexec"
"golang.org/x/sync/errgroup"
"gopkg.in/yaml.v2"
"io"
"os"
"os/exec"
+44 -2
View File
@@ -1,7 +1,9 @@
package tmpl
import (
"encoding/json"
"fmt"
"github.com/google/go-cmp/cmp"
"path/filepath"
"reflect"
"testing"
@@ -47,12 +49,31 @@ func TestReadFile_PassAbsPath(t *testing.T) {
}
}
func TestToYaml_UnsupportedNestedMapKey(t *testing.T) {
expected := ``
vals := Values(map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "BAR",
},
})
actual, err := ToYaml(vals)
if err == nil {
t.Fatalf("expected error but got none")
} else if err.Error() != "error marshaling into JSON: json: unsupported type: map[interface {}]interface {}" {
t.Fatalf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestToYaml(t *testing.T) {
expected := `foo:
bar: BAR
`
vals := Values(map[string]interface{}{
"foo": map[interface{}]interface{}{
"foo": map[string]interface{}{
"bar": "BAR",
},
})
@@ -70,7 +91,7 @@ func TestFromYaml(t *testing.T) {
bar: BAR
`
expected := Values(map[string]interface{}{
"foo": map[interface{}]interface{}{
"foo": map[string]interface{}{
"bar": "BAR",
},
})
@@ -83,6 +104,27 @@ func TestFromYaml(t *testing.T) {
}
}
func TestFromYamlToJson(t *testing.T) {
input := `foo:
bar: BAR
`
want := `{"foo":{"bar":"BAR"}}`
m, err := FromYaml(input)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
got, err := json.Marshal(m)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if d := cmp.Diff(want, string(got)); d != "" {
t.Errorf("unexpected result: want (-), got (+):\n%s", d)
}
}
func TestSetValueAtPath_OneComponent(t *testing.T) {
input := map[string]interface{}{
"foo": "",
+1 -2
View File
@@ -4,8 +4,6 @@ environments:
- environment.values.yaml
---
repositories:
- name: stable
url: https://kubernetes-charts.storage.googleapis.com/
- name: center
url: https://repo.chartcenter.io
@@ -30,6 +28,7 @@ releases:
- name: raw
chart: center/incubator/raw
version: 0.2.3
values:
- mysecret: {{ .Environment.Values.mysecret }}
- values.yaml
+1 -1
View File
@@ -49,7 +49,7 @@ info "Using namespace: ${test_ns}"
if helm version --client 2>/dev/null | grep '"v2\.'; then
helm_major_version=2
info "Using Helm version: $(helm version --short --client | grep -o v.*$)"
${helm} init --wait --override spec.template.spec.automountServiceAccountToken=true
${helm} init --stable-repo-url https://charts.helm.sh/stable --wait --override spec.template.spec.automountServiceAccountToken=true
# helm v3
else
helm_major_version=3