Compare commits

...
302 Commits
Author SHA1 Message Date
Theo Meneau 216c228c0b feat: helm repo add --ca-file via repositories definition (#856)
Resolves #855
2019-09-14 06:23:54 +02:00
KUOKA Yusuke ef63a05513 fix(helm3): delete/destroy/apply/sync unable to detect releases to be deleted (#857)
Fixes #853
2019-09-14 06:23:22 +02:00
art kon 06b0c99a0b Fix recursion for helmfiles pulled from git (#854) 2019-09-14 05:49:59 +02:00
eddycharly fd0133e10a Update documentation and tests for .Values (#839)
Resolves #816
2019-09-14 05:35:16 +02:00
KUOKA Yusuke 9d851cda3b feat: --skip-repos for helmfile deps (#851)
Resolves #661
2019-09-12 19:33:18 +09:00
KUOKA Yusuke f79db2ec8d feat(diff,apply,lint,sync,template): --set k=v for setting adhoc chart values (#850)
Resolves #840
2019-09-12 19:24:43 +09:00
KUOKA Yusuke 94a6fcfb9f feat(diff,apply): --context=N for limiting diff context (#849)
Resolves #787
2019-09-12 18:36:13 +09:00
KUOKA Yusuke fb2041555e feat(diff,apply): --no-color for removing color from output (#848)
Resolves #788
2019-09-12 18:32:30 +09:00
刘相轩 cbf5b8b1e7 Fix helm2 lock file does not get updated (#847)
Ref: https://github.com/helm/helm/issues/2731
2019-09-12 17:58:16 +09:00
matthias-kloeckner 267e0fa1fe adding info about kloeckner-i to users list (#843) 2019-09-11 12:48:00 +09:00
Shane Starcher 5488198d6d fix: allow empty pattern matching and move on (#827)
Ref #778
2019-09-11 12:47:00 +09:00
KUOKA Yusuke 4bbd09ccb2 fix(cmd/deps): make helmfile deps to work w/ helm 3.0.0-beta.3 (#842)
Ref https://github.com/roboll/helmfile/issues/668#issuecomment-529054165
2019-09-07 22:06:29 +09:00
KUOKA Yusuke 4e4f1bee59 feat: Experimental Helm v3 mode (#841)
Set `HELMFILE_HELM3=1` and run `helmfile` like `HELMFILE_HELM3=1 helmfile ...`.

When `HELMFILE_HELM3` is set, `test`, `template`, `delete`, `destroy` behave differently so that it works with Helm 3.

Note that `helmfile diff` doesn't work as `helm-diff` called under the hood doesn't support Helm v3 yet.

Ref #668
2019-09-07 09:40:44 +09:00
a-hat 6dfc9be0d1 doc: Update documentation about layering release values (#837)
Closes #836
2019-09-06 09:29:08 +09:00
chenrui 1823bb1e5b Fix apache thrift location issue (#835)
Relates to Homebrew/homebrew-core#43728
Fixe #834
2019-09-04 12:19:14 +09:00
KUOKA Yusuke 4bc99337b2 Make advanced templating backward-compatible (#833)
This is a follow-up for #823
Ref https://github.com/roboll/helmfile/pull/823#discussion_r319712283
2019-08-31 22:37:46 +09:00
Tibo Beijen 01ae59fedd add helm-tiller plugin to docker image (#825) 2019-08-31 14:38:36 +09:00
Shane Starcher 2a6bd24e3c move context outside to limit duplicate repo updates (#828) 2019-08-31 14:32:48 +09:00
astorath 11d0abba6e feat: Advanced Templating (#823)
1. Added `helmfile build` command to print final state
Motivation: useful for debugging purposes and some CI scenarios

Ref #780 

2. Template interpolation is now recursive (you can cross-reference release fields) like:
```yaml
templates:
  release:
    name: {{`app-{{ .Release.Namespace }}`}}
    namespace: {{`{{ .Release.Labels.ns }}`}}
    labels:
      ns: dev
```
3. Experimental: Added some boolean release fields interpolation in templates:
```yaml
templates:
  release:
    name: {{`app-{{ .Release.Namespace }}`}}
    namespace: dev
    installedTemplate: {{`{{ eq .Release.Namespace "dev" }}`}}
```

Resolves #818

4. Added more template interpolations: Labels, SetValues
5. Added template interpolation for inline Values
6. Added `helmfile list` command to print target releases in simple tabular form
7. Added release names in some `helm` output messages, e.g.: `Comparing release=%v, chart=%v`
2019-08-31 14:31:31 +09:00
Yusuke Kuoka dd58badf81 release: skip building against s390x
Please notice me if anyone is using Helmfile on it
2019-08-24 10:50:18 +09:00
Yusuke Kuoka e24c15324b Revert "release: skip downloading go modules in container image builds (#821)"
This reverts commit ba0891b7b7.
2019-08-24 10:37:01 +09:00
KUOKA Yusuke ba0891b7b7 release: skip downloading go modules in container image builds (#821)
So that the release process can be (probably) 2x faster
2019-08-24 10:29:26 +09:00
KUOKA Yusuke b58e082a8f Merge pull request #820 from roboll/fix-incorrect-remote-cache-on-miss
fix: clean up invalid remote state file cache
2019-08-24 10:01:48 +09:00
Yusuke Kuoka bf9dcc0982 release: Make the release process faster 2019-08-24 09:59:46 +09:00
Yusuke Kuoka cd5d906afb fix: clean up invalid remote state file cache
Fixes #815
2019-08-24 09:47:49 +09:00
Travis Groth a584aeab2e Share helmexec from State Creation (#804)
Closes #444 and #782 

This is the final PR to fully cache and parallelize helm secret decryption.  It threads the shared helmexec.Interface into the StateCreator and HelmState structs to be used during environment secret decryption.  This should effectively cache secrets for the duration of a helmfile run, regardless of where they are first decrypted.
2019-08-15 09:27:55 +09:00
Travis Groth 765bfe6cfd Handle environment secrets concurrently
Ref #782
2019-08-13 09:23:29 +09:00
Jake Hill 622cba9f19 Add kubectl and jq to Dockerfile (#799)
* Add kubectl and jq to Dockerfile

Signed-off-by: Jake Hill <jake@naphta.uk>

* Update kubectl to use checksum verification, and pin to a specific version

Signed-off-by: Jake Hill <jake@naphta.uk>

Resolves #792
2019-08-08 21:24:46 +09:00
Travis Groth 63a337ee57 Base concurrency on items parameter (#798)
Fixes #793
2019-08-08 10:20:05 +09:00
Travis Groth 6baad71b1f Cache secrets and concurrent decryption (#790)
Related to #782 and #444 

- Allows concurrent decryption of different secrets files
- Caches decrypted secrets by original file path and returns decrypted results from memory
- Secrets being run through an instance of helmexec will be cached and run as fast as possible concurrently

NB: This particular PR doesn't make _all_ calls to secrets cached and concurrent.  Environment Secrets in particular seem to not be evaluated with a ScatterGather(), and doesn't use the same helmexec instance as other parts of the code, so it doesn't take advantage of these changes.  Some reworking of the plumbing there would be needed.
2019-08-07 23:00:19 +09:00
OlivierB bce2f4728b fix: pass namespace to helm template command (#771)
Resolves #770
2019-07-30 10:44:42 +09:00
a-hat e5038fb04f print yaml content on error in fromYaml (#765) 2019-07-30 10:43:30 +09:00
bitsofinfo fe5102e777 users update (#750) 2019-07-30 10:42:47 +09:00
KUOKA Yusuke eda961edc2 fix: --state-values-set not setting more than first child in nested paths (#774)
Fixes #773
2019-07-27 22:57:54 +09:00
KUOKA Yusuke b2a6231dcf fix: absolute chart path (#753)
Resolves #743
2019-07-12 22:37:54 +09:00
Marc Sutter 4166b418c9 feat: Adopt existing K8s resources as Helm release on install (#746)
Use with the helm-x support(#673)

This enhances config syntax to accept adopt: [NS/KIND/RESOURCE_NAME] at the release level so that helmfile calls helm-x to transparently import existing resources at the installation time.

Resolves #84
2019-07-11 09:13:34 +09:00
Naseem 408bf7b8bf Update USERS.md (#744) 2019-07-11 09:10:13 +09:00
OlivierB 2f9f52033c feat: add --output-dir on template command (#693)
It generates templates in a subdirectory named "stateFileName-stateFileHash-releaseName"
2019-07-11 09:07:46 +09:00
KUOKA Yusuke 63b5040ec4 fix: fatal error: concurrent map read and map write on concurrency > 1 (#742)
This is a revised version of https://github.com/roboll/helmfile/commit/41e44f74a37ebca0394d9ec4546c27208f8a1adf, which doesn't make CI angry and a more straight-forward(yet smelling) implementation

Fixes #737
2019-07-05 09:20:16 +09:00
bitsofinfo 4cc40cf0f1 Upgrade to sprig 2.20.0 and xstrings 1.2.0 (#735)
Needed for new functions and bug fixes

Resolves #733
Resolves #734
2019-07-02 10:12:14 +09:00
Clément Loiselet 1df9b0e9e9 fix: enabled '=' in a value for --state-values-set (#722)
This adds the possibility to use a "=" in the parameters passed to --state-values-set

previous comportment:
`helmfile --state-values-set 'test="abcde==fg",test2="abcde"'` => outputs 'test="abcde",test2="abcde"'

new comportment:
`helmfile --state-values-set 'test="abcde==fg",test2="abcde"'` => outputs 'test="abcde==fg",test2="abcde"'
2019-06-28 10:12:28 +09:00
KUOKA Yusuke b82f77ee50 fix: Absolute paths for -f not working on Windows (#721)
Fixes #718
2019-06-25 20:06:07 +09:00
KUOKA Yusuke e6bb8282db fix: remote helmfile from git::ssh source (#719)
The remote helmfile feature introduced by #648 was unable to be sourced from private git repositories due to URL parsing issue in helmfile. This fixes that.

Ref https://github.com/roboll/helmfile/issues/469#issuecomment-505236600
2019-06-25 12:45:56 +09:00
Max Williams b6a04d012c Change link in README to newer tag (again) (#717) 2019-06-21 18:31:00 +09:00
Max Williams 9eef44ffe6 Change link in README to newer tag (#716)
Closes #710

Fixed a couple very small typos also.
2019-06-21 18:18:36 +09:00
KUOKA Yusuke cfe309ec21 fix: do not panic on merging env secrets (#713)
Fixes #712
2019-06-21 00:10:11 +09:00
Erik Forsberg 65818cd345 feat: When values.yaml.gotmpl can't be parsed as YAML, print the YAML. (#703)
This helps a lot when debugging go templates.
2019-06-19 08:47:36 +09:00
sgandon ddb5be1b9d feat: optionally show logs for hooks (#699)
Resolves #689 

This adds a new yaml entry for the hook definition to allow the users to specifcy if they want to show the `command` logs or not.

here is an example.
```
releases:
- name: myapp
  chart: mychart
  # *snip*
  hooks:
  - events: ["cleanup"]
    showlogs: true
    command: "kubectl"
    args: ["get", "ingress"]
```
this will display the following output:
```
hook[cleanup] logs | NAME              HOSTS                          ADDRESS   PORTS   AGE
hook[cleanup] logs | catalog-gateway   tdc.foo                                  80      2d6h
hook[cleanup] logs | dataset           foobar.barr.foo.xxxxxxx.com              80      2d6h
hook[cleanup] logs | rating            fooba.barr.foo.xxxxxxx.com               80      2d6h
hook[cleanup] logs | sharing           foobar.barr.foo.xxxxxxx.com              80      2d6h
hook[cleanup] logs | tpsvc-iam-dev     foo.barr.foo.xxxxxxx.com                 80      2d6h
hook[cleanup] logs | tpsvc-iam-front   bar.barr.foo.xxxxxxx.com                 80      2d6h
```
2019-06-19 08:46:32 +09:00
KUOKA Yusuke f61334d9bb fix: helmfile template | kubectl apply -f - should work (#697)
Fixes #685
2019-06-16 16:56:56 +09:00
KUOKA Yusuke 3bf0f7d54e fix: helm diff missing --reset-values (#696)
Fixes #681
2019-06-16 16:52:32 +09:00
KUOKA Yusuke 7dec948950 fix: helm chart referenced by http URL (#695)
We unintentionally broke this since #593.

Fixes #675
Fixes #687
2019-06-16 16:40:17 +09:00
刘相轩 e0d0a1cf7f feat: Disable concurrency if any release is using tillerless (#694)
So that you don't need to remember adding `--concurrency 1`
2019-06-16 12:58:20 +09:00
Cedric Meury 31768b5bac doc: Adding user survey responses while switching to PR style (#692)
* adding survey responses; switch to PR style

* fixing the table

* adding reddit talk
2019-06-14 08:37:42 +09:00
KUOKA Yusuke 1bde201eae feat: --concurrency for delete and destroy (#691)
Resolves #690
2019-06-13 23:14:51 +09:00
KUOKA Yusuke 1831d3375f fix: installed: false not respecting TLS settings and tiller namespace (#683)
It was just missing the appendConnectionFlags call to compose flags passed to `helm delete` there. This fixes it.

Fixes #605
2019-06-12 19:07:31 +09:00
KUOKA Yusuke 2e38f42a40 fix: helmDefaults.kubeContext ignored in helm diff of helmfile apply (#682)
The root cause of this bug was due to that `--kube-context` and `kubeContext` had been treated specifically in code. So on the way I have made it consistent with other per-release settings - by adding `kubeContext` for each release and treating `helmDefaults.kubeContext` as just the default value for per-release setting.

Fixes #674
2019-06-12 16:49:19 +09:00
KUOKA Yusuke 1da3488599 feat: experimental integration with helm-x (#673)
This enhances helmfile so that it can:

- Treat K8s manifests directories and Kustomize projects as charts
- Add adhoc chart dependencies on sync/diff/template without forking or modifying chart(s) (#649)
- Add adhoc patches(JSON Patch or Strategic Merge Patch supported) to be applied to the K8s resources before sync/diff/template, without forking or modifyin chart(s) (#650)

The usage is as outlined in https://github.com/mumoshu/helm-x/tree/master/examples/helmfile.

Add any or all of `dependencies:`, `jsonPatches:` and `strategicMergePatches:` so that it adds additional flags to `helm` calls that is only supported by `helm x`.

```yaml
releases:
- name: kustomize
  chart: ../kustomize
- name: manifests
  chart: ../manifests
- name: foo
  chart: incubator/raw
  dependencies:
  - alias: bar
    chart: incubator/raw
  values:
  - values.yaml
  - bar:
      enabled: true
      resources:
      - apiVersion: v1
        kind: Pod
        metadata:
          name: bar
        spec:
          containers:
          - command:
            - sleep
            - 1000
            image: alpine:3.9.4
            imagePullPolicy: IfNotPresent
            name: bar
  jsonPatches:
  - target:
      version: v1
      kind: Pod
      name: foo
    patch:
    - op: replace
      path: /spec/containers/0/command
      value:
      - sleep
      - "123"
  strategicMergePatches:
  - apiVersion: v1
    kind: Pod
    metadata:
      name: bar
    spec:
      containers:
      - name: bar
        command:
        - sleep
        - "234"
```

You can alternatively provide `source: path/to/patch.yaml` for `jsonPatches` and `strategicMergePatches` items to externalize it. Add `.gotmpl` suffix like you would do for values files for templating.

When running `helmfile` you must point `--helm-binary` to the `helm-x` binary like below:

```
$ helmfile --helm-binary ~/.helm/plugins/helm-x/bin/helm-x --log-level debug apply
```

after installing the [helm-x](https://github.com/mumoshu/helm-x) plugin.

The integration should ideally be automatic. That is, it shouldn't force you to set `--helm-binary`. But I had no other way to not bloat helmfile's codebase to just add this experimental feature.

Resolves #649
Resolves #650
2019-06-12 16:11:34 +09:00
KUOKA Yusuke 78b03e0d92 fix: --state-values-set panic: value of type interface {} is not assignable to type string (#680)
Probably since #647 helmfile has been unable to merge nested maps in environment values if they were loaded from files. This fixes it.

The relevant test is also enhanced so that no further regression like this happens.

Fixes #677
2019-06-12 13:35:04 +09:00
sgandon 11eda66eaa feat: make release version templatized (#671)
Allows the release version to be a Go template.
Resolves #669
2019-06-11 21:52:54 +09:00
KUOKA Yusuke 65d404b276 feat: helmfile -f <go-getter url> (#670)
Extends the remote-helmfile feature to also work when loading the first state file.
This should be useful for people who wants to give helmfile a try without ever opening `$EDITOR`.
2019-06-11 14:28:50 +09:00
Eric Bailey 72425aebfc fix: persist original file path when using bases (#672)
* fix: persist original file path when using bases

Prior to this change, the resulting lock file was called `<bases[0]>.lock`,
instead of `<filename>.lock`.

This change ensures the final, merged state has the correct `.FilePath`.

* test: Assert proper FilePath in layered HelmState
2019-06-11 14:01:27 +09:00
Kevin J. Qiu 34fbfb4fd0 fix: convert path to absolute path when doing helm secret dec (#664)
Fixes #663
2019-06-10 21:43:20 +09:00
sgandon 56c27c2bd9 fix: fixes a regexp issue for release summary (#666)
There was an issue a the regexp reding the version from helm output.

Fixes #665
2019-06-10 09:18:41 +09:00
Vincent Behar 34c793d87e fix: use the same logic as helm-secrets when decoding secrets (#655)
helm-secrets uses the `HELM_SECRETS_DEC_SUFFIX` env var to define the name of the output file
we should have the same logic in helmfile, to come up with the same filename

It only affects people using the `HELM_SECRETS_DEC_SUFFIX` env var

Use-case: if you want to run multiple `helmfile` commands in parallel, without conflicts. in this case, you need to decrypt secrets with different suffixes.
2019-06-06 21:27:04 +09:00
刘相轩 b9862bdae6 Sync repos before update deps (#660) 2019-06-06 15:23:27 +09:00
刘相轩 e3e7905f07 Sort requirements alphabetically by name. (#659) 2019-06-06 12:30:22 +09:00
KUOKA Yusuke 2e57beb448 feat: bundle helm-s3 and helm-git into container images (#651)
Resolves #435 (Git as chart repository)
Resolves #220 (S3 as chart repository)
Resolves #436 (About bundling helm plugins)

A lot of thanks to @aslafy-z for authoring the awesome helm-git plugin and contributing it to the community!
2019-06-04 23:53:11 +09:00
KUOKA Yusuke 3986f6497e Update writing-helmfile.md 2019-06-04 23:41:26 +09:00
KUOKA Yusuke 820abbc06d feat: remote state files (#648)
This change enhances helmfile to accept terraform-module-like URLs in nested state files a.k.a sub-helmfiles.

```yaml
helmfiles:
- # Terraform-module-like URL for importing a remote directory and use a file in it as a nested-state file
  # The nested-state file is locally checked-out along with the remote directory containing it.
  # Therefore all the local paths in the file are resolved relative to the file
  path: git::https://github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.40.0
```

The URL isn't equivalent to terraform module sources. The difference is that we use `@` to distinguish between (1) the path to the repository and directory containing the state file and (2) the path to the state file being loaded. This distinction provides us enough fleibiity to instruct helmfile to check-out necessary and sufficient directory to make the state file works.

Under the hood, it uses [hashicorp/go-getter](https://github.com/hashicorp/go-getter), that is used for [terraform module sources](https://www.terraform.io/docs/modules/sources.html) as well.

Only the git provider without authentication like git-credentials helper is tested. But theoretically any go-getter providers should work. Please feel free to test the provider of your choice and contribute documentation or instruction to use it :)

Resolves #347
2019-06-04 22:59:54 +09:00
KUOKA Yusuke 3710f6233e feat: state values (#647)
This adds `values` to state files as proposed in #640.

```yaml
values:
- key1: val1
- defaults.yaml

environments:
  default:
  - values:
    - environments/default.yaml
  production:
  - values:
    - environments/production.yaml

```

`{{ .Valuese.key1 }}` evaluates to `val1` if and only if it is not overrode via the production or the default env, or command-line args.

Resolves #640
2019-06-04 16:34:02 +09:00
KUOKA Yusuke 2d2b3e486f fix: sprig dict funcs are failing on dicts nested under arrays (#646)
Fixes #643
2019-06-04 13:47:05 +09:00
KUOKA Yusuke 65ee6a2124 fix: "cannot unmarshal !!str <no value> into bool" errors in state templates (#645)
Seems like we are affected by https://github.com/golang/go/issues/24963. That is, even though we internally use the template option `missingkey=zero`, in some cases it still prints `<no value>` instead of zero values, which has been confusing the state yaml parsing.

This fixes the issue by naively replacing all the remaining occurrences of `<no value>` in the rendered text, while printing debug logs to ease debugging in the future when there is unexpected side-effects introduced by this native method.

Fixes #553
2019-06-04 13:23:38 +09:00
KUOKA Yusuke 1d3f5f8a33 feat: override state(former "enviroment") values via command-line args (#644)
The addition of `--set k1=v1,k2=v2` and `--values file1 --values file2` was originally planned in #361.

But it turned out we already had `--values` for existing helmfile commands like `sync`. Duplicated flags doesn't work, obviously.

So this actually add `--state-values-set k1=v1,k2=v2` and `--set-values-file file1 --set-values-file file2`.

They are called "state" values according to the discussion we had at #640

Resolves #361
2019-06-04 11:03:01 +09:00
KUOKA Yusuke e2d6dc4afa feat: helmfile as a go library (#639)
* feat: helmfile as a go library

This removes almost all the dependencies from the helmfile core logic to urfave/cli. `main.go` is now a thin wrapper around the core logic implemented in `pkg/app`.
2019-06-04 09:12:00 +09:00
KUOKA Yusuke f6057a1cca chore: first step towards migration to cobra (#637)
I will start by making as much as possible code in `main.go` independent from urfave/cli, and this illustrates how we can do that by introducing a Config interface that delegates any config value fetch to urfave/cli. We will implement an cobra/pflag impl of the interface when helmfile finally migrates to cobra.
2019-06-01 14:00:26 +09:00
KUOKA Yusuke c68fc5bc50 chore: tidy up pkgs (#636)
for readability and towards potentially making helmfile usable as a go library
2019-06-01 13:36:05 +09:00
KUOKA Yusuke 68b95f14d4 change the nest level of envvals overrides for nested state files(sub-helmfiles) (#635)
We added envvals overrides in the state file via #622 two days ago:

```
helmfiles:
- name: sub.helmfile.yaml
  environment:
    values:
    - mykey: myvalue
```

This change removes the `environment` level in the above cofig, so that it looks like:

```
helmfiles:
- name: sub.helmfile.yaml
  values:
  - mykey: myvalue
``

This is an inevitable breaking change towards #361. But I wanted to break it earlier so that less folks are affected.`

Ref https://github.com/roboll/helmfile/issues/361#issuecomment-497530819
2019-06-01 12:39:31 +09:00
KUOKA Yusuke 9318d8a040 doc: recommend quoting potentially number-like string (#634)
Resolves #608
2019-06-01 11:29:33 +09:00
jonathan cowling msm 44686298f2 Document verify in releases (#632)
Just a one liner in the readme to make it clear there's a `verify` option in the `releases` section 😄
2019-05-31 22:50:45 +09:00
bitsofinfo 206372b7aa feat: upgrade sprig to 2.16.0
Resolves #630
2019-05-31 13:16:49 +09:00
KUOKA Yusuke e1c04971c1 Update README.md 2019-05-30 17:11:42 +09:00
KUOKA Yusuke b3f4586ed8 Fix panic related to env values from files and sprig dict funcs (#625)
* fix: 0.68.0: go panic w/ multiple bases/layers

Fixes #623

* fix: 0.64.1+: getOrNil/hasKey fails on Environment.Values with nested maps

Fixes #624
2019-05-30 15:41:00 +09:00
KUOKA Yusuke 591086dac9 Update README.md 2019-05-29 21:25:59 +09:00
KUOKA Yusuke 1226ea6d1a feat: specify env values from the parent to the nested state (#622)
* feat: specify env values from the parent to the nested state

Adds the `helmfiles[].environment.values` that accepts a mix of file pathes and inline dictes:

```yaml
helmfiles:
- path: path/to/nested/helmfile.yaml
  environment:
    values:
    - key1: val1
    - values.yaml
```

The values files are loaded in the context of the parent state file. For example, in case the above state file is located at `/path/to/helmfile.yaml`,
`values.yaml` is located at `/path/to/values.yaml` instead of `/path/to/nested/values.yaml`.

Resolves #523

* fix: multiple "bases" declarations yields duplicate releases

Fixes #615

* fix regression in double-rendering with env value overrides

The latest commit broke any state files like the below to NOT pass env value overrides at all:

```
helmfiles:
- path: nested/state.yaml
  environment:
    values:
    - overrides.yaml
```

This fixes the issue.
2019-05-29 19:08:51 +09:00
KUOKA Yusuke 681c866ce1 feat: inline environment values (#621)
Resolves #359
2019-05-28 16:26:00 +09:00
KUOKA Yusuke dd70c857a5 Update README.md 2019-05-28 15:38:17 +09:00
KUOKA Yusuke a896f801ab feat: optionally allow missing environment values/secrets files (#620)
```yaml
environments:
  default:
    missingFileHandler: Warn
    values:
    - path/to/values.yaml
    secrets:
    - path/to/secrets.yaml
```

`missingFileHandler` set to `Warn`, `Info`, or `Debug` results in helmfile NOT stop when `path/to/values.yaml` or `path/to/secrets.yaml` is missing.

Resolves #548

While implementing the above feature, I also found a bug that has been causing #559. This also fixes that.

To verify it is actually fixed, create an example helmfile.yaml that looks like the below, and run `helmfile diff`:

```
$ cat helmfile.yaml
environments:
  default:
    secrets:
      - env-secrets.yaml

releases:
  - name: myapp
    chart: nginx
    namespace: default
    secrets: [secrets.yaml]    # Notice this file does not exist
    values:
      - ingress:
          enabled: true

$ helmfile diff
could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read helmfile.yaml.part.0: environment values file matching "env-secrets.yaml" does not exist
in ./helmfile.yaml: failed to read helmfile.yaml: environment values file matching "env-secrets.yaml" does not exist
```

Fixes #559
2019-05-28 15:33:45 +09:00
Raj Perera 4a5996d083 feat: ability to exit zero with no releases match selector (#618)
- Change exit code from 2 to 3 when helmfiles have no releases that match a selector
- Introduces new flag `--allow-no-matching-release` to exit 0 when no releases match a selector.

Resolves: #597
2019-05-27 09:53:24 +09:00
KUOKA Yusuke f6264bfa9d fix: do not complain on missing values files for undesired release (#617)
Fixes #616
2019-05-24 21:40:34 +09:00
KUOKA Yusuke 3b3d3092dd fix scopelint errors / possible races in hooks (#614)
Ref #612
Fixes #514
2019-05-22 18:50:38 +09:00
KUOKA Yusuke 1012256f16 feat: "base" helmfile state gotmpl is rendered with the envvals inherited from the parent (#613)
Resolves #611
2019-05-22 18:28:10 +09:00
KUOKA Yusuke 90390492a3 feat: expand glob pattern in environment values file path (#610)
This enhances helmfile's internal environment values files loader to expand glob patterns (#606)

Fixes the existing bug that helmfile was unable to load environment values file from absolute path (#549)

Resolves #606
Fixes #549
2019-05-21 16:49:57 +09:00
KUOKA Yusuke 4c9c42d3c5 fix: gotmpl layer was unable to reference env values from other layers when env!=default (#609)
Fixes #604
2019-05-21 14:37:03 +09:00
KUOKA Yusuke bd60548f10 feat: add --skip-deps for helmfile lint (#602)
Resolevs #581
2019-05-16 21:31:00 +09:00
KUOKA Yusuke 0534117b62 feat: postsync hooks (#601)
`postsync` events are triggered after each release is applied to the cluster in `helmfile sync` or `helmfile apply`.

This should be a best hook to notify only after each sync failed or succeeded. This can be used for running operations like patching K8s resources managed by helm, but that should be the last-resort. Maybe you should fork/update the chart, or submit a feature request to add `replicated/ship` integration to `helmfile` in that case :)

Resolves #599
2019-05-16 21:24:16 +09:00
KUOKA Yusuke 0104c91fce feat: support for locking the same chart for two or more versions (#600)
Resolves #598
2019-05-16 21:19:30 +09:00
Yusuke KUOKA a769d1a82a fix integration test 2019-05-15 13:21:48 +09:00
KUOKA Yusuke b0179218b9 fix: helmfile panicing on missing lock file (#596)
Fixes #595
2019-05-15 13:16:22 +09:00
KUOKA Yusuke c9a43ad9cb feat: Dependency locking (#593)
In order to maintain predictable deployments, as developer I want to generate and use "lock files" for all chart versions retrieved from a helmfile.

This change solves it by (1)enhancing `helmfile deps` to generate a lock file containing all the direct chart dependencies of each helmfile state file and
(2)making other helmfile sub-commands reads the lock file and merge the locked version numbers to the helmfile state file being processed.

The lock file is named after the helmfile state file being locked, so that you can have multiple set of the helmfile state file and the lock file pairs in a directory.

When `helmfile deps` are not explicitly run before commands like `sync`, all the helmfile behavior should remain as before.

Let's say you have `helmfile.1.yaml`:

```
repositories:
- name: stable
  url: https://kubernetes-charts.storage.googleapis.com

releases:
- name: envoy
  chart: stable/envoy
- name: envoy2
  chart: stable/envoy
```

`helmfile deps` generates `helmfile.1.lock` that looks like:

```
dependencies:
- name: envoy
  repository: https://kubernetes-charts.storage.googleapis.com
  version: 1.5.0
digest: sha256:e43b05c8528ea8ef1560f4980a519719ad2a634658abde0a98daefdb83a104e9
generated: 2019-05-14T16:45:37.78205+09:00
```

Under the hood, `helmfile deps` creates a temporary local helm chart with a dummy `Chart.yaml` and `requirements.yaml` deduced from the `helmfile.yaml` content, then runs `helm dependency update` to produce od update the corresponding `requirements.lock` file.

`helmfile` then renames it to match the name of the targeted helmfile state file and moves it,  so that it becomes adjacent to each `helmfile.yaml`.

Other `helmfile` commands like `sync`, `diiff`, `apply`, `lint` read chart version numbers from the lock file.

Resolves #483
2019-05-15 09:39:12 +09:00
KUOKA Yusuke 9eec72b318 Merge pull request #594 from sstarcher/fetch_devel
fix: Fetch devel

Fixes two issues encountered in https://sweetops.slack.com/archives/CE5NGCB9Q/p1557848927155400
2019-05-15 08:55:54 +09:00
Shane Starcher 081107bb82 error is coming back nil sometimes
this does not fix the root issue that is causing a nil error
2019-05-14 15:51:12 -07:00
Shane Starcher af69d1ba97 fetch should honor devel flag 2019-05-14 15:50:44 -07:00
KUOKA Yusuke 53ad91607a fix: log useful error message when failed walking through the file tree (#592)
Fixes #590
2019-05-14 10:02:50 +09:00
KUOKA Yusuke 255d92064e Merge pull request #587 from roboll/layering-enhancements
feat: helmfile.yaml layering enhancements

The current  [Layering](https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md#layering) system didn't work as documented, as it relies on helmfile to template each "part" of your helmfile.yaml THEN merge them one by one.

The reality was that helmfile template all the parts of your helmfile.yaml at once, and then merge those YAML documents. In https://github.com/roboll/helmfile/issues/388#issuecomment-436186278, @sruon was making a GREAT point that we may need to change helmfile to render templates earlier - that is to evaluate a template per each helmfile.yaml part separated by `---`. Sorry I missed my expertise to follow your great idea last year @sruon  😭 

Anyways, this, in combination with the wrong documentation, has made so many people confused. To finally overcome this situation, here's a fairly large PR that introduces the 2 enhancements:

- `bases:` for easier layering without go template expressions, especially `{{ readFunc "path/to/file" }}`s. This is the first commit of this PR.
- `helmfile.yaml` is splited by the separator `---` at first. Each part is then rendered as a go template(double-render applies as before). Finally, All the results are merged in the order of occurence. I assume this as an enhanced version of @sruon's work. This is the second commit of this PR.

Resolves #388
Resolve #584
Resolves #585 (`HELMFILE_EXPERIMENTA=true -f helmfile.yaml helmfile` disables the whole-file templating, treating the helmfile.yaml as a regular YAML file as the file ext. denotes. Use `helmfile.yaml.gotmpl` or `helmfile.gotmpl` to enable)
Fixes #568 (Use `bases` or `readFile` rather than not importing implicitly with `helmfile.d`
2019-05-14 09:48:20 +09:00
Yusuke KUOKA aef366660b feat: split-render-merge helmfile.yaml parts
This splits your helmfile.yaml by the YAML document separator "---" before evaluating go template expressions as outlined in https://github.com/roboll/helmfile/issues/388#issuecomment-491710348
2019-05-13 21:49:59 +09:00
Yusuke KUOKA 1db205de48 feat: "bases" for easier layerina
This adds the new configuration key `baeses` to your helmfile.yaml files, so that you can layer them without the `readFile` template function, which was a bit unintuitive.

Please see https://github.com/roboll/helmfile/issues/388#issuecomment-491710348 for more context
2019-05-13 21:48:00 +09:00
KUOKA Yusuke 4f83e69bf6 Various U/X improvements for helmfile apply (#586)
* Various U/X improvements for `helmfile apply`

This improves the U/X of `helmfile apply`, by allowing you to selectively apply sub-helmfiles.
When you have two or more sub-helmfiles processed, typing `n` to cancel the first doesn't automatically stop the whole helmfile execution.
Instead, it proceeds by diffing the next sub-helmfile, and asks you to apply it, which should be what the user would expect.

To support this workflow, I have suppressed useless exec logs, correct exit status when diff exists in sub-helmfiles but not in the parent helmfile, and made the final error message emitted by helmfile better.

More concretely, this moves more output from `helm` to STDERR and the `debug` log-level.

The overall output from `helmfile` should be a bit more cleaner especially for `apply`, `sync`, `diff` and perhaps other `helmfile` sub-commands, too.

For example, when one of release failed, `helmfile`'s final error message now includes the error message from the failed `helm` execution, like seen in the last line:

```
List of updated releases :
RELEASE   CHART          VERSION
envoy     stable/envoy     1.5.0

List of releases in error :
RELEASE
envoy2
in ./helmfile.yaml: in .helmfiles[0]: in /Users/c-ykuoka/helmfile/helmfile.1.yaml: failed processing release envoy2: helm exited with status 1:
  Error: UPGRADE FAILED: "envoy2" has no deployed releases
```

This way you can better understand what caused helmfile to finally fail.

`helmfile` has been streaminig a lot of stdout and stderr contents from the `helm` commands regardless of the helmfile's log-level. It has been suppressed by default and moved to the `debug` log-level.
You will see that it helps you focus on what was the cause of a failure.

While working on the above, I found an another bug that made `--detailed-exitcode` useless in some case.
That is, `helmfile diff --detailed-exitcode`, when any diff existed only in sub-helmfiles, has been returning an exit code of `1`. It should return `2` when any release had diff and no release had an error, regardless of the target is a sub-helmfile or a parent helmfile. Why? Because that's what `--detailed-exitcode` meant for!

After this PR gets merged, `helmfile diff --detailed-exitcode`  propery return exit code `2` in such cases.

Fixes #543
Resolves #540
2019-05-12 16:26:11 +09:00
KUOKA Yusuke bae842f234 fix: Send log messages to STDERR (#583)
And only useful outputs from helm commands like ones from `helm template` to STDOUT.

Fixes #551
2019-05-09 14:58:15 +09:00
Raj Perera 55c275b3aa Add presync event hook (#580)
* Add presync hook

* Add note to README about new hook
2019-05-09 10:13:31 +09:00
prakharrr-sl 0fc8ac395b typo in example (#582) 2019-05-09 10:12:14 +09:00
KUOKA Yusuke 272d55e31a fix: the merge mistake in imports (#578) 2019-05-07 09:55:34 +09:00
sgandon d683ea1d7b fix(sync): make error log showing when error (#577)
In debug mode, there was an error log display even when there was no error.
This PR fixes this issue.
2019-05-06 23:28:12 +09:00
Andrey Afoninsky 0ea5960ef2 feat: add .yml extension support (#575)
Resolves #565
2019-05-06 23:26:51 +09:00
sgandon 9a820d7bf2 feat: removes dictionary key for subhelm and uses selectorsInherited (#576)
Removed the usage of subhelmfile path as map key.
I also introduced the selectorsInherited key for explicit parent selector inheritance.

Ref #344
2019-05-06 10:06:32 +09:00
sgandon 4581e004b8 feat(#344): add sub helmfiles explicit selectors (#567)
Fixes #344 by allowing explicit selectors to be specified for composed helmfiles using the following structure

```yaml
helmfiles:
- path: helmfile.d/a*.yaml
  selectors:
  - name=prometheus      
  - name!=zipkin      
- helmfile.d/b*.yaml
- path: helmfile.d/c*.yaml
  selectors: {}
```

2 modes here : 
* legacy mode when no the env var HELMFILE_EXPERIMENTAL is not set to true
  * no selector : inherit from the command line.
  * selector:  is specified then it is used (an emty means no inheritance from command line and take everything).
* experimental when the env var HELMFILE_EXPERIMENTAL=true
  * no selector : nothing is inherited from the command line so use all releases.
  * selector:  is specified then it is used (an emty means no inheritance from command line and take everything).
2019-05-05 13:38:52 +09:00
Yusuke KUOKA d2353b5b70 accept go.mod and go.sum to be updated in pristine
I have currently no way to avoid this from happening in the first place
2019-05-04 22:45:31 +09:00
Yusuke KUOKA fb7e0a360c Update go.mod after running go mod vendor
Not sure why `go mod vendor` updates go.mod but `go build .` not
2019-05-04 22:17:16 +09:00
Yusuke KUOKA 59b254ee45 fix automated releases broken due to workspace location misconfig after go mod migration 2019-05-04 22:10:37 +09:00
sgandon 950bd23d86 fix(struct): renamed Error struct field into Failed (#566) 2019-05-02 20:41:49 +09:00
KUOKA Yusuke 8f030d5eab Bump go to 1.12.4 / Switch to go modules (#564)
* Bump go to 1.12.4 / Switch to go modules

Follow-up for https://github.com/roboll/helmfile/pull/560#issuecomment-486516109
2019-05-02 20:41:36 +09:00
sgandon a31077a1c0 feat(#502): display summary of upgraded, deleted and error releases (#560)
* feat(report): display summary of upgraded, deleted and error releases

* feat(#502): adds dep target in makefile

* feat(#502): removes vendor and fixes pristine in makefile
2019-04-25 13:33:34 +09:00
Aaron Gershman 32588ae319 readme example for failing when values file is missing (#561)
* readme example for failing when values file is missing

Believe this would assist with #548

* Update README.md

Using documentation yanked from searching for the key missingFileHandler https://github.com/roboll/helmfile/search?q=missingFileHandler&unscoped_q=missingFileHandler
2019-04-25 13:12:41 +09:00
Yusuke KUOKA f74963f4b6 fix(ci,release): Use existent base image for docker builds 2019-04-13 22:02:22 +09:00
KUOKA Yusuke d93ec77ea3 fix: with environment secrets (#556)
Since tillerless support we unintentionally broke this, and there isn't a real fix to this.
We must accept a limitation that helmfile needs a tiller installed on your cluster just for decrypting environment secrets.

Fixes #550
2019-04-13 22:01:53 +09:00
KUOKA Yusuke 04a9ea1d2a fix: helmfile apply and sync should properly delete releases (#555)
Since #526, `helmfile apply` have been really able to detect deletion of the last release only, and `sync` has been unable to mark releases with `installed: false` for removal.

Fixes #554
2019-04-13 21:47:27 +09:00
KUOKA Yusuke f53c66749e Bump go to 1.12.1 (#542) 2019-04-08 20:47:36 +09:00
Facundo Guerrero 8a13999de3 Fix link to shared-configuration-across-teams (#547) 2019-04-08 20:46:39 +09:00
Patrick Valsecchi 1acd07fa7e Simple implementation of the tillerless mode (#531)
Ref #449
2019-04-05 19:02:37 +09:00
Yusuke KUOKA 72c43a2f46 fix test 2019-04-03 23:47:08 +09:00
Yusuke KUOKA 08b48a8cca fix test 2019-04-03 23:43:23 +09:00
KUOKA Yusuke 0e00cdf2a5 fix: helmfile destroy should delete releases in the specified tiller namespace (#536)
Actually, 4 helm commands including "list", "diff", "status" and "delete" were not taking tiller-rerelated flags into account in helmfile. This fixes all these commands.

Fixes #534
2019-04-03 23:41:41 +09:00
Erik Osterman f522ba320b Add slack links (#533) 2019-04-03 22:36:17 +09:00
KUOKA Yusuke 8f1a15c9cd feat: helmfile destroy deletes and purges releases (#530)
* feat: `helmfile destroy` deletes and purges releases

This adds `helmfile destroy` that is basically `helmfile delete --purge`.

I've also tweaked the behavior of `delete` and `destroy` for releases with `installed: false`, so that it becomes consistent with other helmfile commands.
It now delete releases only when `installed: true` AND the release is already installed.

**Why an another command?**

Because it's easy to remember, and it also makes it easier to iterate on your helmfile.

We've been using `helmfile delete` from the beginning of helmfile,
and several months have been passed since we've added `--purge` to it.

We noticed that we always prefer to use `--purge` so that we can quickly iterate on helmfile by
e.g. `helmfile delete --purge && helmfile sync`. But making `--purge` default makes the `delete` command inconsistent with the helm's `delete`.

`destroy`, on the other hand, doesn't have such problem, and is still easy to remember for terraform users.

Resolves #511

* Update docs about `helmfile delete` and `helmfile destroy`
2019-04-02 21:17:38 +09:00
Patrick Valsecchi fa95e0dd92 Allow the same release name on different tillerNamespaces (#529)
Closes #528
2019-04-02 18:51:11 +09:00
KUOKA Yusuke eb36e93c5a fix: helmfile should not complain on missing values files for undesired releases (#526)
Fixes #519
2019-04-01 22:29:16 +09:00
KUOKA Yusuke 3edc7d6f33 fix: helmfile diff should not leave temp values files (#525)
Fixes #503 once again. A follow-up to #520 which was incomplete.
2019-04-01 22:24:18 +09:00
sgandon 57df25c3ff doc(team): changed the sub-helmfile as template (#524)
try to make the doc more clear.
2019-04-01 07:41:11 +09:00
KUOKA Yusuke 14a392666e feat: easier way to share configuration across helmfiles (#522)
This basically allows to define both `releases` and `helmfiles` within a helmfile.yaml, so that you can start using sub-helmfiles easily, by extracting only reused releases.

Resolves #445
2019-03-31 22:51:06 +09:00
KUOKA Yusuke 283dac1531 fix: Various helmfile commands should not leave temp values files (#520)
Fixes #504
2019-03-31 14:49:51 +09:00
KUOKA Yusuke f5e565ea3e feat: Tiller configuration per release (#516)
Resolves #486
2019-03-29 23:45:31 +09:00
KUOKA Yusuke 5f8b2e5c7f feat: Enhanced get template function (#513)
`get` is now able to take one more optional argument, that is used as the default value when the value for the key does not exist.

Resolves #465
Fixes #427
Fixes #357
Ref #460
2019-03-29 12:38:16 +09:00
KUOKA Yusuke 870d33f418 fix: helmfile delete should not stop on uninstalled release(s) (#509)
Resolves #481
2019-03-29 10:11:19 +09:00
KUOKA Yusuke d3c5417177 fix: prevent confusing error messages when installed: false (#508)
This removes `release: "your_release_name" not found` errors seen for releases with `installed: false` when running `helmfile sync` and `helmfile apply`.

The problem was that helmfile had been running `helm status` to detect releases to be deleted. helmfile now use `helm list ^YOUR_RELEASE_NAME$` to detect if the release is currently installed or not, which emits no error-like logs on against uninstalled releases.

Fixes #507

Fixes #507
2019-03-28 18:14:29 +09:00
KUOKA Yusuke 0639714136 feat: tpl template function (#504)
Add `tpl` template function that looks exactly like helm's one.

Resolves #420
2019-03-25 18:08:47 +09:00
sgandon 056d150856 doc(readme): changed from chart to helmfile (#501) 2019-03-22 21:35:42 +09:00
Yusuke KUOKA 9bcec38b3b fix sha256 checksum of the helm release archive, for successful helmfile release 2019-03-20 17:57:46 +09:00
Yusuke KUOKA 3e40e90a2a releasing: automatically checkout the master branch back 2019-03-20 17:51:47 +09:00
KUOKA Yusuke acbb416028 fix: absolute paths to helmfiles should work (#498)
Fixes #488
2019-03-20 17:45:48 +09:00
Mike Eves c4eb95f3ec Fix: Release options should override helm defaults (#497)
Options specified in releases (e.g. `recreatePods`) should override the respective options set in `helmDefaults`. Currently, `helmDefaults` takes precedence.

In the below example, `--force` should not be passed as an additional deployment argument:
```
helmDefaults:
  force: true

releases:
  - name: example
    namespace: example
    chart: some/repo
    version: ~1.24.1
    force: false
```

Fixes #492
2019-03-20 11:22:26 +09:00
Mike Eves f2996e2452 Support --atomic as a first class directive in helmfile (#491)
Resolves #487
2019-03-19 19:17:26 +09:00
KUOKA Yusuke 79ae2df71f fix: Always getting deprecated warning (#493)
Fixes #485
2019-03-19 19:15:37 +09:00
Dan O'Brien 8f7ab740bd minor typo (#496) 2019-03-19 19:14:52 +09:00
KUOKA Yusuke 841580f8b3 fix: helmfile sync should NOT continue when secret decryption fails (#490)
Fixes #336
2019-03-15 10:36:24 +09:00
kinzal c6c54fdbed fixes read absolute path woth readFile (#475) 2019-02-15 12:49:35 +09:00
KUOKA Yusuke 35fd5ff117 fix: remote file provided to set.file should not break helmfile (#474)
Fixes #473
2019-02-14 18:13:32 +09:00
KUOKA Yusuke f2bdcb9113 fix: sync should not hang on installed: false (#464)
Fixes #423
2019-02-04 17:16:58 +09:00
Yusuke KUOKA 9cfd6aeae7 Ensure helm-diff is installed for integration testing 2019-02-04 17:13:17 +09:00
Yusuke KUOKA 27da8a6b7a fix integration test 2019-02-04 15:15:56 +09:00
KUOKA Yusuke c6236a15bb feat: Opt-out dep-builds and repo-updates (#463)
* Improve code organization

To make sure it is still readable after upcoming changes to helmfile

* feat: `helmfile deps` to update dependencies of all the local charts

Resolves #450

* feat: helmfile updates repos and build deps by default

But not update deps. Use `helmfile deps` to update deps, and provide `--skip-deps` to skip updating repos and builds deps in sync/diff/apply/template

Resolves #415

* Improve integration test coverage
2019-02-04 12:07:25 +09:00
KUOKA Yusuke b894012c80 feat: Automatically update repos on diffing by default (#462)
Resolves #452
2019-02-04 11:12:34 +09:00
Shane Starcher fb256b0161 support labels for namespace and chart (#459) 2019-01-31 11:25:29 +09:00
Simon Li abeccc5e8e Fix indentation in writing-helmfile.md example (#454) 2019-01-25 23:32:33 +09:00
KUOKA Yusuke f84ed93754 fix: helmfle should not break when run without -f (#453)
Fixes #451
2019-01-25 12:13:56 +09:00
KUOKA Yusuke bf42d2519d fix: Relative path in helmfile not relative to file in multi file setup (#448)
Fixes #431
2019-01-24 10:48:01 +09:00
Maxim Mironenko 99ce8570c7 helmfiles examples link URL fix (#447)
* helmfiles examples link URL fix

* Update README.md

Co-Authored-By: maximmi <mmironenko@chanzuckerberg.com>
2019-01-24 10:46:59 +09:00
KUOKA Yusuke 571f351a8f feat: configurable concurrency for helmfile test (#442)
`helmfile test --concurency N` to set a concurrency number.
It is automatically ceiled at the number of releases just to reduce wasting computing resources.

Also, I've refactored the scatter-gather logic scattered across the code-base.

Resolves #433
2019-01-23 15:46:24 +09:00
KUOKA Yusuke b9a097ed14 fix: first-pass rendering of helmfile.yaml (#441)
There was a regression introduced by #439 that broke the first-pass rendering completely. This fixes that.
2019-01-22 15:55:03 +09:00
Yusuke KUOKA 344c20d790 release: fix make tasks for auto-tagging 2019-01-22 11:25:55 +09:00
KUOKA Yusuke 5f52f96709 fix missingFileHandler (#440)
- Fix panics when `missingFileHandler` is not defined
- Fix `missingFileHandler: Error` had been writing errors at the Warn level
- Add `Info` and `Debug` as valid values

Ref https://github.com/roboll/helmfile/pull/439#issuecomment-456155521
2019-01-22 11:22:53 +09:00
Yusuke KUOKA 171eacfb3c release: fix hack/semtag when there are one or more git remotes before origin 2019-01-22 01:29:28 +09:00
Yusuke KUOKA 7d976e5271 Semi-automate minor/patch version tagging 2019-01-22 01:23:22 +09:00
KUOKA Yusuke f813ac2642 feat: Release Template (#439)
This feature is supposed to help advanced use-cases like Conventional Directory Structure explained in several issues like #428.

Newly added configuration keys `templates`, `missingFileHandler`, and the ability to defer executing template expressions in `values`, `secrets`, `namespace`, and `chart` of releases allows you to abstract away repetitions into a reusable template:

```yaml
templates:
  default: &default
    missingFileHandler: Warn
    namespace: "{{`{{ .Release.Name }}`}}"
    chart: stable/{{`{{ .Release.Name }}`}}
    values:
    - config/{{`{{ .Release.Name }}`}}/values.yaml
    - config/{{`{{ .Release.Name }}`}}/{{`{{ .Environment.Name }}`}}.yaml
    secrets:
    - config/{{`{{ .Release.Name }}`}}/secrets.yaml
    - config/{{`{{ .Release.Name }}`}}/{{`{{ .Environment.Name }}`}}-secrets.yaml

releases:
- name: envoy
  <<: *default
```

See the updated documentation for more details.

Resolves #428
2019-01-22 01:19:07 +09:00
KUOKA Yusuke 23178b398c docs: Remove go get from the recommended installation methods (#437)
Add a contribution guide instead and note about the `go get` way there according to the discussion made in the relevant issue.

Resolves #393
2019-01-17 12:03:58 +09:00
Vincent Behar fea62032e9 fix: cleanup hook executed too early for apply command (#426)
fixes https://github.com/roboll/helmfile/issues/412

the `apply` command first runs the `diff` command, which triggers the execution of the `cleanup` hooks at the end of the `diff`.
2019-01-01 01:33:16 +09:00
Vincent Behar 1f7a8ddbe3 fix: prepare hook is not executed for diff command (#425)
fixes https://github.com/roboll/helmfile/issues/424

note that the "cleanup" hook is already executed as it should
2019-01-01 01:30:25 +09:00
Raphael Sampaio 413ab4f55b adds Homebrew as an installation method (#418) 2018-12-21 12:06:33 +09:00
Hung Tran Duc 39bc28c23f feat: configurable common path prefix for values files (#408)
Removes duplicated declaration path for cleaner helmfile, for example, this configuration.

```
- chart: projectABC
  name: {{ .Environment.Name }}-projectABC
  namespace: abc
  version: {{ .Environment.Values.projectABC.chart.version }}
  valuesPathPrefix: ../values/{{ .Environment.Name }}/
  values:
  - common.yaml
  - project-abc.yaml
  - platform.yaml
  - ...
  secrets:
  - secrets.yaml
```
2018-12-13 18:07:50 +09:00
Matteo Gazzetta 4b22d4fc78 Update Dockerfile and CircleCI (#417) 2018-12-11 07:01:23 +09:00
Eric Chin 2a35b6864c Remove extra appending of chartName for lint and template (#406)
Closes #407
2018-12-11 06:54:57 +09:00
Anatoly Rugalev 4b23213ce4 Added new installation method via AUR (#405) 2018-11-27 14:13:45 -08:00
Anatoly Rugalev 971cbcf1db Added path normalization in - set directive. (#404)
Fixes #399
2018-11-27 14:13:04 -08:00
Anatoly Rugalev d898dfbd0d Updated documentation related to deprecation of context directive (#403) 2018-11-27 14:12:40 -08:00
Anatoly Rugalev b41b44c313 Fixes #383. Deprecated context directive in favor of helmDefaults.kubeContext (#401) 2018-11-22 12:39:03 +09:00
Sam Weston b3e27db8b3 Improve documentation of Environment Secrets (#395) 2018-11-09 16:38:23 +09:00
harmjanblok 9cab7b00f6 Bump helm to v2.11.0 (#394) 2018-11-09 16:36:25 +09:00
Sam Weston bfc86de92d Correct minor spelling and grammar issues in README.md (#385) 2018-10-23 15:21:00 +09:00
Martin Mauch 5ad916003f Add instructions for installing via scoop on Windows (#372) 2018-10-16 16:22:08 +09:00
KUOKA Yusuke b0f9062e08 fix: incomplete environment values merging (#378)
Fixes #377

Ref https://github.com/imdario/mergo/pull/81
2018-10-03 22:39:37 +09:00
KUOKA Yusuke 770c3daa5f feat: get and getOrNil template funcs to allow defaulting in templates (#370)
* feat: `get` and `getOrNil` template funcs to allow defaulting in templates

Ref #357

* Add docs about missing keys and default values in templates
2018-09-28 11:44:49 +09:00
KUOKA Yusuke 6f0dc6e069 bump go to 1.11 (#376)
In go 1.11, the code formatter seems to have changed its formatting rule a bit, that resulted in `make fmt` producing changes introduced in this commit/pr.
2018-09-28 10:48:07 +09:00
KUOKA Yusuke f23a43c80c fix(ci,doc): follow up for default non-interactivity of helmfile (#375)
The integration test was failing due to that it was still using `--auto-approve` flag which was removed. The documentation was still refering to `--auto-approve` and had no explanation about the new `--interactive` flag. This fixes all these issues.

This a follow up for #368
See https://github.com/roboll/helmfile/pull/374#issuecomment-425291468 for more context.
2018-09-28 10:40:57 +09:00
David Genest 942f9a20b0 fix first pass rendering crash on syntax error (#374) 2018-09-28 10:27:11 +09:00
KUOKA Yusuke 429c42d429 doc: mark release as installed/uninstalled (#371)
Resolves #362
2018-09-27 11:17:00 +09:00
KUOKA Yusuke 7d1787d0b6 One Ctrl-C should exit helmfile gracefully (#369)
Perhaps in latest commits I unexpectedly changed it to require double Ctrl-C to actually interrupt. This fixes it so that one Ctrl-C exists helmfile.

On the way to the fix, I changed the exit-code to 128 + SIGNAL(2 for SIGINT, 15 for SIGTERM) according to common *nix commands.
2018-09-27 04:36:34 +09:00
KUOKA Yusuke 98be623701 helmfile should be non-interactive by default (#368)
Request for user confirmation only when a global `--interactive` flag is provided.

Resolves #354
2018-09-27 04:36:14 +09:00
KUOKA Yusuke 76122738c8 fix: cleanup hook commands output should be printed when debug logging enabled (#367)
There was a race condition due to that helmfile had not been waiting for goroutines to exit

Ref #363
2018-09-27 03:21:58 +09:00
KUOKA Yusuke 973d47bfc0 fix: sync repos always failing (#366) 2018-09-27 02:32:56 +09:00
KUOKA Yusuke 2eb13c25c5 fix: helmfile should pass release names to helm template (#365)
Fixes #360
2018-09-27 02:22:18 +09:00
KUOKA Yusuke b94265122f feat: least frequent repository update (#356)
Prevents helmfile from consuming unnecessarily much time in running `helm repo update` over and over.

helmfile now marks which repository was updated, and skip second and further `helm repo update` when all the `repositories` found in a helmfile.yaml was marked as already updated.

Let's say you had two helmfiles, the first one with repositories `foo` and `bar`, an the second one with only `bar`. `helmfile repos` will run `helm update repo` for the first helmfile, marking `foo` and `bar` as already updated. The second helmfile.yaml contains only `bar`, which is marked as already updated. So helmfile won't run `helm repo update` for the second.

This applies to all the helmfile command that results in `helm repo update`, like `repos`, `sync`, `diff` and so on.

Resolves #335
2018-09-27 02:10:11 +09:00
KUOKA Yusuke f2b610afdf feat: helmfile repos updates all repos regardless of selector and releases (#353)
Resolves #338
2018-09-27 01:44:05 +09:00
KUOKA Yusuke 6cba77d4f2 fix(hook): event hooks should run on remote charts, too (#364)
And fixed the bug that "diff" was not triggering "prepare" hooks.

Ref #363
2018-09-27 01:01:25 +09:00
KUOKA Yusuke b9de22b256 feat: prepare and cleanup release event hooks (#349)
Resolves #295
Resolves #330
Resolves #329 (Supports templating of only `releases[].hooks[].command` and `args` right now
Resolves #324
2018-09-21 10:35:12 +09:00
KUOKA Yusuke 4da13b44ee fix: --helm-binary does not work with sync when updating repo (#352)
Fixes #350
2018-09-20 09:31:16 +09:00
KUOKA Yusuke 5d1a92d3a5 fix: Forbid extraneous arguments (#348)
Resolves #296
2018-09-20 09:12:12 +09:00
KUOKA Yusuke 9808849986 feat: Merge multiple docs inside a single helmfile.yaml (#270)
Resolves #254
2018-09-19 15:42:01 +09:00
Dan Helfman 1ade353c1a Use --namespace value within Helmfile template (#343)
* Use --namespace value within Helmfile template (#326).

* Tabs and spaces.

* Tab/spaces fix.

* Pacifying "go fmt".

* Hard-coding namespace in test convenience function.

* MR feedback: Add comment to exported identifier.

* MR feedback: Put comment in correct location.

* fix(ci): fix never-ending build issue by auto-approving `helmfile delete`
2018-09-17 21:28:45 +09:00
Terry Corley 6453768f2e Fix small typo in error string in state.go (#345) 2018-09-17 20:27:39 +09:00
KUOKA Yusuke cf31d02f25 feat: Option to mark release as not installed (#342)
Resolves #197
2018-09-16 22:44:47 +09:00
KUOKA Yusuke c5bc932ba2 feat: Prompt before delete (#341)
Resolves #182
2018-09-16 22:43:29 +09:00
KUOKA Yusuke 81d796c167 fix: readFile and exec should run in the directory the template file resides in (#340)
Fixes #299
2018-09-16 22:43:17 +09:00
KUOKA Yusuke 37ea442515 feat: delete in reverse order (#339)
Resolves #312
2018-09-16 22:42:13 +09:00
David Genest 43a54486a1 docs: add precision on what value files will be rendered (#337) 2018-09-14 21:44:38 +09:00
David Genest b3da9a9a07 fix: evaluate helm-binary BEFORE any helm invocation (#333)
closes #332
2018-09-13 09:59:35 +09:00
David Genest 30155d8742 fix: use channel to harvest download results (#328)
Fixes #327
2018-09-12 09:12:23 +09:00
David Genest 7bfb58c0e4 feat: double render the helmfile (#308)
This allows using the environment values defined in the environments: section of helmfile.yaml to be used from other sections of the file.

This works by having two template renderers, the first-pass and the second-pass renderer.

The first-pass render renders a helmfile.yaml template with replacing template functions has side-effects with noop. So, use only funcs that don't have side-effects to compose your environment values.

Then the second-pass renderer renders the same helmfile.yaml template, but with the environment values loaded by the first-pass renderer.

The implementation uses a buffer instead of re-reading the file twice.

Resolves #297
2018-09-12 08:55:42 +09:00
KUOKA Yusuke 751e549253 fix: prevent panicing on invalid selector format (#323)
Fixes #322
2018-09-09 17:20:44 +09:00
KUOKA Yusuke 4f073167b2 fix: prevent panicing when an undefined environment name was specified via --environment (#321)
Fixes #320
2018-09-09 08:35:15 +09:00
KUOKA Yusuke 7c65b2ed2c fix: unexpected no releases found for any helmfiles on helmfiles: config (#318)
Fixes #315
Fixes #316
2018-09-08 13:50:51 +09:00
Shane Starcher 595c70f85b feat: add devel flag support (#313)
Resolves #161

Changelog:

* add support for devel flag

* allow for explicit disable of the devel default
2018-09-08 08:32:14 +09:00
KUOKA Yusuke 8f96f0aff5 fix: do not panic on invalid yaml (#314)
Fixes #311
2018-09-08 02:15:39 +09:00
Takuma Hashimoto c3934495fe fix: {{ .Environment.Name }} being always blank in helmfile.yaml
Correctly sets the environment name passed via the flag `--environment NAME`, so that it can be referenced by `{{ .Environment.Name }}`
2018-09-06 18:12:32 +09:00
KUOKA Yusuke a205f0d586 fix: concurrency issue in diff and sync (#309)
fixes #301
2018-09-06 10:14:10 +09:00
David Genest 2e44da1b54 ensure helmfiles are searched relative to base helmfile (#307) 2018-09-06 10:08:45 +09:00
Karl Stoney d0a4008965 Add missing repos on lint command. (#305)
https://github.com/roboll/helmfile/issues/303
2018-09-06 10:07:43 +09:00
Karl Stoney af121b85b5 feat: Concurrent chart download in template and lint commands
This enables `helmfile lint` and `helmfile template` commands to fetch and untar all the required charts concurrently. The concurrency is configurable via the `--concurrency` flag, that defaults to `0`.

Ref #292
2018-09-05 08:19:57 +09:00
KUOKA Yusuke 1c3bfcca10 feat(environment): Allow missing environment in helmfile.yaml, partly. (#294)
helmfile as of today ensures that all the targeted helmfile.yaml to have the specified environment defined in it.
That is, `helmfile --environment prod -f helmfile.d/ sync` fails if any helmfile under `helmfile.d/` is missing the `production` environment.

This changes the validation logic, so that helmfile fails only when all the helmfiles miss the environment.

Resolves #279
2018-09-04 22:04:37 +09:00
Hans Kristian Flaatten 7d7ca74a05 docs: update link to documentation for latest released version (#293) 2018-09-04 21:49:21 +09:00
KUOKA Yusuke f1ac50bf44 feat: apply run sync against only affected releases (#291)
Enhance the `diff` functionality to be able to return affected releases that has any changes,
so that the succeeding `sync` can be run against only the affected releases.

This provides us extra idempotency.

Resolves #277
2018-09-04 21:48:43 +09:00
KUOKA Yusuke 60843cc224 fix: Setting concurrency>1 with single release breaks sync (#290)
Fixes #287
2018-09-04 20:59:24 +09:00
Karl Stoney 53dea091e3 Fix set in helmfile template and helmfile lint (#289)
This removes --concurrenty flag from `helmfile template` and `helmfile lint`. YAGNI.
Also separates out the downloadCharts into a function so it can be used by lint too.

Fixes #288
2018-09-04 20:55:32 +09:00
KUOKA Yusuke 5510de4765 feat(apply): update repos and and deps before running diff (#286)
Resolves #271
2018-09-04 16:12:43 +09:00
Karl Stoney 93c5d4c219 feat: helmfile template (#284)
`helmfile template` runs `helm template` over releases within the helmfiles, and provide you a stream of generated yaml documents of Kubernetes resources via stdout.

Resolves #283
2018-09-04 11:31:43 +09:00
KUOKA Yusuke 8a90e5320c feat: Ability to call arbitrary command from a template (#282)
Resolves #244
2018-09-03 16:48:03 +09:00
rmartinez3 6be53b1bcc WIP: adding to be albe to set multiple flags for --set, -f, --value in args (#278)
* adding to be albe to set multiple flags for --set, -f, --value in args

* add test for passing a set flag with two or more key-values
2018-09-03 16:45:02 +09:00
KUOKA Yusuke b0cc7ba96c fix: sync partially run helm upgrade even one of releases failed while loading values.yaml (#281)
Fixes #280
2018-09-03 16:43:57 +09:00
KUOKA Yusuke 54f1567294 feat: --suppress-secrets of diff and apply commands (#272)
Adds `--suppress-secrets` to `helmfile apply` and `helmfile diff`, so that the diff command omits the contents of secrets from its output. This is a security feature that should always be turned on for CI/CD use-cases.

With `--suppress-secrets`, the output when there is any change looks like:

```
Comparing bar stable/grafana
default, baz-grafana, Secret (v1) has changed:
+ Changes suppressed on sensitive content of type Secret
```

Resolves #269
2018-09-02 21:21:29 +09:00
KUOKA Yusuke 046281c70f feat: Environment Secrets (#274)
Resolves #255
2018-09-02 21:07:35 +09:00
KUOKA Yusuke 98617f78e3 fix: --args "--set foo=bar" always result in "unknown flag: --set foo" error (#268)
Ref #248
2018-09-02 21:06:38 +09:00
KUOKA Yusuke ed0854a5c0 feat: Environment and Environment Values (#267)
Resolves #253
2018-08-31 22:59:27 +09:00
KUOKA Yusuke 7c793fdb88 feat: helmfiles: <ordered glob patterns of helmfiles> configuration (#266)
Resolves #247
2018-08-31 12:03:18 +09:00
Dan Wendorf b70956b5d8 Bump Helm 2.9.1 -> 2.10.0 (#265) 2018-08-31 10:17:51 +09:00
KUOKA Yusuke 3840605e04 feat: helmfile apply [--auto-approve] (#263)
This command syncs releases only if there is any difference between the desired and the current state. It asks for an confirmation by default. Provide `--auto-approve` flag after the `apply` command to skip it.

Resolves #205
2018-08-31 10:15:02 +09:00
KUOKA Yusuke bb3b44e511 fix: follow up for the relative paths improvement (#262)
Fixes for the bugs that are introduced by #261, that is values.yaml files specified in `values:` have redundant base path in their prefixes, and remaining .dec files after secrets decryption(https://github.com/roboll/helmfile/issues/251#issuecomment-417285854)
2018-08-30 21:59:59 +09:00
KUOKA Yusuke 421299c883 feat: All the paths are relative to helmfile.yaml (#261)
`helmfile lint` works with relative chart reference (#252)
The tempalte function `readFile` accepts the path relative to helmfile.yaml

Resolves #246
Fixes #252
2018-08-30 19:47:52 +09:00
KUOKA Yusuke 79f0e70ce8 fix: avoid "cross-device link" errors while decrypting secrets (#260)
ref https://github.com/roboll/helmfile/issues/251#issuecomment-417166296
2018-08-30 17:20:38 +09:00
KUOKA Yusuke 0c8a89cbaf fix: coordinate secret decryption (#259)
Fixes #258
2018-08-30 17:03:45 +09:00
KUOKA Yusuke b3ebd4cdd0 fix: Keep backward-compatibility broken after introduction of values file template (#257)
Fixes #249
2018-08-30 16:57:37 +09:00
Shane Starcher 9b71c64ef2 fix: use temporary file for secret file to prevent deletion collision (#250)
Fixes #167
2018-08-30 09:28:26 +09:00
KUOKA Yusuke 822cc13e72 feat: Template functions to replace sets (#242)
Resolves #227
2018-08-28 22:00:51 +09:00
KUOKA Yusuke 0ac8401d1e feat: feat: Configurable default values for --verify, --force, --recreate-pods, and --timeout (#241)
Resolves #230
2018-08-28 15:11:05 +09:00
KUOKA Yusuke 815ee1f85b feat: --force, --recreate-pods, --timeout as first-class citizens in helmfile.yaml (#239)
Resolves #229
2018-08-27 23:06:16 +09:00
KUOKA Yusuke 2c36640ad2 To reduce integration test flakiness 2018-08-27 23:04:16 +09:00
KUOKA Yusuke faaf172615 fix: panic on flag parsing error (#238)
This is a follow-up for #235.

We setup the default logger only after the flags are parsed successfully.
That's because we init the logger according to the --log-level flag value.
2018-08-27 22:58:25 +09:00
KUOKA Yusuke fcbd6aff29 fix: Relative path in KUBECONFIG envvar (#237)
Resolves #173
2018-08-27 22:53:17 +09:00
Shane Starcher 543a653864 improve error handling for value rendering (#235)
Fixes #233

Output on values render error:

```
err: failed to render [/Users/sstarcher/xxx/xxx/values.yaml], because of template: stringTemplate:10:18: executing "stringTemplate" at <requiredEnv "HELM_AC...>: error calling requiredEnv: required env var `HELM_ACCOUNT` is not set
```

Also removes panic and sets the output as `apps.Run()` can and will return errors.  Panic makes no sense.

Changelog:

* improve error handling for value rendering

* only output if error exists

* add exit status
2018-08-27 12:53:25 +09:00
rmartinez3 fa69ac2cfa fix: Args being reordered (#224)
Fixes #212
2018-08-27 10:41:00 +09:00
KUOKA Yusuke 2121fe57b2 Merge pull request #232 from mumoshu/set-array
feat: set array
2018-08-24 17:12:17 +09:00
Yusuke KUOKA cfcafa2e46 feat: set array
Fixes #148
2018-08-24 17:01:34 +09:00
KUOKA Yusuke b6d2ee9d43 Merge pull request #231 from mumoshu/partly-allow-duplicate-release-name
fix: helmfile should fail on duplicate release name after filtered by labels
2018-08-24 16:43:38 +09:00
Yusuke KUOKA 35542b5516 fix: helmfile should fail on duplicate release name after filtered by labels
This is a follow-up for #218, fixes the unintentional degradation that broken the use-case described in https://github.com/roboll/helmfile/issues/193#issuecomment-415434408
2018-08-24 16:18:08 +09:00
KUOKA Yusuke 45d0f1c8e4 Merge pull request #228 from mumoshu/set-file
feat: Support for helm's --set-file
2018-08-24 13:44:07 +09:00
KUOKA Yusuke 96d0781b85 Merge pull request #226 from mumoshu/usage-without-internet
doc: Using helmfile without an Internet connection
2018-08-24 13:43:53 +09:00
Yusuke KUOKA 48e65791c0 feat: Support for helm's --set-file
Resolves #207
2018-08-24 13:06:31 +09:00
Yusuke KUOKA 2398ad2f35 doc: Using helmfile without an Internet connection
Resolves #155
2018-08-24 12:10:18 +09:00
KUOKA Yusuke 013276f576 Merge pull request #225 from mumoshu/unmatched-selector-exit-code
feat: exit code 2 on unmatched selectors
2018-08-24 11:47:10 +09:00
Yusuke KUOKA 3a16b7ed91 feat: exit code 2 on unmatched selectors
Along with the fix for the bug introduced via #217

Resolves #219
2018-08-24 11:40:30 +09:00
KUOKA Yusuke a5d42e741b Merge pull request #223 from mumoshu/show-which-release-failed
feat: emit error message containing which release in which helmfile an upgrade failed
2018-08-23 12:00:15 +09:00
Yusuke KUOKA 313b5de378 feat: emit error message containing which release in which helmfile an upgrade failed
Resolves #66
2018-08-23 11:45:43 +09:00
KUOKA Yusuke 0541731350 Merge pull request #222 from mumoshu/wait-for-release
feat: `wait: true` in each release
2018-08-23 10:43:01 +09:00
Yusuke KUOKA b4623f0515 feat: wait: true in each release
To instruct `helm upgrade` to wait for resources via `--wait`

Resolves #65
2018-08-23 10:37:15 +09:00
KUOKA Yusuke ffa40beff9 Merge pull request #221 from mumoshu/env-files
doc: Instruction to use env files
2018-08-23 09:47:01 +09:00
Yusuke KUOKA 4fae044f0e doc: Instruction to use env files
Closes #203
2018-08-23 09:44:42 +09:00
KUOKA Yusuke d8aa9f305b Merge pull request #218 from mumoshu/should-fail-on-duplicate-release-name
fix: helmfile should fail on duplicate release name
2018-08-23 09:32:03 +09:00
KUOKA Yusuke c402759363 Merge pull request #217 from mumoshu/less-verbose-unmatched-selector-logging
fix: Make logging on unmatched selectors less verbose
2018-08-23 09:27:32 +09:00
Yusuke KUOKA e84ff05a63 fix: helmfile should fail on duplicate release name
Fixes #193
2018-08-23 09:26:39 +09:00
Yusuke KUOKA fcb75fa851 fix: Make logging on unmatched selectors less verbose
Fixes #200
2018-08-23 09:23:55 +09:00
KUOKA Yusuke d609dab848 feat: Template values files (#216)
Resolves #97
2018-08-22 18:53:32 +09:00
KUOKA Yusuke 6c89fac8e5 Merge pull request #215 from mumoshu/diff-target-namespace
feat(diff): Pass namespace to helm-diff
2018-08-22 18:51:10 +09:00
KUOKA Yusuke 3550e5f335 Merge pull request #213 from chadasapp/master
Typofix: required_env -> requiredEnv
2018-08-22 15:05:29 +09:00
Yusuke KUOKA 127dd857df feat(diff): Pass namespace to helm-diff
Tested manually by running `helmfile diff` with debug logging:

```yaml
helmDefaults:
  tillerNamespace: foo

charts:
  - name: grafana
    namespace: grafana
    chart: stable/grafana
```

```console
$ ./helmfile --log-level debug diff
Comparing grafana stable/grafana
exec: helm diff upgrade --allow-unreleased grafana stable/grafana --namespace grafana --tiller-namespace=foo
```

Resolves #179
2018-08-22 11:20:55 +09:00
Chad Horohoe 2cf2dbb868 Typofix: required_env -> requiredEnv
The name is `requiredEnv` and I cannot find any references to `required_env` in the repository anywhere.
2018-08-20 11:10:01 -07:00
KUOKA Yusuke 545a4a3efa Merge pull request #208 from cloudposse/add-examples
Add link to more examples
2018-08-20 21:32:26 +09:00
KUOKA Yusuke 0ed1355573 Merge pull request #206 from InTheCloudDan/patch-1
remove outdated warning.
2018-08-07 15:28:05 +09:00
Erik Osterman 9fef3f7d0f Add link to more examples 2018-08-06 23:20:49 -07:00
KUOKA Yusuke 0d457c4df0 Merge pull request #201 from maver1ck/helmfile-111
Add --args to delete
2018-08-03 09:36:48 +09:00
KUOKA Yusuke 6be52bfbbb Merge pull request #199 from mumoshu/optional-detailed-exitcode-for-diff
feat: Optional detailed exitcodes for `helmfile diff`
2018-08-03 09:36:08 +09:00
Dan O'Brien 089c158084 remove outdated warning. 2018-08-02 20:19:29 -04:00
Maciej Bryński d67573299c Add --args to delete 2018-07-31 10:18:31 +02:00
Yusuke KUOKA 3b4ce90a5a feat: Optional detailed exitcodes for helmfile diff
Adds the `--detailed-exitcode` to the `helmfile diff` command to return `1` on failure, and `2` when no error but diff is seen.

This feature requires the latest `helm-diff` containing https://github.com/databus23/helm-diff/pull/78, and `helm` containing https://github.com/helm/helm/pull/4367.

This is verified to work by manually running commands like the followings:

```bash
./helmfile --helm-binary helm211dev -f ./examples/helmfile.d diff --detailed-exitcode; echo $?
./helmfile --helm-binary helm211dev -f ./examples/helmfile.d diff; echo $?
```

Note that, in above example commands, `helm211dev` is a custom `helm` binary that is built from helm's master branch containing [the necessary enhancement to allow propagate non-zero plugin exit code](https://github.com/helm/helm/pull/4367).
2018-07-26 00:16:47 +09:00
KUOKA Yusuke 92d09a3e92 Merge pull request #198 from mumoshu/fix-ci-flakiness
fix(ci): Reduce flakiness
2018-07-25 23:24:44 +09:00
Yusuke KUOKA 1a5558a306 fix(ci): Reduce flakiness
Fix #103
2018-07-25 11:21:01 +09:00
KUOKA Yusuke ee3632cebc Merge pull request #191 from mumoshu/fix-cmd-descrs
Fix command descriptions
2018-07-19 12:02:07 +09:00
KUOKA Yusuke 2297093f84 Merge pull request #190 from mumoshu/fix-helmfile.d-defaulting
fix: `helmfile sync` doesn't recognize the default `helmfile.d/` directory
2018-07-19 12:01:49 +09:00
KUOKA Yusuke 506de12fcd fix: helmfile -f dir/ -l k=v sync fails on first helmfile w/ no match (#188)
Fixes #187

Changelog:

* fix: `helmfile -f dir/ -l k=v sync` fails on first helmfile w/ no match

* fix(ux): present the filename if the selector did not match any releases

```console
$ helmfile -f helmfile.d/ -l foo=bar sync
2018/07/12 23:37:29 specified selector did not match any releases in helmfile.d/00-backend.yaml
2018/07/12 23:37:29 specified selector did not match any releases in helmfile.d/01-frontend.yaml
2018/07/12 23:37:29 err: specified selector did not match any releases in any helmfile
```
2018-07-19 12:01:14 +09:00
rmartinez3 01fe5b4213 Adding merging for helmDefaults args and --args, adding tillerNamspace and kubeContext to helmDefaults (#186)
Add tillerNamespace and kubeContext to helmDefaults. Also add merging of args capability. 

People can start adding additional fields under helmDefaults on top of this. The additional fields are merged to args as long as they are implemented using the framework introduced by this commit.

Changelog:

* adding merging for helm defaults args and --args, adding tillernamspace, kubecontext to helmdefaults, adding getargs to lint command

* adding docs for helmdefaults

* checking if empty default values
2018-07-19 11:53:09 +09:00
Simon Li 06a1d245ae Multi-level logging (#185)
Closes #93

Summary of changes:

- Adds the [uber-go-/zap](https://github.com/uber-go/zap) library for leveled logging (69040996e0bcbd2f211d0d8d576147027e928e3b)
- A new CLI arg `--log-level` which can take the values `debug` `info` `warn` `error` `fatal` `panic`. The default is `info`, and the existing `--quiet` flag is treated as `--log-level warn`
- All `helm.exec` calls are preceded by a one-line summary. The current information on the full helm command line arguments `exec: helm exec ... ` is only output at `debug` level. This means sensitive command line arguments which may include passwords and `--set` arguments should be hidden by default.
2018-07-19 11:33:57 +09:00
Yusuke KUOKA 438d19852a Fix command descriptions
Resolves #170
2018-07-12 16:09:46 +09:00
Yusuke KUOKA c74db5d281 fix: helmfile sync doesn't recognize the default helmfile.d/ directory
Fixes #189
2018-07-12 14:13:56 +09:00
KUOKA Yusuke 35732f3a93 Bugfix for helm-binary flag. to global (#184)
Two follow-up commits for #181 

* Make the --helm-binary flag global
* Add helm-binary support to the delete command
2018-07-05 11:56:09 +09:00
hori-ryota 7f37c6b39f Add helm-binary opt to delete command 2018-07-04 22:10:17 +09:00
hori-ryota fa0ff32b00 Bugfix for helm-binary flag. to global 2018-07-04 19:01:46 +09:00
Hori Ryota 9f57f8be7d Add the option to specify an alternative helm binary (#181)
`helm` may not be version compatible. Add an option to specify an alternative `helm` command on runtime, possibly according to the version of tiller running on your cluster.
2018-07-03 22:19:02 +09:00
200 changed files with 16268 additions and 25457 deletions
+37 -13
View File
@@ -4,31 +4,51 @@ jobs:
build:
docker:
- image: circleci/golang:1.10.1
working_directory: /go/src/github.com/roboll/helmfile
- image: circleci/golang:1.12.4
environment:
GOPATH: /go
working_directory: /home/circleci/workspace/helmfile
steps:
- checkout
- restore_cache:
keys:
- go-mod-cache-v1-{{ checksum "./go.sum" }}
- go-mod-cache-v1-
- run: go env
- run: make build
- save_cache:
key: go-mod-cache-v1-{{ checksum "./go.sum" }}
paths:
- /go/pkg/mod
- persist_to_workspace:
root: /go/src/github.com/roboll/helmfile
root: ./
paths:
- .
test:
environment:
GOPATH: /go
docker:
- image: circleci/golang:1.10.1
working_directory: /go/src/github.com/roboll/helmfile
- image: circleci/golang:1.12.4
working_directory: /home/circleci/workspace/helmfile
steps:
- checkout
- restore_cache:
keys:
- go-mod-cache-v1-{{ checksum "./go.sum" }}
- go-mod-cache-v1-
- run: make check
- run: make pristine
- run: make test
# thanks to https://raw.githubusercontent.com/weaveworks/launcher/master/.circleci/config.yml
integration_tests:
machine: true
machine:
image: circleci/classic:201808-01
environment:
CHANGE_MINIKUBE_NONE_USER: true
MINIKUBE_WANTUPDATENOTIFICATION: false
MINIKUBE_WANTREPORTERRORPROMPT: false
steps:
- checkout
- run: mkdir ~/build
@@ -38,7 +58,7 @@ jobs:
- run:
name: Install helm
environment:
HELM_VERSION: v2.9.0
HELM_VERSION: v2.13.0
command: |
HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
curl -Lo ${HELM_FILENAME} "https://kubernetes-helm.storage.googleapis.com/${HELM_FILENAME}"
@@ -49,14 +69,13 @@ jobs:
name: Deploy minikube
environment:
CHANGE_MINIKUBE_NONE_USER: true
K8S_VERSION: v1.9.0
MINIKUBE_VERSION: v0.25.2
K8S_VERSION: v1.12.3
MINIKUBE_VERSION: v0.30.0
command: |
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/linux/amd64/kubectl
chmod +x kubectl && sudo mv kubectl /usr/local/bin/
curl -Lo minikube https://storage.googleapis.com/minikube/releases/${MINIKUBE_VERSION}/minikube-linux-amd64
chmod +x minikube && sudo mv minikube /usr/local/bin/
sudo minikube config set WantReportErrorPrompt false
sudo -E minikube start --vm-driver=none --kubernetes-version=${K8S_VERSION}
sudo -E minikube update-context
- run:
@@ -72,16 +91,21 @@ jobs:
release:
docker:
- image: circleci/golang:1.10.1
working_directory: /go/src/github.com/roboll/helmfile
- image: circleci/golang:1.12.4
working_directory: /home/circleci/workspace/helmfile
steps:
- checkout
- restore_cache:
keys:
- go-mod-cache-v1-{{ checksum "./go.sum" }}
- go-mod-cache-v1-
- setup_remote_docker
- run:
command: |
docker login -u="$DOCKER_USER" -p="$DOCKER_PASS" quay.io
make tools
BUILD_URL="$CIRCLE_BUILD_URL" make push release
go mod vendor
ORG=roboll BUILD_URL="$CIRCLE_BUILD_URL" make push release
workflows:
version: 2
+3 -1
View File
@@ -1,3 +1,5 @@
dist/
.idea/
helmfile
helmfile
helmfile.lock
vendor/
+39
View File
@@ -0,0 +1,39 @@
# DCO + License
By contributing to `helmfile`, you accept and agree to the following DCO and license terms and
conditions for your present and future Contributions submitted to the `helmfile` project.
[DCO](https://developercertificate.org/)
[License](https://github.com/roboll/helmfile/blob/master/LICENSE)
# Developing helmfile
Locate your `GOPATH`, usually `~/go`, and run:
```console
$ go get github.com/roboll/helmfile
$ cd $GOPATH/src/github.com/roboll/helmfile
$ git checkout -b your-shiny-new-feature origin/master
...
$ git commit -m 'feat: do whatever for whatever purpose
This adds ... by:
- Adding ...
- Changing ...
- Removing...
Resolves #ISSUE_NUMBER
'
$ hub fork
$ git push YOUR_GITHUB_USER your-shiny-new-feature
$ hub pull-request
```
Note that the above tutorial uses [hub](https://github.com/github/hub) just for ease of explanation.
Please use whatever tool or way to author your pull request!
+22 -9
View File
@@ -1,29 +1,42 @@
FROM golang:1.10.1-alpine3.7 as builder
FROM golang:1.12.4-alpine3.9 as builder
RUN apk add --no-cache make git
WORKDIR /go/src/github.com/roboll/helmfile/
COPY . /go/src/github.com/roboll/helmfile/
WORKDIR /workspace/helmfile
COPY . /workspace/helmfile
RUN make static-linux
# -----------------------------------------------------------------------------
FROM alpine:3.7
FROM alpine:3.8
RUN apk add --no-cache ca-certificates git bash curl
RUN apk add --no-cache ca-certificates git bash curl jq
ARG HELM_VERSION=v2.9.1
ARG HELM_VERSION=v2.13.0
ARG HELM_LOCATION="https://kubernetes-helm.storage.googleapis.com"
ARG HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
ARG HELM_SHA256="56ae2d5d08c68d6e7400d462d6ed10c929effac929fedce18d2636a9b4e166ba"
ARG HELM_SHA256="15eca6ad225a8279de80c7ced42305e24bc5ac60bb7d96f2d2fa4af86e02c794"
RUN wget ${HELM_LOCATION}/${HELM_FILENAME} && \
sha256sum ${HELM_FILENAME} | grep -q "${HELM_SHA256}" && \
tar zxf ${HELM_FILENAME} && mv /linux-amd64/helm /usr/local/bin/ && \
rm ${HELM_FILENAME} && rm -r /linux-amd64
# using the install documentation found at https://kubernetes.io/docs/tasks/tools/install-kubectl/
# for now but in a future version of alpine (in the testing version at the time of writing)
# we should be able to install using apk add.
ENV KUBECTL_VERSION="v1.14.5"
ENV KUBECTL_SHA256="26681319de56820a8467c9407e9203d5b15fb010ffc75ac5b99c9945ad0bd28c"
RUN curl -LO "https://storage.googleapis.com/kubernetes-release/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl" && \
sha256sum kubectl | grep ${KUBECTL_SHA256} && \
chmod +x kubectl && \
mv kubectl /usr/local/bin/kubectl
RUN mkdir -p "$(helm home)/plugins"
RUN helm plugin install https://github.com/databus23/helm-diff && \
helm plugin install https://github.com/futuresimple/helm-secrets
helm plugin install https://github.com/futuresimple/helm-secrets && \
helm plugin install https://github.com/hypnoglow/helm-s3.git && \
helm plugin install https://github.com/aslafy-z/helm-git.git && \
helm plugin install https://github.com/rimusz/helm-tiller
COPY --from=builder /go/src/github.com/roboll/helmfile/dist/helmfile_linux_amd64 /usr/local/bin/helmfile
COPY --from=builder /workspace/helmfile/dist/helmfile_linux_amd64 /usr/local/bin/helmfile
CMD ["/usr/local/bin/helmfile"]
Generated
-65
View File
@@ -1,65 +0,0 @@
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
name = "github.com/Masterminds/semver"
packages = ["."]
revision = "8d82589cda2d7b5b9167396841975c4535c9bec6"
version = "v1.4.1"
[[projects]]
name = "github.com/Masterminds/sprig"
packages = ["."]
revision = "6b2a58267f6a8b1dc8e2eb5519b984008fa85e8c"
version = "v2.15.0"
[[projects]]
name = "github.com/aokoli/goutils"
packages = ["."]
revision = "3391d3790d23d03408670993e957e8f408993c34"
version = "v1.0.1"
[[projects]]
name = "github.com/google/uuid"
packages = ["."]
revision = "064e2069ce9c359c118179501254f67d7d37ba24"
version = "0.2"
[[projects]]
name = "github.com/huandu/xstrings"
packages = ["."]
revision = "2bf18b218c51864a87384c06996e40ff9dcff8e1"
version = "v1.0.0"
[[projects]]
name = "github.com/imdario/mergo"
packages = ["."]
revision = "9d5f1277e9a8ed20c3684bda8fde67c05628518c"
version = "v0.3.4"
[[projects]]
name = "github.com/urfave/cli"
packages = ["."]
revision = "6011f165dc288c72abd8acd7722f837c5c64198d"
[[projects]]
branch = "master"
name = "golang.org/x/crypto"
packages = [
"pbkdf2",
"scrypt"
]
revision = "b2aa35443fbc700ab74c586ae79b81c171851023"
[[projects]]
name = "gopkg.in/yaml.v2"
packages = ["."]
revision = "5420a8b6744d3b0345ab293f6fcba19c978f1183"
version = "v2.2.1"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "2f414b2f5156dc8144a90cdbbb9685d5b8af323024f986bb52e66601f02f3c0f"
solver-name = "gps-cdcl"
solver-version = 1
-11
View File
@@ -1,11 +0,0 @@
[[constraint]]
name = "github.com/Masterminds/sprig"
version = "2.15.0"
[[constraint]]
branch = "v2"
name = "gopkg.in/yaml.v2"
[prune]
go-tests = true
unused-packages = true
+25 -4
View File
@@ -26,19 +26,24 @@ integration:
.PHONY: integration
cross:
env CGO_ENABLED=0 gox -os '!freebsd !netbsd' -arch '!arm' -output "dist/{{.Dir}}_{{.OS}}_{{.Arch}}" -ldflags '-X main.Version=${TAG}' ${TARGETS}
env CGO_ENABLED=0 gox -os '!openbsd !freebsd !netbsd' -arch '!arm !mips !mipsle !mips64 !mips64le !s390x' -output "dist/{{.Dir}}_{{.OS}}_{{.Arch}}" -ldflags '-X main.Version=${TAG}' ${TARGETS}
.PHONY: cross
static-linux:
env CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o "dist/helmfile_linux_amd64" -ldflags '-X main.Version=${TAG}' ${TARGETS}
.PHONY: linux
env CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOFLAGS=-mod=vendor go build -o "dist/helmfile_linux_amd64" -ldflags '-X main.Version=${TAG}' ${TARGETS}
.PHONY: static-linux
install:
env CGO_ENABLED=0 go install -ldflags '-X main.Version=${TAG}' ${TARGETS}
.PHONY: install
clean:
rm dist/helmfile_*
.PHONY: clean
pristine: generate fmt
git ls-files --exclude-standard --modified --deleted --others | diff /dev/null -
git diff | cat
git ls-files --exclude-standard --modified --deleted --others -x vendor | grep -v '^go.' | diff /dev/null -
.PHONY: pristine
release: pristine cross
@@ -61,3 +66,19 @@ tools:
TAG = $(shell git describe --tags --abbrev=0 HEAD)
LAST = $(shell git describe --tags --abbrev=0 HEAD^)
BODY = "`git log ${LAST}..HEAD --oneline --decorate` `printf '\n\#\#\# [Build Info](${BUILD_URL})'`"
release/minor:
git checkout master
git pull --rebase origin master
bash -c 'if git branch | grep autorelease; then git branch -D autorelease; else echo no branch to be cleaned; fi'
git checkout -b autorelease origin/master
bash -c 'SEMTAG_REMOTE=origin hack/semtag final -s minor'
git checkout master
release/patch:
git checkout master
git pull --rebase origin master
bash -c 'if git branch | grep autorelease; then git branch -D autorelease; else echo no branch to be cleaned; fi'
git checkout -b autorelease origin/master
bash -c 'SEMTAG_REMOTE=origin hack/semtag final -s patch'
git checkout master
+817 -57
View File
@@ -1,18 +1,19 @@
# helmfile [![CircleCI](https://circleci.com/gh/roboll/helmfile.svg?style=svg)](https://circleci.com/gh/roboll/helmfile)
# Helmfile [![CircleCI](https://circleci.com/gh/roboll/helmfile.svg?style=svg)](https://circleci.com/gh/roboll/helmfile)
Deploy Kubernetes Helm Charts
[![Docker Repository on Quay](https://quay.io/repository/roboll/helmfile/status "Docker Repository on Quay")](https://quay.io/repository/roboll/helmfile)
[![Slack Community #helmfile](https://slack.sweetops.com/badge.svg)](https://slack.sweetops.com)
## status
## Status
Even though Helmfile is used in production environments [across multiple organizations](USERS.md), it is still in its early stage of development, hence versioned 0.x.
Helmfile complies to Semantic Versioning 2.0.0 in which v0.x means that there could be backward-incompatible changes for every release.
Note that we will try our best to document any backward incompatibility.
Note that we will try our best to document any backward incompatibility. And in reality, helmfile had no breaking change for a year or so.
## about
## About
Helmfile is a declarative spec for deploying helm charts. It lets you...
@@ -22,23 +23,78 @@ Helmfile is a declarative spec for deploying helm charts. It lets you...
To avoid upgrades for each iteration of `helm`, the `helmfile` executable delegates to `helm` - as a result, `helm` must be installed.
## configuration syntax
## Highlights
**CAUTION**: This documentation is for the development version of helmfile. If you are looking for the documentation for any of releases, please switch to the corresponding branch like [v0.12.0](https://github.com/roboll/helmfile/tree/v0.12.0).
**Declarative**: Write, version-control, apply the desired state file for visibility and reproducibility.
**Modules**: Modularize common patterns of your infrastructure, distribute it via Git, S3, etc. to be reused across the entire company (See [#648](https://github.com/roboll/helmfile/pull/648))
**Versatility**: Manage your cluster consisting of charts, [kustomizations](https://github.com/kubernetes-sigs/kustomize), and directories of Kubernetes resources, turning everything to Helm releases (See [#673](https://github.com/roboll/helmfile/pull/673))
**Patch**: JSON/Strategic-Merge Patch Kubernetes resources before `helm-install`ing, without forking upstream charts (See [#673](https://github.com/roboll/helmfile/pull/673))
## Configuration
**CAUTION**: This documentation is for the development version of Helmfile. If you are looking for the documentation for any of releases, please switch to the corresponding release tag like [v0.79.1](https://github.com/roboll/helmfile/tree/v0.79.1).
The default helmfile is `helmfile.yaml`:
```yaml
# Chart repositories used from within this state file
#
# Use `helm-s3` and `helm-git` and whatever Helm Downloader plugins
# to use repositories other than the official repository or one backend by chartmuseum.
repositories:
- name: roboll
url: http://roboll.io/charts
certFile: optional_client_cert
keyFile: optional_client_key
username: optional_username
password: optional_password
# To use official "stable" charts a.k.a https://github.com/helm/charts/tree/master/stable
- name: stable
url: https://kubernetes-charts.storage.googleapis.com
# To use official "incubator" charts a.k.a https://github.com/helm/charts/tree/master/incubator
- name: incubator
url: https://kubernetes-charts-incubator.storage.googleapis.com
# helm-git powered repository: You can treat any Git repository as a charts repository
- name: polaris
url: git+https://github.com/reactiveops/polaris@deploy/helm?ref=master
# Advanced configuration: You can setup basic or tls auth
- name: roboll
url: http://roboll.io/charts
certFile: optional_client_cert
keyFile: optional_client_key
username: optional_username
password: optional_password
# Advanced configuration: You can use a ca bundle to use an https repo
# with a self-signed certificate
- name: insecure
url: https://charts.my-insecure-domain.com
caFile: optional_ca_crt
context: kube-context # kube-context (--kube-context)
# context: kube-context # this directive is deprecated, please consider using helmDefaults.kubeContext
#default values to set for args along with dedicated keys that can be set by contributers, cli args take precedence over these
helmDefaults:
tillerNamespace: tiller-namespace #dedicated default key for tiller-namespace
tillerless: false #dedicated default key for tillerless
kubeContext: kube-context #dedicated default key for kube-context (--kube-context)
# additional and global args passed to helm
args:
- "--set k=v"
# defaults for verify, wait, force, timeout and recreatePods under releases[]
verify: true
wait: true
timeout: 600
recreatePods: true
force: true
# enable TLS for request to Tiller
tls: true
# path to TLS CA certificate file (default "$HELM_HOME/ca.pem")
tlsCACert: "path/to/ca.pem"
# path to TLS certificate file (default "$HELM_HOME/cert.pem")
tlsCert: "path/to/cert.pem"
# path to TLS key file (default "$HELM_HOME/key.pem")
tlsKey: "path/to/key.pem"
# The desired states of Helm releases.
#
# Helmfile runs various helm commands to converge the current state in the live cluster to the desired state defined here.
releases:
# Published chart example
- name: vault # name of this release
@@ -47,31 +103,160 @@ releases:
foo: bar
chart: roboll/vault-secret-manager # the chart being installed to create this release, referenced by `repository/chart` syntax
version: ~1.24.1 # the semver of the chart. range constraint is supported
missingFileHandler: Warn # set to either "Error" or "Warn". "Error" instructs helmfile to fail when unable to find a values or secrets file. When "Warn", it prints the file and continues.
# Values files used for rendering the chart
values:
- vault.yaml # value files (--values)
- db: # inline values. Passed via a temporary values file (--values)
# Value files passed via --values
- vault.yaml
# Inline values, passed via a temporary values file and --values, so that it doesn't suffer from type issues like --set
- address: https://vault.example.com
# Go template available in inline values and values files.
- image:
# The end result is more or less YAML. So do `quote` to prevent number-like strings from accidentally parsed into numbers!
# See https://github.com/roboll/helmfile/issues/608
tag: {{ requiredEnv "IMAGE_TAG" | quote }}
# Otherwise:
# tag: "{{ requiredEnv "IMAGE_TAG" }}"
# tag: !!string {{ requiredEnv "IMAGE_TAG" }}
db:
username: {{ requiredEnv "DB_USERNAME" }}
# value taken from environment variable. Quotes are necessary. Will throw an error if the environment variable is not set. $DB_PASSWORD needs to be set in the calling environment ex: export DB_PASSWORD='password1'
password: {{ requiredEnv "DB_PASSWORD" }}
proxy:
# Interpolate environment variable with a fixed string
domain: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
scheme: {{ env "SCHEME" | default "https" }}
# Use `values` whenever possible!
# `set` translates to helm's `--set key=val`, that is known to suffer from type issues like https://github.com/roboll/helmfile/issues/608
set:
# single value loaded from a local file, translates to --set-file foo.config=path/to/file
- name: foo.config
file: path/to/file
# set a single array value in an array, translates to --set bar[0]={1,2}
- name: bar[0]
values:
- 1
- 2
# set a templated value
- name: namespace
value: {{ .Namespace }}
# will attempt to decrypt it using helm-secrets plugin
secrets:
- vault_secret.yaml # will attempt to decrypt it using helm-secrets plugin
set: # values (--set)
- name: address
value: https://vault.example.com
- name: db.password
value: {{ requiredEnv "DB_PASSWORD" }} # value taken from environment variable. Quotes are necessary. Will throw an error if the environment variable is not set. $DB_PASSWORD needs to be set in the calling environment ex: export DB_PASSWORD='password1'
- name: proxy.domain
value: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com # Interpolate environment variable with a fixed string
- name: proxy.scheme
value: {{ env "SCHEME" | default "https" }}
- vault_secret.yaml
# verify the chart before upgrading (only works with packaged charts not directories)
verify: true
# wait for k8s resources via --wait. Defaults to `false`
wait: true
# time in seconds to wait for any individual Kubernetes operation (like Jobs for hooks, and waits on pod/pvc/svc/deployment readiness) (default 300)
timeout: 60
# performs pods restart for the resource if applicable
recreatePods: true
# forces resource update through delete/recreate if needed
force: true
# set `false` to uninstall on sync
installed: true
# restores previous state in case of failed release
atomic: true
# name of the tiller namespace
tillerNamespace: vault
# if true, will use the helm-tiller plugin
tillerless: false
# enable TLS for request to Tiller
tls: true
# path to TLS CA certificate file (default "$HELM_HOME/ca.pem")
tlsCACert: "path/to/ca.pem"
# path to TLS certificate file (default "$HELM_HOME/cert.pem")
tlsCert: "path/to/cert.pem"
# path to TLS key file (default "$HELM_HOME/key.pem")
tlsKey: "path/to/key.pem"
# --kube-context to be passed to helm commands
# CAUTION: this doesn't work as expected for `tilerless: true`.
# See https://github.com/roboll/helmfile/issues/642
kubeContext: kube-context
# Local chart example
- name: grafana # name of this release
namespace: another # target namespace
chart: ../my-charts/grafana # the chart being installed to create this release, referenced by relative path to local chart
chart: ../my-charts/grafana # the chart being installed to create this release, referenced by relative path to local helmfile
values:
- "../../my-values/grafana/values.yaml" # Values file (relative path to manifest)
- ./values/{{ requiredEnv "PLATFORM_ENV" }}/config.yaml # Values file taken from path with environment variable. $PLATFORM_ENV must be set in the calling environment.
wait: true
#
# Advanced Configuration: Nested States
#
helmfiles:
- # Path to the helmfile state file being processed BEFORE releases in this state file
path: path/to/subhelmfile.yaml
# Label selector used for filtering releases in the nested state.
# For example, `name=prometheus` in this context is equivalent to processing the nested state like
# helmfile -f path/to/subhelmfile.yaml -l name=prometheus sync
selectors:
- name=prometheus
# Override state values
values:
# Values files merged into the nested state's values
- additiona.values.yaml
# Inline state values merged into the nested state's values
- key1: val1
- # All the nested state files under `helmfiles:` is processed in the order of definition.
# So it can be used for preparation for your main `releases`. An example would be creating CRDs required by `releases` in the parent state file.
path: path/to/mycrd.helmfile.yaml
- # Terraform-module-like URL for importing a remote directory and use a file in it as a nested-state file
# The nested-state file is locally checked-out along with the remote directory containing it.
# Therefore all the local paths in the file are resolved relative to the file
path: git::https://github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.40.0
#
# Advanced Configuration: Environments
#
# The list of environments managed by helmfile.
#
# The default is `environments: {"default": {}}` which implies:
#
# - `{{ .Environment.Name }}` evaluates to "default"
# - `{{ .Values }}` being empty
environments:
# The "default" environment is available and used when `helmfile` is run without `--environment NAME`.
default:
# Everything from the values.yaml is available via `{{ .Values.KEY }}`.
# Suppose `{"foo": {"bar": 1}}` contained in the values.yaml below,
# `{{ .Values.foo.bar }}` is evaluated to `1`.
values:
- environments/default/values.yaml
# Each entry in values can be either a file path or inline values.
# The below is an example of inline values, which is merged to the `.Values`
- myChartVer: 1.0.0-dev
# Any environment other than `default` is used only when `helmfile` is run with `--environment NAME`.
# That is, the "production" env below is used when and only when it is run like `helmfile --environment production sync`.
production:
values:
- environment/production/values.yaml
- myChartVer: 1.0.0
## `secrets.yaml` is decrypted by `helm-secrets` and available via `{{ .Environment.Secrets.KEY }}`
secrets:
- environment/production/secrets.yaml
# Instructs helmfile to fail when unable to find a environment values file listed under `environments.NAME.values`.
#
# Possible values are "Error", "Warn", "Info", "Debug". The default is "Error".
#
# Use "Warn", "Info", or "Debug" if you want helmfile to not fail when a values file is missing, while just leaving
# a message about the missing file at the log-level.
missingFileHandler: Error
#
# Advanced Configuration: Layering
#
# Helmfile merges all the "base" state files and this state file before processing.
#
# Assuming this state file is named `helmfile.yaml`, all the files are merged in the order of:
# environments.yaml <- defaults.yaml <- templates.yaml <- helmfile.yaml
bases:
- environments.yaml
- defaults.yaml
- templates.yaml
```
## Templating
@@ -79,7 +264,7 @@ releases:
Helmfile uses [Go templates](https://godoc.org/text/template) for templating your helmfile.yaml. While go ships several built-in functions, we have added all of the functions in the [Sprig library](https://godoc.org/github.com/Masterminds/sprig).
We also added one special template function: `requiredEnv`.
The `required_env` function allows you to declare a particular environment variable as required for template rendering.
The `requiredEnv` function allows you to declare a particular environment variable as required for template rendering.
If the environment variable is unset or empty, the template rendering will fail with an error message.
## Using environment variables
@@ -89,7 +274,7 @@ Environment variables can be used in most places for templating the helmfile. Cu
Examples:
```yaml
respositories:
repositories:
- name: your-private-git-repo-hosted-charts
url: https://{{ requiredEnv "GITHUB_TOKEN"}}@raw.githubusercontent.com/kmzfs/helm-repo-in-github/master/
```
@@ -112,9 +297,11 @@ releases:
## installation
- `go get github.com/roboll/helmfile` or
- download one of [releases](https://github.com/roboll/helmfile/releases) or
- run as a [container](https://quay.io/roboll/helmfile)
- run as a [container](https://quay.io/roboll/helmfile) or
- install from [AUR](https://aur.archlinux.org/packages/kubernetes-helmfile-bin/) for Archlinux or
- Windows (using [scoop](https://scoop.sh/)): `scoop install helmfile`
- macOS (using [homebrew](https://brew.sh/)): `brew install helmfile`
## getting started
@@ -135,12 +322,16 @@ releases:
Sync your Kubernetes cluster state to the desired one by running:
```console
helmfile sync
helmfile apply
```
Congratulations! You now have your first Prometheus deployment running inside your cluster.
Iterate on the `helmfile.yaml` by referencing the [configuration syntax](#configuration-syntax) and the [cli reference](#cli-reference).
Iterate on the `helmfile.yaml` by referencing:
- [Configuration syntax](#configuration-syntax)
- [CLI reference](#cli-reference).
- [Helmfile Best Practices Guide](https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md)
## cli reference
@@ -151,37 +342,65 @@ NAME:
USAGE:
helmfile [global options] command [command options] [arguments...]
VERSION:
v0.70.0
COMMANDS:
repos sync repositories from state file (helm repo add && helm repo update)
charts sync charts from state file (helm upgrade --install)
diff diff charts from state file against env (helm diff)
lint lint charts from state file (helm lint)
sync sync all resources from state file (repos, charts and local chart deps)
status retrieve status of releases in state file
delete delete charts from state file (helm delete)
test tets releases from state file (helm test)
deps update charts based on the contents of requirements.yaml
repos sync repositories from state file (helm repo add && helm repo update)
charts DEPRECATED: sync releases from state file (helm upgrade --install)
diff diff releases from state file against env (helm diff)
template template releases from state file against env (helm template)
lint lint charts from state file (helm lint)
sync sync all resources from state file (repos, releases and chart deps)
apply apply all resources from state file only when there are changes
status retrieve status of releases in state file
delete DEPRECATED: delete releases from state file (helm delete)
destroy deletes and then purges releases
test test releases from state file (helm test)
GLOBAL OPTIONS:
--file FILE, -f FILE load config from FILE (default: "helmfile.yaml")
--quiet, -q silence output
--namespace value, -n value Set namespace. Uses the namespace set in the context by default
--selector,l value Only run using the releases that match labels. Labels can take the form of foo=bar or foo!=bar.
A release must match all labels in a group in order to be used. Multiple groups can be specified at once.
--selector tier=frontend,tier!=proxy --selector tier=backend. Will match all frontend, non-proxy releases AND all backend releases.
The name of a release can be used as a label. --selector name=myrelease
--kube-context value Set kubectl context. Uses current context by default
--help, -h show help
--version, -v print the version
--helm-binary value, -b value path to helm binary
--file helmfile.yaml, -f helmfile.yaml load config from file or directory. defaults to helmfile.yaml or `helmfile.d`(means `helmfile.d/*.yaml`) in this preference
--environment default, -e default specify the environment name. defaults to default
--state-values-set value set state values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--state-values-file value specify state values in a YAML file
--quiet, -q Silence output. Equivalent to log-level warn
--kube-context value Set kubectl context. Uses current context by default
--log-level value Set log level, default info
--namespace value, -n value Set namespace. Uses the namespace set in the context by default, and is available in templates as {{ .Namespace }}
--selector value, -l value Only run using the releases that match labels. Labels can take the form of foo=bar or foo!=bar.
A release must match all labels in a group in order to be used. Multiple groups can be specified at once.
--selector tier=frontend,tier!=proxy --selector tier=backend. Will match all frontend, non-proxy releases AND all backend releases.
The name of a release can be used as a label. --selector name=myrelease
--allow-no-matching-release Do not exit with an error code if the provided selector has no matching releases.
--interactive, -i Request confirmation before attempting to modify clusters
--help, -h show help
--version, -v print the version
```
### sync
The `helmfile sync` sub-command sync your cluster state as described in your `helmfile`. The default helmfile is `helmfile.yaml`, but any yaml file can be passed by specifying a `--file path/to/your/yaml/file` flag.
The `helmfile sync` sub-command sync your cluster state as described in your `helmfile`. The default helmfile is `helmfile.yaml`, but any YAML file can be passed by specifying a `--file path/to/your/yaml/file` flag.
Under the covers, Helmfile executes `helm upgrade --install` for each `release` declared in the manifest, by optionally decrypting [secrets](#secrets) to be consumed as helm chart values. It also updates specified chart repositories and updates the
dependencies of any referenced local charts.
For Helm 2.9+ you can use a username and password to authenticate to a remote repository. WARNING - repository password will be exposed unmasked in console using literal value or environment variable.
For Helm 2.9+ you can use a username and password to authenticate to a remote repository.
### deps
The `helmfile deps` sub-command locks your helmfile state and local charts dependencies.
It basically runs `helm dependency update` on your helmfile state file and all the referenced local charts, so that you get a "lock" file per each helmfile state or local chart.
All the other `helmfile` sub-commands like `sync` use chart versions recorded in the lock files, so that e.g. untested chart versions won't suddenly get deployed to the production environment.
For example, the lock file for a helmfile state file named `helmfile.1.yaml` will be `helmfile.1.lock`. The lock file for a local chart would be `requirements.lock`, which is the same as `helm`.
It is recommended to version-control all the lock files, so that they can be used in the production deployment pipeline for extra reproducibility.
To bring in chart updates systematically, it would also be a good idea to run `helmfile deps` regularly, test it, and then update the lock files in the version-control system.
### diff
@@ -192,12 +411,29 @@ To supply the diff functionality Helmfile needs the [helm-diff](https://github.c
you should be able to simply execute `helm plugin install https://github.com/databus23/helm-diff`. For more details
please look at their [documentation](https://github.com/databus23/helm-diff#helm-diff-plugin).
### delete
### apply
The `helmfile delete` sub-command deletes all the releases defined in the manfiests
The `helmfile apply` sub-command begins by executing `diff`. If `diff` finds that there is any changes, `sync` is executed. Adding `--interactive` instructs Helmfile to request your confirmation before `sync`.
An expected use-case of `apply` is to schedule it to run periodically, so that you can auto-fix skews between the desired and the current state of your apps running on Kubernetes clusters.
### destroy
The `helmfile destroy` sub-command deletes and purges all the releases defined in the manifests.
`helmfile --interactive destroy` instructs Helmfile to request your confirmation before actually deleting releases.
`destroy` basically runs `helm delete --purge` on all the targeted releases. If you don't want purging, use `helmfile delete` instead.
### delete (DEPRECATED)
The `helmfile delete` sub-command deletes all the releases defined in the manifests.
`helmfile --interactive delete` instructs Helmfile to request your confirmation before actually deleting releases.
Note that `delete` doesn't purge releases. So `helmfile delete && helmfile sync` results in sync failed due to that releases names are not deleted but preserved for future references. If you really want to remove releases for reuse, add `--purge` flag to run it like `helmfile delete --purge`.
### secrets
The `secrets` parameter in a `helmfile.yaml` causes the [helm-secrets](https://github.com/futuresimple/helm-secrets) plugin to be executed to decrypt the file.
@@ -222,13 +458,13 @@ Using manifest files in conjunction with command line argument can be a bit conf
A few rules to clear up this ambiguity:
- Absolute paths are always resolved as absolute paths
- Relative paths referenced *in* the helmfile manifest itself are relative to that manifest
- Relative paths referenced *in* the Helmfile manifest itself are relative to that manifest
- Relative paths referenced on the command line are relative to the current working directory the user is in
For additional context, take a look at [paths examples](PATHS.md)
## Labels Overview
A selector can be used to only target a subset of releases when running helmfile. This is useful for large helmfiles with releases that are logically grouped together.
A selector can be used to only target a subset of releases when running Helmfile. This is useful for large helmfiles with releases that are logically grouped together.
Labels are simple key value pairs that are an optional field of the release spec. When selecting by label, the search can be inverted. `tier!=backend` would match all releases that do NOT have the `tier: backend` label. `tier=fronted` would only match releases with the `tier: frontend` label.
@@ -238,6 +474,530 @@ The `selector` parameter can be specified multiple times. Each parameter is reso
`--selector tier=frontend --selector tier=backend` will select all the charts
In addition to user supplied labels, the name, the namespace, and the chart are available to be used as selectors. The chart will just be the chart name excluding the repository (Example `stable/filebeat` would be selected using `--selector chart=filebeat`).
## Templates
You can use go's text/template expressions in `helmfile.yaml` and `values.yaml.gotmpl` (templated helm values files). `values.yaml` references will be used verbatim. In other words:
- for value files ending with `.gotmpl`, template expressions will be rendered
- for plain value files (ending in `.yaml`), content will be used as-is
In addition to built-in ones, the following custom template functions are available:
- `readFile` reads the specified local file and generate a golang string
- `fromYaml` reads a golang string and generates a map
- `setValueAtPath PATH NEW_VALUE` traverses a golang map, replaces the value at the PATH with NEW_VALUE
- `toYaml` marshals a map into a string
### Values Files Templates
You can reference a template of values file in your `helmfile.yaml` like below:
```yaml
releases
- name: myapp
chart: mychart
values:
- values.yaml.gotmpl
```
Every values file whose file extension is `.gotmpl` is considered as a template file.
Suppose `values.yaml.gotmpl` was something like:
```yaml
{{ readFile "values.yaml" | fromYaml | setValueAtPath "foo.bar" "FOO_BAR" | toYaml }}
```
And `values.yaml` was:
```yaml
foo:
bar: ""
```
The resulting, temporary values.yaml that is generated from `values.yaml.tpl` would become:
```yaml
foo:
# Notice `setValueAtPath "foo.bar" "FOO_BAR"` in the template above
bar: FOO_BAR
```
## Refactoring `helmfile.yaml` with values files templates
One of expected use-cases of values files templates is to keep `helmfile.yaml` small and concise.
See the example `helmfile.yaml` below:
```yaml
releases:
- name: {{ requiredEnv "NAME" }}-vault
namespace: {{ requiredEnv "NAME" }}
chart: roboll/vault-secret-manager
values:
- db:
username: {{ requiredEnv "DB_USERNAME" }}
password: {{ requiredEnv "DB_PASSWORD" }}
set:
- name: proxy.domain
value: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
- name: proxy.scheme
value: {{ env "SCHEME" | default "https" }}
```
The `values` and `set` sections of the config file can be separated out into a template:
`helmfile.yaml`:
```yaml
releases:
- name: {{ requiredEnv "NAME" }}-vault
namespace: {{ requiredEnv "NAME" }}
chart: roboll/vault-secret-manager
values:
- values.yaml.tmpl
```
`values.yaml.tmpl`:
```yaml
db:
username: {{ requiredEnv "DB_USERNAME" }}
password: {{ requiredEnv "DB_PASSWORD" }}
proxy:
domain: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
scheme: {{ env "SCHEME" | default "https" }}
```
## Environment
When you want to customize the contents of `helmfile.yaml` or `values.yaml` files per environment, use this feature.
You can define as many environments as you want under `environments` in `helmfile.yaml`.
The environment name defaults to `default`, that is, `helmfile sync` implies the `default` environment.
The selected environment name can be referenced from `helmfile.yaml` and `values.yaml.gotmpl` by `{{ .Environment.Name }}`.
If you want to specify a non-default environment, provide a `--environment NAME` flag to `helmfile` like `helmfile --environment production sync`.
The below example shows how to define a production-only release:
```yaml
environments:
default:
production:
releases:
{{ if eq .Environment.Name "production" }}
- name: newrelic-agent
# snip
{{ end }}
- name: myapp
# snip
```
## Environment Values
Environment Values allows you to inject a set of values specific to the selected environment, into values.yaml templates.
Use it to inject common values from the environment to multiple values files, to make your configuration DRY.
Suppose you have three files `helmfile.yaml`, `production.yaml` and `values.yaml.gotmpl`:
`helmfile.yaml`
```yaml
environments:
production:
values:
- production.yaml
releases:
- name: myapp
values:
- values.yaml.gotmpl
```
`production.yaml`
```yaml
domain: prod.example.com
releaseName: prod
```
`values.yaml.gotmpl`
```yaml
domain: {{ .Values | getOrNil "my.domain" | default "dev.example.com" }}
```
`helmfile sync` installs `myapp` with the value `domain=dev.example.com`,
whereas `helmfile --environment production sync` installs the app with the value `domain=production.example.com`.
For even more flexibility, you can now use values declared in the `environments:` section in other parts of your helmfiles:
consider:
`default.yaml`
```yaml
domain: dev.example.com
releaseName: dev
```
```yaml
environments:
default:
values:
- default.yaml
production:
values:
- production.yaml # bare .yaml file, content will be used verbatim
- other.yaml.gotmpl # template directives with potential side-effects like `exec` and `readFile` will be honoured
releases:
- name: myapp-{{ .Values.releaseName }} # release name will be one of `dev` or `prod` depending on selected environment
values:
- values.yaml.gotmpl
{{ if eq (.Values.releaseName "prod" ) }}
# this release would be installed only if selected environment is `production`
- name: production-specific-release
...
{{ end }}
```
### Note
The `{{ .Values.foo }}` syntax is the recommended way of using environment values.
Prior to this [pull request](https://github.com/roboll/helmfile/pull/647), environment values were made available through the `{{ .Environment.Values.foo }}` syntax.
This is still working but is **deprecated** and the new `{{ .Values.foo }}` syntax should be used instead.
You can read more infos about the feature proposal [here](https://github.com/roboll/helmfile/issues/640).
## Environment Secrets
Environment Secrets (not to be confused with Kubernetes Secrets) are encrypted versions of `Environment Values`.
You can list any number of `secrets.yaml` files created using `helm secrets` or `sops`, so that
Helmfile could automatically decrypt and merge the secrets into the environment values.
First you must have the [helm-secrets](https://github.com/futuresimple/helm-secrets) plugin installed along with a
`.sops.yaml` file to configure the method of encryption (this can be in the same directory as your helmfile or
in the sub-directory containing your secrets files).
Then suppose you have a a foo.bar secret defined in `environments/production/secrets.yaml`:
```yaml
foo.bar: "mysupersecretstring"
```
You can then encrypt it with `helm secrets enc environments/production/secrets.yaml`
Then reference that encrypted file in `helmfile.yaml`:
```yaml
environments:
production:
secrets:
- environments/production/secrets.yaml
releases:
- name: myapp
chart: mychart
values:
- values.yaml.gotmpl
```
Then the environment secret `foo.bar` can be referenced by the below template expression in your `values.yaml.gotmpl`:
```yaml
{{ .Values.foo.bar }}
```
## Tillerless
With the [helm-tiller](https://github.com/rimusz/helm-tiller) plugin installed, you can work without tiller installed.
To enable this mode, you need to define `tillerless: true` and set the `tillerNamespace` in the `helmDefaults` section
or in the `releases` entries.
## Separating helmfile.yaml into multiple independent files
Once your `helmfile.yaml` got to contain too many releases,
split it into multiple yaml files.
Recommended granularity of helmfile.yaml files is "per microservice" or "per team".
And there are two ways to organize your files.
- Single directory
- Glob patterns
### Single directory
`helmfile -f path/to/directory` loads and runs all the yaml files under the specified directory, each file as an independent helmfile.yaml.
The default helmfile directory is `helmfile.d`, that is,
in case helmfile is unable to locate `helmfile.yaml`, it tries to locate `helmfile.d/*.yaml`.
All the yaml files under the specified directory are processed in the alphabetical order. For example, you can use a `<two digit number>-<microservice>.yaml` naming convention to control the sync order.
- `helmfile.d`/
- `00-database.yaml`
- `00-backend.yaml`
- `01-frontend.yaml`
### Glob patterns
In case you want more control over how multiple `helmfile.yaml` files are organized, use `helmfiles:` configuration key in the `helmfile.yaml`:
Suppose you have multiple microservices organized in a Git repository that looks like:
- `myteam/` (sometimes it is equivalent to a k8s ns, that is `kube-system` for `clusterops` team)
- `apps/`
- `filebeat/`
- `helmfile.yaml` (no `charts/` exists, because it depends on the stable/filebeat chart hosted on the official helm charts repository)
- `README.md` (each app managed by my team has a dedicated README maintained by the owners of the app)
- `metricbeat/`
- `helmfile.yaml`
- `README.md`
- `elastalert-operator/`
- `helmfile.yaml`
- `README.md`
- `charts/`
- `elastalert-operator/`
- `<the content of the local helm chart>`
The benefits of this structure is that you can run `git diff` to locate in which directory=microservice a git commit has changes.
It allows your CI system to run a workflow for the changed microservice only.
A downside of this is that you don't have an obvious way to sync all microservices at once. That is, you have to run:
```bash
for d in apps/*; do helmfile -f $d diff; if [ $? -eq 2 ]; then helmfile -f $d sync; fi; done
```
At this point, you'll start writing a `Makefile` under `myteam/` so that `make sync-all` will do the job.
It does work, but you can rely on the Helmfile feature instead.
Put `myteam/helmfile.yaml` that looks like:
```yaml
helmfiles:
- apps/*/helmfile.yaml
```
So that you can get rid of the `Makefile` and the bash snippet.
Just run `helmfile sync` inside `myteam/`, and you are done.
All the files are sorted alphabetically per group = array item inside `helmfiles:`, so that you have granular control over ordering, too.
#### selectors
When composing helmfiles you can use selectors from the command line as well as explicit selectors inside the parent helmfile to filter the releases to be used.
```yaml
helmfiles:
- apps/*/helmfile.yaml
- path: apps/a-helmfile.yaml
selectors: # list of selectors
- name=prometheus
- tier=frontend
- path: apps/b-helmfile.yaml # no selector, so all releases are used
selectors: []
- path: apps/c-helmfile.yaml # parent selector to be used or cli selector for the initial helmfile
selectorsInherited: true
```
* When a selector is specified, only this selector applies and the parents or CLI selectors are ignored.
* When not selector is specified there are 2 modes for the selector inheritance because we would like to change the current inheritance behavior (see [issue #344](https://github.com/roboll/helmfile/issues/344) ).
* Legacy mode, sub-helmfiles without selectors inherit selectors from their parent helmfile. The initial helmfiles inherit from the command line selectors.
* explicit mode, sub-helmfile without selectors do not inherit from their parent or the CLI selector. If you want them to inherit from their parent selector then use `selectorsInherited: true`. To enable this explicit mode you need to set the following environment variable `HELMFILE_EXPERIMENTAL=explicit-selector-inheritance` (see [experimental](#experimental-features)).
* Using `selector: []` will select all releases regardless of the parent selector or cli for the initial helmfile
* using `selectorsInherited: true` make the sub-helmfile selects releases with the parent selector or the cli for the initial helmfile. You cannot specify an explicit selector while using `selectorsInherited: true`
## Importing values from any source
The `exec` template function that is available in `values.yaml.gotmpl` is useful for importing values from any source
that is accessible by running a command:
A usual usage of `exec` would look like this:
```
mysetting: |
{{ exec "./mycmd" (list "arg1" "arg2" "--flag1") | indent 2 }}
```
Or even with a pipeline:
```
mysetting: |
{{ yourinput | exec "./mycmd-consume-stdin" (list "arg1" "arg2") | indent 2 }}
```
The possibility is endless. Try importing values from your golang app, bash script, jsonnet, or anything!
## Hooks
A Helmfile hook is a per-release extension point that is composed of:
- `events`
- `command`
- `args`
- `showlogs`
Helmfile triggers various `events` while it is running.
Once `events` are triggered, associated `hooks` are executed, by running the `command` with `args`. The standard output of the `command` will be displayed if `showlogs` is set and it's value is `true`.
Currently supported `events` are:
- `prepare`
- `presync`
- `postsync`
- `cleanup`
Hooks associated to `prepare` events are triggered after each release in your helmfile is loaded from YAML, before execution.
Hooks associated to `cleanup` events are triggered after each release is processed.
Hooks associated to `presync` events are triggered before each release is applied to the remote cluster. This is the ideal event to execute any commands that may mutate the cluster state as it will not be run for read-only operations like `lint`, `diff` or `template`.
Hooks associated to `postsync` events are triggered after each release is applied to the remote cluster. This is the ideal event to execute any commands that may mutate the cluster state as it will not be run for read-only operations like `lint`, `diff` or `template`.
The following is an example hook that just prints the contextual information provided to hook:
```
releases:
- name: myapp
chart: mychart
# *snip*
hooks:
- events: ["prepare", "cleanup"]
showlogs: true
command: "echo"
args: ["{{`{{.Environment.Name}}`}}", "{{`{{.Release.Name}}`}}", "{{`{{.HelmfileCommand}}`}}\
"]
```
Let's say you ran `helmfile --environment prod sync`, the above hook results in executing:
```
echo {{Environment.Name}} {{.Release.Name}} {{.HelmfileCommand}}
```
Whereas the template expressions are executed thus the command becomes:
```
echo prod myapp sync
```
Now, replace `echo` with any command you like, and rewrite `args` that actually conforms to the command, so that you can integrate any command that does:
- templating
- linting
- testing
For templating, imagine that you created a hook that generates a helm chart on-the-fly by running an external tool like ksonnet, kustomize, or your own template engine.
It will allow you to write your helm releases with any language you like, while still leveraging goodies provided by helm.
### Helmfile + Kustomize
Do you prefer `kustomize` to write and organize your Kubernetes apps, but still want to leverage helm's useful features
like rollback, history, and so on? This section is for you!
The combination of `hooks` and [helmify-kustomize](https://gist.github.com/mumoshu/f9d0bd98e0eb77f636f79fc2fb130690)
enables you to integrate [kustomize](https://github.com/kubernetes-sigs/kustomize) into Helmfile.
That is, you can use `kustommize` to build a local helm chart from a kustomize overlay.
Let's assume you have a kustomize project named `foo-kustomize` like this:
```
foo-kustomize/
├── base
│   ├── configMap.yaml
│   ├── deployment.yaml
│   ├── kustomization.yaml
│   └── service.yaml
└── overlays
├── default
│   ├── kustomization.yaml
│   └── map.yaml
├── production
│   ├── deployment.yaml
│   └── kustomization.yaml
└── staging
├── kustomization.yaml
└── map.yaml
5 directories, 10 files
```
Write `helmfile.yaml`:
```yaml
- name: kustomize
chart: ./foo
hooks:
- events: ["prepare", "cleanup"]
command: "../helmify"
args: ["{{`{{if eq .Event.Name \"prepare\"}}build{{else}}clean{{end}}`}}", "{{`{{.Release.Ch\
art}}`}}", "{{`{{.Environment.Name}}`}}"]
```
Run `helmfile --environment staging sync` and see it results in helmfile running `kustomize build foo-kustomize/overlays/staging > foo/templates/all.yaml`.
Voilà! You can mix helm releases that are backed by remote charts, local charts, and even kustomize overlays.
## Guides
Use the [Helmfile Best Practices Guide](/docs/writing-helmfile.md) to write advanced helmfiles that feature:
- Default values
- Layering
We also have dedicated documentation on the following topics which might interest you:
- [Shared Configurations Across Teams](/docs/shared-configuration-across-teams.md)
Or join our friendly slack community in the [`#helmfile`](https://slack.sweetops.com) channel to ask questions and get help. Check out our [slack archive](https://archive.sweetops.com/helmfile/) for good examples of how others are using it.
## Using env files
Helmfile itself doesn't have an ability to load env files. But you can write some bash script to achieve the goal:
```console
set -a; . .env; set +a; helmfile sync
```
Please see #203 for more context.
## Running helmfile interactively
`helmfile --interactive [apply|destroy]` requests confirmation from you before actually modifying your cluster.
Use it when you're running `helmfile` manually on your local machine or a kind of secure administrative hosts.
For your local use-case, aliasing it like `alias hi='helmfile --interactive'` would be convenient.
## Running Helmfile without an Internet connection
Once you download all required charts into your machine, you can run `helmfile charts` to deploy your apps.
It basically run only `helm upgrade --install` with your already-downloaded charts, hence no Internet connection is required.
See #155 for more information on this topic.
## Experimental features
Some experimental features may be available for testing in perspective of being (or not) included in a future release.
Those features are set using the environment variable `HELMFILE_EXPERIMENTAL`. Here is the current experimental feature :
* `explicit-selector-inheritance` : remove today implicit cli selectors inheritance for composed helmfiles, see [composition selector](#selectors)
If you want to enable all experimental features set the env var to `HELMFILE_EXPERIMENTAL=true`
## Examples
For more examples, see [examples/README.md](https://github.com/roboll/helmfile/blob/master/examples/README.md).
For more examples, see the [examples/README.md](https://github.com/roboll/helmfile/blob/master/examples/README.md) or the [`helmfile`](https://github.com/cloudposse/helmfiles/tree/master/releases) distribution by [Cloud Posse](https://github.com/cloudposse/).
# Attribution
We use:
- [semtag](https://github.com/pnikosis/semtag) for automated semver tagging. I greatly appreciate the author(pnikosis)'s effort on creating it and their kindness to share it!
+16 -13
View File
@@ -1,18 +1,21 @@
# Helmfile Users
The following table is compiled from public information or from survey responses. If you are using Helmfile in your
company or organization, we would like to invite you to [participate in our user survey](https://docs.google.com/forms/d/e/1FAIpQLSfrF5kR9nXsr5yWqNZGgN-KquM1mBTuYNn5lRmEVXEvhq0VIQ/viewform)
to help us extend this list.
Who's actually using Helmfile? The following table is compiled from public information, survey response and PRs. It's
likely this is only a small selection of actual users, but it's a start.
If you are using Helmfile in your company or organization, we would like to invite you to create a PR to add your
information to this file.
<!-- TABLE_START -->
| Organization | Workload | More Info | Location |
|---|---|---|---|
| [reddit.com](https://www.reddit.com/) | production | [GitHub issue](https://github.com/roboll/helmfile/issues/96#issue-312012249) | San Francisco, CA, US |
| [ricardo.ch](https://www.ricardo.ch/) | proof-of-concept | | Zug, Switzerland |
| Organization | Workload | More Info | Location | Added |
|---|---|---|---|---|
| [reddit.com](https://www.reddit.com/) | production | [GitHub issue](https://github.com/roboll/helmfile/issues/96#issue-312012249), [Talk](https://www.slideshare.net/GregoryTaylor11/helm-at-reddit-from-local-dev-staging-to-production) | San Francisco, CA | April 2018 |
| [ricardo.ch](https://www.ricardo.ch/) | production | We're deploying our complete application platform using Helmfile. | Zug, Switzerland | April 2018 |
| [Sight Machine](https://sightmachine.com/) | production | We don't have anything publicly posted about it, but have been using it for quite a while in production. | San Francisco, CA and Ann Arbor, MI | December 2018 |
| [VSHN – The DevOps Company](https://vshn.ch) | production | | Zurich, Switzerland | March 2019 |
| [Vlocity](https://vlocity.com/) | proof-of-concept | | Melbourne, Australia | March 2019 |
| [transit](https://transit.app/) | production | [Blog post](https://medium.com/@naseem_60378/helmfile-its-like-a-helm-for-your-helm-74a908581599). | Montreal, Canada | March 2019 |
| [uniqkey](https://uniqkey.eu/) | production | [Wiki Page](https://ocd-scm.github.io/ocd-meta/) | Copenhagen, Denmark | April 2019 |
| [bitsofinfo](https://github.com/bitsofinfo/helmfile-deploy) | proof-of-concept, stage | Used with [helmfile-deploy](https://github.com/bitsofinfo/helmfile-deploy) to manage releases for dozens of apps | USA | July 2019 |
| [kloeckner-i](https://www.kloeckner-i.com/) | production | We are deploying our standard tools via helmfile to all our clusters. | Berlin, Germany | September 2019 |
<!-- TABLE_END -->
## Contact
Please fill out [the Google form](https://docs.google.com/forms/d/e/1FAIpQLSfrF5kR9nXsr5yWqNZGgN-KquM1mBTuYNn5lRmEVXEvhq0VIQ/viewform) to add your organization instead of opening a PR.
You can also [reach the maintainer on Twitter](https://twitter.com/DerCed).
+152
View File
@@ -0,0 +1,152 @@
# Shared Configuration Across Teams
Assume you have a two or more teams, each works for a different internal or external service, like:
- Product 1
- Product 2
- Observability
The simplest `helmfile.yaml` that declares the whole cluster that is composed of the three services would look like the below:
```yaml
releases:
- name: product1-api
chart: product1-charts/api
# snip
- name: product1-web
chart: product1-charts/web
# snip
- name: product2-api
chart: saas-charts/api
# snip
- name: product2-web
chart: product2-charts/web
# snip
- name: observability-prometheus-operator
chart: stable/prometheus-operator
# snip
- name: observability-process-exporter
chart: stable/prometheus-operator
# snip
```
This works, but what if you wanted to a separate cluster per service to achieve smaller blast radius?
Let's start by creating a `helmfile.yaml` for each service.
`product1/helmfile.yaml`:
```yaml
releases:
- name: product1-api
chart: product1-charts/api
# snip
- name: product1-web
chart: product1-charts/web
# snip
- name: observability-prometheus-operator
chart: stable/prometheus-operator
# snip
- name: observability-process-exporter
chart: stable/prometheus-operator
# snip
```
`product2/helmfile.yaml`:
```yaml
releases:
- name: product2-api
chart: product2-charts/api
# snip
- name: product2-web
chart: product2-charts/web
# snip
- name: observability-prometheus-operator
chart: stable/prometheus-operator
# snip
- name: observability-process-exporter
chart: stable/prometheus-operator
# snip
```
You will (of course!) notice this isn't DRY.
To remove the duplication of observability stack between the two helmfiles, create a "sub-helmfile" for the observability stack.
`observability/helmfile.yaml`:
```yaml
- name: observability-prometheus-operator
chart: stable/prometheus-operator
# snip
- name: observability-process-exporter
chart: stable/prometheus-operator
# snip
```
As you might have imagined, the observability helmfile can be reused from the two product helmfiles by declaring `helmfiles`.
`product1/helmfile.yaml`:
```yaml
helmfiles:
- ../observability/helmfile.yaml
releases:
- name: product1-api
chart: product1-charts/api
# snip
- name: product1-web
chart: product1-charts/web
# snip
```
`product2/helmfile.yaml`:
```yaml
helmfiles:
- ../observability/helmfile.yaml
releases:
- name: product2-api
chart: product2-charts/api
# snip
- name: product2-web
chart: product2-charts/web
# snip
```
## Using sub-helmfile as a template
You can go even further by generalizing the product related releases as a pair of `api` and `web`:
`shared/helmfile.yaml`:
```yaml
releases:
- name: product{{ env "PRODUCT_ID" }}-api
chart: product{{ env "PRODUCT_ID" }}-charts/api
# snip
- name: product{{ env "PRODUCT_ID" }}-web
chart: product{{ env "PRODUCT_ID" }}-charts/web
# snip
```
Then you only need one single product helmfile
`product/helmfile.yaml`:
```yaml
helmfiles:
- ../observability/helmfile.yaml
- ../shared/helmfile.yaml
```
Now that we use the environment variable `PRODUCT_ID` to as the parameters of release names, you need to set it before running `helmfile`, so that it produces the differently named releases per product:
```console
$ PRODUCT_ID=1 helmfile -f product/helmfile.yaml apply
$ PRODUCT_ID=2 helmfile -f product/helmfile.yaml apply
```
+358
View File
@@ -0,0 +1,358 @@
# The Helmfile Best Practices Guide
This guide covers the Helmfile’s considered patterns for writing advanced helmfiles. It focuses on how helmfile should be structured and executed.
## Missing keys and Default values
helmfile tries its best to inform users for noticing potential mistakes.
One example of how helmfile achieves it is that, `helmfile` fails when you tried to access missing keys in environment values.
That is, the following example let `helmfile` fail when you have no `eventApi.replicas` defined in environment values.
```
{{ .Values.eventApi.replicas | default 1 }}
```
In case it isn't a mistake and you do want to allow missing keys, use the `getOrNil` template function:
```
{{ .Values | getOrNil "eventApi.replicas" }}
```
This result in printing `<no value` in your template, that may or may not result in a failure.
If you want a kind of default values that is used when a missing key was referenced, use `default` like:
```
{{ .Values | getOrNil "eventApi.replicas" | default 1 }}
```
Now, you get `1` when there is no `eventApi.replicas` defined in environment values.
## Release Template / Conventional Directory Structure
Introducing helmfile into a large-scale project that involes dozens of releases often results in a lot of repetitions in `helmfile.yaml` files.
The example below shows repetitions in `namespace`, `chart`, `values`, and `secrets`:
```yaml
releases:
# *snip*
- name: heapster
namespace: kube-system
chart: stable/heapster
version: 0.3.2
values:
- "./config/heapster/values.yaml"
- "./config/heapster/{{ .Environment.Name }}.yaml"
secrets:
- "./config/heapster/secrets.yaml"
- "./config/heapster/{{ .Environment.Name }}-secrets.yaml"
- name: kubernetes-dashboard
namespace: kube-system
chart: stable/kubernetes-dashboard
version: 0.10.0
values:
- "./config/kubernetes-dashboard/values.yaml"
- "./config/kubernetes-dashboard/{{ .Environment.Name }}.yaml"
values:
- "./config/kubernetes-dashboard/secrets.yaml"
- "./config/kubernetes-dashboard/{{ .Environment.Name }}-secrets.yaml"
```
This is where Helmfile's advanced feature called Release Template comes handy.
It allows you to abstract away the repetitions in releases into a template, which is then included and executed by using YAML anchor/alias:
```yaml
templates:
default: &default
chart: stable/{{`{{ .Release.Name }}`}}
namespace: kube-system
# This prevents helmfile exiting when it encounters a missing file
# Valid values are "Error", "Warn", "Info", "Debug". The default is "Error"
# Use "Debug" to make missing files errors invisible at the default log level(--log-level=INFO)
missingFileHandler: Warn
values:
- config/{{`{{ .Release.Name }}`}}/values.yaml
- config/{{`{{ .Release.Name }}`}}/{{`{{ .Environment.Name }}`}}.yaml
secrets:
- config/{{`{{ .Release.Name }}`}}/secrets.yaml
- config/{{`{{ .Release.Name }}`}}/{{`{{ .Environment.Name }}`}}-secrets.yaml
releases:
- name: heapster
<<: *default
- name: kubernetes-dashboard
<<: *default
```
Release Templating supports the following parts of release definition:
- basic fields: `name`, `namespace`, `chart`, `version`
- boolean fields: `installed`, `wait`, `tillerless`, `verify` by the means of additional text
fields designed for templating only: `installedTemplate`, `waitTemplate`, `tillerlessTemplate`, `verifyTemplate`
```yaml
# ...
installedTemplate: '{{`{{ eq .Release.Namespace "kube-system" }}`}}'
waitTemplate: '{{`{{ eq .Release.Labels.tag "safe" | not }}`}}'
# ...
```
- `set` block values:
```yaml
# ...
setTemplate:
- name: '{{`{{ .Release.Name }}`}}'
values: '{{`{{ .Release.Namespace }}`}}'
# ...
```
- `values` and `secrets` file paths:
```yaml
# ...
valuesTemplate:
- config/{{`{{ .Release.Name }}`}}/values.yaml
secrets:
- config/{{`{{ .Release.Name }}`}}/secrets.yaml
# ...
```
- inline `values` map:
```yaml
# ...
valuesTemplate:
- image:
tag: `{{ .Release.Labels.tag }}`
# ...
```
See the [issue 428](https://github.com/roboll/helmfile/issues/428) for more context on how this is supposed to work.
## Layering Release Values
Please note, that it is not possible to layer `values` sections. If `values` is defined in the release and in the release template, only the `values` defined in the release will be considered. The same applies to `secrets` and `set`.
## Layering State Files
> See **Layering State Template Files** if you're layering templates.
You may occasionally end up with many helmfiles that shares common parts like which repositories to use, and whichi release to be bundled by default.
Use Layering to extract the common parts into a dedicated *library helmfile*s, so that each helmfile becomes DRY.
Let's assume that your `helmfile.yaml` looks like:
```
bases:
- environments.yaml
releases:
- name: metricbaet
chart: stable/metricbeat
- name: myapp
chart: mychart
```
Whereas `environments.yaml` contained well-known environments:
```yaml
environments:
development:
production:
```
At run time, `bases` in your `helmfile.yaml` are evaluated to produce:
```yaml
---
# environments.yaml
environments:
development:
production:
---
# helmfile.yaml
releases:
- name: myapp
chart: mychart
- name: metricbaet
chart: stable/metricbeat
```
Finally the resulting YAML documents are merged in the order of occurrence,
so that your `helmfile.yaml` becomes:
```yaml
environments:
development:
production:
releases:
- name: metricbaet
chart: stable/metricbeat
- name: myapp
chart: mychart
```
Great!
Now, repeat the above steps for each your `helmfile.yaml`, so that all your helmfiles becomes DRY.
Please also see [the discussion in the issue 388](https://github.com/roboll/helmfile/issues/388#issuecomment-491710348) for more advanced layering examples.
## Merging Arrays in Layers
Helmfile doesn't merge arrays across layers. That is, the below example doesn't work as you might have expected:
```yaml
releases:
- name: metricbaet
chart: stable/metricbeat
---
releases:
- name: myapp
chart: mychart
```
Helmfile overrides the `releases` array with the latest layer so the resulting state file will be:
```yaml
releases:
# metricbeat release disappeared! but that's how helmfiel works
- name: myapp
chart: mychart
```
A work-around is to treat the state file as a go template and use `readFile` template function to import the common part of your state file as a plain text:
`common.yaml`:
```yaml
templates:
metricbeat: &metricbeat
- name: metricbeat
chart: stable/metricbeat
```
`helmfile.yaml`:
```yaml
{{ readFile "common.yaml" }}
releases:
- <<: *metricbeat
- name: myapp
chart: mychart
```
## Layering State Template Files
Do you need to make your state file even more DRY?
Turned out layering state files wasn't enough for you?
Helmfile supports an advanced feature that allows you to compose state "template" files to generate the final state to be processed.
In the following example `helmfile.yaml.gotmpl`, each `---` separated part of the file is a go template.
`helmfile.yaml.gotmpl`:
```yaml
# Part 1: Reused Enviroment Values
bases:
- myenv.yaml
---
# Part 2: Reused Defaults
bases:
- mydefaults.yaml
---
# Part 3: Dynamic Releases
releases:
- name: test1
chart: mychart-{{ .Values.myname }}
values:
replicaCount: 1
image:
repository: "nginx"
tag: "latest"
```
Suppose the `myenv.yaml` and `test.env.yaml` loaded in the first part looks like:
`myenv.yaml`:
```yaml
environments:
test:
values:
- test.env.yaml
```
`test.env.yaml`:
```yaml
kubeContext: test
wait: false
cvOnly: false
myname: "dog"
```
Where the gotmpl file loaded in the second part looks like:
`mydefaults.yaml.gotmpl`:
```yaml
helmDefaults:
tillerNamespace: kube-system
kubeContext: {{ .Values.kubeContext }}
verify: false
{{ if .Values.wait }}
wait: true
{{ else }}
wait: false
{{ end }}
timeout: 600
recreatePods: false
force: true
```
Each go template is rendered in the context where `.Values` is inherited from the previous part.
So in `mydefaults.yaml.gotmpl`, both `.Values.kubeContext` and `.Values.wait` are valid as they do exist in the environment values inherited from the previous part(=the first part) of your `helmfile.yaml.gotmpl`, and therefore the template is rendered to:
```yaml
helmDefaults:
tillerNamespace: kube-system
kubeContext: test
verify: false
wait: false
timeout: 600
recreatePods: false
force: true
```
Similarly, the third part of the top-level `helmfile.yaml.gotmpl`, `.Values.myname` is valid as it is included in the environment values inherited from the previous parts:
```yaml
# Part 3: Dynamic Releases
releases:
- name: test1
chart: mychart-{{ .Values.myname }}
values:
replicaCount: 1
image:
repository: "nginx"
tag: "latest"
````
hence rendered to:
```yaml
# Part 3: Dynamic Releases
releases:
- name: test1
chart: mychart-dog
values:
replicaCount: 1
image:
repository: "nginx"
tag: "latest"
```
+27
View File
@@ -0,0 +1,27 @@
module github.com/roboll/helmfile
go 1.12
require (
github.com/Masterminds/goutils v1.1.0 // indirect
github.com/Masterminds/semver v1.4.1
github.com/Masterminds/sprig v2.20.0+incompatible
github.com/go-test/deep v1.0.3
github.com/google/go-cmp v0.3.0
github.com/google/uuid v0.0.0-20161128191214-064e2069ce9c // indirect
github.com/gosuri/uitable v0.0.3
github.com/hashicorp/go-getter v1.3.0
github.com/huandu/xstrings v1.2.0 // indirect
github.com/imdario/mergo v0.3.6
github.com/pkg/errors v0.8.1 // indirect
github.com/r3labs/diff v0.0.0-20190801153147-a71de73c46ad
github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939
github.com/urfave/cli v0.0.0-20160620154522-6011f165dc28
go.uber.org/atomic v1.3.2 // indirect
go.uber.org/multierr v1.1.0
go.uber.org/zap v1.8.0
gopkg.in/yaml.v2 v2.2.1
gotest.tools v2.2.0+incompatible
)
replace git.apache.org/thrift.git => github.com/apache/thrift v0.12.0
+218
View File
@@ -0,0 +1,218 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.31.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.36.0 h1:+aCSj7tOo2LODWVEuZDZeGCckdt6MlSF+X/rB3wUiS8=
cloud.google.com/go v0.36.0/go.mod h1:RUoy9p/M4ge0HzT8L+SDZ8jg+Q6fth0CiBuhFJpSV40=
dmitri.shuralyov.com/app/changes v0.0.0-20180602232624-0a106ad413e3/go.mod h1:Yl+fi1br7+Rr3LqpNJf1/uxUdtRUV+Tnj0o93V2B9MU=
dmitri.shuralyov.com/html/belt v0.0.0-20180602232347-f7d459c86be0/go.mod h1:JLBrvjyP0v+ecvNYvCpyZgu5/xkfAUhi6wJj28eUfSU=
dmitri.shuralyov.com/service/change v0.0.0-20181023043359-a85b471d5412/go.mod h1:a1inKt/atXimZ4Mv927x+r7UpyzRUf4emIoiiSC2TN4=
dmitri.shuralyov.com/state v0.0.0-20180228185332-28bcc343414c/go.mod h1:0PRwlb0D6DFvNNtx+9ybjezNCa8XF0xaYcETyp6rHWU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/Masterminds/goutils v1.1.0 h1:zukEsf/1JZwCMgHiK3GZftabmxiCw4apj3a28RPBiVg=
github.com/Masterminds/goutils v1.1.0/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU=
github.com/Masterminds/semver v1.4.1 h1:CaDA1wAoM3rj9sAFyyZP37LloExUzxFGYt+DqJ870JA=
github.com/Masterminds/semver v1.4.1/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF078ddwwvV3Y=
github.com/Masterminds/sprig v2.20.0+incompatible h1:dJTKKuUkYW3RMFdQFXPU/s6hg10RgctmTjRcbZ98Ap8=
github.com/Masterminds/sprig v2.20.0+incompatible/go.mod h1:y6hNFY5UBTIWBxnzTeuNhlNS5hqE0NB0E6fgfo2Br3o=
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c=
github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/aws/aws-sdk-go v1.15.78 h1:LaXy6lWR0YK7LKyuU0QWy2ws/LWTPfYV/UgfiBu4tvY=
github.com/aws/aws-sdk-go v1.15.78/go.mod h1:E3/ieXAlvM0XWO57iftYVDLLvQ824smPP3ATZkfNZeM=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d h1:xDfNPAt8lFiC1UJrqV3uuy861HCTo708pDMbjHHdCas=
github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d/go.mod h1:6QX/PXZ00z/TKoufEY6K/a0k6AhaJrQKdFe6OfVXsa4=
github.com/bradfitz/go-smtpd v0.0.0-20170404230938-deb6d6237625/go.mod h1:HYsPBTaaSFSlLx/70C2HPIMNZpVV8+vt/A+FMnYP11g=
github.com/cheggaaa/pb v1.0.27/go.mod h1:pQciLPpbU0oxA0h+VJYYLxO+XeDQb5pZijXscXHm81s=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/coreos/go-systemd v0.0.0-20181012123002-c6f51f82210d/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gliderlabs/ssh v0.1.1/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0=
github.com/go-test/deep v1.0.3 h1:ZrJSEWsXzPOxaZnFteGEfooLba+ju3FYIbOrS+rQd68=
github.com/go-test/deep v1.0.3/go.mod h1:wGDj63lr65AM2AQyKZd/NYHGb0R+1RLqB8NKt3aSFNA=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/lint v0.0.0-20180702182130-06c8688daad7/go.mod h1:tluoj9z5200jBnyusfRPU2LqT6J+DAorxEvtC7LHB+E=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-github v17.0.0+incompatible/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/martian v2.1.0+incompatible h1:/CP5g8u/VJHijgedC/Legn3BAbAaWPgecwXBIDzw5no=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/uuid v0.0.0-20161128191214-064e2069ce9c h1:jWtZjFEUE/Bz0IeIhqCnyZ3HG6KRXSntXe4SjtuTH7c=
github.com/google/uuid v0.0.0-20161128191214-064e2069ce9c/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go v2.0.0+incompatible h1:j0GKcs05QVmm7yesiZq2+9cxHkNK9YM6zKx4D2qucQU=
github.com/googleapis/gax-go v2.0.0+incompatible/go.mod h1:SFVmujtThgffbyetf+mdk2eWhX2bMyUtNHzFKcPA9HY=
github.com/googleapis/gax-go/v2 v2.0.3 h1:siORttZ36U2R/WjiJuDz8znElWBiAlO9rVt+mqJt0Cc=
github.com/googleapis/gax-go/v2 v2.0.3/go.mod h1:LLvjysVCY1JZeum8Z6l8qUty8fiNwE08qbEPm1M08qg=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gosuri/uitable v0.0.3 h1:9ZY4qCODg6JL1Ui4dL9LqCF4ghWnAOSV2h7xG98SkHE=
github.com/gosuri/uitable v0.0.3/go.mod h1:tKR86bXuXPZazfOTG1FIzvjIdXzd0mo4Vtn16vt0PJo=
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=
github.com/grpc-ecosystem/grpc-gateway v1.5.0/go.mod h1:RSKVYQBd5MCa4OVpNdGskqpgL2+G+NZTnrVHpWWfpdw=
github.com/hashicorp/go-cleanhttp v0.5.0 h1:wvCrVc9TjDls6+YGAF2hAifE1E5U1+b4tH6KdvN3Gig=
github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-getter v1.3.0 h1:pFMSFlI9l5NaeuzkpE3L7BYk9qQ9juTAgXW/H0cqxcU=
github.com/hashicorp/go-getter v1.3.0/go.mod h1:/O1k/AizTN0QmfEKknCYGvICeyKUDqCYA8vvWtGWDeQ=
github.com/hashicorp/go-safetemp v1.0.0 h1:2HR189eFNrjHQyENnQMMpCiBAsRxzbTMIgBhEyExpmo=
github.com/hashicorp/go-safetemp v1.0.0/go.mod h1:oaerMy3BhqiTbVye6QuFhFtIceqFoDHxNAB65b+Rj1I=
github.com/hashicorp/go-version v1.1.0 h1:bPIoEKD27tNdebFGGxxYwcL4nepeY4j1QP23PFRGzg0=
github.com/hashicorp/go-version v1.1.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/huandu/xstrings v1.2.0 h1:yPeWdRnmynF7p+lLYz0H2tthW9lqhMJrQV/U7yy4wX0=
github.com/huandu/xstrings v1.2.0/go.mod h1:DvyZB1rfVYsBIigL8HwpZgxHwXozlTgGqn63UyNX5k4=
github.com/imdario/mergo v0.3.6 h1:xTNEAn+kxVO7dTZGu0CegyqKZmoWFI0rF8UxjlB2d28=
github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/jellevandenhooff/dkim v0.0.0-20150330215556-f50fe3d243e1/go.mod h1:E0B/fFc00Y+Rasa88328GlI/XbtyysCtTHZS8h7IrBU=
github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8 h1:12VvqtR6Aowv3l/EQUlocDHW2Cp4G9WJVH7uyH8QFJE=
github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.3/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/mattn/go-colorable v0.0.9 h1:UVL0vNpWh04HeJXV0KLcaT7r06gOH2l4OW6ddYRUIY4=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-isatty v0.0.4 h1:bnP0vzxcAdeI1zdubAl5PjU6zsERjGZb7raWodagDYs=
github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-runewidth v0.0.4 h1:2BvfKmzob6Bmd4YsL0zygOqfdFnK7GR4QL06Do4/p7Y=
github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/microcosm-cc/bluemonday v1.0.1/go.mod h1:hsXNsILzKxV+sX77C5b8FSuKF00vh2OMYv+xgHpAMF4=
github.com/mitchellh/go-homedir v1.0.0 h1:vKb8ShqSby24Yrqr/yDYkuFz8d0WUjys40rvnGC8aR0=
github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mitchellh/go-testing-interface v1.0.0 h1:fzU/JVNcaqHQEcVFAKeR41fkiLdIPrefOvVG1VZ96U0=
github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=
github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86/go.mod h1:kHJEU3ofeGjhHklVoIGuVj85JJwZ6kWPaJwCIxgnFmo=
github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab/go.mod h1:Qr6/a/Q4r9LP1IltGz7tA7iOK1WonHEYhu1HRBA7ZiM=
github.com/openzipkin/zipkin-go v0.1.1/go.mod h1:NtoC/o8u3JlF1lSlyPNswIbeQH9bJTmOf0Erfk+hxe8=
github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v0.8.0/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/common v0.0.0-20180801064454-c7de2306084e/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
github.com/prometheus/procfs v0.0.0-20180725123919-05ee40e3a273/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/r3labs/diff v0.0.0-20190801153147-a71de73c46ad h1:j5pg/OewZJyE6i3hIG4v3eQUvUyFdQkC8Nd/mjaEkxE=
github.com/r3labs/diff v0.0.0-20190801153147-a71de73c46ad/go.mod h1:ozniNEFS3j1qCwHKdvraMn1WJOsUxHd7lYfukEIS4cs=
github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/shurcooL/component v0.0.0-20170202220835-f88ec8f54cc4/go.mod h1:XhFIlyj5a1fBNx5aJTbKoIq0mNaPvOagO+HjB3EtxrY=
github.com/shurcooL/events v0.0.0-20181021180414-410e4ca65f48/go.mod h1:5u70Mqkb5O5cxEA8nxTsgrgLehJeAw6Oc4Ab1c/P1HM=
github.com/shurcooL/github_flavored_markdown v0.0.0-20181002035957-2122de532470/go.mod h1:2dOwnU2uBioM+SGy2aZoq1f/Sd1l9OkAeAUvjSyvgU0=
github.com/shurcooL/go v0.0.0-20180423040247-9e1955d9fb6e/go.mod h1:TDJrrUr11Vxrven61rcy3hJMUqaf/CLWYhHNPmT14Lk=
github.com/shurcooL/go-goon v0.0.0-20170922171312-37c2f522c041/go.mod h1:N5mDOmsrJOB+vfqUK+7DmDyjhSLIIBnXo9lvZJj3MWQ=
github.com/shurcooL/gofontwoff v0.0.0-20180329035133-29b52fc0a18d/go.mod h1:05UtEgK5zq39gLST6uB0cf3NEHjETfB4Fgr3Gx5R9Vw=
github.com/shurcooL/gopherjslib v0.0.0-20160914041154-feb6d3990c2c/go.mod h1:8d3azKNyqcHP1GaQE/c6dDgjkgSx2BZ4IoEi4F1reUI=
github.com/shurcooL/highlight_diff v0.0.0-20170515013008-09bb4053de1b/go.mod h1:ZpfEhSmds4ytuByIcDnOLkTHGUI6KNqRNPDLHDk+mUU=
github.com/shurcooL/highlight_go v0.0.0-20181028180052-98c3abbbae20/go.mod h1:UDKB5a1T23gOMUJrI+uSuH0VRDStOiUVSjBTRDVBVag=
github.com/shurcooL/home v0.0.0-20181020052607-80b7ffcb30f9/go.mod h1:+rgNQw2P9ARFAs37qieuu7ohDNQ3gds9msbT2yn85sg=
github.com/shurcooL/htmlg v0.0.0-20170918183704-d01228ac9e50/go.mod h1:zPn1wHpTIePGnXSHpsVPWEktKXHr6+SS6x/IKRb7cpw=
github.com/shurcooL/httperror v0.0.0-20170206035902-86b7830d14cc/go.mod h1:aYMfkZ6DWSJPJ6c4Wwz3QtW22G7mf/PEgaB9k/ik5+Y=
github.com/shurcooL/httpfs v0.0.0-20171119174359-809beceb2371/go.mod h1:ZY1cvUeJuFPAdZ/B6v7RHavJWZn2YPVFQ1OSXhCGOkg=
github.com/shurcooL/httpgzip v0.0.0-20180522190206-b1c53ac65af9/go.mod h1:919LwcH0M7/W4fcZ0/jy0qGght1GIhqyS/EgWGH2j5Q=
github.com/shurcooL/issues v0.0.0-20181008053335-6292fdc1e191/go.mod h1:e2qWDig5bLteJ4fwvDAc2NHzqFEthkqn7aOZAOpj+PQ=
github.com/shurcooL/issuesapp v0.0.0-20180602232740-048589ce2241/go.mod h1:NPpHK2TI7iSaM0buivtFUc9offApnI0Alt/K8hcHy0I=
github.com/shurcooL/notifications v0.0.0-20181007000457-627ab5aea122/go.mod h1:b5uSkrEVM1jQUspwbixRBhaIjIzL2xazXp6kntxYle0=
github.com/shurcooL/octicon v0.0.0-20181028054416-fa4f57f9efb2/go.mod h1:eWdoE5JD4R5UVWDucdOPg1g2fqQRq78IQa9zlOV1vpQ=
github.com/shurcooL/reactions v0.0.0-20181006231557-f2e0b4ca5b82/go.mod h1:TCR1lToEk4d2s07G3XGfz2QrgHXg4RJBvjrOozvoWfk=
github.com/shurcooL/sanitized_anchor_name v0.0.0-20170918181015-86672fcb3f95/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/shurcooL/users v0.0.0-20180125191416-49c67e49c537/go.mod h1:QJTqeLYEDaXHZDBsXlPCDqdhQuJkuw4NOtaxYe3xii4=
github.com/shurcooL/webdavfs v0.0.0-20170829043945-18c3829fa133/go.mod h1:hKmq5kWdCj2z2KEozexVbfEZIWiTjhE0+UjmZgPqehw=
github.com/sourcegraph/annotate v0.0.0-20160123013949-f4cad6c6324d/go.mod h1:UdhH50NIW0fCiwBSr0co2m7BnFLdv4fQTgdqdJTHFeE=
github.com/sourcegraph/syntaxhighlight v0.0.0-20170531221838-bd320f5d308e/go.mod h1:HuIsMU8RRBOtsCgI77wP899iHVBQpCmg4ErYMZB+2IA=
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07/go.mod h1:kDXzergiv9cbyO7IOYJZWg1U88JhDg3PB6klq9Hg2pA=
github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939 h1:BhIUXV2ySTLrKgh/Hnts+QTQlIbWtomXt3LMdzME0A0=
github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939/go.mod h1:omGxs4/6hNjxPKUTjmaNkPzehSnNJOJN6pMEbrlYIT4=
github.com/ulikunitz/xz v0.5.5 h1:pFrO0lVpTBXLpYw+pnLj6TbvHuyjXMfjGeCwSqCVwok=
github.com/ulikunitz/xz v0.5.5/go.mod h1:2bypXElzHzzJZwzH67Y6wb67pO62Rzfn7BSiF4ABRW8=
github.com/urfave/cli v0.0.0-20160620154522-6011f165dc28 h1:Yf7/DcGM+61oLBGXQV2Q+ztEGBcOe3EUnbKSOn4fQuE=
github.com/urfave/cli v0.0.0-20160620154522-6011f165dc28/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
go.opencensus.io v0.18.0 h1:Mk5rgZcggtbvtAun5aJzAtjKKN/t0R3jJPlWILlv938=
go.opencensus.io v0.18.0/go.mod h1:vKdFvxhtzZ9onBp9VKHK8z/sRpBMnKAsufL7wlDrCOA=
go.uber.org/atomic v1.3.2 h1:2Oa65PReHzfn29GpvgsYwloV9AVFHPDk8tYxt2c2tr4=
go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/multierr v1.1.0 h1:HoEmRHQPVSqub6w2z2d2EOVs2fjyFRGyofhKuyDq0QI=
go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
go.uber.org/zap v1.8.0 h1:r6Za1Rii8+EGOYRDLvpooNOF6kP3iyDnkpzbw67gCQ8=
go.uber.org/zap v1.8.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
go4.org v0.0.0-20180809161055-417644f6feb5/go.mod h1:MkTOUMDaeVYJUOUsaDXIhWPZYa1yOyC1qaOBpL57BhE=
golang.org/x/build v0.0.0-20190111050920-041ab4dc3f9d/go.mod h1:OWs+y06UdEOHN4y+MfF/py+xQ/tYqIWW03b70/CG9Rw=
golang.org/x/crypto v0.0.0-20181030102418-4d3f4d9ffa16 h1:y6ce7gCWtnH+m3dCjzQ1PCuwl28DDIc3VNnvY29DlIA=
golang.org/x/crypto v0.0.0-20181030102418-4d3f4d9ffa16/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20180702182130-06c8688daad7/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181029044818-c44066c5c816/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181106065722-10aee1819953/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a h1:gOpx8G595UYyvj8UK4+OFyY4rx037g3fmfhe5SasG3U=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181017192945-9dcd33a902f4/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890 h1:uESlIz09WIHT2I+pasSXcpLYqYK8wHcdCetU3VuMBJE=
golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/perf v0.0.0-20180704124530-6e6d33e29852/go.mod h1:JLpeXjPJfIyPr5TlbXLkXWLhP8nz10XfvxElABhCtcw=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f h1:Bl/8QSvNqXvPGPGXa2z5xUTmV7VDcZyvRZ+QQXkXTZQ=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181029174526-d69651ed3497/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181213200352-4d1cda033e06 h1:0oC8rFnE+74kEmuHZ46F6KHsMr5Gx2gUQPuNz28iQZM=
golang.org/x/sys v0.0.0-20181213200352-4d1cda033e06/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2 h1:z99zHgr7hKfrUcX/KsoJk5FJfjTceCKIp96+biqP4To=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20181030000716-a0a13e073c7b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
google.golang.org/api v0.0.0-20180910000450-7ca32eb868bf/go.mod h1:4mhQ8q/RsB7i+udVvVy5NUi08OU8ZlA0gRVgrF7VFY0=
google.golang.org/api v0.0.0-20181030000543-1d582fd0359e/go.mod h1:4mhQ8q/RsB7i+udVvVy5NUi08OU8ZlA0gRVgrF7VFY0=
google.golang.org/api v0.1.0 h1:K6z2u68e86TPdSdefXdzvXgR1zEMa+459vBSfWYAZkI=
google.golang.org/api v0.1.0/go.mod h1:UGEZY7KEX120AnNLIHFMKIo4obdJhkp2tPbaPlQx13Y=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.3.0 h1:FBSsiFRMz3LBeXIomRnVzrQwSDj4ibvcRexLG0LZGQk=
google.golang.org/appengine v1.3.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20180831171423-11092d34479b/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20181029155118-b69ba1387ce2/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20181202183823-bd91e49a0898/go.mod h1:7Ep/1NZk928CDR8SjdVbjWNpdIf6nzjE3BTgJDr2Atg=
google.golang.org/genproto v0.0.0-20190201180003-4b09977fb922 h1:mBVYJnbrXLA/ZCBTCe7PtEgAUP+1bg92qTaFoPHdz+8=
google.golang.org/genproto v0.0.0-20190201180003-4b09977fb922/go.mod h1:L3J43x8/uS+qIUoksaLKe6OS3nUKxOKuIFz1sl2/jx4=
google.golang.org/grpc v1.14.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
google.golang.org/grpc v1.16.0/go.mod h1:0JHn/cJsOMiMfNA9+DeHDlAU7KAAB5GDlYFpa9MZMio=
google.golang.org/grpc v1.17.0 h1:TRJYBgMclJvGYn2rIMjj+h9KtMt5r1Ij7ODVRIZkwhk=
google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/cheggaaa/pb.v1 v1.0.27/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/yaml.v2 v2.2.1 h1:mUhvW9EsL+naU5Q3cakzfE91YhliOondGd6ZrsDBHQE=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo=
gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw=
grpc.go4.org v0.0.0-20170609214715-11d0a25b4919/go.mod h1:77eQGdRu53HpSqPFJFmuJdjuHRquDANNeA4x7B8WQ9o=
honnef.co/go/tools v0.0.0-20180728063816-88497007e858/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
sourcegraph.com/sourcegraph/go-diff v0.5.0/go.mod h1:kuch7UrkMzY0X+p9CRK03kfuPQ2zzQcaEFbx8wA8rck=
sourcegraph.com/sqs/pbtypes v0.0.0-20180604144634-d3ebe8f20ae4/go.mod h1:ketZ/q3QxT9HOBeFhu6RdvsftgpsbFHBF5Cas6cDKZ0=
Executable
+623
View File
@@ -0,0 +1,623 @@
#!/usr/bin/env bash
PROG=semtag
PROG_VERSION="v0.1.0"
SEMVER_REGEX="^v?(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(\-[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$"
IDENTIFIER_REGEX="^\-([0-9A-Za-z-]+)\.([0-9A-Za-z-]+)*$"
# Global variables
FIRST_VERSION="v0.0.0"
finalversion=$FIRST_VERSION
lastversion=$FIRST_VERSION
hasversiontag="false"
scope="patch"
displayonly="false"
forcetag="false"
forcedversion=
versionname=
identifier=
HELP="\
Usage:
$PROG
$PROG getlast
$PROG getfinal
$PROG (final|alpha|beta|candidate) [-s <scope> (major|minor|patch|auto) | -o]
$PROG --help
$PROG --version
Options:
-s The scope that must be increased, can be major, minor or patch.
The resulting version will match X.Y.Z(-PRERELEASE)(+BUILD)
where X, Y and Z are positive integers, PRERELEASE is an optionnal
string composed of alphanumeric characters describing if the build is
a release candidate, alpha or beta version, with a number.
BUILD is also an optional string composed of alphanumeric
characters and hyphens.
Setting the scope as 'auto', the script will chose the scope between
'minor' and 'patch', depending on the amount of lines added (<10% will
choose patch).
-v Specifies manually the version to be tagged, must be a valid semantic version
in the format X.Y.Z where X, Y and Z are positive integers.
-o Output the version only, shows the bumped version, but doesn't tag.
-f Forces to tag, even if there are unstaged or uncommited changes.
Commands:
--help Print this help message.
--version Prints the program's version.
get Returns both current final version and last tagged version.
getlast Returns the latest tagged version.
getfinal Returns the latest tagged final version.
getcurrent Returns the current version, based on the latest one, if there are uncommited or
unstaged changes, they will be reflected in the version, adding the number of
pending commits, current branch and commit hash.
final Tags the current build as a final version, this only can be done on the master branch.
candidate Tags the current build as a release candidate, the tag will contain all
the commits from the last final version.
alpha Tags the current build as an alpha version, the tag will contain all
the commits from the last final version.
beta Tags the current build as a beta version, the tag will contain all
the commits from the last final version."
# Commands and options
ACTION="getlast"
ACTION="$1"
shift
# We get the parameters
while getopts "v:s:of" opt; do
case $opt in
v)
forcedversion="$OPTARG"
;;
s)
scope="$OPTARG"
;;
o)
displayonly="true"
;;
f)
forcetag="true"
;;
\?)
echo "Invalid option: -$OPTARG" >&2
exit 1
;;
:)
echo "Option -$OPTARG requires an argument." >&2
exit 1
;;
esac
done
# Gets a string with the version and returns an array of maximum size of 5 with all the parts of the sematinc version
# $1 The string containing the version in semantic format
# $2 The variable to store the result array:
# position 0: major number
# position 1: minor number
# position 2: patch number
# position 3: identifier (or prerelease identifier)
# position 4: build info
function explode_version {
local __version=$1
local __result=$2
if [[ $__version =~ $SEMVER_REGEX ]] ; then
local __major=${BASH_REMATCH[1]}
local __minor=${BASH_REMATCH[2]}
local __patch=${BASH_REMATCH[3]}
local __prere=${BASH_REMATCH[4]}
local __build=${BASH_REMATCH[5]}
eval "$__result=(\"$__major\" \"$__minor\" \"$__patch\" \"$__prere\" \"$__build\")"
else
eval "$__result="
fi
}
# Compare two versions and returns -1, 0 or 1
# $1 The first version to compare
# $2 The second version to compare
# $3 The variable where to store the result
function compare_versions {
local __first
local __second
explode_version $1 __first
explode_version $2 __second
local lv=$3
# Compares MAJOR, MINOR and PATCH
for i in 0 1 2; do
local __numberfirst=${__first[$i]}
local __numbersecond=${__second[$i]}
case $(($__numberfirst - $__numbersecond)) in
0)
;;
-[0-9]*)
eval "$lv=-1"
return 0
;;
[0-9]*)
eval "$lv=1"
return 0
;;
esac
done
# Identifiers should compare with the ASCII order.
local __identifierfirst=${__first[3]}
local __identifiersecond=${__second[3]}
if [[ -n "$__identifierfirst" ]] && [[ -n "$__identifiersecond" ]]; then
if [[ "$__identifierfirst" > "$__identifiersecond" ]]; then
eval "$lv=1"
return 0
elif [[ "$__identifierfirst" < "$__identifiersecond" ]]; then
eval "$lv=-1"
return 0
fi
elif [[ -z "$__identifierfirst" ]] && [[ -n "$__identifiersecond" ]]; then
eval "$lv=1"
return 0
elif [[ -n "$__identifierfirst" ]] && [[ -z "$__identifiersecond" ]]; then
eval "$lv=-1"
return 0
fi
eval "$lv=0"
}
# Returns the last version of two
# $1 The first version to compare
# $2 The second version to compare
# $3 The variable where to store the last one
function get_latest_of_two {
local __first=$1
local __second=$2
local __result
local __latest=$3
compare_versions $__first $__second __result
case $__result in
0)
eval "$__latest=$__second"
;;
-1)
eval "$__latest=$__second"
;;
1)
eval "$__latest=$__first"
;;
esac
}
# Assigns a 2 size array with the identifier, having the identifier at pos 0, and the number in pos 1
# $1 The identifier in the format -id.#
# $2 The vferiable where to store the 2 size array
function explode_identifier {
local __identifier=$1
local __result=$2
if [[ $__identifier =~ $IDENTIFIER_REGEX ]] ; then
local __id=${BASH_REMATCH[1]}
local __number=${BASH_REMATCH[2]}
if [[ -z "$__number" ]]; then
__number=1
fi
eval "$__result=(\"$__id\" \"$__number\")"
else
eval "$__result="
fi
}
# Gets a list of tags and assigns the base and latest versions
# Receives an array with the tags containing the versions
# Assigns to the global variables finalversion and lastversion the final version and the latest version
function get_latest {
local __taglist=("$@")
local __tagsnumber=${#__taglist[@]}
local __current
case $__tagsnumber in
0)
finalversion=$FIRST_VERSION
lastversion=$FIRST_VERSION
;;
1)
__current=${__taglist[0]}
explode_version $__current ver
if [ -n "$ver" ]; then
if [ -n "${ver[3]}" ]; then
finalversion=$FIRST_VERSION
else
finalversion=$__current
fi
lastversion=$__current
else
finalversion=$FIRST_VERSION
lastversion=$FIRST_VERSION
fi
;;
*)
local __lastpos=$(($__tagsnumber-1))
for i in $(seq 0 $__lastpos)
do
__current=${__taglist[i]}
explode_version ${__taglist[i]} ver
if [ -n "$ver" ]; then
if [ -z "${ver[3]}" ]; then
get_latest_of_two $finalversion $__current finalversion
get_latest_of_two $lastversion $finalversion lastversion
else
get_latest_of_two $lastversion $__current lastversion
fi
fi
done
;;
esac
if git rev-parse -q --verify "refs/tags/$lastversion" >/dev/null; then
hasversiontag="true"
else
hasversiontag="false"
fi
}
# Gets the next version given the provided scope
# $1 The version that is going to be bumped
# $2 The scope to bump
# $3 The variable where to stoer the result
function get_next_version {
local __exploded
local __fromversion=$1
local __scope=$2
local __result=$3
explode_version $__fromversion __exploded
case $__scope in
major)
__exploded[0]=$((${__exploded[0]}+1))
__exploded[1]=0
__exploded[2]=0
;;
minor)
__exploded[1]=$((${__exploded[1]}+1))
__exploded[2]=0
;;
patch)
__exploded[2]=$((${__exploded[2]}+1))
;;
esac
eval "$__result=v${__exploded[0]}.${__exploded[1]}.${__exploded[2]}"
}
function bump_version {
## First we try to get the next version based on the existing last one
if [ "$scope" == "auto" ]; then
get_scope_auto scope
fi
local __candidatefromlast=$FIRST_VERSION
local __explodedlast
explode_version $lastversion __explodedlast
if [[ -n "${__explodedlast[3]}" ]]; then
# Last version is not final
local __idlast
explode_identifier ${__explodedlast[3]} __idlast
# We get the last, given the desired id based on the scope
__candidatefromlast="v${__explodedlast[0]}.${__explodedlast[1]}.${__explodedlast[2]}"
if [[ -n "$identifier" ]]; then
local __nextid="$identifier.1"
if [ "$identifier" == "${__idlast[0]}" ]; then
# We target the same identifier as the last so we increase one
__nextid="$identifier.$(( ${__idlast[1]}+1 ))"
__candidatefromlast="$__candidatefromlast-$__nextid"
else
# Different identifiers, we make sure we are assigning a higher identifier, if not, we increase the version
__candidatefromlast="$__candidatefromlast-$__nextid"
local __comparedwithlast
compare_versions $__candidatefromlast $lastversion __comparedwithlast
if [ "$__comparedwithlast" == -1 ]; then
get_next_version $__candidatefromlast $scope __candidatefromlast
__candidatefromlast="$__candidatefromlast-$__nextid"
fi
fi
fi
fi
# Then we try to get the version based on the latest final one
local __candidatefromfinal=$FIRST_VERSION
get_next_version $finalversion $scope __candidatefromfinal
if [[ -n "$identifier" ]]; then
__candidatefromfinal="$__candidatefromfinal-$identifier.1"
fi
# Finally we compare both candidates
local __resultversion
local __result
compare_versions $__candidatefromlast $__candidatefromfinal __result
case $__result in
0)
__resultversion=$__candidatefromlast
;;
-1)
__resultversion="$__candidatefromfinal"
;;
1)
__resultversion=$__candidatefromlast
;;
esac
eval "$1=$__resultversion"
}
function increase_version {
local __version=
if [ -z $forcedversion ]; then
bump_version __version
else
if [[ $forcedversion =~ $SEMVER_REGEX ]] ; then
compare_versions $forcedversion $lastversion __result
if [ $__result -le 0 ]; then
echo "Version can't be lower than last version: $lastversion"
exit 1
fi
else
echo "Non valid version to bump"
exit 1
fi
__version=$forcedversion
fi
if [ "$displayonly" == "true" ]; then
echo "$__version"
else
if [ "$forcetag" == "false" ]; then
check_git_dirty_status
fi
local __commitlist
if [ "$finalversion" == "$FIRST_VERSION" ] || [ "$hasversiontag" != "true" ]; then
__commitlist="$(git log --pretty=oneline | cat)"
else
__commitlist="$(git log --pretty=oneline $finalversion... | cat)"
fi
# If we are forcing a bump, we add bump to the commit list
if [[ -z $__commitlist && "$forcetag" == "true" ]]; then
__commitlist="bump"
fi
if [[ -z $__commitlist ]]; then
echo "No commits since the last final version, not bumping version"
else
if [[ -z $versionname ]]; then
versionname=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
fi
local __message="$versionname
$__commitlist"
# We check we have info on the user
local __username=$(git config user.name)
if [ -z "$__username" ]; then
__username=$(id -u -n)
git config user.name $__username
fi
local __useremail=$(git config user.email)
if [ -z "$__useremail" ]; then
__useremail=$(hostname)
git config user.email "$__username@$__useremail"
fi
git tag -a $__version -m "$__message"
# If we have a remote, we push there
local __remote=${SEMTAG_REMOTE:-$(git remote)}
if [[ -n $__remote ]]; then
git push $__remote $__version > /dev/null
if [ $? -eq 0 ]; then
echo "$__version"
else
echo "Error pushing the tag $__version to $__remote"
exit 1
fi
else
echo "$__version"
fi
fi
fi
}
function check_git_dirty_status {
local __repostatus=
get_work_tree_status __repostatus
if [ "$__repostatus" == "uncommitted" ]; then
echo "ERROR: You have uncommitted changes"
git status --porcelain
exit 1
fi
if [ "$__repostatus" == "unstaged" ]; then
echo "ERROR: You have unstaged changes"
git status --porcelain
exit 1
fi
}
# Get the total amount of lines of code in the repo
function get_total_lines {
local __empty_id="$(git hash-object -t tree /dev/null)"
local __changes="$(git diff --numstat $__empty_id | cat)"
local __added_deleted=$1
get_changed_lines "$__changes" $__added_deleted
}
# Get the total amount of lines of code since the provided tag
function get_sincetag_lines {
local __sincetag=$1
local __changes="$(git diff --numstat $__sincetag | cat)"
local __added_deleted=$2
get_changed_lines "$__changes" $__added_deleted
}
function get_changed_lines {
local __changes_numstat=$1
local __result=$2
IFS=$'\n' read -rd '' -a __changes_array <<<"$__changes_numstat"
local __diff_regex="^([0-9]+)[[:space:]]+([0-9]+)[[:space:]]+.+$"
local __total_added=0
local __total_deleted=0
for i in "${__changes_array[@]}"
do
if [[ $i =~ $__diff_regex ]] ; then
local __added=${BASH_REMATCH[1]}
local __deleted=${BASH_REMATCH[2]}
__total_added=$(( $__total_added+$__added ))
__total_deleted=$(( $__total_deleted+$__deleted ))
fi
done
eval "$2=( $__total_added $__total_deleted )"
}
function get_scope_auto {
local __verbose=$2
local __total=0
local __since=0
local __scope=
get_total_lines __total
get_sincetag_lines $finalversion __since
local __percentage=0
if [ "$__total" != "0" ]; then
local __percentage=$(( 100*$__since/$__total ))
if [ $__percentage -gt "10" ]; then
__scope="minor"
else
__scope="patch"
fi
fi
eval "$1=$__scope"
if [[ -n "$__verbose" ]]; then
echo "[Auto Scope] Percentage of lines changed: $__percentage"
echo "[Auto Scope] : $__scope"
fi
}
function get_work_tree_status {
# Update the index
git update-index -q --ignore-submodules --refresh > /dev/null
eval "$1="
if ! git diff-files --quiet --ignore-submodules -- > /dev/null
then
eval "$1=unstaged"
fi
if ! git diff-index --cached --quiet HEAD --ignore-submodules -- > /dev/null
then
eval "$1=uncommitted"
fi
}
function get_current {
if [ "$hasversiontag" == "true" ]; then
local __commitcount="$(git rev-list $lastversion.. --count)"
else
local __commitcount="$(git rev-list --count HEAD)"
fi
local __status=
get_work_tree_status __status
if [ "$__commitcount" == "0" ] && [ -z "$__status" ]; then
eval "$1=$lastversion"
else
local __buildinfo="$(git rev-parse --short HEAD)"
local __currentbranch="$(git rev-parse --abbrev-ref HEAD)"
if [ "$__currentbranch" != "master" ]; then
__buildinfo="$__currentbranch.$__buildinfo"
fi
local __suffix=
if [ "$__commitcount" != "0" ]; then
if [ -n "$__suffix" ]; then
__suffix="$__suffix."
fi
__suffix="$__suffix$__commitcount"
fi
if [ -n "$__status" ]; then
if [ -n "$__suffix" ]; then
__suffix="$__suffix."
fi
__suffix="$__suffix$__status"
fi
__suffix="$__suffix+$__buildinfo"
if [ "$lastversion" == "$finalversion" ]; then
scope="patch"
identifier=
local __bumped=
bump_version __bumped
eval "$1=$__bumped-dev.$__suffix"
else
eval "$1=$lastversion.$__suffix"
fi
fi
}
function init {
git fetch > /dev/null
TAGS="$(git tag)"
IFS=$'\n' read -rd '' -a TAG_ARRAY <<<"$TAGS"
get_latest ${TAG_ARRAY[@]}
currentbranch="$(git rev-parse --abbrev-ref HEAD)"
}
case $ACTION in
--help)
echo -e "$HELP"
;;
--version)
echo -e "${PROG}: $PROG_VERSION"
;;
final)
init
diff=$(git diff master | cat)
if [ "$forcetag" == "false" ]; then
if [ -n "$diff" ]; then
echo "ERROR: Branch must be updated with master for final versions"
exit 1
fi
fi
increase_version
;;
alpha|beta)
init
identifier="$ACTION"
increase_version
;;
candidate)
init
identifier="rc"
increase_version
;;
getlast)
init
echo "$lastversion"
;;
getfinal)
init
echo "$finalversion"
;;
getcurrent)
init
get_current current
echo "$current"
;;
get)
init
echo "Current final version: $finalversion"
echo "Last tagged version: $lastversion"
;;
*)
echo "'$ACTION' is not a valid command, see --help for available commands."
;;
esac
-125
View File
@@ -1,125 +0,0 @@
package helmexec
import (
"fmt"
"io"
"strings"
)
const (
command = "helm"
)
type execer struct {
runner Runner
writer io.Writer
kubeContext string
extra []string
}
// New for running helm commands
func New(writer io.Writer, kubeContext string) *execer {
return &execer{
writer: writer,
kubeContext: kubeContext,
runner: &ShellRunner{},
}
}
func (helm *execer) SetExtraArgs(args ...string) {
helm.extra = args
}
func (helm *execer) AddRepo(name, repository, certfile, keyfile, username, password string) error {
var args []string
args = append(args, "repo", "add", name, repository)
if certfile != "" && keyfile != "" {
args = append(args, "--cert-file", certfile, "--key-file", keyfile)
}
if username != "" && password != "" {
args = append(args, "--username", username, "--password", password)
}
out, err := helm.exec(args...)
helm.write(out)
return err
}
func (helm *execer) UpdateRepo() error {
out, err := helm.exec("repo", "update")
helm.write(out)
return err
}
func (helm *execer) UpdateDeps(chart string) error {
out, err := helm.exec("dependency", "update", chart)
helm.write(out)
return err
}
func (helm *execer) SyncRelease(name, chart string, flags ...string) error {
out, err := helm.exec(append([]string{"upgrade", "--install", "--reset-values", name, chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) ReleaseStatus(name string) error {
out, err := helm.exec(append([]string{"status", name})...)
if helm.writer != nil {
helm.writer.Write(out)
}
return err
}
func (helm *execer) DecryptSecret(name string) (string, error) {
out, err := helm.exec(append([]string{"secrets", "dec", name})...)
helm.write(out)
return name + ".dec", err
}
func (helm *execer) DiffRelease(name, chart string, flags ...string) error {
out, err := helm.exec(append([]string{"diff", "upgrade", "--allow-unreleased", name, chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) Lint(chart string, flags ...string) error {
out, err := helm.exec(append([]string{"lint", chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) Fetch(chart string, flags ...string) error {
out, err := helm.exec(append([]string{"fetch", chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) DeleteRelease(name string, flags ...string) error {
out, err := helm.exec(append([]string{"delete", name}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) TestRelease(name string, flags ...string) error {
out, err := helm.exec(append([]string{"test", name}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) exec(args ...string) ([]byte, error) {
cmdargs := args
if len(helm.extra) > 0 {
cmdargs = append(cmdargs, helm.extra...)
}
if helm.kubeContext != "" {
cmdargs = append(cmdargs, "--kube-context", helm.kubeContext)
}
helm.write([]byte(fmt.Sprintf("exec: helm %s\n", strings.Join(cmdargs, " "))))
return helm.runner.Execute(command, cmdargs)
}
func (helm *execer) write(out []byte) {
if helm.writer != nil {
helm.writer.Write(out)
}
}
-260
View File
@@ -1,260 +0,0 @@
package helmexec
import (
"bytes"
"io"
"reflect"
"testing"
)
// Mocking the command-line runner
type mockRunner struct {
output []byte
err error
}
func (mock *mockRunner) Execute(cmd string, args []string) ([]byte, error) {
return []byte{}, nil
}
func MockExecer(writer io.Writer, kubeContext string) *execer {
execer := New(writer, kubeContext)
execer.runner = &mockRunner{}
return execer
}
// Test methods
func TestNewHelmExec(t *testing.T) {
buffer := bytes.NewBufferString("something")
helm := New(buffer, "dev")
if helm.kubeContext != "dev" {
t.Error("helmexec.New() - kubeContext")
}
if buffer.String() != "something" {
t.Error("helmexec.New() - changed buffer")
}
if len(helm.extra) != 0 {
t.Error("helmexec.New() - extra args not empty")
}
}
func Test_SetExtraArgs(t *testing.T) {
helm := New(new(bytes.Buffer), "dev")
helm.SetExtraArgs()
if len(helm.extra) != 0 {
t.Error("helmexec.SetExtraArgs() - passing no arguments should not change extra field")
}
helm.SetExtraArgs("foo")
if !reflect.DeepEqual(helm.extra, []string{"foo"}) {
t.Error("helmexec.SetExtraArgs() - one extra argument missing")
}
helm.SetExtraArgs("alpha", "beta")
if !reflect.DeepEqual(helm.extra, []string{"alpha", "beta"}) {
t.Error("helmexec.SetExtraArgs() - two extra arguments missing (overwriting the previous value)")
}
}
func Test_AddRepo(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.AddRepo("myRepo", "https://repo.example.com/", "cert.pem", "key.pem", "", "")
expected := "exec: helm repo add myRepo https://repo.example.com/ --cert-file cert.pem --key-file key.pem --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "", "")
expected = "exec: helm repo add myRepo https://repo.example.com/ --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "example_user", "example_password")
expected = "exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_UpdateRepo(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.UpdateRepo()
expected := "exec: helm repo update --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.UpdateRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_SyncRelease(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.SyncRelease("release", "chart", "--timeout 10", "--wait")
expected := "exec: helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SyncRelease("release", "chart")
expected = "exec: helm upgrade --install --reset-values release chart --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_UpdateDeps(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.UpdateDeps("./chart/foo")
expected := "exec: helm dependency update ./chart/foo --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SetExtraArgs("--verify")
helm.UpdateDeps("./chart/foo")
expected = "exec: helm dependency update ./chart/foo --verify --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DecryptSecret(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.DecryptSecret("secretName")
expected := "exec: helm secrets dec secretName --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.DecryptSecret()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DiffRelease(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.DiffRelease("release", "chart", "--timeout 10", "--wait")
expected := "exec: helm diff upgrade --allow-unreleased release chart --timeout 10 --wait --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.DiffRelease("release", "chart")
expected = "exec: helm diff upgrade --allow-unreleased release chart --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DeleteRelease(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.DeleteRelease("release")
expected := "exec: helm delete release --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.DeleteRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DeleteRelease_Flags(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.DeleteRelease("release", "--purge")
expected := "exec: helm delete release --purge --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.DeleteRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_TestRelease(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.TestRelease("release")
expected := "exec: helm test release --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.TestRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_TestRelease_Flags(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.TestRelease("release", "--cleanup", "--timeout", "60")
expected := "exec: helm test release --cleanup --timeout 60 --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.TestRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_ReleaseStatus(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.ReleaseStatus("myRelease")
expected := "exec: helm status myRelease --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.ReleaseStatus()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_exec(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "")
helm.exec("version")
expected := "exec: helm version\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
helm = MockExecer(nil, "dev")
ret, _ := helm.exec("diff")
if len(ret) != 0 {
t.Error("helmexec.exec() - expected empty return value")
}
buffer.Reset()
helm = MockExecer(&buffer, "dev")
helm.exec("diff", "release", "chart", "--timeout 10", "--wait")
expected = "exec: helm diff release chart --timeout 10 --wait --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.exec("version")
expected = "exec: helm version --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SetExtraArgs("foo")
helm.exec("version")
expected = "exec: helm version foo --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_Lint(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.Lint("path/to/chart", "--values", "file.yml")
expected := "exec: helm lint path/to/chart --values file.yml --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.Lint()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_Fetch(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.Fetch("chart", "--version", "1.2.3", "--untar", "--untardir", "/tmp/dir")
expected := "exec: helm fetch chart --version 1.2.3 --untar --untardir /tmp/dir --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.Lint()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
-19
View File
@@ -1,19 +0,0 @@
package helmexec
// Interface for executing helm commands
type Interface interface {
SetExtraArgs(args ...string)
AddRepo(name, repository, certfile, keyfile, username, password string) error
UpdateRepo() error
UpdateDeps(chart string) error
SyncRelease(name, chart string, flags ...string) error
DiffRelease(name, chart string, flags ...string) error
Fetch(chart string, flags ...string) error
Lint(chart string, flags ...string) error
ReleaseStatus(name string) error
DeleteRelease(name string, flags ...string) error
TestRelease(name string, flags ...string) error
DecryptSecret(name string) (string, error)
}
-33
View File
@@ -1,33 +0,0 @@
package helmexec
import (
"io/ioutil"
"os"
"os/exec"
)
const (
tmpPrefix = "helmfile-"
tmpSuffix = "-exec"
)
// Runner interface for shell commands
type Runner interface {
Execute(cmd string, args []string) ([]byte, error)
}
// ShellRunner implemention for shell commands
type ShellRunner struct{}
// Execute a shell command
func (shell ShellRunner) Execute(cmd string, args []string) ([]byte, error) {
dir, err := ioutil.TempDir("", tmpPrefix+cmd+tmpSuffix)
if err != nil {
return nil, err
}
defer os.RemoveAll(dir)
preparedCmd := exec.Command(cmd, args...)
preparedCmd.Dir = dir
return preparedCmd.CombinedOutput()
}
+444 -273
View File
@@ -2,51 +2,90 @@ package main
import (
"fmt"
"io"
"log"
"os"
"os/signal"
"strings"
"syscall"
"github.com/roboll/helmfile/helmexec"
"github.com/roboll/helmfile/state"
"github.com/roboll/helmfile/pkg/app"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/maputil"
"github.com/roboll/helmfile/pkg/state"
"github.com/urfave/cli"
"path/filepath"
"sort"
)
const (
DefaultHelmfile = "helmfile.yaml"
DeprecatedHelmfile = "charts.yaml"
DefaultHelmfileDirectory = "helmfile.d"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
)
var Version string
var logger *zap.SugaredLogger
func configureLogging(c *cli.Context) error {
// Valid levels:
// https://github.com/uber-go/zap/blob/7e7e266a8dbce911a49554b945538c5b950196b8/zapcore/level.go#L126
logLevel := c.GlobalString("log-level")
if c.GlobalBool("quiet") {
logLevel = "warn"
}
var level zapcore.Level
err := level.Set(logLevel)
if err != nil {
return err
}
logger = helmexec.NewLogger(os.Stderr, logLevel)
if c.App.Metadata == nil {
// Auto-initialised in 1.19.0
// https://github.com/urfave/cli/blob/master/CHANGELOG.md#1190---2016-11-19
c.App.Metadata = make(map[string]interface{})
}
c.App.Metadata["logger"] = logger
return nil
}
func main() {
app := cli.NewApp()
app.Name = "helmfile"
app.Usage = ""
app.Version = Version
app.Flags = []cli.Flag{
cliApp := cli.NewApp()
cliApp.Name = "helmfile"
cliApp.Usage = ""
cliApp.Version = Version
cliApp.Flags = []cli.Flag{
cli.StringFlag{
Name: "helm-binary, b",
Usage: "path to helm binary",
},
cli.StringFlag{
Name: "file, f",
Value: DefaultHelmfile,
Usage: "load config from `FILE`",
Usage: "load config from file or directory. defaults to `helmfile.yaml` or `helmfile.d`(means `helmfile.d/*.yaml`) in this preference",
},
cli.StringFlag{
Name: "environment, e",
Usage: "specify the environment name. defaults to `default`",
},
cli.StringSliceFlag{
Name: "state-values-set",
Usage: "set state values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)",
},
cli.StringSliceFlag{
Name: "state-values-file",
Usage: "specify state values in a YAML file",
},
cli.BoolFlag{
Name: "quiet, q",
Usage: "silence output",
Usage: "Silence output. Equivalent to log-level warn",
},
cli.StringFlag{
Name: "kube-context",
Usage: "Set kubectl context. Uses current context by default",
},
cli.BoolFlag{
Name: "no-color",
Usage: "Output without color",
},
cli.StringFlag{
Name: "log-level",
Usage: "Set log level, default info",
},
cli.StringFlag{
Name: "namespace, n",
Usage: "Set namespace. Uses the namespace set in the context by default",
Usage: "Set namespace. Uses the namespace set in the context by default, and is available in templates as {{ .Namespace }}",
},
cli.StringSliceFlag{
Name: "selector, l",
@@ -55,9 +94,36 @@ func main() {
--selector tier=frontend,tier!=proxy --selector tier=backend. Will match all frontend, non-proxy releases AND all backend releases.
The name of a release can be used as a label. --selector name=myrelease`,
},
cli.BoolFlag{
Name: "allow-no-matching-release",
Usage: `Do not exit with an error code if the provided selector has no matching releases.`,
},
cli.BoolFlag{
Name: "interactive, i",
Usage: "Request confirmation before attempting to modify clusters",
},
}
app.Commands = []cli.Command{
cliApp.Before = configureLogging
cliApp.Commands = []cli.Command{
{
Name: "deps",
Usage: "update charts based on the contents of requirements.yaml",
Flags: []cli.Flag{
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
cli.BoolFlag{
Name: "skip-repos",
Usage: "skip running `helm repo update` before running `helm dependency build`",
},
},
Action: action(func(run *app.App, c configImpl) error {
return run.Deps(c)
}),
},
{
Name: "repos",
Usage: "sync repositories from state file (helm repo add && helm repo update)",
@@ -68,26 +134,23 @@ func main() {
Usage: "pass args to helm exec",
},
},
Action: func(c *cli.Context) error {
return eachDesiredStateDo(c, func(state *state.HelmState, helm helmexec.Interface) []error {
args := getArgs(c, state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
return state.SyncRepos(helm)
})
},
Action: action(func(run *app.App, c configImpl) error {
return run.Repos(c)
}),
},
{
Name: "charts",
Usage: "sync charts from state file (helm upgrade --install)",
Usage: "DEPRECATED: sync releases from state file (helm upgrade --install)",
Flags: []cli.Flag{
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
cli.StringSliceFlag{
Name: "set",
Usage: "additional values to be merged into the command",
},
cli.StringSliceFlag{
Name: "values",
Usage: "additional value files to be merged into the command",
@@ -98,62 +161,88 @@ func main() {
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
},
Action: func(c *cli.Context) error {
return eachDesiredStateDo(c, func(state *state.HelmState, helm helmexec.Interface) []error {
args := getArgs(c, state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
values := c.StringSlice("values")
workers := c.Int("concurrency")
return state.SyncReleases(helm, values, workers)
})
},
Action: action(func(run *app.App, c configImpl) error {
return run.DeprecatedSyncCharts(c)
}),
},
{
Name: "diff",
Usage: "diff charts from state file against env (helm diff)",
Usage: "diff releases from state file against env (helm diff)",
Flags: []cli.Flag{
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
cli.StringSliceFlag{
Name: "set",
Usage: "additional values to be merged into the command",
},
cli.StringSliceFlag{
Name: "values",
Usage: "additional value files to be merged into the command",
},
cli.BoolFlag{
Name: "sync-repos",
Usage: "enable a repo sync prior to diffing",
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
},
cli.BoolFlag{
Name: "detailed-exitcode",
Usage: "return a non-zero exit code when there are changes",
},
cli.BoolFlag{
Name: "suppress-secrets",
Usage: "suppress secrets in the output. highly recommended to specify on CI/CD use-cases",
},
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
cli.IntFlag{
Name: "context",
Value: 0,
Usage: "output NUM lines of context around changes",
},
},
Action: func(c *cli.Context) error {
return eachDesiredStateDo(c, func(state *state.HelmState, helm helmexec.Interface) []error {
args := getArgs(c, state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
if c.Bool("sync-repos") {
if errs := state.SyncRepos(helm); errs != nil && len(errs) > 0 {
return errs
}
}
values := c.StringSlice("values")
workers := c.Int("concurrency")
return state.DiffReleases(helm, values, workers)
})
Action: action(func(run *app.App, c configImpl) error {
return run.Diff(c)
}),
},
{
Name: "template",
Usage: "template releases from state file against env (helm template)",
Flags: []cli.Flag{
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm template",
},
cli.StringSliceFlag{
Name: "set",
Usage: "additional values to be merged into the command",
},
cli.StringSliceFlag{
Name: "values",
Usage: "additional value files to be merged into the command",
},
cli.StringFlag{
Name: "output-dir",
Usage: "output directory to pass to helm template (helm template --output-dir)",
},
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent downloads of release charts",
},
cli.BoolFlag{
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
},
},
Action: action(func(run *app.App, c configImpl) error {
return run.Template(c)
}),
},
{
Name: "lint",
@@ -164,6 +253,10 @@ func main() {
Value: "",
Usage: "pass args to helm exec",
},
cli.StringSliceFlag{
Name: "set",
Usage: "additional values to be merged into the command",
},
cli.StringSliceFlag{
Name: "values",
Usage: "additional value files to be merged into the command",
@@ -171,27 +264,25 @@ func main() {
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
Usage: "maximum number of concurrent downloads of release charts",
},
cli.BoolFlag{
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
},
},
Action: func(c *cli.Context) error {
return eachDesiredStateDo(c, func(state *state.HelmState, helm helmexec.Interface) []error {
args := c.String("args")
if len(args) > 0 {
helm.SetExtraArgs(strings.Split(args, " ")...)
}
values := c.StringSlice("values")
workers := c.Int("concurrency")
return state.LintReleases(helm, values, workers)
})
},
Action: action(func(run *app.App, c configImpl) error {
return run.Lint(c)
}),
},
{
Name: "sync",
Usage: "sync all resources from state file (repos, charts and local chart deps)",
Usage: "sync all resources from state file (repos, releases and chart deps)",
Flags: []cli.Flag{
cli.StringSliceFlag{
Name: "set",
Usage: "additional values to be merged into the command",
},
cli.StringSliceFlag{
Name: "values",
Usage: "additional value files to be merged into the command",
@@ -206,28 +297,54 @@ func main() {
Value: "",
Usage: "pass args to helm exec",
},
cli.BoolFlag{
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
},
},
Action: func(c *cli.Context) error {
return eachDesiredStateDo(c, func(state *state.HelmState, helm helmexec.Interface) []error {
if errs := state.SyncRepos(helm); errs != nil && len(errs) > 0 {
return errs
}
if errs := state.UpdateDeps(helm); errs != nil && len(errs) > 0 {
return errs
}
args := getArgs(c, state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
values := c.StringSlice("values")
workers := c.Int("concurrency")
return state.SyncReleases(helm, values, workers)
})
Action: action(func(run *app.App, c configImpl) error {
return run.Sync(c)
}),
},
{
Name: "apply",
Usage: "apply all resources from state file only when there are changes",
Flags: []cli.Flag{
cli.StringSliceFlag{
Name: "set",
Usage: "additional values to be merged into the command",
},
cli.StringSliceFlag{
Name: "values",
Usage: "additional value files to be merged into the command",
},
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
cli.IntFlag{
Name: "context",
Value: 0,
Usage: "output NUM lines of context around changes",
},
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
cli.BoolFlag{
Name: "suppress-secrets",
Usage: "suppress secrets in the diff output. highly recommended to specify on CI/CD use-cases",
},
cli.BoolFlag{
Name: "skip-deps",
Usage: "skip running `helm repo update` and `helm dependency build`",
},
},
Action: action(func(run *app.App, c configImpl) error {
return run.Apply(c)
}),
},
{
Name: "status",
@@ -244,35 +361,51 @@ func main() {
Usage: "pass args to helm exec",
},
},
Action: func(c *cli.Context) error {
return eachDesiredStateDo(c, func(state *state.HelmState, helm helmexec.Interface) []error {
workers := c.Int("concurrency")
args := getArgs(c, state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
return state.ReleaseStatuses(helm, workers)
})
},
Action: action(func(run *app.App, c configImpl) error {
return run.Status(c)
}),
},
{
Name: "delete",
Usage: "delete releases from state file (helm delete)",
Usage: "DEPRECATED: delete releases from state file (helm delete)",
Flags: []cli.Flag{
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
cli.BoolFlag{
Name: "purge",
Usage: "purge releases i.e. free release names and histories",
},
},
Action: func(c *cli.Context) error {
return eachDesiredStateDo(c, func(state *state.HelmState, helm helmexec.Interface) []error {
purge := c.Bool("purge")
return state.DeleteReleases(helm, purge)
})
Action: action(func(run *app.App, c configImpl) error {
return run.Delete(c)
}),
},
{
Name: "destroy",
Usage: "deletes and then purges releases",
Flags: []cli.Flag{
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
},
Action: action(func(run *app.App, c configImpl) error {
return run.Destroy(c)
}),
},
{
Name: "test",
@@ -292,181 +425,219 @@ func main() {
Value: 300,
Usage: "maximum time for tests to run before being considered failed",
},
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
},
Action: func(c *cli.Context) error {
return eachDesiredStateDo(c, func(state *state.HelmState, helm helmexec.Interface) []error {
cleanup := c.Bool("cleanup")
timeout := c.Int("timeout")
args := getArgs(c, state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
return state.TestReleases(helm, cleanup, timeout)
})
},
Action: action(func(run *app.App, c configImpl) error {
return run.Test(c)
}),
},
{
Name: "build",
Usage: "output compiled helmfile state(s) as YAML",
Flags: []cli.Flag{},
Action: action(func(run *app.App, c configImpl) error {
return run.PrintState(c)
}),
},
{
Name: "list",
Usage: "list releases defined in state file",
Flags: []cli.Flag{},
Action: action(func(run *app.App, c configImpl) error {
return run.ListReleases(c)
}),
},
}
err := app.Run(os.Args)
err := cliApp.Run(os.Args)
if err != nil {
log.Printf("err: %s", err.Error())
os.Exit(1)
fmt.Fprintf(os.Stderr, "%v\n", err)
os.Exit(3)
}
}
func eachDesiredStateDo(c *cli.Context, converge func(*state.HelmState, helmexec.Interface) []error) error {
fileOrDirPath := c.GlobalString("file")
desiredStateFiles, err := findDesiredStateFiles(fileOrDirPath)
if err != nil {
return err
}
for _, f := range desiredStateFiles {
state, helm, err := loadDesiredStateFromFile(c, f)
if err != nil {
return err
}
errs := converge(state, helm)
if err := clean(state, errs); err != nil {
return err
}
}
return nil
type configImpl struct {
c *cli.Context
set map[string]interface{}
}
func findDesiredStateFiles(specifiedPath string) ([]string, error) {
var helmfileDir string
if specifiedPath != "" {
if fileExistsAt(specifiedPath) {
return []string{specifiedPath}, nil
} else if directoryExistsAt(specifiedPath) {
helmfileDir = specifiedPath
} else {
return []string{}, fmt.Errorf("specified state file %s is not found", specifiedPath)
}
} else {
var defaultFile string
if fileExistsAt(DefaultHelmfile) {
defaultFile = DefaultHelmfile
} else if fileExistsAt(DeprecatedHelmfile) {
log.Printf(
"warn: %s is being loaded: %s is deprecated in favor of %s. See https://github.com/roboll/helmfile/issues/25 for more information",
DeprecatedHelmfile,
DeprecatedHelmfile,
DefaultHelmfile,
)
defaultFile = DeprecatedHelmfile
}
func NewUrfaveCliConfigImpl(c *cli.Context) (configImpl, error) {
if c.NArg() > 0 {
cli.ShowAppHelp(c)
return configImpl{}, fmt.Errorf("err: extraneous arguments: %s", strings.Join(c.Args(), ", "))
}
if directoryExistsAt(DefaultHelmfileDirectory) {
if defaultFile != "" {
return []string{}, fmt.Errorf("configuration conlict error: you can have either %s or %s, but not both", defaultFile, DefaultHelmfileDirectory)
conf := configImpl{
c: c,
}
optsSet := c.GlobalStringSlice("state-values-set")
if len(optsSet) > 0 {
set := map[string]interface{}{}
for i := range optsSet {
ops := strings.Split(optsSet[i], ",")
for j := range ops {
op := strings.SplitN(ops[j], "=", 2)
k := strings.Split(op[0], ".")
v := op[1]
set = maputil.Set(set, k, v)
}
helmfileDir = DefaultHelmfileDirectory
} else if defaultFile != "" {
return []string{defaultFile}, nil
} else {
return []string{}, fmt.Errorf("no state file found. It must be named %s/*.yaml, %s, or %s, or otherwise specified with the --file flag", DefaultHelmfileDirectory, DefaultHelmfile, DeprecatedHelmfile)
}
conf.set = set
}
files, err := filepath.Glob(filepath.Join(helmfileDir, "*.yaml"))
if err != nil {
return []string{}, err
}
sort.Slice(files, func(i, j int) bool {
return files[i] < files[j]
})
return files, nil
return conf, nil
}
func fileExistsAt(path string) bool {
fileInfo, err := os.Stat(path)
return err == nil && fileInfo.Mode().IsRegular()
func (c configImpl) Set() []string {
return c.c.StringSlice("set")
}
func directoryExistsAt(path string) bool {
fileInfo, err := os.Stat(path)
return err == nil && fileInfo.Mode().IsDir()
func (c configImpl) SkipRepos() bool {
return c.c.Bool("skip-repos")
}
func loadDesiredStateFromFile(c *cli.Context, file string) (*state.HelmState, helmexec.Interface, error) {
quiet := c.GlobalBool("quiet")
kubeContext := c.GlobalString("kube-context")
namespace := c.GlobalString("namespace")
labels := c.GlobalStringSlice("selector")
func (c configImpl) Values() []string {
return c.c.StringSlice("values")
}
st, err := state.ReadFromFile(file)
if err != nil {
return nil, nil, fmt.Errorf("failed to read %s: %v", file, err)
func (c configImpl) Args() string {
return c.c.String("args")
}
func (c configImpl) OutputDir() string {
return c.c.String("output-dir")
}
func (c configImpl) Concurrency() int {
return c.c.Int("concurrency")
}
func (c configImpl) HasCommandName(name string) bool {
return c.c.Command.HasName(name)
}
// DiffConfig
func (c configImpl) SkipDeps() bool {
return c.c.Bool("skip-deps")
}
func (c configImpl) DetailedExitcode() bool {
return c.c.Bool("detailed-exitcode")
}
func (c configImpl) SuppressSecrets() bool {
return c.c.Bool("suppress-secrets")
}
// DeleteConfig
func (c configImpl) Purge() bool {
return c.c.Bool("purge")
}
// TestConfig
func (c configImpl) Cleanup() bool {
return c.c.Bool("cleanup")
}
func (c configImpl) Timeout() int {
return c.c.Int("timeout")
}
// GlobalConfig
func (c configImpl) HelmBinary() string {
return c.c.GlobalString("helm-binary")
}
func (c configImpl) KubeContext() string {
return c.c.GlobalString("kube-context")
}
func (c configImpl) Namespace() string {
return c.c.GlobalString("namespace")
}
func (c configImpl) FileOrDir() string {
return c.c.GlobalString("file")
}
func (c configImpl) Selectors() []string {
return c.c.GlobalStringSlice("selector")
}
func (c configImpl) StateValuesSet() map[string]interface{} {
return c.set
}
func (c configImpl) StateValuesFiles() []string {
return c.c.GlobalStringSlice("state-values-file")
}
func (c configImpl) Interactive() bool {
return c.c.GlobalBool("interactive")
}
func (c configImpl) NoColor() bool {
return c.c.GlobalBool("no-color")
}
func (c configImpl) Context() int {
return c.c.Int("context")
}
func (c configImpl) Logger() *zap.SugaredLogger {
return c.c.App.Metadata["logger"].(*zap.SugaredLogger)
}
func (c configImpl) Env() string {
env := c.c.GlobalString("environment")
if env == "" {
env = state.DefaultEnv
}
return env
}
if st.Context != "" {
if kubeContext != "" {
log.Printf("err: Cannot use option --kube-context and set attribute context.")
os.Exit(1)
}
kubeContext = st.Context
}
if namespace != "" {
if st.Namespace != "" {
log.Printf("err: Cannot use option --namespace and set attribute namespace.")
os.Exit(1)
}
st.Namespace = namespace
}
if len(labels) > 0 {
err = st.FilterReleases(labels)
func action(do func(*app.App, configImpl) error) func(*cli.Context) error {
return func(implCtx *cli.Context) error {
conf, err := NewUrfaveCliConfigImpl(implCtx)
if err != nil {
log.Print(err)
os.Exit(1)
return err
}
a := app.New(conf)
a.ErrorHandler = func(err error) error {
return toCliError(implCtx, err)
}
return do(a, conf)
}
}
func toCliError(c *cli.Context, err error) error {
if err != nil {
switch e := err.(type) {
case *app.NoMatchingHelmfileError:
noMatchingExitCode := 3
if c.GlobalBool("allow-no-matching-release") {
noMatchingExitCode = 0
}
return cli.NewExitError(e.Error(), noMatchingExitCode)
case *app.Error:
return cli.NewExitError(e.Error(), e.Code())
default:
panic(fmt.Errorf("BUG: please file an github issue for this unhandled error: %T: %v", e, e))
}
}
var writer io.Writer
if !quiet {
writer = os.Stdout
}
sigs := make(chan os.Signal, 1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
go func() {
sig := <-sigs
errs := []error{fmt.Errorf("Received [%s] to shutdown ", sig)}
clean(st, errs)
}()
return st, helmexec.New(writer, kubeContext), nil
}
func clean(state *state.HelmState, errs []error) error {
if errs == nil {
errs = []error{}
}
cleanErrs := state.Clean()
if cleanErrs != nil {
errs = append(errs, cleanErrs...)
}
if errs != nil && len(errs) > 0 {
for _, err := range errs {
fmt.Printf("err: %s\n", err.Error())
}
os.Exit(1)
}
return nil
}
func getArgs(c *cli.Context, state *state.HelmState) []string {
args := c.String("args")
if len(args) > 0 {
state.HelmDefaults.Args = strings.Split(args, " ")
}
return state.HelmDefaults.Args
return err
}
+669
View File
@@ -0,0 +1,669 @@
package app
import (
"fmt"
"io/ioutil"
"log"
"os"
"os/signal"
"strings"
"syscall"
"github.com/gosuri/uitable"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/remote"
"github.com/roboll/helmfile/pkg/state"
"go.uber.org/zap"
"path/filepath"
"sort"
)
type App struct {
KubeContext string
Logger *zap.SugaredLogger
Reverse bool
Env string
Namespace string
Selectors []string
HelmBinary string
Args string
ValuesFiles []string
Set map[string]interface{}
FileOrDir string
ErrorHandler func(error) error
readFile func(string) ([]byte, error)
fileExists func(string) (bool, error)
glob func(string) ([]string, error)
abs func(string) (string, error)
fileExistsAt func(string) bool
directoryExistsAt func(string) bool
getwd func() (string, error)
chdir func(string) error
remote *remote.Remote
helmExecer helmexec.Interface
}
func New(conf ConfigProvider) *App {
return Init(&App{
KubeContext: conf.KubeContext(),
Logger: conf.Logger(),
Env: conf.Env(),
Namespace: conf.Namespace(),
Selectors: conf.Selectors(),
HelmBinary: conf.HelmBinary(),
Args: conf.Args(),
FileOrDir: conf.FileOrDir(),
ValuesFiles: conf.StateValuesFiles(),
Set: conf.StateValuesSet(),
helmExecer: helmexec.New(conf.Logger(), conf.KubeContext(), &helmexec.ShellRunner{
Logger: conf.Logger(),
}),
})
}
func Init(app *App) *App {
app.readFile = ioutil.ReadFile
app.glob = filepath.Glob
app.abs = filepath.Abs
app.getwd = os.Getwd
app.chdir = os.Chdir
app.fileExistsAt = fileExistsAt
app.fileExists = fileExists
app.directoryExistsAt = directoryExistsAt
return app
}
func (a *App) Deps(c DepsConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
return run.Deps(c)
})
}
func (a *App) Repos(c ReposConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
return run.Repos(c)
})
}
func (a *App) reverse() *App {
new := *a
new.Reverse = true
return &new
}
func (a *App) DeprecatedSyncCharts(c DeprecatedChartsConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
return run.DeprecatedSyncCharts(c)
})
}
func (a *App) Diff(c DiffConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
return run.Diff(c)
})
}
func (a *App) Template(c TemplateConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
return run.Template(c)
})
}
func (a *App) Lint(c LintConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
return run.Lint(c)
})
}
func (a *App) Sync(c SyncConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
return run.Sync(c)
})
}
func (a *App) Apply(c ApplyConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
return run.Apply(c)
})
}
func (a *App) Status(c StatusesConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
return run.Status(c)
})
}
func (a *App) Delete(c DeleteConfigProvider) error {
return a.reverse().ForEachState(func(run *Run) []error {
return run.Delete(c)
})
}
func (a *App) Destroy(c DestroyConfigProvider) error {
return a.reverse().ForEachState(func(run *Run) []error {
return run.Destroy(c)
})
}
func (a *App) Test(c TestConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
return run.Test(c)
})
}
func (a *App) PrintState(c StateConfigProvider) error {
return a.ForEachState(func(run *Run) []error {
state, err := run.state.ToYaml()
if err != nil {
return []error{err}
}
fmt.Printf("---\n# Source: %s\n\n%+v", run.state.FilePath, state)
return []error{}
})
}
func (a *App) ListReleases(c StateConfigProvider) error {
table := uitable.New()
table.AddRow("NAME", "NAMESPACE", "INSTALLED", "LABELS")
err := a.ForEachState(func(run *Run) []error {
//var releases m
for _, r := range run.state.Releases {
labels := ""
for k, v := range r.Labels {
labels = fmt.Sprintf("%s,%s:%s", labels, k, v)
}
installed := r.Installed == nil || *r.Installed
table.AddRow(r.Name, r.Namespace, fmt.Sprintf("%t", installed), strings.Trim(labels, ","))
}
return []error{}
})
fmt.Println(table.String())
return err
}
func (a *App) within(dir string, do func() error) error {
if dir == "." {
return do()
}
prev, err := a.getwd()
if err != nil {
return fmt.Errorf("failed getting current working direcotyr: %v", err)
}
absDir, err := a.abs(dir)
if err != nil {
return err
}
a.Logger.Debugf("changing working directory to \"%s\"", absDir)
if err := a.chdir(absDir); err != nil {
return fmt.Errorf("failed changing working directory to \"%s\": %v", absDir, err)
}
appErr := do()
a.Logger.Debugf("changing working directory back to \"%s\"", prev)
if chdirBackErr := a.chdir(prev); chdirBackErr != nil {
if appErr != nil {
a.Logger.Warnf("%v", appErr)
}
return fmt.Errorf("failed chaging working directory back to \"%s\": %v", prev, chdirBackErr)
}
return appErr
}
func (a *App) visitStateFiles(fileOrDir string, do func(string, string) error) error {
desiredStateFiles, err := a.findDesiredStateFiles(fileOrDir)
if err != nil {
return appError("", err)
}
for _, relPath := range desiredStateFiles {
var file string
var dir string
if a.directoryExistsAt(relPath) {
file = relPath
dir = relPath
} else {
file = filepath.Base(relPath)
dir = filepath.Dir(relPath)
}
a.Logger.Debugf("processing file \"%s\" in directory \"%s\"", file, dir)
err := a.within(dir, func() error {
absd, err := a.abs(dir)
if err != nil {
return err
}
return do(file, absd)
})
if err != nil {
return appError(fmt.Sprintf("in %s/%s", dir, file), err)
}
}
return nil
}
func (a *App) loadDesiredStateFromYaml(file string, opts ...LoadOpts) (*state.HelmState, error) {
ld := &desiredStateLoader{
readFile: a.readFile,
fileExists: a.fileExists,
env: a.Env,
namespace: a.Namespace,
logger: a.Logger,
abs: a.abs,
Reverse: a.Reverse,
KubeContext: a.KubeContext,
glob: a.glob,
helm: a.helmExecer,
}
var op LoadOpts
if len(opts) > 0 {
op = opts[0]
}
return ld.Load(file, op)
}
func (a *App) visitStates(fileOrDir string, defOpts LoadOpts, converge func(*state.HelmState, helmexec.Interface) (bool, []error)) error {
noMatchInHelmfiles := true
err := a.visitStateFiles(fileOrDir, func(f, d string) error {
opts := defOpts.DeepCopy()
if opts.CalleePath == "" {
opts.CalleePath = f
}
st, err := a.loadDesiredStateFromYaml(f, opts)
sigs := make(chan os.Signal, 1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
go func() {
sig := <-sigs
errs := []error{fmt.Errorf("Received [%s] to shutdown ", sig)}
_ = context{a, st}.clean(errs)
// See http://tldp.org/LDP/abs/html/exitcodes.html
switch sig {
case syscall.SIGINT:
os.Exit(130)
case syscall.SIGTERM:
os.Exit(143)
}
}()
ctx := context{a, st}
helm := a.helmExecer
if err != nil {
switch stateLoadErr := err.(type) {
// Addresses https://github.com/roboll/helmfile/issues/279
case *state.StateLoadError:
switch stateLoadErr.Cause.(type) {
case *state.UndefinedEnvError:
return nil
default:
return ctx.wrapErrs(err)
}
default:
return ctx.wrapErrs(err)
}
}
st.Selectors = opts.Selectors
if len(st.Helmfiles) > 0 {
noMatchInSubHelmfiles := true
for i, m := range st.Helmfiles {
optsForNestedState := LoadOpts{
CalleePath: filepath.Join(d, f),
Environment: m.Environment,
}
//assign parent selector to sub helm selector in legacy mode or do not inherit in experimental mode
if (m.Selectors == nil && !isExplicitSelectorInheritanceEnabled()) || m.SelectorsInherited {
optsForNestedState.Selectors = opts.Selectors
} else {
optsForNestedState.Selectors = m.Selectors
}
if err := a.visitStates(m.Path, optsForNestedState, converge); err != nil {
switch err.(type) {
case *NoMatchingHelmfileError:
default:
return appError(fmt.Sprintf("in .helmfiles[%d]", i), err)
}
} else {
noMatchInSubHelmfiles = false
}
}
noMatchInHelmfiles = noMatchInHelmfiles && noMatchInSubHelmfiles
}
templated, tmplErr := st.ExecuteTemplates()
if tmplErr != nil {
return appError(fmt.Sprintf("failed executing release templates in \"%s\"", f), tmplErr)
}
processed, errs := converge(templated, helm)
noMatchInHelmfiles = noMatchInHelmfiles && !processed
return context{a, templated}.clean(errs)
})
if err != nil {
return err
}
if noMatchInHelmfiles {
return &NoMatchingHelmfileError{selectors: a.Selectors, env: a.Env}
}
return nil
}
func (a *App) ForEachState(do func(*Run) []error) error {
ctx := NewContext()
err := a.VisitDesiredStatesWithReleasesFiltered(a.FileOrDir, func(st *state.HelmState, helm helmexec.Interface) []error {
run := NewRun(st, helm, ctx)
return do(run)
})
if err != nil && a.ErrorHandler != nil {
return a.ErrorHandler(err)
}
return err
}
func (a *App) VisitDesiredStatesWithReleasesFiltered(fileOrDir string, converge func(*state.HelmState, helmexec.Interface) []error) error {
opts := LoadOpts{
Selectors: a.Selectors,
}
envvals := []interface{}{}
if a.ValuesFiles != nil {
for i := range a.ValuesFiles {
envvals = append(envvals, a.ValuesFiles[i])
}
}
if a.Set != nil {
envvals = append(envvals, a.Set)
}
if len(envvals) > 0 {
opts.Environment.OverrideValues = envvals
}
dir, err := a.getwd()
if err != nil {
return err
}
getter := &remote.GoGetter{Logger: a.Logger}
remote := &remote.Remote{
Logger: a.Logger,
Home: dir,
Getter: getter,
ReadFile: a.readFile,
DirExists: a.directoryExistsAt,
FileExists: a.fileExistsAt,
}
a.remote = remote
return a.visitStates(fileOrDir, opts, func(st *state.HelmState, helm helmexec.Interface) (bool, []error) {
if len(st.Selectors) > 0 {
err := st.FilterReleases()
if err != nil {
return false, []error{err}
}
}
if a.HelmBinary != "" {
helm.SetHelmBinary(a.HelmBinary)
}
type Key struct {
TillerNamespace, Name string
}
releaseNameCounts := map[Key]int{}
for _, r := range st.Releases {
tillerNamespace := st.HelmDefaults.TillerNamespace
if r.TillerNamespace != "" {
tillerNamespace = r.TillerNamespace
}
releaseNameCounts[Key{tillerNamespace, r.Name}]++
}
for name, c := range releaseNameCounts {
if c > 1 {
return false, []error{fmt.Errorf("duplicate release \"%s\" found in \"%s\": there were %d releases named \"%s\" matching specified selector", name.Name, name.TillerNamespace, c, name.Name)}
}
}
errs := converge(st, helm)
processed := len(st.Releases) != 0 && len(errs) == 0
return processed, errs
})
}
func (a *App) findStateFilesInAbsPaths(specifiedPath string) ([]string, error) {
rels, err := a.findDesiredStateFiles(specifiedPath)
if err != nil {
return rels, err
}
files := make([]string, len(rels))
for i := range rels {
files[i], err = filepath.Abs(rels[i])
if err != nil {
return []string{}, err
}
}
return files, nil
}
func (a *App) findDesiredStateFiles(specifiedPath string) ([]string, error) {
path, err := a.remote.Locate(specifiedPath)
if err != nil {
return nil, fmt.Errorf("locate: %v", err)
}
if specifiedPath != path {
a.Logger.Debugf("fetched remote \"%s\" to local cache \"%s\" and loading the latter...", specifiedPath, path)
}
specifiedPath = path
var helmfileDir string
if specifiedPath != "" {
if a.fileExistsAt(specifiedPath) {
return []string{specifiedPath}, nil
} else if a.directoryExistsAt(specifiedPath) {
helmfileDir = specifiedPath
} else {
return []string{}, fmt.Errorf("specified state file %s is not found", specifiedPath)
}
} else {
var defaultFile string
if a.fileExistsAt(DefaultHelmfile) {
defaultFile = DefaultHelmfile
} else if a.fileExistsAt(DeprecatedHelmfile) {
log.Printf(
"warn: %s is being loaded: %s is deprecated in favor of %s. See https://github.com/roboll/helmfile/issues/25 for more information",
DeprecatedHelmfile,
DeprecatedHelmfile,
DefaultHelmfile,
)
defaultFile = DeprecatedHelmfile
}
if a.directoryExistsAt(DefaultHelmfileDirectory) {
if defaultFile != "" {
return []string{}, fmt.Errorf("configuration conlict error: you can have either %s or %s, but not both", defaultFile, DefaultHelmfileDirectory)
}
helmfileDir = DefaultHelmfileDirectory
} else if defaultFile != "" {
return []string{defaultFile}, nil
} else {
return []string{}, fmt.Errorf("no state file found. It must be named %s/*.{yaml,yml}, %s, or %s, or otherwise specified with the --file flag", DefaultHelmfileDirectory, DefaultHelmfile, DeprecatedHelmfile)
}
}
files, err := a.glob(filepath.Join(helmfileDir, "*.y*ml"))
if err != nil {
return []string{}, err
}
sort.Slice(files, func(i, j int) bool {
return files[i] < files[j]
})
return files, nil
}
func fileExistsAt(path string) bool {
fileInfo, err := os.Stat(path)
return err == nil && fileInfo.Mode().IsRegular()
}
func fileExists(path string) (bool, error) {
_, err := os.Stat(path)
if err != nil {
if os.IsNotExist(err) {
return false, nil
}
return false, err
}
return true, nil
}
func directoryExistsAt(path string) bool {
fileInfo, err := os.Stat(path)
return err == nil && fileInfo.Mode().IsDir()
}
type Error struct {
msg string
Errors []error
}
func (e *Error) Error() string {
var cause string
if e.Errors == nil {
return e.msg
}
if len(e.Errors) == 1 {
if e.Errors[0] == nil {
panic(fmt.Sprintf("[bug] assertion error: unexpected state: e.Errors: %v", e.Errors))
}
cause = e.Errors[0].Error()
} else {
msgs := []string{}
for i, err := range e.Errors {
if err == nil {
continue
}
msgs = append(msgs, fmt.Sprintf("err %d: %v", i, err.Error()))
}
cause = fmt.Sprintf("%d errors:\n%s", len(e.Errors), strings.Join(msgs, "\n"))
}
msg := ""
if e.msg != "" {
msg = fmt.Sprintf("%s: %s", e.msg, cause)
} else {
msg = cause
}
return msg
}
func (e *Error) Code() int {
allDiff := false
anyNonZero := false
for _, err := range e.Errors {
switch ee := err.(type) {
case *state.ReleaseError:
if anyNonZero {
allDiff = allDiff && ee.Code == 2
} else {
allDiff = ee.Code == 2
}
case *Error:
if anyNonZero {
allDiff = allDiff && ee.Code() == 2
} else {
allDiff = ee.Code() == 2
}
}
anyNonZero = true
}
if anyNonZero {
if allDiff {
return 2
}
return 1
}
panic(fmt.Sprintf("[bug] assertion error: unexpected state: unable to handle errors: %v", e.Errors))
}
func appError(msg string, err error) error {
return &Error{msg, []error{err}}
}
func (c context) clean(errs []error) error {
if errs == nil {
errs = []error{}
}
cleanErrs := c.st.Clean()
if cleanErrs != nil {
errs = append(errs, cleanErrs...)
}
return c.wrapErrs(errs...)
}
type context struct {
app *App
st *state.HelmState
}
func (c context) wrapErrs(errs ...error) error {
if errs != nil && len(errs) > 0 {
for _, err := range errs {
switch e := err.(type) {
case *state.ReleaseError:
c.app.Logger.Debugf("err: release \"%s\" in \"%s\" failed: %v", e.Name, c.st.FilePath, e)
default:
c.app.Logger.Debugf("err: %v", e)
}
}
return &Error{Errors: errs}
}
return nil
}
+2134
View File
File diff suppressed because it is too large Load Diff
+33
View File
@@ -0,0 +1,33 @@
package app
import (
"bufio"
"fmt"
"log"
"os"
"strings"
)
// Copyright (c) 2017 Roland Singer [roland.singer@desertbit.com]
//
// Shamelessly borrowed from @r0l1's awesome work that is available at https://gist.github.com/r0l1/3dcbb0c8f6cfe9c66ab8008f55f8f28b
func AskForConfirmation(s string) bool {
reader := bufio.NewReader(os.Stdin)
for {
fmt.Printf("%s [y/n]: ", s)
response, err := reader.ReadString('\n')
if err != nil {
log.Fatal(err)
}
response = strings.ToLower(strings.TrimSpace(response))
if response == "y" || response == "yes" {
return true
} else if response == "n" || response == "no" {
return false
}
}
}
+147
View File
@@ -0,0 +1,147 @@
package app
import "go.uber.org/zap"
type ConfigProvider interface {
Args() string
HelmBinary() string
FileOrDir() string
KubeContext() string
Namespace() string
Selectors() []string
StateValuesSet() map[string]interface{}
StateValuesFiles() []string
Env() string
loggingConfig
}
type DeprecatedChartsConfigProvider interface {
Values() []string
concurrencyConfig
loggingConfig
}
type DepsConfigProvider interface {
Args() string
SkipRepos() bool
}
type ReposConfigProvider interface {
Args() string
}
type ApplyConfigProvider interface {
Args() string
Values() []string
Set() []string
SkipDeps() bool
SuppressSecrets() bool
NoColor() bool
Context() int
concurrencyConfig
interactive
loggingConfig
}
type SyncConfigProvider interface {
Args() string
Values() []string
Set() []string
SkipDeps() bool
concurrencyConfig
loggingConfig
}
type DiffConfigProvider interface {
Args() string
Values() []string
Set() []string
SkipDeps() bool
SuppressSecrets() bool
DetailedExitcode() bool
NoColor() bool
Context() int
concurrencyConfig
}
type DeleteConfigProvider interface {
Args() string
Purge() bool
interactive
loggingConfig
concurrencyConfig
}
type DestroyConfigProvider interface {
Args() string
interactive
loggingConfig
concurrencyConfig
}
type TestConfigProvider interface {
Args() string
Timeout() int
Cleanup() bool
concurrencyConfig
}
type LintConfigProvider interface {
Args() string
Values() []string
Set() []string
SkipDeps() bool
concurrencyConfig
}
type TemplateConfigProvider interface {
Args() string
Values() []string
Set() []string
SkipDeps() bool
OutputDir() string
concurrencyConfig
}
type StatusesConfigProvider interface {
Args() string
concurrencyConfig
}
type StateConfigProvider interface {
}
type concurrencyConfig interface {
Concurrency() int
}
type loggingConfig interface {
Logger() *zap.SugaredLogger
}
type interactive interface {
Interactive() bool
}
+22
View File
@@ -0,0 +1,22 @@
package app
import (
"os"
"strings"
)
const (
DefaultHelmfile = "helmfile.yaml"
DeprecatedHelmfile = "charts.yaml"
DefaultHelmfileDirectory = "helmfile.d"
ExperimentalEnvVar = "HELMFILE_EXPERIMENTAL" // environment variable for experimental features, expecting "true" lower case
ExperimentalSelectorExplicit = "explicit-selector-inheritance" // value to remove default selector inheritance to sub-helmfiles and use the explicit one
)
func experimentalModeEnabled() bool {
return os.Getenv(ExperimentalEnvVar) == "true"
}
func isExplicitSelectorInheritanceEnabled() bool {
return experimentalModeEnabled() || strings.Contains(os.Getenv(ExperimentalEnvVar), ExperimentalSelectorExplicit)
}
+27
View File
@@ -0,0 +1,27 @@
package app
import (
"gotest.tools/assert"
is "gotest.tools/assert/cmp"
"gotest.tools/env"
"os"
"testing"
)
func TestIsExplicitSelectorInheritanceEnabled(t *testing.T) {
//env var ExperimentalEnvVar is set
assert.Assert(t, is.Equal(os.Getenv(ExperimentalEnvVar), ""))
assert.Check(t, !isExplicitSelectorInheritanceEnabled())
//check for env var ExperimentalEnvVar set to true
defer env.Patch(t, ExperimentalEnvVar, "true")()
assert.Check(t, isExplicitSelectorInheritanceEnabled())
//check for env var ExperimentalEnvVar set to anything
defer env.Patch(t, ExperimentalEnvVar, "foo")()
assert.Check(t, !isExplicitSelectorInheritanceEnabled())
//check for env var ExperimentalEnvVar set to ExperimentalSelectorExplicit
defer env.Patch(t, ExperimentalEnvVar, ExperimentalSelectorExplicit)()
assert.Check(t, isExplicitSelectorInheritanceEnabled())
}
+35
View File
@@ -0,0 +1,35 @@
package app
import (
"github.com/roboll/helmfile/pkg/state"
)
type Context struct {
updatedRepos map[string]struct{}
}
func NewContext() Context {
return Context{
updatedRepos: map[string]struct{}{},
}
}
func (ctx Context) SyncReposOnce(st *state.HelmState, helm state.RepoUpdater) []error {
var errs []error
allUpdated := true
for _, r := range st.Repositories {
_, exists := ctx.updatedRepos[r.Name]
allUpdated = allUpdated && exists
}
if !allUpdated {
errs = st.SyncRepos(helm)
for _, r := range st.Repositories {
ctx.updatedRepos[r.Name] = struct{}{}
}
}
return errs
}
+205
View File
@@ -0,0 +1,205 @@
package app
import (
"bytes"
"errors"
"fmt"
"path/filepath"
"sort"
"github.com/imdario/mergo"
"github.com/roboll/helmfile/pkg/environment"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/state"
"go.uber.org/zap"
)
type desiredStateLoader struct {
KubeContext string
Reverse bool
env string
namespace string
readFile func(string) ([]byte, error)
fileExists func(string) (bool, error)
abs func(string) (string, error)
glob func(string) ([]string, error)
logger *zap.SugaredLogger
helm helmexec.Interface
}
func (ld *desiredStateLoader) Load(f string, opts LoadOpts) (*state.HelmState, error) {
var overrodeEnv *environment.Environment
args := opts.Environment.OverrideValues
if len(args) > 0 {
if opts.CalleePath == "" {
return nil, fmt.Errorf("bug: opts.CalleePath was nil: f=%s, opts=%v", f, opts)
}
storage := state.NewStorage(opts.CalleePath, ld.logger, ld.glob)
envld := state.NewEnvironmentValuesLoader(storage, ld.readFile, ld.logger)
handler := state.MissingFileHandlerError
vals, err := envld.LoadEnvironmentValues(&handler, args)
if err != nil {
return nil, err
}
overrodeEnv = &environment.Environment{
Name: ld.env,
Values: vals,
}
}
st, err := ld.loadFileWithOverrides(nil, overrodeEnv, filepath.Dir(f), filepath.Base(f), true)
if err != nil {
return nil, err
}
if ld.Reverse {
rev := func(i, j int) bool {
return j < i
}
sort.Slice(st.Releases, rev)
sort.Slice(st.Helmfiles, rev)
}
if ld.KubeContext != "" {
if st.HelmDefaults.KubeContext != "" {
return nil, errors.New("err: Cannot use option --kube-context and set attribute helmDefaults.kubeContext.")
}
st.HelmDefaults.KubeContext = ld.KubeContext
}
if ld.namespace != "" {
if st.Namespace != "" {
return nil, errors.New("err: Cannot use option --namespace and set attribute namespace.")
}
st.Namespace = ld.namespace
}
return st, nil
}
func (ld *desiredStateLoader) loadFile(inheritedEnv *environment.Environment, baseDir, file string, evaluateBases bool) (*state.HelmState, error) {
return ld.loadFileWithOverrides(inheritedEnv, nil, baseDir, file, evaluateBases)
}
func (ld *desiredStateLoader) loadFileWithOverrides(inheritedEnv, overrodeEnv *environment.Environment, baseDir, file string, evaluateBases bool) (*state.HelmState, error) {
var f string
if filepath.IsAbs(file) {
f = file
} else {
f = filepath.Join(baseDir, file)
}
fileBytes, err := ld.readFile(f)
if err != nil {
return nil, err
}
ext := filepath.Ext(f)
var self *state.HelmState
if !experimentalModeEnabled() || ext == ".gotmpl" {
self, err = ld.renderAndLoad(
inheritedEnv,
overrodeEnv,
baseDir,
f,
fileBytes,
evaluateBases,
)
} else {
self, err = ld.load(
fileBytes,
baseDir,
file,
evaluateBases,
inheritedEnv,
overrodeEnv,
)
}
return self, err
}
func (a *desiredStateLoader) underlying() *state.StateCreator {
c := state.NewCreator(a.logger, a.readFile, a.fileExists, a.abs, a.glob, a.helm)
c.LoadFile = a.loadFile
return c
}
func (a *desiredStateLoader) load(yaml []byte, baseDir, file string, evaluateBases bool, env, overrodeEnv *environment.Environment) (*state.HelmState, error) {
merged, err := env.Merge(overrodeEnv)
if err != nil {
return nil, err
}
st, err := a.underlying().ParseAndLoad(yaml, baseDir, file, a.env, evaluateBases, merged)
if err != nil {
return nil, err
}
helmfiles, err := st.ExpandedHelmfiles()
if err != nil {
return nil, err
}
st.Helmfiles = helmfiles
return st, nil
}
func (ld *desiredStateLoader) renderAndLoad(env, overrodeEnv *environment.Environment, baseDir, filename string, content []byte, evaluateBases bool) (*state.HelmState, error) {
parts := bytes.Split(content, []byte("\n---\n"))
var finalState *state.HelmState
for i, part := range parts {
var yamlBuf *bytes.Buffer
var err error
id := fmt.Sprintf("%s.part.%d", filename, i)
if env == nil && overrodeEnv == nil {
yamlBuf, err = ld.renderTemplatesToYaml(baseDir, id, part)
if err != nil {
return nil, fmt.Errorf("error during %s parsing: %v", id, err)
}
} else {
yamlBuf, err = ld.renderTemplatesToYamlWithEnv(baseDir, id, part, env, overrodeEnv)
if err != nil {
return nil, fmt.Errorf("error during %s parsing: %v", id, err)
}
}
currentState, err := ld.load(
yamlBuf.Bytes(),
baseDir,
filename,
evaluateBases,
env,
overrodeEnv,
)
if err != nil {
return nil, err
}
if finalState == nil {
finalState = currentState
} else {
if err := mergo.Merge(finalState, currentState, mergo.WithOverride); err != nil {
return nil, err
}
}
env = &finalState.Env
ld.logger.Debugf("merged environment: %v", env)
}
return finalState, nil
}
+19
View File
@@ -0,0 +1,19 @@
package app
import (
"fmt"
"strings"
)
type NoMatchingHelmfileError struct {
selectors []string
env string
}
func (e *NoMatchingHelmfileError) Error() string {
return fmt.Sprintf(
"err: no releases found that matches specified selector(%s) and environment(%s), in any helmfile",
strings.Join(e.selectors, ", "),
e.env,
)
}
+28
View File
@@ -0,0 +1,28 @@
package app
import (
"github.com/roboll/helmfile/pkg/state"
"gopkg.in/yaml.v2"
)
type LoadOpts struct {
Selectors []string
Environment state.SubhelmfileEnvironmentSpec
// CalleePath is the absolute path to the file being loaded
CalleePath string
}
func (o LoadOpts) DeepCopy() LoadOpts {
bytes, err := yaml.Marshal(o)
if err != nil {
panic(err)
}
new := LoadOpts{}
if err := yaml.Unmarshal(bytes, &new); err != nil {
panic(err)
}
return new
}
+332
View File
@@ -0,0 +1,332 @@
package app
import (
"fmt"
"strings"
"github.com/roboll/helmfile/pkg/argparser"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/state"
)
type Run struct {
state *state.HelmState
helm helmexec.Interface
ctx Context
Ask func(string) bool
}
func NewRun(st *state.HelmState, helm helmexec.Interface, ctx Context) *Run {
return &Run{state: st, helm: helm, ctx: ctx}
}
func (r *Run) askForConfirmation(msg string) bool {
if r.Ask != nil {
return r.Ask(msg)
}
return AskForConfirmation(msg)
}
func (r *Run) Deps(c DepsConfigProvider) []error {
r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
if !c.SkipRepos() {
if errs := r.ctx.SyncReposOnce(r.state, r.helm); errs != nil && len(errs) > 0 {
return errs
}
}
return r.state.UpdateDeps(r.helm)
}
func (r *Run) Repos(c ReposConfigProvider) []error {
r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
return r.ctx.SyncReposOnce(r.state, r.helm)
}
func (r *Run) DeprecatedSyncCharts(c DeprecatedChartsConfigProvider) []error {
st := r.state
helm := r.helm
affectedReleases := state.AffectedReleases{}
errs := st.SyncReleases(&affectedReleases, helm, c.Values(), c.Concurrency())
affectedReleases.DisplayAffectedReleases(c.Logger())
return errs
}
func (r *Run) Status(c StatusesConfigProvider) []error {
workers := c.Concurrency()
r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
return r.state.ReleaseStatuses(r.helm, workers)
}
func (r *Run) Delete(c DeleteConfigProvider) []error {
affectedReleases := state.AffectedReleases{}
purge := c.Purge()
errs := []error{}
names := make([]string, len(r.state.Releases))
for i, r := range r.state.Releases {
names[i] = fmt.Sprintf(" %s (%s)", r.Name, r.Chart)
}
msg := fmt.Sprintf(`Affected releases are:
%s
Do you really want to delete?
Helmfile will delete all your releases, as shown above.
`, strings.Join(names, "\n"))
interactive := c.Interactive()
if !interactive || interactive && r.askForConfirmation(msg) {
r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
errs = r.state.DeleteReleases(&affectedReleases, r.helm, c.Concurrency(), purge)
}
affectedReleases.DisplayAffectedReleases(c.Logger())
return errs
}
func (r *Run) Destroy(c DestroyConfigProvider) []error {
errs := []error{}
affectedReleases := state.AffectedReleases{}
names := make([]string, len(r.state.Releases))
for i, r := range r.state.Releases {
names[i] = fmt.Sprintf(" %s (%s)", r.Name, r.Chart)
}
msg := fmt.Sprintf(`Affected releases are:
%s
Do you really want to delete?
Helmfile will delete all your releases, as shown above.
`, strings.Join(names, "\n"))
interactive := c.Interactive()
if !interactive || interactive && r.askForConfirmation(msg) {
r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
errs = r.state.DeleteReleases(&affectedReleases, r.helm, c.Concurrency(), true)
}
affectedReleases.DisplayAffectedReleases(c.Logger())
return errs
}
func (r *Run) Apply(c ApplyConfigProvider) []error {
st := r.state
helm := r.helm
ctx := r.ctx
affectedReleases := state.AffectedReleases{}
if !c.SkipDeps() {
if errs := ctx.SyncReposOnce(st, helm); errs != nil && len(errs) > 0 {
return errs
}
if errs := st.BuildDeps(helm); errs != nil && len(errs) > 0 {
return errs
}
}
if errs := st.PrepareReleases(helm, "apply"); errs != nil && len(errs) > 0 {
return errs
}
// helm must be 2.11+ and helm-diff should be provided `--detailed-exitcode` in order for `helmfile apply` to work properly
detailedExitCode := true
diffOpts := &state.DiffOpts{
NoColor: c.NoColor(),
Context: c.Context(),
Set: c.Set(),
}
releases, errs := st.DiffReleases(helm, c.Values(), c.Concurrency(), detailedExitCode, c.SuppressSecrets(), false, diffOpts)
releasesToBeDeleted, err := st.DetectReleasesToBeDeleted(helm)
if err != nil {
errs = append(errs, err)
}
fatalErrs := []error{}
noError := true
for _, e := range errs {
switch err := e.(type) {
case *state.ReleaseError:
if err.Code != 2 {
noError = false
fatalErrs = append(fatalErrs, e)
}
default:
noError = false
fatalErrs = append(fatalErrs, e)
}
}
// sync only when there are changes
if noError {
if len(releases) == 0 && len(releasesToBeDeleted) == 0 {
// TODO better way to get the logger
logger := c.Logger()
logger.Infof("")
logger.Infof("No affected releases")
} else {
names := []string{}
for _, r := range releases {
names = append(names, fmt.Sprintf(" %s (%s) UPDATED", r.Name, r.Chart))
}
for _, r := range releasesToBeDeleted {
names = append(names, fmt.Sprintf(" %s (%s) DELETED", r.Name, r.Chart))
}
msg := fmt.Sprintf(`Affected releases are:
%s
Do you really want to apply?
Helmfile will apply all your changes, as shown above.
`, strings.Join(names, "\n"))
interactive := c.Interactive()
if !interactive || interactive && r.askForConfirmation(msg) {
rs := []state.ReleaseSpec{}
for _, r := range releases {
rs = append(rs, *r)
}
for _, r := range releasesToBeDeleted {
rs = append(rs, *r)
}
r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
st.Releases = rs
syncOpts := &state.SyncOpts{
Set: c.Set(),
}
return st.SyncReleases(&affectedReleases, helm, c.Values(), c.Concurrency(), syncOpts)
}
}
}
affectedReleases.DisplayAffectedReleases(c.Logger())
return fatalErrs
}
func (r *Run) Diff(c DiffConfigProvider) []error {
st := r.state
helm := r.helm
ctx := r.ctx
if !c.SkipDeps() {
if errs := ctx.SyncReposOnce(st, helm); errs != nil && len(errs) > 0 {
return errs
}
if errs := st.BuildDeps(helm); errs != nil && len(errs) > 0 {
return errs
}
}
if errs := st.PrepareReleases(helm, "diff"); errs != nil && len(errs) > 0 {
return errs
}
r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
opts := &state.DiffOpts{
Context: c.Context(),
NoColor: c.NoColor(),
Set: c.Set(),
}
_, errs := st.DiffReleases(helm, c.Values(), c.Concurrency(), c.DetailedExitcode(), c.SuppressSecrets(), true, opts)
return errs
}
func (r *Run) Sync(c SyncConfigProvider) []error {
st := r.state
helm := r.helm
ctx := r.ctx
affectedReleases := state.AffectedReleases{}
if !c.SkipDeps() {
if errs := ctx.SyncReposOnce(st, helm); errs != nil && len(errs) > 0 {
return errs
}
if errs := st.BuildDeps(helm); errs != nil && len(errs) > 0 {
return errs
}
}
if errs := st.PrepareReleases(helm, "sync"); errs != nil && len(errs) > 0 {
return errs
}
r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
opts := &state.SyncOpts{
Set: c.Set(),
}
errs := st.SyncReleases(&affectedReleases, helm, c.Values(), c.Concurrency(), opts)
affectedReleases.DisplayAffectedReleases(c.Logger())
return errs
}
func (r *Run) Template(c TemplateConfigProvider) []error {
st := r.state
helm := r.helm
ctx := r.ctx
if !c.SkipDeps() {
if errs := ctx.SyncReposOnce(st, helm); errs != nil && len(errs) > 0 {
return errs
}
if errs := st.BuildDeps(helm); errs != nil && len(errs) > 0 {
return errs
}
}
if errs := st.PrepareReleases(helm, "template"); errs != nil && len(errs) > 0 {
return errs
}
args := argparser.GetArgs(c.Args(), st)
opts := &state.TemplateOpts{
Set: c.Set(),
}
return st.TemplateReleases(helm, c.OutputDir(), c.Values(), args, c.Concurrency(), opts)
}
func (r *Run) Test(c TestConfigProvider) []error {
cleanup := c.Cleanup()
timeout := c.Timeout()
concurrency := c.Concurrency()
r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
return r.state.TestReleases(r.helm, cleanup, timeout, concurrency)
}
func (r *Run) Lint(c LintConfigProvider) []error {
st := r.state
helm := r.helm
ctx := r.ctx
values := c.Values()
args := argparser.GetArgs(c.Args(), st)
workers := c.Concurrency()
if !c.SkipDeps() {
if errs := ctx.SyncReposOnce(st, helm); errs != nil && len(errs) > 0 {
return errs
}
if errs := st.BuildDeps(helm); errs != nil && len(errs) > 0 {
return errs
}
}
if errs := st.PrepareReleases(helm, "lint"); errs != nil && len(errs) > 0 {
return errs
}
opts := &state.LintOpts{
Set: c.Set(),
}
return st.LintReleases(helm, values, args, workers, opts)
}
+145
View File
@@ -0,0 +1,145 @@
package app
import (
"bytes"
"fmt"
"github.com/google/go-cmp/cmp"
"github.com/roboll/helmfile/pkg/environment"
"github.com/roboll/helmfile/pkg/state"
"github.com/roboll/helmfile/pkg/tmpl"
"strings"
)
func prependLineNumbers(text string) string {
buf := bytes.NewBufferString("")
lines := strings.Split(text, "\n")
for i, line := range lines {
buf.WriteString(fmt.Sprintf("%2d: %s\n", i, line))
}
return buf.String()
}
func (r *desiredStateLoader) renderPrestate(firstPassEnv *environment.Environment, baseDir, filename string, content []byte) (*environment.Environment, *state.HelmState) {
tmplData := state.EnvironmentTemplateData{
Environment: *firstPassEnv,
Namespace: r.namespace,
Values: map[string]interface{}{},
}
firstPassRenderer := tmpl.NewFirstPassRenderer(baseDir, tmplData)
// parse as much as we can, tolerate errors, this is a preparse
yamlBuf, err := firstPassRenderer.RenderTemplateContentToBuffer(content)
if err != nil && r.logger != nil {
r.logger.Debugf("first-pass rendering input of \"%s\":\n%s", filename, prependLineNumbers(string(content)))
if yamlBuf == nil { // we have a template syntax error, let the second parse report
r.logger.Debugf("template syntax error: %v", err)
return firstPassEnv, nil
}
}
yamlData := yamlBuf.String()
// Work-around for https://github.com/golang/go/issues/24963
sanitized := strings.ReplaceAll(yamlData, "<no value>", "")
if len(yamlData) != len(sanitized) {
msg := "replaced <no value>s to workaround https://github.com/golang/go/issues/24963 to address https://github.com/roboll/helmfile/issues/553:\n%s"
r.logger.Debugf(msg, cmp.Diff(yamlData, sanitized))
}
c := r.underlying()
c.Strict = false
// create preliminary state, as we may have an environment. Tolerate errors.
prestate, err := c.ParseAndLoad([]byte(sanitized), baseDir, filename, r.env, false, firstPassEnv)
if err != nil && r.logger != nil {
switch err.(type) {
case *state.StateLoadError:
r.logger.Infof("could not deduce `environment:` block, configuring only .Environment.Name. error: %v", err)
}
r.logger.Debugf("error in first-pass rendering: result of \"%s\":\n%s", filename, prependLineNumbers(yamlBuf.String()))
}
if prestate != nil {
firstPassEnv = &prestate.Env
}
return firstPassEnv, prestate
}
type RenderOpts struct {
}
func (r *desiredStateLoader) renderTemplatesToYaml(baseDir, filename string, content []byte) (*bytes.Buffer, error) {
env := &environment.Environment{Name: r.env, Values: map[string]interface{}(nil)}
return r.renderTemplatesToYamlWithEnv(baseDir, filename, content, env, nil)
}
func (r *desiredStateLoader) renderTemplatesToYamlWithEnv(baseDir, filename string, content []byte, inherited, overrode *environment.Environment) (*bytes.Buffer, error) {
return r.twoPassRenderTemplateToYaml(inherited, overrode, baseDir, filename, content)
}
func (r *desiredStateLoader) twoPassRenderTemplateToYaml(inherited, overrode *environment.Environment, baseDir, filename string, content []byte) (*bytes.Buffer, error) {
// try a first pass render. This will always succeed, but can produce a limited env
if r.logger != nil {
r.logger.Debugf("first-pass rendering starting for \"%s\": inherited=%v, overrode=%v", filename, inherited, overrode)
}
initEnv, err := inherited.Merge(overrode)
if err != nil {
return nil, err
}
if r.logger != nil {
r.logger.Debugf("first-pass uses: %v", initEnv)
}
renderedEnv, prestate := r.renderPrestate(initEnv, baseDir, filename, content)
if r.logger != nil {
r.logger.Debugf("first-pass produced: %v", renderedEnv)
}
finalEnv, err := inherited.Merge(renderedEnv)
if err != nil {
return nil, err
}
finalEnv, err = finalEnv.Merge(overrode)
if err != nil {
return nil, err
}
if r.logger != nil {
r.logger.Debugf("first-pass rendering result of \"%s\": %v", filename, *finalEnv)
}
vals := map[string]interface{}{}
if prestate != nil {
prestate.Env = *finalEnv
vals, err = prestate.Values()
if err != nil {
return nil, err
}
}
if prestate != nil {
r.logger.Debugf("vals:\n%v\ndefaultVals:%v", vals, prestate.DefaultValues)
}
tmplData := state.EnvironmentTemplateData{
Environment: *finalEnv,
Namespace: r.namespace,
Values: vals,
}
secondPassRenderer := tmpl.NewFileRenderer(r.readFile, baseDir, tmplData)
yamlBuf, err := secondPassRenderer.RenderTemplateContentToBuffer(content)
if err != nil {
if r.logger != nil {
r.logger.Debugf("second-pass rendering failed, input of \"%s\":\n%s", filename, prependLineNumbers(string(content)))
}
return nil, err
}
if r.logger != nil {
r.logger.Debugf("second-pass rendering result of \"%s\":\n%s", filename, prependLineNumbers(yamlBuf.String()))
}
return yamlBuf, nil
}
+234
View File
@@ -0,0 +1,234 @@
package app
import (
"os"
"strings"
"testing"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/state"
"github.com/roboll/helmfile/pkg/testhelper"
"gopkg.in/yaml.v2"
)
func makeLoader(files map[string]string, env string) (*desiredStateLoader, *testhelper.TestFs) {
testfs := testhelper.NewTestFs(files)
return &desiredStateLoader{
env: env,
namespace: "namespace",
logger: helmexec.NewLogger(os.Stdout, "debug"),
readFile: testfs.ReadFile,
fileExists: testfs.FileExists,
abs: testfs.Abs,
glob: testfs.Glob,
}, testfs
}
func TestReadFromYaml_MakeEnvironmentHasNoSideEffects(t *testing.T) {
yamlContent := []byte(`
environments:
staging:
values:
- default/values.yaml
production:
releases:
- name: {{ readFile "other/default/values.yaml" }}
chart: mychart1
`)
files := map[string]string{
"/path/to/default/values.yaml": ``,
"/path/to/other/default/values.yaml": `SecondPass`,
}
r, testfs := makeLoader(files, "staging")
yamlBuf, err := r.renderTemplatesToYaml("", "", yamlContent)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
var state state.HelmState
err = yaml.Unmarshal(yamlBuf.Bytes(), &state)
if testfs.FileReaderCalls() > 2 {
t.Error("reader should be called only twice")
}
if state.Releases[0].Name != "SecondPass" {
t.Errorf("release name should have ben set as SecondPass")
}
}
func TestReadFromYaml_RenderTemplate(t *testing.T) {
defaultValuesYaml := `
releaseName: "hello"
conditionalReleaseTag: "yes"
`
yamlContent := []byte(`
environments:
staging:
values:
- default/values.yaml
production:
releases:
- name: {{ .Environment.Values.releaseName }}
chart: mychart1
{{ if (eq .Environment.Values.conditionalReleaseTag "yes") }}
- name: conditionalRelease
{{ end }}
`)
files := map[string]string{
"/path/to/default/values.yaml": defaultValuesYaml,
}
r, _ := makeLoader(files, "staging")
// test the double rendering
yamlBuf, err := r.renderTemplatesToYaml("", "", yamlContent)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
var state state.HelmState
err = yaml.Unmarshal(yamlBuf.Bytes(), &state)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if len(state.Releases) != 2 {
t.Fatal("there should be 2 releases")
}
if state.Releases[0].Name != "hello" {
t.Errorf("release name should be hello")
}
if state.Releases[1].Name != "conditionalRelease" {
t.Error("conditional release should have been present")
}
}
func TestReadFromYaml_RenderTemplateWithValuesReferenceError(t *testing.T) {
defaultValuesYaml := ``
yamlContent := []byte(`
environments:
staging:
values:
- default/values.yaml
production:
{{ if (eq .Environment.Values.releaseName "a") }} # line 8
releases:
- name: a
chart: mychart1
{{ end }}
`)
files := map[string]string{
"/path/to/default/values.yaml": defaultValuesYaml,
}
r, _ := makeLoader(files, "staging")
// test the double rendering
_, err := r.renderTemplatesToYaml("", "", yamlContent)
if !strings.Contains(err.Error(), "stringTemplate:8") {
t.Fatalf("error should contain a stringTemplate error (reference to unknow key) %v", err)
}
}
// This test shows that a gotmpl reference will get rendered correctly
// even if the pre-render disables the readFile and exec functions.
// This does not apply to .gotmpl files, which is a nice side-effect.
func TestReadFromYaml_RenderTemplateWithGotmpl(t *testing.T) {
defaultValuesYamlGotmpl := `
releaseName: {{ readFile "nonIgnoredFile" }}
`
yamlContent := []byte(`
environments:
staging:
values:
- values.yaml.gotmpl
production:
{{ if (eq .Environment.Values.releaseName "release-a") }} # line 8
releases:
- name: a
chart: mychart1
{{ end }}
`)
files := map[string]string{
"/path/to/nonIgnoredFile": `release-a`,
"/path/to/values.yaml.gotmpl": defaultValuesYamlGotmpl,
}
r, _ := makeLoader(files, "staging")
rendered, _ := r.renderTemplatesToYaml("", "", yamlContent)
var state state.HelmState
yaml.Unmarshal(rendered.Bytes(), &state)
if len(state.Releases) != 1 {
t.Fatal("there should be 1 release")
}
if state.Releases[0].Name != "a" {
t.Fatal("release should have been declared")
}
}
func TestReadFromYaml_RenderTemplateWithNamespace(t *testing.T) {
yamlContent := []byte(`releases:
- name: {{ .Namespace }}-myrelease
chart: mychart
`)
files := map[string]string{}
r, _ := makeLoader(files, "staging")
yamlBuf, err := r.renderTemplatesToYaml("", "", yamlContent)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
var state state.HelmState
err = yaml.Unmarshal(yamlBuf.Bytes(), &state)
if state.Releases[0].Name != "namespace-myrelease" {
t.Errorf("release name should be namespace-myrelease")
}
}
func TestReadFromYaml_HelmfileShouldBeResilentToTemplateErrors(t *testing.T) {
yamlContent := []byte(`environments:
staging:
production:
releases:
{{ if (eq .Environment.Name "production" }} # notice syntax error: unclosed left paren
- name: prod-myrelease
{{ else }}
- name: myapp
{{ end }}
chart: mychart
`)
r, _ := makeLoader(map[string]string{}, "staging")
_, err := r.renderTemplatesToYaml("", "", yamlContent)
if err == nil {
t.Fatalf("wanted error, none returned")
}
}
+107
View File
@@ -0,0 +1,107 @@
package argparser
import (
"fmt"
"github.com/roboll/helmfile/pkg/state"
"strings"
)
type keyVal struct {
key string
val string
spaceFlag bool
}
type argMap struct {
//m map[string]string
m map[string][]*keyVal
flags []string
}
func (a *argMap) SetArg(flag, arg string, isSpace bool) {
if _, exists := a.m[flag]; !exists {
keyarg := &keyVal{key: flag, val: arg, spaceFlag: isSpace}
a.m[flag] = append(a.m[flag], keyarg)
a.flags = append(a.flags, flag)
} else if flag == "--set" || flag == "-f" || flag == "--values" {
keyarg := &keyVal{key: flag, val: arg, spaceFlag: isSpace}
a.m[flag] = append(a.m[flag], keyarg)
}
}
func newArgMap() *argMap {
return &argMap{m: map[string][]*keyVal{}}
}
func GetArgs(args string, state *state.HelmState) []string {
argsMap := newArgMap()
if len(args) > 0 {
argsVals := strings.Split(args, " ")
for index, arg := range argsVals {
if strings.HasPrefix(arg, "--") {
argVal := strings.SplitN(arg, "=", 2)
if len(argVal) > 1 {
arg := argVal[0]
value := argVal[1]
argsMap.SetArg(arg, value, false)
} else {
//check if next value is arg to flag
if index+1 < len(argsVals) {
nextVal := argsVals[index+1]
if strings.HasPrefix(nextVal, "--") {
argsMap.SetArg(arg, "", false)
} else {
argsMap.SetArg(arg, nextVal, true)
}
} else {
argsMap.SetArg(arg, "", false)
}
}
}
}
}
if len(state.HelmDefaults.Args) > 0 {
for _, arg := range state.HelmDefaults.Args {
var flag string
var val string
argsNum, _ := fmt.Sscanf(arg, "--%s %s", &flag, &val)
if argsNum == 2 {
argsMap.SetArg(flag, val, true)
} else {
argVal := strings.SplitN(arg, "=", 2)
argFirst := argVal[0]
if len(argVal) > 1 {
val = argVal[1]
argsMap.SetArg(argFirst, val, false)
} else {
argsMap.SetArg(argFirst, "", false)
}
}
}
}
var argArr []string
for _, flag := range argsMap.flags {
val := argsMap.m[flag]
for _, obj := range val {
if obj.val != "" {
if obj.spaceFlag {
argArr = append(argArr, obj.key, obj.val)
} else {
argArr = append(argArr, fmt.Sprintf("%s=%s", obj.key, obj.val))
}
} else {
argArr = append(argArr, fmt.Sprintf("%s", obj.key))
}
}
}
state.HelmDefaults.Args = argArr
return state.HelmDefaults.Args
}
+45
View File
@@ -0,0 +1,45 @@
package argparser
import (
"github.com/roboll/helmfile/pkg/state"
"strings"
"testing"
)
func TestGetArgs(t *testing.T) {
args := "--timeout=3600 --set app1.bootstrap=true --set app2.bootstrap=false --tiller-namespace ns"
defaultArgs := []string{"--recreate-pods", "--force"}
Helmdefaults := state.HelmSpec{KubeContext: "test", TillerNamespace: "test-namespace", Args: defaultArgs}
testState := &state.HelmState{HelmDefaults: Helmdefaults}
receivedArgs := GetArgs(args, testState)
expectedOutput := "--timeout=3600 --set app1.bootstrap=true --set app2.bootstrap=false --tiller-namespace ns --recreate-pods --force"
if compareArgs(expectedOutput, receivedArgs) == false {
t.Errorf("expected %s, got %s", expectedOutput, strings.Join(receivedArgs, " "))
}
}
func Test2(t *testing.T) {
args := "--timeout=3600 --set app1.bootstrap=true --set app2.bootstrap=false,app3.bootstrap=true --tiller-namespace ns"
defaultArgs := []string{"--recreate-pods", "--force"}
Helmdefaults := state.HelmSpec{KubeContext: "test", TillerNamespace: "test-namespace", Args: defaultArgs}
testState := &state.HelmState{HelmDefaults: Helmdefaults}
receivedArgs := GetArgs(args, testState)
expectedOutput := "--timeout=3600 --set app1.bootstrap=true --set app2.bootstrap=false,app3.bootstrap=true --tiller-namespace ns --recreate-pods --force"
if compareArgs(expectedOutput, receivedArgs) == false {
t.Errorf("expected %s, got %s", expectedOutput, strings.Join(receivedArgs, " "))
}
}
func compareArgs(expectedArgs string, args []string) bool {
if strings.Compare(strings.Join(args, " "), expectedArgs) != 0 {
return false
}
return true
}
+66
View File
@@ -0,0 +1,66 @@
package environment
import (
"github.com/imdario/mergo"
"github.com/roboll/helmfile/pkg/maputil"
"gopkg.in/yaml.v2"
)
type Environment struct {
Name string
Values map[string]interface{}
Defaults map[string]interface{}
}
var EmptyEnvironment Environment
func (e Environment) DeepCopy() Environment {
valuesBytes, err := yaml.Marshal(e.Values)
if err != nil {
panic(err)
}
var values map[string]interface{}
if err := yaml.Unmarshal(valuesBytes, &values); err != nil {
panic(err)
}
values, err = maputil.CastKeysToStrings(values)
if err != nil {
panic(err)
}
defaultsBytes, err := yaml.Marshal(e.Defaults)
if err != nil {
panic(err)
}
var defaults map[string]interface{}
if err := yaml.Unmarshal(defaultsBytes, &defaults); err != nil {
panic(err)
}
defaults, err = maputil.CastKeysToStrings(defaults)
if err != nil {
panic(err)
}
return Environment{
Name: e.Name,
Values: values,
Defaults: defaults,
}
}
func (e *Environment) Merge(other *Environment) (*Environment, error) {
if e == nil {
if other != nil {
copy := other.DeepCopy()
return &copy, nil
}
return nil, nil
}
copy := e.DeepCopy()
if other != nil {
if err := mergo.Merge(&copy, other, mergo.WithOverride); err != nil {
return nil, err
}
}
return &copy, nil
}
+109
View File
@@ -0,0 +1,109 @@
package event
import (
"fmt"
"os"
"strings"
"github.com/roboll/helmfile/pkg/environment"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/tmpl"
"go.uber.org/zap"
)
type Hook struct {
Name string `yaml:"name"`
Events []string `yaml:"events"`
Command string `yaml:"command"`
Args []string `yaml:"args"`
ShowLogs bool `yaml:"showlogs"`
}
type event struct {
Name string
}
type Bus struct {
Runner helmexec.Runner
Hooks []Hook
BasePath string
StateFilePath string
Namespace string
Env environment.Environment
ReadFile func(string) ([]byte, error)
Logger *zap.SugaredLogger
}
func (bus *Bus) Trigger(evt string, context map[string]interface{}) (bool, error) {
if bus.Runner == nil {
bus.Runner = helmexec.ShellRunner{
Dir: bus.BasePath,
}
}
executed := false
for _, hook := range bus.Hooks {
contained := false
for _, e := range hook.Events {
contained = contained || e == evt
}
if !contained {
continue
}
var err error
name := hook.Name
if name == "" {
name = hook.Command
}
fmt.Fprintf(os.Stderr, "%s: basePath=%s\n", bus.StateFilePath, bus.BasePath)
data := map[string]interface{}{
"Environment": bus.Env,
"Namespace": bus.Namespace,
"Event": event{
Name: evt,
},
}
for k, v := range context {
data[k] = v
}
render := tmpl.NewTextRenderer(bus.ReadFile, bus.BasePath, data)
bus.Logger.Debugf("hook[%s]: triggered by event \"%s\"\n", name, evt)
command, err := render.RenderTemplateText(hook.Command)
if err != nil {
return false, fmt.Errorf("hook[%s]: %v", name, err)
}
args := make([]string, len(hook.Args))
for i, raw := range hook.Args {
args[i], err = render.RenderTemplateText(raw)
if err != nil {
return false, fmt.Errorf("hook[%s]: %v", name, err)
}
}
bytes, err := bus.Runner.Execute(command, args, map[string]string{})
bus.Logger.Debugf("hook[%s]: %s\n", name, string(bytes))
if hook.ShowLogs {
prefix := fmt.Sprintf("\nhook[%s] logs | ", evt)
bus.Logger.Infow(prefix + strings.ReplaceAll(string(bytes), "\n", prefix))
}
if err != nil {
return false, fmt.Errorf("hook[%s]: command `%s` failed: %v", name, command, err)
}
executed = true
}
return executed, nil
}
+128
View File
@@ -0,0 +1,128 @@
package event
import (
"fmt"
"os"
"testing"
"github.com/roboll/helmfile/pkg/environment"
"github.com/roboll/helmfile/pkg/helmexec"
"go.uber.org/zap"
"go.uber.org/zap/zaptest/observer"
)
var logger = helmexec.NewLogger(os.Stdout, "warn")
type runner struct {
}
func (r *runner) Execute(cmd string, args []string, env map[string]string) ([]byte, error) {
if cmd == "ng" {
return nil, fmt.Errorf("cmd failed due to invalid cmd: %s", cmd)
}
for _, a := range args {
if a == "ng" {
return nil, fmt.Errorf("cmd failed due to invalid arg: %s", a)
}
}
return []byte(""), nil
}
func TestTrigger(t *testing.T) {
cases := []struct {
name string
hook *Hook
triggeredEvt string
expectedResult bool
expectedErr string
}{
{
"okhook1",
&Hook{"okhook1", []string{"foo"}, "ok", []string{}, true},
"foo",
true,
"",
},
{
"okhooké",
&Hook{"okhook2", []string{"foo"}, "ok", []string{}, false},
"foo",
true,
"",
},
{
"missinghook1",
&Hook{"okhook1", []string{"foo"}, "ok", []string{}, false},
"bar",
false,
"",
},
{
"nohook1",
nil,
"bar",
false,
"",
},
{
"nghook1",
&Hook{"nghook1", []string{"foo"}, "ng", []string{}, false},
"foo",
false,
"hook[nghook1]: command `ng` failed: cmd failed due to invalid cmd: ng",
},
{
"nghook2",
&Hook{"nghook2", []string{"foo"}, "ok", []string{"ng"}, false},
"foo",
false,
"hook[nghook2]: command `ok` failed: cmd failed due to invalid arg: ng",
},
}
readFile := func(filename string) ([]byte, error) {
return nil, fmt.Errorf("unexpected call to readFile: %s", filename)
}
for _, c := range cases {
hooks := []Hook{}
if c.hook != nil {
hooks = append(hooks, *c.hook)
}
observer, observedLogs := observer.New(zap.InfoLevel)
zeLogger := zap.New(observer).Sugar()
bus := &Bus{
Hooks: hooks,
StateFilePath: "path/to/helmfile.yaml",
BasePath: "path/to",
Namespace: "myns",
Env: environment.Environment{Name: "prod"},
Logger: zeLogger,
ReadFile: readFile,
}
bus.Runner = &runner{}
data := map[string]interface{}{
"Release": "myrel",
"HelmfileCommand": "mycmd",
}
ok, err := bus.Trigger(c.triggeredEvt, data)
if ok != c.expectedResult {
t.Errorf("unexpected result for case \"%s\": expected=%v, actual=%v", c.name, c.expectedResult, ok)
}
if c.expectedErr != "" {
if err == nil {
t.Error("error expected, but not occurred")
} else if err.Error() != c.expectedErr {
t.Errorf("unexpected error for case \"%s\": expected=%s, actual=%v", c.name, c.expectedErr, err)
}
} else {
if err != nil {
t.Errorf("unexpected error for case \"%s\": %v", c.name, err)
}
}
if observedLogs.Len() != 0 && !hooks[0].ShowLogs {
t.Errorf("unexpected error for case \"%s\": Logs should not be created : %v", c.name, observedLogs.All())
}
}
}
+44
View File
@@ -0,0 +1,44 @@
package helmexec
import (
"os"
"path/filepath"
)
type HelmContext struct {
Tillerless bool
TillerNamespace string
WorkerIndex int
}
func (context *HelmContext) GetTillerlessArgs(helmBinary string) []string {
if context.Tillerless {
if context.TillerNamespace != "" {
return []string{"tiller", "run", context.TillerNamespace, "--", helmBinary}
} else {
return []string{"tiller", "run", "--", helmBinary}
}
} else {
return []string{}
}
}
func (context *HelmContext) getTillerlessEnv() map[string]string {
if context.Tillerless {
result := map[string]string{
"HELM_TILLER_SILENT": "true",
// Changing the TILLER port doesn't really work: https://github.com/helm/helm/issues/3159
// So this is not used for the moment.
// "HELM_TILLER_PORT": strconv.Itoa(44134 + context.WorkerIndex),
}
if config := os.Getenv("KUBECONFIG"); config != "" {
absConfig, err := filepath.Abs(config)
if err == nil {
result["KUBECONFIG"] = absConfig
}
}
return result
} else {
return map[string]string{}
}
}
+321
View File
@@ -0,0 +1,321 @@
package helmexec
import (
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"strings"
"sync"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
)
const (
command = "helm"
)
type decryptedSecret struct {
mutex sync.RWMutex
bytes []byte
}
type execer struct {
helmBinary string
runner Runner
logger *zap.SugaredLogger
kubeContext string
extra []string
decryptedSecretMutex sync.Mutex
decryptedSecrets map[string]*decryptedSecret
}
func NewLogger(writer io.Writer, logLevel string) *zap.SugaredLogger {
var cfg zapcore.EncoderConfig
cfg.MessageKey = "message"
out := zapcore.AddSync(writer)
var level zapcore.Level
err := level.Set(logLevel)
if err != nil {
panic(err)
}
core := zapcore.NewCore(
zapcore.NewConsoleEncoder(cfg),
out,
level,
)
return zap.New(core).Sugar()
}
// New for running helm commands
func New(logger *zap.SugaredLogger, kubeContext string, runner Runner) *execer {
return &execer{
helmBinary: command,
logger: logger,
kubeContext: kubeContext,
runner: runner,
decryptedSecrets: make(map[string]*decryptedSecret),
}
}
func (helm *execer) SetExtraArgs(args ...string) {
helm.extra = args
}
func (helm *execer) SetHelmBinary(bin string) {
helm.helmBinary = bin
}
func (helm *execer) AddRepo(name, repository, cafile, certfile, keyfile, username, password string) error {
var args []string
args = append(args, "repo", "add", name, repository)
if certfile != "" && keyfile != "" {
args = append(args, "--cert-file", certfile, "--key-file", keyfile)
}
if cafile != "" {
args = append(args, "--ca-file", cafile)
}
if username != "" && password != "" {
args = append(args, "--username", username, "--password", password)
}
helm.logger.Infof("Adding repo %v %v", name, repository)
out, err := helm.exec(args, map[string]string{})
helm.info(out)
return err
}
func (helm *execer) UpdateRepo() error {
helm.logger.Info("Updating repo")
out, err := helm.exec([]string{"repo", "update"}, map[string]string{})
helm.info(out)
return err
}
func (helm *execer) BuildDeps(name, chart string) error {
helm.logger.Infof("Building dependency release=%v, chart=%v", name, chart)
out, err := helm.exec([]string{"dependency", "build", chart}, map[string]string{})
helm.info(out)
return err
}
func (helm *execer) UpdateDeps(chart string) error {
helm.logger.Infof("Updating dependency %v", chart)
out, err := helm.exec([]string{"dependency", "update", chart}, map[string]string{})
helm.info(out)
return err
}
func (helm *execer) SyncRelease(context HelmContext, name, chart string, flags ...string) error {
helm.logger.Infof("Upgrading release=%v, chart=%v", name, chart)
preArgs := context.GetTillerlessArgs(helm.helmBinary)
env := context.getTillerlessEnv()
out, err := helm.exec(append(append(preArgs, "upgrade", "--install", "--reset-values", name, chart), flags...), env)
helm.write(out)
return err
}
func (helm *execer) ReleaseStatus(context HelmContext, name string, flags ...string) error {
helm.logger.Infof("Getting status %v", name)
preArgs := context.GetTillerlessArgs(helm.helmBinary)
env := context.getTillerlessEnv()
out, err := helm.exec(append(append(preArgs, "status", name), flags...), env)
helm.write(out)
return err
}
func (helm *execer) List(context HelmContext, filter string, flags ...string) (string, error) {
helm.logger.Infof("Listing releases matching %v", filter)
preArgs := context.GetTillerlessArgs(helm.helmBinary)
env := context.getTillerlessEnv()
var args []string
if helm.isHelm3() {
args = []string{"list", "--filter", filter}
} else {
args = []string{"list", filter}
}
out, err := helm.exec(append(append(preArgs, args...), flags...), env)
helm.write(out)
return string(out), err
}
func (helm *execer) DecryptSecret(context HelmContext, name string, flags ...string) (string, error) {
absPath, err := filepath.Abs(name)
if err != nil {
return "", err
}
helm.logger.Debugf("Preparing to decrypt secret %v", absPath)
helm.decryptedSecretMutex.Lock()
secret, ok := helm.decryptedSecrets[absPath]
// Cache miss
if !ok {
secret = &decryptedSecret{}
helm.decryptedSecrets[absPath] = secret
secret.mutex.Lock()
defer secret.mutex.Unlock()
helm.decryptedSecretMutex.Unlock()
helm.logger.Infof("Decrypting secret %v", absPath)
preArgs := context.GetTillerlessArgs(helm.helmBinary)
env := context.getTillerlessEnv()
out, err := helm.exec(append(append(preArgs, "secrets", "dec", absPath), flags...), env)
helm.info(out)
if err != nil {
return "", err
}
// HELM_SECRETS_DEC_SUFFIX is used by the helm-secrets plugin to define the output file
decSuffix := os.Getenv("HELM_SECRETS_DEC_SUFFIX")
if len(decSuffix) == 0 {
decSuffix = ".yaml.dec"
}
decFilename := strings.Replace(absPath, ".yaml", decSuffix, 1)
secretBytes, err := ioutil.ReadFile(decFilename)
if err != nil {
return "", err
}
secret.bytes = secretBytes
if err := os.Remove(decFilename); err != nil {
return "", err
}
} else {
// Cache hit
helm.logger.Debugf("Found secret in cache %v", absPath)
secret.mutex.RLock()
helm.decryptedSecretMutex.Unlock()
defer secret.mutex.RUnlock()
}
tmpFile, err := ioutil.TempFile("", "secret")
if err != nil {
return "", err
}
_, err = tmpFile.Write(secret.bytes)
if err != nil {
return "", err
}
return tmpFile.Name(), err
}
func (helm *execer) TemplateRelease(name string, chart string, flags ...string) error {
helm.logger.Infof("Templating release=%v, chart=%v", name, chart)
var args []string
if helm.isHelm3() {
args = []string{"template", name, chart}
} else {
args = []string{"template", chart, "--name", name}
}
out, err := helm.exec(append(args, flags...), map[string]string{})
helm.write(out)
return err
}
func (helm *execer) DiffRelease(context HelmContext, name, chart string, flags ...string) error {
helm.logger.Infof("Comparing release=%v, chart=%v", name, chart)
preArgs := context.GetTillerlessArgs(helm.helmBinary)
env := context.getTillerlessEnv()
out, err := helm.exec(append(append(preArgs, "diff", "upgrade", "--reset-values", "--allow-unreleased", name, chart), flags...), env)
// Do our best to write STDOUT only when diff existed
// Unfortunately, this works only when you run helmfile with `--detailed-exitcode`
detailedExitcodeEnabled := false
for _, f := range flags {
if strings.Contains(f, "detailed-exitcode") {
detailedExitcodeEnabled = true
break
}
}
if detailedExitcodeEnabled {
switch e := err.(type) {
case ExitError:
if e.ExitStatus() == 2 {
helm.write(out)
return err
}
}
} else {
helm.write(out)
}
return err
}
func (helm *execer) Lint(name, chart string, flags ...string) error {
helm.logger.Infof("Linting release=%v, chart=%v", name, chart)
out, err := helm.exec(append([]string{"lint", chart}, flags...), map[string]string{})
helm.write(out)
return err
}
func (helm *execer) Fetch(chart string, flags ...string) error {
helm.logger.Infof("Fetching %v", chart)
out, err := helm.exec(append([]string{"fetch", chart}, flags...), map[string]string{})
helm.info(out)
return err
}
func (helm *execer) DeleteRelease(context HelmContext, name string, flags ...string) error {
helm.logger.Infof("Deleting %v", name)
preArgs := context.GetTillerlessArgs(helm.helmBinary)
env := context.getTillerlessEnv()
out, err := helm.exec(append(append(preArgs, "delete", name), flags...), env)
helm.write(out)
return err
}
func (helm *execer) TestRelease(context HelmContext, name string, flags ...string) error {
helm.logger.Infof("Testing %v", name)
preArgs := context.GetTillerlessArgs(helm.helmBinary)
env := context.getTillerlessEnv()
var args []string
if helm.isHelm3() {
args = []string{"test", "run", name}
} else {
args = []string{"test", name}
}
out, err := helm.exec(append(append(preArgs, args...), flags...), env)
helm.write(out)
return err
}
func (helm *execer) exec(args []string, env map[string]string) ([]byte, error) {
cmdargs := args
if len(helm.extra) > 0 {
cmdargs = append(cmdargs, helm.extra...)
}
if helm.kubeContext != "" {
cmdargs = append(cmdargs, "--kube-context", helm.kubeContext)
}
cmd := fmt.Sprintf("exec: %s %s", helm.helmBinary, strings.Join(cmdargs, " "))
helm.logger.Debug(cmd)
bytes, err := helm.runner.Execute(helm.helmBinary, cmdargs, env)
helm.logger.Debugf("%s: %s", cmd, bytes)
return bytes, err
}
func (helm *execer) info(out []byte) {
if len(out) > 0 {
helm.logger.Infof("%s", out)
}
}
func (helm *execer) write(out []byte) {
if len(out) > 0 {
fmt.Printf("%s\n", out)
}
}
func (helm *execer) isHelm3() bool {
return os.Getenv("HELMFILE_HELM3") != ""
}
+519
View File
@@ -0,0 +1,519 @@
package helmexec
import (
"bytes"
"fmt"
"os"
"path"
"path/filepath"
"reflect"
"testing"
"go.uber.org/zap"
)
// Mocking the command-line runner
type mockRunner struct {
output []byte
err error
}
func (mock *mockRunner) Execute(cmd string, args []string, env map[string]string) ([]byte, error) {
return []byte{}, nil
}
func MockExecer(logger *zap.SugaredLogger, kubeContext string) *execer {
execer := New(logger, kubeContext, &mockRunner{})
return execer
}
// Test methods
func TestNewHelmExec(t *testing.T) {
buffer := bytes.NewBufferString("something")
logger := NewLogger(buffer, "debug")
helm := New(logger, "dev", &ShellRunner{
Logger: logger,
})
if helm.kubeContext != "dev" {
t.Error("helmexec.New() - kubeContext")
}
if buffer.String() != "something" {
t.Error("helmexec.New() - changed buffer")
}
if len(helm.extra) != 0 {
t.Error("helmexec.New() - extra args not empty")
}
}
func Test_SetExtraArgs(t *testing.T) {
buffer := bytes.NewBufferString("something")
logger := NewLogger(buffer, "debug")
helm := New(NewLogger(os.Stdout, "info"), "dev", &ShellRunner{
Logger: logger,
})
helm.SetExtraArgs()
if len(helm.extra) != 0 {
t.Error("helmexec.SetExtraArgs() - passing no arguments should not change extra field")
}
helm.SetExtraArgs("foo")
if !reflect.DeepEqual(helm.extra, []string{"foo"}) {
t.Error("helmexec.SetExtraArgs() - one extra argument missing")
}
helm.SetExtraArgs("alpha", "beta")
if !reflect.DeepEqual(helm.extra, []string{"alpha", "beta"}) {
t.Error("helmexec.SetExtraArgs() - two extra arguments missing (overwriting the previous value)")
}
}
func Test_SetHelmBinary(t *testing.T) {
buffer := bytes.NewBufferString("something")
logger := NewLogger(buffer, "debug")
helm := New(NewLogger(os.Stdout, "info"), "dev", &ShellRunner{
Logger: logger,
})
if helm.helmBinary != "helm" {
t.Error("helmexec.command - default command is not helm")
}
helm.SetHelmBinary("foo")
if helm.helmBinary != "foo" {
t.Errorf("helmexec.SetHelmBinary() - actual = %s expect = foo", helm.helmBinary)
}
}
func Test_AddRepo(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.AddRepo("myRepo", "https://repo.example.com/", "", "cert.pem", "key.pem", "", "")
expected := `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --cert-file cert.pem --key-file key.pem --kube-context dev
exec: helm repo add myRepo https://repo.example.com/ --cert-file cert.pem --key-file key.pem --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.AddRepo("myRepo", "https://repo.example.com/", "ca.crt", "", "", "", "")
expected = `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --ca-file ca.crt --kube-context dev
exec: helm repo add myRepo https://repo.example.com/ --ca-file ca.crt --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "", "", "")
expected = `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --kube-context dev
exec: helm repo add myRepo https://repo.example.com/ --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "", "example_user", "example_password")
expected = `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password --kube-context dev
exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_UpdateRepo(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.UpdateRepo()
expected := `Updating repo
exec: helm repo update --kube-context dev
exec: helm repo update --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.UpdateRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_SyncRelease(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.SyncRelease(HelmContext{}, "release", "chart", "--timeout 10", "--wait")
expected := `Upgrading release=release, chart=chart
exec: helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev
exec: helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SyncRelease(HelmContext{}, "release", "chart")
expected = `Upgrading release=release, chart=chart
exec: helm upgrade --install --reset-values release chart --kube-context dev
exec: helm upgrade --install --reset-values release chart --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_SyncReleaseTillerless(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.SyncRelease(HelmContext{Tillerless: true, TillerNamespace: "foo"}, "release", "chart",
"--timeout 10", "--wait")
expected := `Upgrading release=release, chart=chart
exec: helm tiller run foo -- helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev
exec: helm tiller run foo -- helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_UpdateDeps(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.UpdateDeps("./chart/foo")
expected := `Updating dependency ./chart/foo
exec: helm dependency update ./chart/foo --kube-context dev
exec: helm dependency update ./chart/foo --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.UpdateDeps()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SetExtraArgs("--verify")
helm.UpdateDeps("./chart/foo")
expected = `Updating dependency ./chart/foo
exec: helm dependency update ./chart/foo --verify --kube-context dev
exec: helm dependency update ./chart/foo --verify --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_BuildDeps(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.BuildDeps("foo", "./chart/foo")
expected := `Building dependency release=foo, chart=./chart/foo
exec: helm dependency build ./chart/foo --kube-context dev
exec: helm dependency build ./chart/foo --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.BuildDeps()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SetExtraArgs("--verify")
helm.BuildDeps("foo", "./chart/foo")
expected = `Building dependency release=foo, chart=./chart/foo
exec: helm dependency build ./chart/foo --verify --kube-context dev
exec: helm dependency build ./chart/foo --verify --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.BuildDeps()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DecryptSecret(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DecryptSecret(HelmContext{}, "secretName")
cwd, err := filepath.Abs(".")
if err != nil {
t.Errorf("Error: %v", err)
}
// Run again for caching
helm.DecryptSecret(HelmContext{}, "secretName")
expected := fmt.Sprintf(`Preparing to decrypt secret %v/secretName
Decrypting secret %s/secretName
exec: helm secrets dec %s/secretName --kube-context dev
exec: helm secrets dec %s/secretName --kube-context dev:
Preparing to decrypt secret %s/secretName
Found secret in cache %s/secretName
`, cwd, cwd, cwd, cwd, cwd, cwd)
if buffer.String() != expected {
t.Errorf("helmexec.DecryptSecret()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DiffRelease(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DiffRelease(HelmContext{}, "release", "chart", "--timeout 10", "--wait")
expected := `Comparing release=release, chart=chart
exec: helm diff upgrade --reset-values --allow-unreleased release chart --timeout 10 --wait --kube-context dev
exec: helm diff upgrade --reset-values --allow-unreleased release chart --timeout 10 --wait --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.DiffRelease(HelmContext{}, "release", "chart")
expected = `Comparing release=release, chart=chart
exec: helm diff upgrade --reset-values --allow-unreleased release chart --kube-context dev
exec: helm diff upgrade --reset-values --allow-unreleased release chart --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DiffReleaseTillerless(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DiffRelease(HelmContext{Tillerless: true}, "release", "chart", "--timeout 10", "--wait")
expected := `Comparing release=release, chart=chart
exec: helm tiller run -- helm diff upgrade --reset-values --allow-unreleased release chart --timeout 10 --wait --kube-context dev
exec: helm tiller run -- helm diff upgrade --reset-values --allow-unreleased release chart --timeout 10 --wait --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DeleteRelease(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DeleteRelease(HelmContext{}, "release")
expected := `Deleting release
exec: helm delete release --kube-context dev
exec: helm delete release --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.DeleteRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DeleteRelease_Flags(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DeleteRelease(HelmContext{}, "release", "--purge")
expected := `Deleting release
exec: helm delete release --purge --kube-context dev
exec: helm delete release --purge --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.DeleteRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_TestRelease(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.TestRelease(HelmContext{}, "release")
expected := `Testing release
exec: helm test release --kube-context dev
exec: helm test release --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.TestRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_TestRelease_Flags(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.TestRelease(HelmContext{}, "release", "--cleanup", "--timeout", "60")
expected := `Testing release
exec: helm test release --cleanup --timeout 60 --kube-context dev
exec: helm test release --cleanup --timeout 60 --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.TestRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_ReleaseStatus(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.ReleaseStatus(HelmContext{}, "myRelease")
expected := `Getting status myRelease
exec: helm status myRelease --kube-context dev
exec: helm status myRelease --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.ReleaseStatus()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_exec(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "")
env := map[string]string{}
helm.exec([]string{"version"}, env)
expected := `exec: helm version
exec: helm version:
`
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
helm = MockExecer(logger, "dev")
ret, _ := helm.exec([]string{"diff"}, env)
if len(ret) != 0 {
t.Error("helmexec.exec() - expected empty return value")
}
buffer.Reset()
helm = MockExecer(logger, "dev")
helm.exec([]string{"diff", "release", "chart", "--timeout 10", "--wait"}, env)
expected = `exec: helm diff release chart --timeout 10 --wait --kube-context dev
exec: helm diff release chart --timeout 10 --wait --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.exec([]string{"version"}, env)
expected = `exec: helm version --kube-context dev
exec: helm version --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SetExtraArgs("foo")
helm.exec([]string{"version"}, env)
expected = `exec: helm version foo --kube-context dev
exec: helm version foo --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm = MockExecer(logger, "")
helm.SetHelmBinary("overwritten")
helm.exec([]string{"version"}, env)
expected = `exec: overwritten version
exec: overwritten version:
`
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_Lint(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.Lint("release", "path/to/chart", "--values", "file.yml")
expected := `Linting release=release, chart=path/to/chart
exec: helm lint path/to/chart --values file.yml --kube-context dev
exec: helm lint path/to/chart --values file.yml --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.Lint()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_Fetch(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.Fetch("chart", "--version", "1.2.3", "--untar", "--untardir", "/tmp/dir")
expected := `Fetching chart
exec: helm fetch chart --version 1.2.3 --untar --untardir /tmp/dir --kube-context dev
exec: helm fetch chart --version 1.2.3 --untar --untardir /tmp/dir --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.Lint()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
var logLevelTests = map[string]string{
"debug": `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password
exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password:
`,
"info": `Adding repo myRepo https://repo.example.com/
`,
"warn": ``,
}
func Test_LogLevels(t *testing.T) {
var buffer bytes.Buffer
for logLevel, expected := range logLevelTests {
buffer.Reset()
logger := NewLogger(&buffer, logLevel)
helm := MockExecer(logger, "")
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "", "example_user", "example_password")
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
}
func Test_getTillerlessEnv(t *testing.T) {
context := HelmContext{Tillerless: true, TillerNamespace: "foo", WorkerIndex: 1}
os.Unsetenv("KUBECONFIG")
actual := context.getTillerlessEnv()
if val, found := actual["HELM_TILLER_SILENT"]; !found || val != "true" {
t.Errorf("getTillerlessEnv() HELM_TILLER_SILENT\nactual = %s\nexpect = true", val)
}
// This feature is disabled until it is fixed in helm
/*if val, found := actual["HELM_TILLER_PORT"]; !found || val != "44135" {
t.Errorf("getTillerlessEnv() HELM_TILLER_PORT\nactual = %s\nexpect = 44135", val)
}*/
if val, found := actual["KUBECONFIG"]; found {
t.Errorf("getTillerlessEnv() KUBECONFIG\nactual = %s\nexpect = nil", val)
}
os.Setenv("KUBECONFIG", "toto")
actual = context.getTillerlessEnv()
cwd, _ := os.Getwd()
expected := path.Join(cwd, "toto")
if val, found := actual["KUBECONFIG"]; !found || val != expected {
t.Errorf("getTillerlessEnv() KUBECONFIG\nactual = %s\nexpect = %s", val, expected)
}
os.Unsetenv("KUBECONFIG")
}
func Test_mergeEnv(t *testing.T) {
actual := env2map(mergeEnv([]string{"A=1", "B=c=d", "E=2"}, map[string]string{"B": "3", "F": "4"}))
expected := map[string]string{"A": "1", "B": "3", "E": "2", "F": "4"}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("mergeEnv()\nactual = %v\nexpect = %v", actual, expected)
}
}
func Test_Template(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.TemplateRelease("release", "path/to/chart", "--values", "file.yml")
expected := `Templating release=release, chart=path/to/chart
exec: helm template path/to/chart --name release --values file.yml --kube-context dev
exec: helm template path/to/chart --name release --values file.yml --kube-context dev:
`
if buffer.String() != expected {
t.Errorf("helmexec.Template()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
+36
View File
@@ -0,0 +1,36 @@
package helmexec
import (
"fmt"
"path/filepath"
"strings"
)
func newExitError(helmCmdPath string, exitStatus int, errorMessage string) ExitError {
return ExitError{
msg: fmt.Sprintf("%s exited with status %d:\n%s", filepath.Base(helmCmdPath), exitStatus, indent(strings.TrimSpace(errorMessage))),
exitStatus: exitStatus,
}
}
func indent(text string) string {
lines := strings.Split(text, "\n")
for i := range lines {
lines[i] = " " + lines[i]
}
return strings.Join(lines, "\n")
}
// ExitError is created whenever your shell command exits with a non-zero exit status
type ExitError struct {
msg string
exitStatus int
}
func (e ExitError) Error() string {
return e.msg
}
func (e ExitError) ExitStatus() int {
return e.exitStatus
}
+26
View File
@@ -0,0 +1,26 @@
package helmexec
// Interface for executing helm commands
type Interface interface {
SetExtraArgs(args ...string)
SetHelmBinary(bin string)
AddRepo(name, repository, cafile, certfile, keyfile, username, password string) error
UpdateRepo() error
BuildDeps(name, chart string) error
UpdateDeps(chart string) error
SyncRelease(context HelmContext, name, chart string, flags ...string) error
DiffRelease(context HelmContext, name, chart string, flags ...string) error
TemplateRelease(name, chart string, flags ...string) error
Fetch(chart string, flags ...string) error
Lint(name, chart string, flags ...string) error
ReleaseStatus(context HelmContext, name string, flags ...string) error
DeleteRelease(context HelmContext, name string, flags ...string) error
TestRelease(context HelmContext, name string, flags ...string) error
List(context HelmContext, filter string, flags ...string) (string, error)
DecryptSecret(context HelmContext, name string, flags ...string) (string, error)
}
type DependencyUpdater interface {
UpdateDeps(chart string) error
}
+99
View File
@@ -0,0 +1,99 @@
package helmexec
import (
"bytes"
"errors"
"fmt"
"go.uber.org/zap"
"os"
"os/exec"
"strings"
"syscall"
)
const (
tmpPrefix = "helmfile-"
tmpSuffix = "-exec"
)
// Runner interface for shell commands
type Runner interface {
Execute(cmd string, args []string, env map[string]string) ([]byte, error)
}
// ShellRunner implemention for shell commands
type ShellRunner struct {
Dir string
Logger *zap.SugaredLogger
}
// Execute a shell command
func (shell ShellRunner) Execute(cmd string, args []string, env map[string]string) ([]byte, error) {
preparedCmd := exec.Command(cmd, args...)
preparedCmd.Dir = shell.Dir
preparedCmd.Env = mergeEnv(os.Environ(), env)
return combinedOutput(preparedCmd, shell.Logger)
}
func combinedOutput(c *exec.Cmd, logger *zap.SugaredLogger) ([]byte, error) {
if c.Stdout != nil {
return nil, errors.New("exec: Stdout already set")
}
if c.Stderr != nil {
return nil, errors.New("exec: Stderr already set")
}
var stdout bytes.Buffer
var stderr bytes.Buffer
c.Stdout = &stdout
c.Stderr = &stderr
err := c.Run()
o := stdout.Bytes()
e := stderr.Bytes()
if err != nil {
// TrimSpace is necessary, because otherwise helmfile prints the redundant new-lines after each error like:
//
// err: release "envoy2" in "helmfile.yaml" failed: exit status 1: Error: could not find a ready tiller pod
// <redundant new line!>
// err: release "envoy" in "helmfile.yaml" failed: exit status 1: Error: could not find a ready tiller pod
switch ee := err.(type) {
case *exec.ExitError:
// Propagate any non-zero exit status from the external command, rather than throwing it away,
// so that helmfile could return its own exit code accordingly
waitStatus := ee.Sys().(syscall.WaitStatus)
exitStatus := waitStatus.ExitStatus()
err = newExitError(c.Path, exitStatus, string(e))
default:
panic(fmt.Sprintf("unexpected error: %v", err))
}
}
return o, err
}
func mergeEnv(orig []string, new map[string]string) []string {
wanted := env2map(orig)
for k, v := range new {
wanted[k] = v
}
return map2env(wanted)
}
func map2env(wanted map[string]string) []string {
result := []string{}
for k, v := range wanted {
result = append(result, k+"="+v)
}
return result
}
func env2map(env []string) map[string]string {
wanted := map[string]string{}
for _, cur := range env {
pair := strings.SplitN(cur, "=", 2)
wanted[pair[0]] = pair[1]
}
return wanted
}
+91
View File
@@ -0,0 +1,91 @@
package maputil
import "fmt"
func CastKeysToStrings(s interface{}) (map[string]interface{}, error) {
new := map[string]interface{}{}
switch src := s.(type) {
case map[interface{}]interface{}:
for k, v := range src {
var str_k string
switch typed_k := k.(type) {
case string:
str_k = typed_k
default:
return nil, fmt.Errorf("unexpected type of key in map: expected string, got %T: value=%v, map=%v", typed_k, typed_k, src)
}
casted_v, err := recursivelyStringifyMapKey(v)
if err != nil {
return nil, err
}
new[str_k] = casted_v
}
case map[string]interface{}:
for k, v := range src {
casted_v, err := recursivelyStringifyMapKey(v)
if err != nil {
return nil, err
}
new[k] = casted_v
}
}
return new, nil
}
func recursivelyStringifyMapKey(v interface{}) (interface{}, error) {
var casted_v interface{}
switch typed_v := v.(type) {
case map[interface{}]interface{}, map[string]interface{}:
tmp, err := CastKeysToStrings(typed_v)
if err != nil {
return nil, err
}
casted_v = tmp
case []interface{}:
a := []interface{}{}
for i := range typed_v {
res, err := recursivelyStringifyMapKey(typed_v[i])
if err != nil {
return nil, err
}
a = append(a, res)
}
casted_v = a
default:
casted_v = typed_v
}
return casted_v, nil
}
func Set(m map[string]interface{}, key []string, value string) map[string]interface{} {
if len(key) == 0 {
panic(fmt.Errorf("bug: unexpected length of key: %d", len(key)))
}
k := key[0]
if len(key) == 1 {
m[k] = value
return m
}
remain := key[1:]
nested, ok := m[k]
if !ok {
nested = map[string]interface{}{}
}
switch t := nested.(type) {
case map[string]interface{}:
nested = Set(t, remain, value)
default:
panic(fmt.Errorf("unexpected type: %v(%T)", t, t))
}
m[k] = nested
return m
}
+61
View File
@@ -0,0 +1,61 @@
package maputil
import "testing"
func TestMapUtil_StrKeys(t *testing.T) {
m := map[string]interface{}{
"a": []interface{}{
map[string]interface{}{
"b": []interface{}{
map[string]interface{}{
"c": "C",
},
},
},
},
}
r, err := CastKeysToStrings(m)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
a := r["a"].([]interface{})
a0 := a[0].(map[string]interface{})
b := a0["b"].([]interface{})
b0 := b[0].(map[string]interface{})
c := b0["c"]
if c != "C" {
t.Errorf("unexpected c: expected=C, got=%s", c)
}
}
func TestMapUtil_IFKeys(t *testing.T) {
m := map[interface{}]interface{}{
"a": []interface{}{
map[interface{}]interface{}{
"b": []interface{}{
map[interface{}]interface{}{
"c": "C",
},
},
},
},
}
r, err := CastKeysToStrings(m)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
a := r["a"].([]interface{})
a0 := a[0].(map[string]interface{})
b := a0["b"].([]interface{})
b0 := b[0].(map[string]interface{})
c := b0["c"]
if c != "C" {
t.Errorf("unexpected c: expected=C, got=%s", c)
}
}
+262
View File
@@ -0,0 +1,262 @@
package remote
import (
"context"
"encoding/json"
"fmt"
"github.com/hashicorp/go-getter"
"github.com/hashicorp/go-getter/helper/url"
"go.uber.org/multierr"
"go.uber.org/zap"
"gopkg.in/yaml.v2"
"os"
"path/filepath"
"strings"
)
const DefaultCacheDir = ".helmfile/cache"
type Remote struct {
Logger *zap.SugaredLogger
// Home is the home directory for helmfile. Usually this points to $HOME of the user running helmfile.
// Helmfile saves fetched remote files into .helmfile/cache under home
Home string
// Getter is the underlying implementation of getter used for fetching remote files
Getter Getter
// ReadFile is the implementation of the file reader that reads a local file from the specified path.
// Inject any implementation of your choice, like an im-memory impl for testing, ioutil.ReadFile for the real-world use.
ReadFile func(string) ([]byte, error)
DirExists func(string) bool
FileExists func(string) bool
}
func (r *Remote) Unmarshal(src string, dst interface{}) error {
bytes, err := r.GetBytes(src)
if err != nil {
return err
}
strs := strings.Split(src, "/")
file := strs[len(strs)-1]
ext := filepath.Ext(file)
{
r.Logger.Debugf("unmarshalling %s", string(bytes))
var err error
switch ext {
case "json":
err = json.Unmarshal(bytes, dst)
default:
err = yaml.Unmarshal(bytes, dst)
}
r.Logger.Debugf("unmarshalled to %v", dst)
if err != nil {
return err
}
}
return nil
}
func (r *Remote) GetBytes(goGetterSrc string) ([]byte, error) {
f, err := r.Fetch(goGetterSrc)
if err != nil {
return nil, err
}
bytes, err := r.ReadFile(f)
if err != nil {
return nil, fmt.Errorf("read file: %v", err)
}
return bytes, nil
}
// Locate takes an URL to a remote file or a path to a local file.
// If the argument was an URL, it fetches the remote directory contained within the URL,
// and returns the path to the file in the fetched directory
func (r *Remote) Locate(urlOrPath string) (string, error) {
if r.FileExists(urlOrPath) || r.DirExists(urlOrPath) {
return urlOrPath, nil
}
fetched, err := r.Fetch(urlOrPath)
if err != nil {
switch err.(type) {
case InvalidURLError:
return urlOrPath, nil
}
return "", err
}
return fetched, nil
}
type InvalidURLError struct {
err string
}
func (e InvalidURLError) Error() string {
return e.err
}
type Source struct {
Getter, Scheme, User, Host, Dir, File, RawQuery string
}
func IsRemote(goGetterSrc string) bool {
if _, err := Parse(goGetterSrc); err != nil {
return false
}
return true
}
func Parse(goGetterSrc string) (*Source, error) {
items := strings.Split(goGetterSrc, "::")
var getter string
switch len(items) {
case 2:
getter = items[0]
goGetterSrc = items[1]
}
u, err := url.Parse(goGetterSrc)
if err != nil {
return nil, InvalidURLError{err: fmt.Sprintf("parse url: %v", err)}
}
if u.Scheme == "" {
return nil, InvalidURLError{err: fmt.Sprintf("parse url: missing scheme - probably this is a local file path? %s", goGetterSrc)}
}
pathComponents := strings.Split(u.Path, "@")
if len(pathComponents) != 2 {
return nil, fmt.Errorf("invalid src format: it must be `[<getter>::]<scheme>://<host>/<path/to/dir>@<path/to/file>?key1=val1&key2=val2: got %s", goGetterSrc)
}
return &Source{
Getter: getter,
User: u.User.String(),
Scheme: u.Scheme,
Host: u.Host,
Dir: pathComponents[0],
File: pathComponents[1],
RawQuery: u.RawQuery,
}, nil
}
func (r *Remote) Fetch(goGetterSrc string) (string, error) {
u, err := Parse(goGetterSrc)
if err != nil {
return "", err
}
srcDir := fmt.Sprintf("%s://%s%s", u.Scheme, u.Host, u.Dir)
file := u.File
r.Logger.Debugf("getter: %s", u.Getter)
r.Logger.Debugf("scheme: %s", u.Scheme)
r.Logger.Debugf("user: %s", u.User)
r.Logger.Debugf("host: %s", u.Host)
r.Logger.Debugf("dir: %s", u.Dir)
r.Logger.Debugf("file: %s", u.File)
// This should be shared across variant commands, so that they can share cache for the shared imports
cacheBaseDir := DefaultCacheDir
query := u.RawQuery
var cacheKey string
replacer := strings.NewReplacer(":", "", "//", "_", "/", "_", ".", "_")
dirKey := replacer.Replace(srcDir)
if len(query) > 0 {
paramsKey := strings.Replace(query, "&", "_", -1)
cacheKey = fmt.Sprintf("%s.%s", dirKey, paramsKey)
} else {
cacheKey = dirKey
}
cached := false
// e.g. .helmfile/cache/https_github_com_cloudposse_helmfiles_git.ref=0.xx.0
getterDst := filepath.Join(cacheBaseDir, cacheKey)
// e.g. $PWD/.helmfile/cache/https_github_com_cloudposse_helmfiles_git.ref=0.xx.0
cacheDirPath := filepath.Join(r.Home, getterDst)
r.Logger.Debugf("home: %s", r.Home)
r.Logger.Debugf("getter dest: %s", getterDst)
r.Logger.Debugf("cached dir: %s", cacheDirPath)
{
if r.FileExists(cacheDirPath) {
return "", fmt.Errorf("%s is not directory. please remove it so that variant could use it for dependency caching", getterDst)
}
if r.DirExists(cacheDirPath) {
cached = true
}
}
if !cached {
var getterSrc string
if u.User != "" {
getterSrc = fmt.Sprintf("%s://%s@%s%s", u.Scheme, u.User, u.Host, u.Dir)
} else {
getterSrc = fmt.Sprintf("%s://%s%s", u.Scheme, u.Host, u.Dir)
}
if len(query) > 0 {
getterSrc = strings.Join([]string{getterSrc, query}, "?")
}
if u.Getter != "" {
getterSrc = u.Getter + "::" + getterSrc
}
r.Logger.Debugf("downloading %s to %s", getterSrc, getterDst)
if err := r.Getter.Get(r.Home, getterSrc, cacheDirPath); err != nil {
rmerr := os.RemoveAll(cacheDirPath)
if rmerr != nil {
return "", multierr.Append(err, rmerr)
}
return "", err
}
}
return filepath.Join(cacheDirPath, file), nil
}
type Getter interface {
Get(wd, src, dst string) error
}
type GoGetter struct {
Logger *zap.SugaredLogger
}
func (g *GoGetter) Get(wd, src, dst string) error {
ctx := context.Background()
get := &getter.Client{
Ctx: ctx,
Src: src,
Dst: dst,
Pwd: wd,
Mode: getter.ClientModeDir,
Options: []getter.ClientOption{},
}
g.Logger.Debugf("client: %+v", *get)
if err := get.Get(); err != nil {
return fmt.Errorf("get: %v", err)
}
return nil
}
+165
View File
@@ -0,0 +1,165 @@
package remote
import (
"fmt"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/testhelper"
"os"
"testing"
)
func TestRemote_HttpsGitHub(t *testing.T) {
cleanfs := map[string]string{
"path/to/home": "",
}
cachefs := map[string]string{
"path/to/home/.helmfile/cache/https_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml": "foo: bar",
}
type testcase struct {
files map[string]string
expectCacheHit bool
}
testcases := []testcase{
{files: cleanfs, expectCacheHit: false},
{files: cachefs, expectCacheHit: true},
}
for i := range testcases {
testcase := testcases[i]
t.Run(fmt.Sprintf("case %d", i), func(t *testing.T) {
testfs := testhelper.NewTestFs(testcase.files)
hit := true
get := func(wd, src, dst string) error {
if wd != "path/to/home" {
return fmt.Errorf("unexpected wd: %s", wd)
}
if src != "git::https://github.com/cloudposse/helmfiles.git?ref=0.40.0" {
return fmt.Errorf("unexpected src: %s", src)
}
hit = false
return nil
}
getter := &testGetter{
get: get,
}
remote := &Remote{
Logger: helmexec.NewLogger(os.Stderr, "debug"),
Home: "path/to/home",
Getter: getter,
ReadFile: testfs.ReadFile,
FileExists: testfs.FileExistsAt,
DirExists: testfs.DirectoryExistsAt,
}
// FYI, go-getter in the `dir` mode accepts URL like the below. So helmfile expects URLs similar to it:
// go-getter -mode dir git::https://github.com/cloudposse/helmfiles.git?ref=0.40.0 gettertest1/b
// We use `@` to separate dir and the file path. This is a good idea borrowed from helm-git:
// https://github.com/aslafy-z/helm-git
url := "git::https://github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.40.0"
file, err := remote.Locate(url)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if file != "path/to/home/.helmfile/cache/https_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml" {
t.Errorf("unexpected file located: %s", file)
}
if testcase.expectCacheHit && !hit {
t.Errorf("unexpected result: unexpected cache miss")
}
if !testcase.expectCacheHit && hit {
t.Errorf("unexpected result: unexpected cache hit")
}
})
}
}
func TestRemote_SShGitHub(t *testing.T) {
cleanfs := map[string]string{
"path/to/home": "",
}
cachefs := map[string]string{
"path/to/home/.helmfile/cache/ssh_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml": "foo: bar",
}
type testcase struct {
files map[string]string
expectCacheHit bool
}
testcases := []testcase{
{files: cleanfs, expectCacheHit: false},
{files: cachefs, expectCacheHit: true},
}
for i := range testcases {
testcase := testcases[i]
t.Run(fmt.Sprintf("case %d", i), func(t *testing.T) {
testfs := testhelper.NewTestFs(testcase.files)
hit := true
get := func(wd, src, dst string) error {
if wd != "path/to/home" {
return fmt.Errorf("unexpected wd: %s", wd)
}
if src != "git::ssh://git@github.com/cloudposse/helmfiles.git?ref=0.40.0" {
return fmt.Errorf("unexpected src: %s", src)
}
hit = false
return nil
}
getter := &testGetter{
get: get,
}
remote := &Remote{
Logger: helmexec.NewLogger(os.Stderr, "debug"),
Home: "path/to/home",
Getter: getter,
ReadFile: testfs.ReadFile,
FileExists: testfs.FileExistsAt,
DirExists: testfs.DirectoryExistsAt,
}
url := "git::ssh://git@github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.40.0"
file, err := remote.Locate(url)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if file != "path/to/home/.helmfile/cache/ssh_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml" {
t.Errorf("unexpected file located: %s", file)
}
if testcase.expectCacheHit && !hit {
t.Errorf("unexpected result: unexpected cache miss")
}
if !testcase.expectCacheHit && hit {
t.Errorf("unexpected result: unexpected cache hit")
}
})
}
}
type testGetter struct {
get func(wd, src, dst string) error
}
func (t *testGetter) Get(wd, src, dst string) error {
return t.get(wd, src, dst)
}
+432
View File
@@ -0,0 +1,432 @@
package state
import (
"fmt"
"github.com/Masterminds/semver"
"github.com/r3labs/diff"
"github.com/roboll/helmfile/pkg/helmexec"
"go.uber.org/zap"
"gopkg.in/yaml.v2"
"io/ioutil"
"os"
"path/filepath"
"sort"
"strings"
)
type ChartMeta struct {
Name string `yaml:"name"`
}
type unresolvedChartDependency struct {
// ChartName identifies the dependant chart. In Helmfile, ChartName for `chart: stable/envoy` would be just `envoy`.
// It can't be collided with other charts referenced in the same helmfile spec.
// That is, collocating `chart: incubator/foo` and `chart: stable/foo` isn't allowed. Name them differently for a work-around.
ChartName string `yaml:"name"`
// Repository contains the URL for the helm chart repository that hosts the chart identified by ChartName
Repository string `yaml:"repository"`
// VersionConstraint is the version constraint of the dependent chart. "*" means the latest version.
VersionConstraint string `yaml:"version"`
}
type ResolvedChartDependency struct {
// ChartName identifies the dependant chart. In Helmfile, ChartName for `chart: stable/envoy` would be just `envoy`.
// It can't be collided with other charts referenced in the same helmfile spec.
// That is, collocating `chart: incubator/foo` and `chart: stable/foo` isn't allowed. Name them differently for a work-around.
ChartName string `yaml:"name"`
// Repository contains the URL for the helm chart repository that hosts the chart identified by ChartName
Repository string `yaml:"repository"`
// Version is the version number of the dependent chart.
// In the context of helmfile this can be omitted. When omitted, it is considered `*` which results helm/helmfile fetching the latest version.
Version string `yaml:"version"`
}
type UnresolvedDependencies struct {
deps map[string][]unresolvedChartDependency
}
type ChartRequirements struct {
UnresolvedDependencies []unresolvedChartDependency `yaml:"dependencies"`
}
type ChartLockedRequirements struct {
ResolvedDependencies []ResolvedChartDependency `yaml:"dependencies"`
Digest string `yaml:"digest"`
Generated string `yaml:"generated"`
}
func (d *UnresolvedDependencies) Add(chart, url, versionConstraint string) error {
dep := unresolvedChartDependency{
ChartName: chart,
Repository: url,
VersionConstraint: versionConstraint,
}
return d.add(dep)
}
func (d *UnresolvedDependencies) add(dep unresolvedChartDependency) error {
deps := d.deps[dep.ChartName]
if deps == nil {
deps = []unresolvedChartDependency{dep}
} else {
deps = append(deps, dep)
}
d.deps[dep.ChartName] = deps
return nil
}
func (d *UnresolvedDependencies) ToChartRequirements() *ChartRequirements {
deps := []unresolvedChartDependency{}
for _, ds := range d.deps {
for _, d := range ds {
if d.VersionConstraint == "" {
d.VersionConstraint = "*"
}
deps = append(deps, d)
}
}
return &ChartRequirements{UnresolvedDependencies: deps}
}
type ResolvedDependencies struct {
deps map[string][]ResolvedChartDependency
}
func (d *ResolvedDependencies) add(dep ResolvedChartDependency) error {
deps := d.deps[dep.ChartName]
if deps == nil {
deps = []ResolvedChartDependency{dep}
} else {
deps = append(deps, dep)
}
d.deps[dep.ChartName] = deps
return nil
}
func (d *ResolvedDependencies) Get(chart, versionConstraint string) (string, error) {
if versionConstraint == "" {
versionConstraint = "*"
}
deps, exists := d.deps[chart]
if exists {
for _, dep := range deps {
constraint, err := semver.NewConstraint(versionConstraint)
if err != nil {
return "", err
}
version, err := semver.NewVersion(dep.Version)
if err != nil {
return "", err
}
if constraint.Check(version) {
return dep.Version, nil
}
}
}
return "", fmt.Errorf("no resolved dependency found for \"%s\"", chart)
}
func (st *HelmState) mergeLockedDependencies() (*HelmState, error) {
filename, unresolved, err := getUnresolvedDependenciess(st)
if err != nil {
return nil, err
}
if len(unresolved.deps) == 0 {
return st, nil
}
depMan := NewChartDependencyManager(filename, st.logger)
if st.readFile != nil {
depMan.readFile = st.readFile
}
return resolveDependencies(st, depMan, unresolved)
}
func resolveDependencies(st *HelmState, depMan *chartDependencyManager, unresolved *UnresolvedDependencies) (*HelmState, error) {
resolved, lockfileExists, err := depMan.Resolve(unresolved)
if err != nil {
return nil, fmt.Errorf("unable to resolve %d deps: %v", len(unresolved.deps), err)
}
if !lockfileExists {
return st, nil
}
repoToURL := map[string]string{}
for _, r := range st.Repositories {
repoToURL[r.Name] = r.URL
}
updated := *st
for i, r := range updated.Releases {
repo, chart, ok := resolveRemoteChart(r.Chart)
if !ok {
continue
}
_, ok = repoToURL[repo]
// Skip this chart from dependency management, as there's no matching `repository` in the helmfile state,
// which may imply that this is a local chart within a directory, like `charts/myapp`
if !ok {
continue
}
ver, err := resolved.Get(chart, r.Version)
if err != nil {
return nil, err
}
updated.Releases[i].Version = ver
}
return &updated, nil
}
func (st *HelmState) updateDependenciesInTempDir(shell helmexec.DependencyUpdater, tempDir func(string, string) (string, error)) (*HelmState, error) {
filename, unresolved, err := getUnresolvedDependenciess(st)
if err != nil {
return nil, err
}
if len(unresolved.deps) == 0 {
return st, nil
}
d, err := tempDir("", "")
if err != nil {
return nil, fmt.Errorf("unable to create dir: %v", err)
}
defer os.RemoveAll(d)
return updateDependencies(st, shell, unresolved, filename, d)
}
func getUnresolvedDependenciess(st *HelmState) (string, *UnresolvedDependencies, error) {
repoToURL := map[string]string{}
for _, r := range st.Repositories {
repoToURL[r.Name] = r.URL
}
unresolved := &UnresolvedDependencies{deps: map[string][]unresolvedChartDependency{}}
//if err := unresolved.Add("stable/envoy", "https://kubernetes-charts.storage.googleapis.com", ""); err != nil {
// panic(err)
//}
for _, r := range st.Releases {
repo, chart, ok := resolveRemoteChart(r.Chart)
if !ok {
continue
}
url, ok := repoToURL[repo]
// Skip this chart from dependency management, as there's no matching `repository` in the helmfile state,
// which may imply that this is a local chart within a directory, like `charts/myapp`
if !ok {
continue
}
if err := unresolved.Add(chart, url, r.Version); err != nil {
return "", nil, err
}
}
filename := filepath.Base(st.FilePath)
filename = strings.TrimSuffix(filename, ".gotmpl")
filename = strings.TrimSuffix(filename, ".yaml")
filename = strings.TrimSuffix(filename, ".yml")
return filename, unresolved, nil
}
func updateDependencies(st *HelmState, shell helmexec.DependencyUpdater, unresolved *UnresolvedDependencies, filename, wd string) (*HelmState, error) {
depMan := NewChartDependencyManager(filename, st.logger)
_, err := depMan.Update(shell, wd, unresolved)
if err != nil {
return nil, fmt.Errorf("unable to resolve %d deps: %v", len(unresolved.deps), err)
}
return resolveDependencies(st, depMan, unresolved)
}
type chartDependencyManager struct {
Name string
logger *zap.SugaredLogger
readFile func(string) ([]byte, error)
writeFile func(string, []byte, os.FileMode) error
}
func NewChartDependencyManager(name string, logger *zap.SugaredLogger) *chartDependencyManager {
return &chartDependencyManager{
Name: name,
readFile: ioutil.ReadFile,
writeFile: ioutil.WriteFile,
logger: logger,
}
}
func (m *chartDependencyManager) lockFileName() string {
return fmt.Sprintf("%s.lock", m.Name)
}
func (m *chartDependencyManager) Update(shell helmexec.DependencyUpdater, wd string, unresolved *UnresolvedDependencies) (*ResolvedDependencies, error) {
if isHelm3() {
return m.updateHelm3(shell, wd, unresolved)
}
return m.updateHelm2(shell, wd, unresolved)
}
func (m *chartDependencyManager) updateHelm3(shell helmexec.DependencyUpdater, wd string, unresolved *UnresolvedDependencies) (*ResolvedDependencies, error) {
// Generate `Chart.yaml` of the temporary local chart
chartMetaContent := fmt.Sprintf("name: %s\nversion: 1.0.0\n", m.Name)
// Generate `requirements.yaml` of the temporary local chart from the helmfile state
reqsContent, err := yaml.Marshal(unresolved.ToChartRequirements())
if err != nil {
return nil, err
}
if err := m.writeBytes(filepath.Join(wd, "Chart.yaml"), []byte(chartMetaContent+string(reqsContent))); err != nil {
return nil, err
}
return m.doUpdate("Chart.lock", unresolved, shell, wd)
}
func (m *chartDependencyManager) updateHelm2(shell helmexec.DependencyUpdater, wd string, unresolved *UnresolvedDependencies) (*ResolvedDependencies, error) {
// Generate `Chart.yaml` of the temporary local chart
if err := m.writeBytes(filepath.Join(wd, "Chart.yaml"), []byte(fmt.Sprintf("name: %s\nversion: 1.0.0\n", m.Name))); err != nil {
return nil, err
}
// Generate `requirements.yaml` of the temporary local chart from the helmfile state
reqsContent, err := yaml.Marshal(unresolved.ToChartRequirements())
if err != nil {
return nil, err
}
if err := m.writeBytes(filepath.Join(wd, "requirements.yaml"), reqsContent); err != nil {
return nil, err
}
return m.doUpdate("requirements.lock", unresolved, shell, wd)
}
func (m *chartDependencyManager) doUpdate(chartLockFile string, unresolved *UnresolvedDependencies, shell helmexec.DependencyUpdater, wd string) (*ResolvedDependencies, error) {
// Generate `requirements.lock` of the temporary local chart by coping `<basename>.lock`
lockFile := m.lockFileName()
originalLockFileContent, err := m.readBytes(lockFile)
if err != nil && !os.IsNotExist(err) {
return nil, err
}
if isHelm3() && originalLockFileContent != nil {
if err := m.writeBytes(filepath.Join(wd, chartLockFile), originalLockFileContent); err != nil {
return nil, err
}
}
// Update the lock file by running `helm dependency update`
if err := shell.UpdateDeps(wd); err != nil {
return nil, err
}
updatedLockFileContent, err := m.readBytes(filepath.Join(wd, chartLockFile))
if err != nil {
return nil, err
}
// Sort requirements alphabetically by name.
lockedReqs := &ChartLockedRequirements{}
if err := yaml.Unmarshal(updatedLockFileContent, lockedReqs); err != nil {
return nil, err
}
sort.Slice(lockedReqs.ResolvedDependencies, func(i, j int) bool {
return lockedReqs.ResolvedDependencies[i].ChartName < lockedReqs.ResolvedDependencies[j].ChartName
})
// Don't update lock file if no dependency updated.
if !isHelm3() && originalLockFileContent != nil {
originalLockedReqs := &ChartLockedRequirements{}
if err := yaml.Unmarshal(originalLockFileContent, originalLockedReqs); err != nil {
return nil, err
}
changes, err := diff.Diff(originalLockedReqs.ResolvedDependencies, lockedReqs.ResolvedDependencies)
if err != nil {
return nil, err
}
if len(changes) == 0 {
lockedReqs.Generated = originalLockedReqs.Generated
}
}
updatedLockFileContent, err = yaml.Marshal(lockedReqs)
if err != nil {
return nil, err
}
// Commit the lock file if and only if everything looks ok
if err := m.writeBytes(lockFile, updatedLockFileContent); err != nil {
return nil, err
}
resolved, _, err := m.Resolve(unresolved)
return resolved, err
}
func (m *chartDependencyManager) Resolve(unresolved *UnresolvedDependencies) (*ResolvedDependencies, bool, error) {
updatedLockFileContent, err := m.readBytes(m.lockFileName())
if err != nil {
if os.IsNotExist(err) {
return nil, false, nil
}
return nil, false, err
}
// Load resolved dependencies into memory
lockedReqs := &ChartLockedRequirements{}
if err := yaml.Unmarshal(updatedLockFileContent, lockedReqs); err != nil {
return nil, false, err
}
resolved := &ResolvedDependencies{deps: map[string][]ResolvedChartDependency{}}
for _, d := range lockedReqs.ResolvedDependencies {
if err := resolved.add(d); err != nil {
return nil, false, err
}
}
return resolved, true, nil
}
func (m *chartDependencyManager) readBytes(filename string) ([]byte, error) {
bytes, err := m.readFile(filename)
if err != nil {
return nil, err
}
m.logger.Debugf("readBytes: read from %s:\n%s", filename, bytes)
return bytes, nil
}
func (m *chartDependencyManager) writeBytes(filename string, data []byte) error {
err := m.writeFile(filename, data, 0644)
if err != nil {
return err
}
m.logger.Debugf("writeBytes: wrote to %s:\n%s", filename, data)
return nil
}
+317
View File
@@ -0,0 +1,317 @@
package state
import (
"bytes"
"errors"
"fmt"
"io"
"os"
"github.com/imdario/mergo"
"github.com/roboll/helmfile/pkg/environment"
"github.com/roboll/helmfile/pkg/helmexec"
"github.com/roboll/helmfile/pkg/maputil"
"go.uber.org/zap"
"gopkg.in/yaml.v2"
)
type StateLoadError struct {
msg string
Cause error
}
func (e *StateLoadError) Error() string {
return fmt.Sprintf("%s: %v", e.msg, e.Cause)
}
type UndefinedEnvError struct {
msg string
}
func (e *UndefinedEnvError) Error() string {
return e.msg
}
type StateCreator struct {
logger *zap.SugaredLogger
readFile func(string) ([]byte, error)
fileExists func(string) (bool, error)
abs func(string) (string, error)
glob func(string) ([]string, error)
helm helmexec.Interface
Strict bool
LoadFile func(inheritedEnv *environment.Environment, baseDir, file string, evaluateBases bool) (*HelmState, error)
}
func NewCreator(logger *zap.SugaredLogger, readFile func(string) ([]byte, error), fileExists func(string) (bool, error), abs func(string) (string, error), glob func(string) ([]string, error), helm helmexec.Interface) *StateCreator {
return &StateCreator{
logger: logger,
readFile: readFile,
fileExists: fileExists,
abs: abs,
glob: glob,
Strict: true,
helm: helm,
}
}
// Parse parses YAML into HelmState
func (c *StateCreator) Parse(content []byte, baseDir, file string) (*HelmState, error) {
var state HelmState
state.FilePath = file
state.basePath = baseDir
state.helm = c.helm
decoder := yaml.NewDecoder(bytes.NewReader(content))
if !c.Strict {
decoder.SetStrict(false)
} else {
decoder.SetStrict(true)
}
i := 0
for {
i++
var intermediate HelmState
err := decoder.Decode(&intermediate)
if err == io.EOF {
break
} else if err != nil {
return nil, &StateLoadError{fmt.Sprintf("failed to read %s: reading document at index %d", file, i), err}
}
if err := mergo.Merge(&state, &intermediate, mergo.WithAppendSlice); err != nil {
return nil, &StateLoadError{fmt.Sprintf("failed to read %s: merging document at index %d", file, i), err}
}
}
if len(state.DeprecatedReleases) > 0 {
if len(state.Releases) > 0 {
return nil, fmt.Errorf("failed to parse %s: you can't specify both `charts` and `releases` sections", file)
}
state.Releases = state.DeprecatedReleases
state.DeprecatedReleases = []ReleaseSpec{}
}
if state.DeprecatedContext != "" && state.HelmDefaults.KubeContext == "" {
state.HelmDefaults.KubeContext = state.DeprecatedContext
}
state.logger = c.logger
state.readFile = c.readFile
state.removeFile = os.Remove
state.fileExists = c.fileExists
state.glob = c.glob
return &state, nil
}
// LoadEnvValues loads environment values files relative to the `baseDir`
func (c *StateCreator) LoadEnvValues(target *HelmState, env string, ctxEnv *environment.Environment) (*HelmState, error) {
state := *target
e, err := state.loadEnvValues(env, ctxEnv, c.readFile, c.glob)
if err != nil {
return nil, &StateLoadError{fmt.Sprintf("failed to read %s", state.FilePath), err}
}
e.Defaults, err = state.loadValuesEntries(nil, state.DefaultValues)
if err != nil {
return nil, err
}
state.Env = *e
return &state, nil
}
// Parses YAML into HelmState, while loading environment values files relative to the `baseDir`
func (c *StateCreator) ParseAndLoad(content []byte, baseDir, file string, envName string, evaluateBases bool, envValues *environment.Environment) (*HelmState, error) {
state, err := c.Parse(content, baseDir, file)
if err != nil {
return nil, err
}
if !evaluateBases {
if len(state.Bases) > 0 {
return nil, errors.New("nested `base` helmfile is unsupported. please submit a feature request if you need this!")
}
}
state, err = c.loadBases(envValues, state, baseDir)
if err != nil {
return nil, err
}
state, err = c.LoadEnvValues(state, envName, envValues)
if err != nil {
return nil, err
}
state.FilePath = file
return state, nil
}
func (c *StateCreator) loadBases(envValues *environment.Environment, st *HelmState, baseDir string) (*HelmState, error) {
layers := []*HelmState{}
for _, b := range st.Bases {
base, err := c.LoadFile(envValues, baseDir, b, false)
if err != nil {
return nil, err
}
layers = append(layers, base)
}
layers = append(layers, st)
for i := 1; i < len(layers); i++ {
if err := mergo.Merge(layers[0], layers[i], mergo.WithAppendSlice); err != nil {
return nil, err
}
}
return layers[0], nil
}
func (st *HelmState) loadEnvValues(name string, ctxEnv *environment.Environment, readFile func(string) ([]byte, error), glob func(string) ([]string, error)) (*environment.Environment, error) {
envVals := map[string]interface{}{}
envSpec, ok := st.Environments[name]
if ok {
var err error
envVals, err = st.loadValuesEntries(envSpec.MissingFileHandler, envSpec.Values)
if err != nil {
return nil, err
}
if len(envSpec.Secrets) > 0 {
var envSecretFiles []string
for _, urlOrPath := range envSpec.Secrets {
resolved, skipped, err := st.storage().resolveFile(envSpec.MissingFileHandler, "environment values", urlOrPath)
if err != nil {
return nil, err
}
if skipped {
continue
}
envSecretFiles = append(envSecretFiles, resolved...)
}
if err = st.scatterGatherEnvSecretFiles(envSecretFiles, envVals, readFile); err != nil {
return nil, err
}
}
} else if ctxEnv == nil && name != DefaultEnv {
return nil, &UndefinedEnvError{msg: fmt.Sprintf("environment \"%s\" is not defined", name)}
}
newEnv := &environment.Environment{Name: name, Values: envVals}
if ctxEnv != nil {
intEnv := *ctxEnv
if err := mergo.Merge(&intEnv, newEnv, mergo.WithOverride); err != nil {
return nil, fmt.Errorf("error while merging environment values for \"%s\": %v", name, err)
}
newEnv = &intEnv
}
return newEnv, nil
}
func (st *HelmState) scatterGatherEnvSecretFiles(envSecretFiles []string, envVals map[string]interface{}, readFile func(string) ([]byte, error)) error {
var errs []error
inputs := envSecretFiles
inputsSize := len(inputs)
type secretResult struct {
result map[string]interface{}
err error
path string
}
secrets := make(chan string, inputsSize)
results := make(chan secretResult, inputsSize)
helm := st.helm
st.scatterGather(0, inputsSize,
func() {
for _, secretFile := range envSecretFiles {
secrets <- secretFile
}
close(secrets)
},
func(id int) {
for path := range secrets {
release := &ReleaseSpec{}
flags := st.appendConnectionFlags([]string{}, release)
decFile, err := helm.DecryptSecret(st.createHelmContext(release, 0), path, flags...)
if err != nil {
results <- secretResult{nil, err, path}
continue
}
bytes, err := readFile(decFile)
if err != nil {
results <- secretResult{nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", path, err), path}
continue
}
m := map[string]interface{}{}
if err := yaml.Unmarshal(bytes, &m); err != nil {
results <- secretResult{nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", path, err), path}
continue
}
// All the nested map key should be string. Otherwise we get strange errors due to that
// mergo or reflect is unable to merge map[interface{}]interface{} with map[string]interface{} or vice versa.
// See https://github.com/roboll/helmfile/issues/677
vals, err := maputil.CastKeysToStrings(m)
if err != nil {
results <- secretResult{nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", path, err), path}
continue
}
results <- secretResult{vals, nil, path}
}
},
func() {
for i := 0; i < inputsSize; i++ {
result := <-results
if result.err != nil {
errs = append(errs, result.err)
} else {
if err := mergo.Merge(&envVals, &result.result, mergo.WithOverride); err != nil {
errs = append(errs, fmt.Errorf("failed to load environment secrets file \"%s\": %v", result.path, err))
}
}
}
close(results)
},
)
if len(errs) > 1 {
for _, err := range errs {
st.logger.Error(err)
}
return fmt.Errorf("Failed loading environment secrets with %d errors", len(errs))
}
return nil
}
func (st *HelmState) loadValuesEntries(missingFileHandler *string, entries []interface{}) (map[string]interface{}, error) {
envVals := map[string]interface{}{}
valuesEntries := append([]interface{}{}, entries...)
ld := NewEnvironmentValuesLoader(st.storage(), st.readFile, st.logger)
var err error
envVals, err = ld.LoadEnvironmentValues(missingFileHandler, valuesEntries)
if err != nil {
return nil, err
}
return envVals, nil
}
+351
View File
@@ -0,0 +1,351 @@
package state
import (
"io/ioutil"
"path/filepath"
"reflect"
"testing"
"github.com/roboll/helmfile/pkg/testhelper"
"go.uber.org/zap"
. "gotest.tools/assert"
"gotest.tools/assert/cmp"
)
func createFromYaml(content []byte, file string, env string, logger *zap.SugaredLogger) (*HelmState, error) {
c := &StateCreator{
logger: logger,
readFile: ioutil.ReadFile,
abs: filepath.Abs,
Strict: true,
}
return c.ParseAndLoad(content, filepath.Dir(file), file, env, false, nil)
}
func TestReadFromYaml(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease
namespace: mynamespace
chart: mychart
`)
state, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err != nil {
t.Errorf("unxpected error: %v", err)
}
if state.Releases[0].Name != "myrelease" {
t.Errorf("unexpected release name: expected=myrelease actual=%s", state.Releases[0].Name)
}
if state.Releases[0].Namespace != "mynamespace" {
t.Errorf("unexpected chart namespace: expected=mynamespace actual=%s", state.Releases[0].Chart)
}
if state.Releases[0].Chart != "mychart" {
t.Errorf("unexpected chart name: expected=mychart actual=%s", state.Releases[0].Chart)
}
}
func TestReadFromYaml_InexistentEnv(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease
namespace: mynamespace
chart: mychart
`)
_, err := createFromYaml(yamlContent, yamlFile, "production", logger)
if err == nil {
t.Error("expected error")
}
}
func TestReadFromYaml_NonDefaultEnv(t *testing.T) {
yamlFile := "/example/path/to/helmfile.yaml"
yamlContent := []byte(`environments:
production:
values:
- foo.yaml
- bar.yaml.gotmpl
releases:
- name: myrelease
namespace: mynamespace
chart: mychart
values:
- values.yaml.gotmpl
`)
fooYamlFile := "/example/path/to/foo.yaml"
fooYamlContent := []byte(`foo: foo
# As this file doesn't have an file extension ".gotmpl", this template expression should not be evaluated
baz: "{{ readFile \"baz.txt\" }}"`)
barYamlFile := "/example/path/to/bar.yaml.gotmpl"
barYamlContent := []byte(`foo: FOO
bar: {{ readFile "bar.txt" }}
`)
barTextFile := "/example/path/to/bar.txt"
barTextContent := []byte("BAR")
expected := map[string]interface{}{
"foo": "FOO",
"bar": "BAR",
// As the file doesn't have an file extension ".gotmpl", this template expression should not be evaluated
"baz": "{{ readFile \"baz.txt\" }}",
}
valuesFile := "/example/path/to/values.yaml.gotmpl"
valuesContent := []byte(`env: {{ .Environment.Name }}`)
expectedValues := `env: production`
testFs := testhelper.NewTestFs(map[string]string{
fooYamlFile: string(fooYamlContent),
barYamlFile: string(barYamlContent),
barTextFile: string(barTextContent),
valuesFile: string(valuesContent),
})
testFs.Cwd = "/example/path/to"
state, err := NewCreator(logger, testFs.ReadFile, testFs.FileExists, testFs.Abs, testFs.Glob, nil).ParseAndLoad(yamlContent, filepath.Dir(yamlFile), yamlFile, "production", false, nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
actual := state.Env.Values
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected environment values: expected=%v, actual=%v", expected, actual)
}
actualValuesData, err := state.RenderValuesFileToBytes(valuesFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actualValues := string(actualValuesData)
if !reflect.DeepEqual(expectedValues, actualValues) {
t.Errorf("unexpected values: expected=%v, actual=%v", expectedValues, actualValues)
}
}
func TestReadFromYaml_StrictUnmarshalling(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease
namespace: mynamespace
releases: mychart
`)
_, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err == nil {
t.Error("expected an error for wrong key 'releases' which is not in struct")
}
}
func TestReadFromYaml_DeprecatedReleaseReferences(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`charts:
- name: myrelease
chart: mychart
`)
state, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err != nil {
t.Errorf("unxpected error: %v", err)
}
if state.Releases[0].Name != "myrelease" {
t.Errorf("unexpected release name: expected=myrelease actual=%s", state.Releases[0].Name)
}
if state.Releases[0].Chart != "mychart" {
t.Errorf("unexpected chart name: expected=mychart actual=%s", state.Releases[0].Chart)
}
}
func TestReadFromYaml_ConflictingReleasesConfig(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`charts:
- name: myrelease1
chart: mychart1
releases:
- name: myrelease2
chart: mychart2
`)
_, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err == nil {
t.Error("expected error")
}
}
func TestReadFromYaml_FilterReleasesOnLabels(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease1
chart: mychart1
labels:
tier: frontend
foo: bar
- name: myrelease2
chart: mychart2
labels:
tier: frontend
- name: myrelease3
chart: mychart3
labels:
tier: backend
`)
cases := []struct {
filter LabelFilter
results []bool
}{
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}}},
[]bool{true, true, false}},
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}, []string{"foo", "bar"}}},
[]bool{true, false, false}},
{LabelFilter{negativeLabels: [][]string{[]string{"tier", "frontend"}}},
[]bool{false, false, true}},
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}}, negativeLabels: [][]string{[]string{"foo", "bar"}}},
[]bool{false, true, false}},
}
state, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
for idx, c := range cases {
for idx2, expected := range c.results {
if f := c.filter.Match(state.Releases[idx2]); f != expected {
t.Errorf("[case: %d][outcome: %d] Unexpected outcome wanted %t, got %t", idx, idx2, expected, f)
}
}
}
}
func TestReadFromYaml_FilterNegatives(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease1
chart: mychart1
labels:
stage: pre
foo: bar
- name: myrelease2
chart: mychart2
labels:
stage: post
- name: myrelease3
chart: mychart3
`)
cases := []struct {
filter LabelFilter
results []bool
}{
{LabelFilter{positiveLabels: [][]string{[]string{"stage", "pre"}}},
[]bool{true, false, false}},
{LabelFilter{positiveLabels: [][]string{[]string{"stage", "post"}}},
[]bool{false, true, false}},
{LabelFilter{negativeLabels: [][]string{[]string{"stage", "pre"}, []string{"stage", "post"}}},
[]bool{false, false, true}},
}
state, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
for idx, c := range cases {
for idx2, expected := range c.results {
if f := c.filter.Match(state.Releases[idx2]); f != expected {
t.Errorf("[case: %d][outcome: %d] Unexpected outcome wanted %t, got %t", idx, idx2, expected, f)
}
}
}
}
func TestReadFromYaml_Helmfiles_Selectors(t *testing.T) {
tests := []struct {
path string
content []byte
wantErr bool
helmfiles []SubHelmfileSpec
}{
{
path: "working/selector",
content: []byte(`helmfiles:
- simple/helmfile.yaml
- path: path/prefix/selector.yaml
selectors:
- name=zorba
- foo=bar
- path: path/prefix/empty/selector.yaml
selectors: []
- path: path/prefix/inherits/selector.yaml
selectorsInherited: true
`),
wantErr: false,
helmfiles: []SubHelmfileSpec{{Path: "simple/helmfile.yaml", Selectors: nil, SelectorsInherited: false},
{Path: "path/prefix/selector.yaml", Selectors: []string{"name=zorba", "foo=bar"}, SelectorsInherited: false},
{Path: "path/prefix/empty/selector.yaml", Selectors: []string{}, SelectorsInherited: false},
{Path: "path/prefix/inherits/selector.yaml", Selectors: nil, SelectorsInherited: true},
},
},
{
path: "failing2/selector",
content: []byte(`helmfiles:
- path: failing2/helmfile.yaml
wrongkey:
`),
wantErr: true,
},
{
path: "failing3/selector",
content: []byte(`helmfiles:
- path: failing3/helmfile.yaml
selectors: foo
`),
wantErr: true,
},
{
path: "failing4/selector",
content: []byte(`helmfiles:
- path: failing4/helmfile.yaml
selectors:
`),
wantErr: true,
},
{
path: "failing4/selector",
content: []byte(`helmfiles:
- path: failing4/helmfile.yaml
selectors:
- colon: not-authorized
`),
wantErr: true,
},
{
path: "failing6/selector",
content: []byte(`helmfiles:
- selectors:
- whatever
`),
wantErr: true,
},
{
path: "failing7/selector",
content: []byte(`helmfiles:
- path: foo/bar
selectors:
- foo=bar
selectorsInherited: true
`),
wantErr: true,
},
}
for _, test := range tests {
st, err := createFromYaml(test.content, test.path, DefaultEnv, logger)
if err != nil {
if test.wantErr {
continue
} else {
t.Error("unexpected error:", err)
}
}
Assert(t, cmp.DeepEqual(st.Helmfiles, test.helmfiles), "for path %v", test.path)
}
}
+15
View File
@@ -0,0 +1,15 @@
package state
type EnvironmentSpec struct {
Values []interface{} `yaml:"values,omitempty"`
Secrets []string `yaml:"secrets,omitempty"`
// MissingFileHandler instructs helmfile to fail when unable to find a environment values file listed
// under `environments.NAME.values`.
//
// Possible values are "Error", "Warn", "Info", "Debug". The default is "Error".
//
// Use "Warn", "Info", or "Debug" if you want helmfile to not fail when a values file is missing, while just leaving
// a message about the missing file at the log-level.
MissingFileHandler *string `yaml:"missingFileHandler,omitempty"`
}
+83
View File
@@ -0,0 +1,83 @@
package state
import (
"fmt"
"github.com/imdario/mergo"
"github.com/roboll/helmfile/pkg/environment"
"github.com/roboll/helmfile/pkg/maputil"
"github.com/roboll/helmfile/pkg/tmpl"
"go.uber.org/zap"
"gopkg.in/yaml.v2"
"path/filepath"
)
type EnvironmentValuesLoader struct {
storage *Storage
readFile func(string) ([]byte, error)
logger *zap.SugaredLogger
}
func NewEnvironmentValuesLoader(storage *Storage, readFile func(string) ([]byte, error), logger *zap.SugaredLogger) *EnvironmentValuesLoader {
return &EnvironmentValuesLoader{
storage: storage,
readFile: readFile,
logger: logger,
}
}
func (ld *EnvironmentValuesLoader) LoadEnvironmentValues(missingFileHandler *string, valuesEntries []interface{}) (map[string]interface{}, error) {
result := map[string]interface{}{}
for _, entry := range valuesEntries {
maps := []interface{}{}
switch strOrMap := entry.(type) {
case string:
urlOrPath := strOrMap
files, skipped, err := ld.storage.resolveFile(missingFileHandler, "environment values", urlOrPath)
if err != nil {
return nil, err
}
if skipped {
continue
}
for _, f := range files {
tmplData := EnvironmentTemplateData{environment.EmptyEnvironment, "", map[string]interface{}{}}
r := tmpl.NewFileRenderer(ld.readFile, filepath.Dir(f), tmplData)
bytes, err := r.RenderToBytes(f)
if err != nil {
return nil, fmt.Errorf("failed to load environment values file \"%s\": %v", f, err)
}
m := map[string]interface{}{}
if err := yaml.Unmarshal(bytes, &m); err != nil {
return nil, fmt.Errorf("failed to load environment values file \"%s\": %v\n\nOffending YAML:\n%s", f, err, bytes)
}
maps = append(maps, m)
if ld.logger != nil {
ld.logger.Debugf("envvals_loader: loaded %s:%v", strOrMap, m)
}
}
case map[interface{}]interface{}:
maps = append(maps, strOrMap)
default:
return nil, fmt.Errorf("unexpected type of value: value=%v, type=%T", strOrMap, strOrMap)
}
for _, m := range maps {
// All the nested map key should be string. Otherwise we get strange errors due to that
// mergo or reflect is unable to merge map[interface{}]interface{} with map[string]interface{} or vice versa.
// See https://github.com/roboll/helmfile/issues/677
vals, err := maputil.CastKeysToStrings(m)
if err != nil {
return nil, err
}
if err := mergo.Merge(&result, &vals, mergo.WithOverride); err != nil {
return nil, fmt.Errorf("failed to merge %v: %v", m, err)
}
}
}
return result, nil
}
+55
View File
@@ -0,0 +1,55 @@
package state
type Dependency struct {
Chart string `yaml:"chart"`
Version string `yaml:"version"`
Alias string `yaml:"alias"`
}
func (st *HelmState) appendHelmXFlags(flags []string, release *ReleaseSpec) ([]string, error) {
for _, d := range release.Dependencies {
var dep string
if d.Alias != "" {
dep += d.Alias + "="
}
dep += d.Chart
if d.Version != "" {
dep += ":" + d.Version
}
flags = append(flags, "--dependency", dep)
}
for _, adopt := range release.Adopt {
flags = append(flags, "--adopt", adopt)
}
jsonPatches := release.JSONPatches
if len(jsonPatches) > 0 {
generatedFiles, err := st.generateTemporaryValuesFiles(jsonPatches, release.MissingFileHandler)
if err != nil {
return nil, err
}
for _, f := range generatedFiles {
flags = append(flags, "--json-patch", f)
}
release.generatedValues = append(release.generatedValues, generatedFiles...)
}
strategicMergePatches := release.StrategicMergePatches
if len(strategicMergePatches) > 0 {
generatedFiles, err := st.generateTemporaryValuesFiles(strategicMergePatches, release.MissingFileHandler)
if err != nil {
return nil, err
}
for _, f := range generatedFiles {
flags = append(flags, "--strategic-merge-patch", f)
}
release.generatedValues = append(release.generatedValues, generatedFiles...)
}
return flags, nil
}
+204
View File
@@ -0,0 +1,204 @@
package state
import (
"fmt"
"github.com/roboll/helmfile/pkg/tmpl"
"gopkg.in/yaml.v2"
)
func (r ReleaseSpec) ExecuteTemplateExpressions(renderer *tmpl.FileRenderer) (*ReleaseSpec, error) {
var result *ReleaseSpec
var err error
result, err = r.Clone()
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\": %v", r.Name, err)
}
{
ts := result.Name
result.Name, err = renderer.RenderTemplateContentToString([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".name = \"%s\": %v", r.Name, ts, err)
}
}
{
ts := result.Chart
result.Chart, err = renderer.RenderTemplateContentToString([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".chart = \"%s\": %v", r.Name, ts, err)
}
}
{
ts := result.Namespace
result.Namespace, err = renderer.RenderTemplateContentToString([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".namespace = \"%s\": %v", r.Name, ts, err)
}
}
{
ts := result.Version
result.Version, err = renderer.RenderTemplateContentToString([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".version = \"%s\": %v", r.Name, ts, err)
}
}
if result.WaitTemplate != nil {
ts := *result.WaitTemplate
resultTmpl, err := renderer.RenderTemplateContentToString([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".version = \"%s\": %v", r.Name, ts, err)
}
result.WaitTemplate = &resultTmpl
}
if result.InstalledTemplate != nil {
ts := *result.InstalledTemplate
resultTmpl, err := renderer.RenderTemplateContentToString([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".version = \"%s\": %v", r.Name, ts, err)
}
result.InstalledTemplate = &resultTmpl
}
if result.TillerlessTemplate != nil {
ts := *result.TillerlessTemplate
resultTmpl, err := renderer.RenderTemplateContentToString([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".version = \"%s\": %v", r.Name, ts, err)
}
result.TillerlessTemplate = &resultTmpl
}
if result.VerifyTemplate != nil {
ts := *result.VerifyTemplate
resultTmpl, err := renderer.RenderTemplateContentToString([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".version = \"%s\": %v", r.Name, ts, err)
}
result.VerifyTemplate = &resultTmpl
}
for key, val := range result.Labels {
ts := val
s, err := renderer.RenderTemplateContentToBuffer([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".labels[%s] = \"%s\": %v", r.Name, key, ts, err)
}
result.Labels[key] = s.String()
}
if result.ValuesTemplate != nil && len(result.ValuesTemplate) > 0 {
for i, t := range result.ValuesTemplate {
switch ts := t.(type) {
case map[interface{}]interface{}:
serialized, err := yaml.Marshal(ts)
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".values[%d] = \"%v\": %v", r.Name, i, ts, err)
}
s, err := renderer.RenderTemplateContentToBuffer([]byte(serialized))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".values[%d] = \"%v\": %v", r.Name, i, serialized, err)
}
var deserialized map[interface{}]interface{}
if err := yaml.Unmarshal(s.Bytes(), &deserialized); err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".values[%d] = \"%v\": %v", r.Name, i, ts, err)
}
result.ValuesTemplate[i] = deserialized
}
}
result.Values = result.ValuesTemplate
}
for i, t := range result.Values {
switch ts := t.(type) {
case string:
s, err := renderer.RenderTemplateContentToBuffer([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".values[%d] = \"%s\": %v", r.Name, i, ts, err)
}
result.Values[i] = s.String()
}
}
for i, ts := range result.Secrets {
s, err := renderer.RenderTemplateContentToBuffer([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".secrets[%d] = \"%s\": %v", r.Name, i, ts, err)
}
result.Secrets[i] = s.String()
}
if result.SetValuesTemplate != nil && len(result.SetValuesTemplate) > 0 {
for i, val := range result.SetValuesTemplate {
{
// name
ts := val.Name
s, err := renderer.RenderTemplateContentToBuffer([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".set[%d].name = \"%s\": %v", r.Name, i, ts, err)
}
result.SetValuesTemplate[i].Name = s.String()
}
{
// value
ts := val.Value
s, err := renderer.RenderTemplateContentToBuffer([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".set[%d].value = \"%s\": %v", r.Name, i, ts, err)
}
result.SetValuesTemplate[i].Value = s.String()
}
{
// file
ts := val.File
s, err := renderer.RenderTemplateContentToBuffer([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".set[%d].file = \"%s\": %v", r.Name, i, ts, err)
}
result.SetValuesTemplate[i].File = s.String()
}
for j, ts := range val.Values {
// values
s, err := renderer.RenderTemplateContentToBuffer([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".set[%d].values[%d] = \"%s\": %v", r.Name, i, j, ts, err)
}
result.SetValuesTemplate[i].Values[j] = s.String()
}
}
result.SetValues = result.SetValuesTemplate
}
return result, nil
}
func (r ReleaseSpec) Clone() (*ReleaseSpec, error) {
serialized, err := yaml.Marshal(r)
if err != nil {
return nil, fmt.Errorf("failed cloning release \"%s\": %v", r.Name, err)
}
var deserialized ReleaseSpec
if err := yaml.Unmarshal(serialized, &deserialized); err != nil {
return nil, fmt.Errorf("failed cloning release \"%s\": %v", r.Name, err)
}
return &deserialized, nil
}
func (r ReleaseSpec) Desired() bool {
return r.Installed == nil || *r.Installed
}
+21
View File
@@ -0,0 +1,21 @@
package state
import (
"fmt"
)
const ReleaseErrorCodeFailure = 1
type ReleaseError struct {
*ReleaseSpec
err error
Code int
}
func (e *ReleaseError) Error() string {
return fmt.Sprintf("failed processing release %s: %v", e.Name, e.err.Error())
}
func newReleaseError(release *ReleaseSpec, err error) *ReleaseError {
return &ReleaseError{release, err, ReleaseErrorCodeFailure}
}
@@ -62,7 +62,7 @@ func ParseLabels(l string) (LabelFilter, error) {
kv := strings.Split(label, "=")
lf.positiveLabels = append(lf.positiveLabels, kv)
} else { // malformed case
err = fmt.Errorf("Malformed label: %s. Expected label in form k=v or k!=v", label)
return lf, fmt.Errorf("Malformed label: %s. Expected label in form k=v or k!=v", label)
}
}
return lf, err
+1735
View File
File diff suppressed because it is too large Load Diff
+134
View File
@@ -0,0 +1,134 @@
package state
import (
"fmt"
"reflect"
"github.com/imdario/mergo"
"github.com/roboll/helmfile/pkg/maputil"
"github.com/roboll/helmfile/pkg/tmpl"
"gopkg.in/yaml.v2"
)
func (st *HelmState) Values() (map[string]interface{}, error) {
vals := map[string]interface{}{}
if err := mergo.Merge(&vals, st.Env.Defaults, mergo.WithOverride); err != nil {
return nil, err
}
if err := mergo.Merge(&vals, st.Env.Values, mergo.WithOverride); err != nil {
return nil, err
}
vals, err := maputil.CastKeysToStrings(vals)
if err != nil {
return nil, err
}
return vals, nil
}
func (st *HelmState) mustLoadVals() map[string]interface{} {
vals, err := st.Values()
if err != nil {
panic(err)
}
return vals
}
func (st *HelmState) valuesFileTemplateData() EnvironmentTemplateData {
return EnvironmentTemplateData{
Environment: st.Env,
Namespace: st.Namespace,
Values: st.mustLoadVals(),
}
}
func getBoolRefFromStringTemplate(templateRef string) (*bool, error) {
var result bool
if err := yaml.Unmarshal([]byte(templateRef), &result); err != nil {
return nil, fmt.Errorf("failed deserialising string %s: %v", templateRef, err)
}
return &result, nil
}
func updateBoolTemplatedValues(r *ReleaseSpec) error {
if r.InstalledTemplate != nil {
if installed, err := getBoolRefFromStringTemplate(*r.InstalledTemplate); err != nil {
return fmt.Errorf("installedTemplate: %v", err)
} else {
r.InstalledTemplate = nil
r.Installed = installed
}
}
if r.WaitTemplate != nil {
if wait, err := getBoolRefFromStringTemplate(*r.WaitTemplate); err != nil {
return fmt.Errorf("waitTemplate: %v", err)
} else {
r.WaitTemplate = nil
r.Wait = wait
}
}
if r.TillerlessTemplate != nil {
if tillerless, err := getBoolRefFromStringTemplate(*r.TillerlessTemplate); err != nil {
return fmt.Errorf("tillerlessTemplate: %v", err)
} else {
r.TillerlessTemplate = nil
r.Tillerless = tillerless
}
}
if r.VerifyTemplate != nil {
if verify, err := getBoolRefFromStringTemplate(*r.VerifyTemplate); err != nil {
return fmt.Errorf("verifyTemplate: %v", err)
} else {
r.VerifyTemplate = nil
r.Verify = verify
}
}
return nil
}
func (st *HelmState) ExecuteTemplates() (*HelmState, error) {
r := *st
vals, err := st.Values()
if err != nil {
return nil, err
}
for i, rt := range st.Releases {
successFlag := false
for it, prev := 0, &rt; it < 6; it++ {
tmplData := releaseTemplateData{
Environment: st.Env,
Release: *prev,
Values: vals,
}
renderer := tmpl.NewFileRenderer(st.readFile, st.basePath, tmplData)
r, err := rt.ExecuteTemplateExpressions(renderer)
if err != nil {
return nil, fmt.Errorf("failed executing templates in release \"%s\".\"%s\": %v", st.FilePath, rt.Name, err)
}
if reflect.DeepEqual(prev, r) {
successFlag = true
if err := updateBoolTemplatedValues(r); err != nil {
return nil, fmt.Errorf("failed executing templates in release \"%s\".\"%s\": %v", st.FilePath, rt.Name, err)
}
st.Releases[i] = *r
break
}
prev = r
}
if !successFlag {
return nil, fmt.Errorf("failed executing templates in release \"%s\".\"%s\": %s", st.FilePath, rt.Name,
"recursive references can't be resolved")
}
}
return &r, nil
}
+256
View File
@@ -0,0 +1,256 @@
package state
import (
"fmt"
"reflect"
"strings"
"testing"
"github.com/go-test/deep"
"github.com/roboll/helmfile/pkg/environment"
)
func boolPtrToString(ptr *bool) string {
if ptr == nil {
return "<nil>"
}
return fmt.Sprintf("&%t", *ptr)
}
func ptr(v interface{}) interface{} {
r := v
return reflect.ValueOf(r).Addr().Interface()
}
func TestHelmState_executeTemplates(t *testing.T) {
tests := []struct {
name string
input ReleaseSpec
want ReleaseSpec
}{
{
name: "Has template expressions in chart, values, secrets, version, labels",
input: ReleaseSpec{
Chart: "test-charts/{{ .Release.Name }}",
Version: "{{ .Release.Name }}-0.1",
Name: "test-app",
Namespace: "test-namespace-{{ .Release.Name }}",
ValuesTemplate: []interface{}{"config/{{ .Environment.Name }}/{{ .Release.Name }}/values.yaml"},
Secrets: []string{"config/{{ .Environment.Name }}/{{ .Release.Name }}/secrets.yaml"},
Labels: map[string]string{"id": "{{ .Release.Name }}"},
},
want: ReleaseSpec{
Chart: "test-charts/test-app",
Version: "test-app-0.1",
Name: "test-app",
Namespace: "test-namespace-test-app",
Values: []interface{}{"config/test_env/test-app/values.yaml"},
Secrets: []string{"config/test_env/test-app/secrets.yaml"},
Labels: map[string]string{"id": "test-app"},
},
},
{
name: "Has template expressions in name with recursive refs",
input: ReleaseSpec{
Chart: "test-chart",
Name: "{{ .Release.Labels.id }}-{{ .Release.Namespace }}",
Namespace: "dev",
Labels: map[string]string{"id": "{{ .Release.Chart }}"},
},
want: ReleaseSpec{
Chart: "test-chart",
Name: "test-chart-dev",
Namespace: "dev",
Labels: map[string]string{"id": "test-chart"},
},
},
{
name: "Has template expressions in boolean values",
input: ReleaseSpec{
Chart: "test-chart",
Name: "app-dev",
Namespace: "dev",
Labels: map[string]string{"id": "app"},
InstalledTemplate: func(i string) *string { return &i }(`{{ eq .Release.Labels.id "app" | ternary "yes" "no" }}`),
VerifyTemplate: func(i string) *string { return &i }(`{{ true }}`),
Verify: func(i bool) *bool { return &i }(false),
WaitTemplate: func(i string) *string { return &i }(`{{ false }}`),
TillerlessTemplate: func(i string) *string { return &i }(`yes`),
},
want: ReleaseSpec{
Chart: "test-chart",
Name: "app-dev",
Namespace: "dev",
Labels: map[string]string{"id": "app"},
Installed: func(i bool) *bool { return &i }(true),
Verify: func(i bool) *bool { return &i }(true),
Wait: func(i bool) *bool { return &i }(false),
Tillerless: func(i bool) *bool { return &i }(true),
},
},
{
name: "Has template in set-values",
input: ReleaseSpec{
Chart: "test-charts/chart",
Name: "test-app",
Namespace: "dev",
SetValuesTemplate: []SetValue{
SetValue{Name: "val1", Value: "{{ .Release.Name }}-val1"},
SetValue{Name: "val2", File: "{{ .Release.Name }}.yml"},
SetValue{Name: "val3", Values: []string{"{{ .Release.Name }}-val2", "{{ .Release.Name }}-val3"}},
},
},
want: ReleaseSpec{
Chart: "test-charts/chart",
Name: "test-app",
Namespace: "dev",
SetValues: []SetValue{
SetValue{Name: "val1", Value: "test-app-val1"},
SetValue{Name: "val2", File: "test-app.yml"},
SetValue{Name: "val3", Values: []string{"test-app-val2", "test-app-val3"}},
},
},
},
{
name: "Has template in values (map)",
input: ReleaseSpec{
Chart: "test-charts/chart",
Verify: nil,
Name: "app",
Namespace: "dev",
ValuesTemplate: []interface{}{map[string]string{"key": "{{ .Release.Name }}-val0"}},
},
want: ReleaseSpec{
Chart: "test-charts/chart",
Verify: nil,
Name: "app",
Namespace: "dev",
Values: []interface{}{map[interface{}]interface{}{"key": "app-val0"}},
},
},
}
for i := range tests {
tt := tests[i]
t.Run(tt.name, func(t *testing.T) {
state := &HelmState{
basePath: ".",
HelmDefaults: HelmSpec{
KubeContext: "test_context",
},
Env: environment.Environment{Name: "test_env"},
Namespace: "test-namespace_",
Repositories: nil,
Releases: []ReleaseSpec{
tt.input,
},
}
r, err := state.ExecuteTemplates()
if err != nil {
t.Errorf("Unexpected error: %v", err)
t.FailNow()
}
actual := r.Releases[0]
if !reflect.DeepEqual(actual.Name, tt.want.Name) {
t.Errorf("expected Name %+v, got %+v", tt.want.Name, actual.Name)
}
if !reflect.DeepEqual(actual.Chart, tt.want.Chart) {
t.Errorf("expected Chart %+v, got %+v", tt.want.Chart, actual.Chart)
}
if !reflect.DeepEqual(actual.Namespace, tt.want.Namespace) {
t.Errorf("expected Namespace %+v, got %+v", tt.want.Namespace, actual.Namespace)
}
if diff := deep.Equal(actual.Values, tt.want.Values); diff != nil && len(actual.Values) > 0 {
t.Errorf("Values differs \n%+v", strings.Join(diff, "\n"))
}
if diff := deep.Equal(actual.Secrets, tt.want.Secrets); diff != nil && len(actual.Secrets) > 0 {
t.Errorf("Secrets differs \n%+v", strings.Join(diff, "\n"))
}
if diff := deep.Equal(actual.SetValues, tt.want.SetValues); diff != nil && len(actual.SetValues) > 0 {
t.Errorf("SetValues differs \n%+v", strings.Join(diff, "\n"))
}
if diff := deep.Equal(actual.Labels, tt.want.Labels); diff != nil && len(actual.Labels) > 0 {
t.Errorf("Labels differs \n%+v", strings.Join(diff, "\n"))
}
if !reflect.DeepEqual(actual.Version, tt.want.Version) {
t.Errorf("expected Version %+v, got %+v", tt.want.Version, actual.Version)
}
if !reflect.DeepEqual(actual.Installed, tt.want.Installed) {
t.Errorf("expected actual.Installed %+v, got %+v",
boolPtrToString(tt.want.Installed), boolPtrToString(actual.Installed),
)
}
if !reflect.DeepEqual(actual.Tillerless, tt.want.Tillerless) {
t.Errorf("expected actual.Tillerless %+v, got %+v",
boolPtrToString(tt.want.Tillerless), boolPtrToString(actual.Tillerless),
)
}
if !reflect.DeepEqual(actual.Verify, tt.want.Verify) {
t.Errorf("expected actual.Verify %+v, got %+v",
boolPtrToString(tt.want.Verify), boolPtrToString(actual.Verify),
)
}
if !reflect.DeepEqual(actual.Wait, tt.want.Wait) {
t.Errorf("expected actual.Wait %+v, got %+v",
boolPtrToString(tt.want.Wait), boolPtrToString(actual.Wait),
)
}
})
}
}
func TestHelmState_recursiveRefsTemplates(t *testing.T) {
tests := []struct {
name string
input ReleaseSpec
}{
{
name: "Has reqursive references",
input: ReleaseSpec{
Chart: "test-charts/{{ .Release.Name }}",
Verify: nil,
Name: "{{ .Release.Labels.id }}",
Namespace: "dev",
Labels: map[string]string{"id": "app-{{ .Release.Name }}"},
},
},
{
name: "Has unresolvable boolean templates",
input: ReleaseSpec{
Name: "app-dev",
Chart: "test-charts/app",
Verify: nil,
Namespace: "dev",
WaitTemplate: func(i string) *string { return &i }("hi"),
},
},
}
for i := range tests {
tt := tests[i]
t.Run(tt.name, func(t *testing.T) {
state := &HelmState{
basePath: ".",
HelmDefaults: HelmSpec{
KubeContext: "test_context",
},
Env: environment.Environment{Name: "test_env"},
Namespace: "test-namespace_",
Repositories: nil,
Releases: []ReleaseSpec{
tt.input,
},
}
r, err := state.ExecuteTemplates()
if err == nil {
t.Errorf("Expected error, got valid response: %v", r)
t.FailNow()
}
})
}
}
+98
View File
@@ -0,0 +1,98 @@
package state
import (
"fmt"
"sync"
"github.com/roboll/helmfile/pkg/helmexec"
)
type result struct {
release ReleaseSpec
err error
}
func (st *HelmState) scatterGather(concurrency int, items int, produceInputs func(), receiveInputsAndProduceIntermediates func(int), aggregateIntermediates func()) {
if concurrency < 1 || concurrency > items {
concurrency = items
}
for _, r := range st.Releases {
if r.Tillerless != nil {
if *r.Tillerless {
concurrency = 1
}
} else if st.HelmDefaults.Tillerless {
concurrency = 1
}
}
// WaitGroup is required to wait until goroutine per job in job queue cleanly stops.
var waitGroup sync.WaitGroup
waitGroup.Add(concurrency)
go produceInputs()
for w := 1; w <= concurrency; w++ {
go func(id int) {
st.logger.Debugf("worker %d/%d started", id, concurrency)
receiveInputsAndProduceIntermediates(id)
st.logger.Debugf("worker %d/%d finished", id, concurrency)
waitGroup.Done()
}(w)
}
aggregateIntermediates()
// Wait until all the goroutines to gracefully finish
waitGroup.Wait()
}
func (st *HelmState) scatterGatherReleases(helm helmexec.Interface, concurrency int,
do func(ReleaseSpec, int) error) []error {
var errs []error
inputs := st.Releases
inputsSize := len(inputs)
releases := make(chan ReleaseSpec)
results := make(chan result)
st.scatterGather(
concurrency,
inputsSize,
func() {
for _, release := range inputs {
releases <- release
}
close(releases)
},
func(id int) {
for release := range releases {
err := do(release, id)
st.logger.Debugf("sending result for release: %s\n", release.Name)
results <- result{release: release, err: err}
st.logger.Debugf("sent result for release: %s\n", release.Name)
}
},
func() {
for i := range inputs {
st.logger.Debugf("receiving result %d", i)
r := <-results
if r.err != nil {
errs = append(errs, fmt.Errorf("release \"%s\" failed: %v", r.release.Name, r.err))
} else {
st.logger.Debugf("received result for release \"%s\"", r.release.Name)
}
st.logger.Debugf("received result for %d", i)
}
},
)
if len(errs) != 0 {
return errs
}
return nil
}
File diff suppressed because it is too large Load Diff
+99
View File
@@ -0,0 +1,99 @@
package state
import (
"fmt"
"go.uber.org/zap"
"net/url"
"path/filepath"
"sort"
)
type Storage struct {
logger *zap.SugaredLogger
FilePath string
basePath string
glob func(string) ([]string, error)
}
func NewStorage(forFile string, logger *zap.SugaredLogger, glob func(string) ([]string, error)) *Storage {
return &Storage{
FilePath: forFile,
basePath: filepath.Dir(forFile),
logger: logger,
glob: glob,
}
}
func (st *Storage) resolveFile(missingFileHandler *string, tpe, path string) ([]string, bool, error) {
title := fmt.Sprintf("%s file", tpe)
files, err := st.ExpandPaths(path)
if err != nil {
return nil, false, err
}
var handlerId string
if missingFileHandler != nil {
handlerId = *missingFileHandler
} else {
handlerId = MissingFileHandlerError
}
if len(files) == 0 {
switch handlerId {
case MissingFileHandlerError:
return nil, false, fmt.Errorf("%s matching \"%s\" does not exist in \"%s\"", title, path, st.basePath)
case MissingFileHandlerWarn:
st.logger.Warnf("skipping missing %s matching \"%s\"", title, path)
return nil, true, nil
case MissingFileHandlerInfo:
st.logger.Infof("skipping missing %s matching \"%s\"", title, path)
return nil, true, nil
case MissingFileHandlerDebug:
st.logger.Debugf("skipping missing %s matching \"%s\"", title, path)
return nil, true, nil
default:
available := []string{
MissingFileHandlerError,
MissingFileHandlerWarn,
MissingFileHandlerInfo,
MissingFileHandlerDebug,
}
return nil, false, fmt.Errorf("invalid missing file handler \"%s\" while processing \"%s\" in \"%s\": it must be one of %s", handlerId, path, st.FilePath, available)
}
}
return files, false, nil
}
func (st *Storage) ExpandPaths(globPattern string) ([]string, error) {
result := []string{}
absPathPattern := st.normalizePath(globPattern)
matches, err := st.glob(absPathPattern)
if err != nil {
return nil, fmt.Errorf("failed processing %s: %v", globPattern, err)
}
sort.Strings(matches)
result = append(result, matches...)
return result, nil
}
// normalizes relative path to absolute one
func (st *Storage) normalizePath(path string) string {
u, _ := url.Parse(path)
if u.Scheme != "" || filepath.IsAbs(path) {
return path
} else {
return st.JoinBase(path)
}
}
// JoinBase returns an absolute path in the form basePath/relative
func (st *Storage) JoinBase(relPath string) string {
return filepath.Join(st.basePath, relPath)
}
+30
View File
@@ -0,0 +1,30 @@
package state
import (
"github.com/roboll/helmfile/pkg/environment"
)
// TemplateSpec defines the structure of a reusable and composable template for helm releases.
type TemplateSpec struct {
ReleaseSpec `yaml:",inline"`
}
// EnvironmentTemplateData provides variables accessible while executing golang text/template expressions in helmfile and values YAML files
type EnvironmentTemplateData struct {
// Environment is accessible as `.Environment` from any template executed by the renderer
Environment environment.Environment
// Namespace is accessible as `.Namespace` from any non-values template executed by the renderer
Namespace string
// Values is accessible as `.Values` and it contains default state values overrode by environment values and override values.
Values map[string]interface{}
}
// releaseTemplateData provides variables accessible while executing golang text/template expressions in releases of a helmfile YAML file
type releaseTemplateData struct {
// Environment is accessible as `.Environment` from any template expression executed by the renderer
Environment environment.Environment
// Release is accessible as `.Release` from any template expression executed by the renderer
Release ReleaseSpec
// Values is accessible as `.Values` and it contains default state values overrode by environment values and override values.
Values map[string]interface{}
}
+52
View File
@@ -0,0 +1,52 @@
package state
import (
"path/filepath"
"regexp"
"strings"
)
func isLocalChart(chart string) bool {
regex, _ := regexp.Compile("^[.]?./")
matched := regex.MatchString(chart)
if matched {
return true
}
uriLike := strings.Index(chart, "://") > -1
if uriLike {
return false
}
return chart == "" ||
chart[0] == '/' ||
strings.Index(chart, "/") == -1 ||
len(strings.Split(chart, "/")) != 2
}
func resolveRemoteChart(repoAndChart string) (string, string, bool) {
if isLocalChart(repoAndChart) {
return "", "", false
}
parts := strings.Split(repoAndChart, "/")
if len(parts) != 2 {
return "", "", false
}
repo := parts[0]
chart := parts[1]
return repo, chart, true
}
// normalizeChart allows for the distinction between a file path reference and repository references.
// - Any single (or double character) followed by a `/` will be considered a local file reference and
// be constructed relative to the `base path`.
// - Everything else is assumed to be an absolute path or an actual <repository>/<chart> reference.
func normalizeChart(basePath, chart string) string {
if !isLocalChart(chart) || chart[0] == '/' {
return chart
}
return filepath.Join(basePath, chart)
}
+147
View File
@@ -0,0 +1,147 @@
package state
import "testing"
func TestIsLocalChart(t *testing.T) {
testcases := []struct {
input string
expected bool
}{
{
input: "mychart",
expected: true,
},
{
input: "stable/mysql",
expected: false,
},
{
input: "./charts/myapp",
expected: true,
},
{
input: "charts/mysubsystem/myapp",
expected: true,
},
{
input: "./charts/mysubsystem/myapp",
expected: true,
},
{
input: "/charts/mysubsystem/myapp",
expected: true,
},
{
// Regression test case for:
// * https://github.com/roboll/helmfile/issues/675
// * https://github.com/roboll/helmfile/issues/687
input: "https://github.com/arangodb/kube-arangodb/releases/download/0.3.11/kube-arangodb-crd.tgz",
expected: false,
},
{
input: "https://example.com/bar.tgz",
expected: false,
},
}
for i := range testcases {
testcase := testcases[i]
actual := isLocalChart(testcase.input)
if testcase.expected != actual {
t.Errorf("unexpected result: isLocalChart(\"%s\"): expected=%v, got=%v", testcase.input, testcase.expected, actual)
}
}
}
func TestResolveRemortChart(t *testing.T) {
testcases := []struct {
input string
repo string
chart string
remote bool
}{
{
input: "mychart",
remote: false,
},
{
input: "stable/mysql",
repo: "stable",
chart: "mysql",
remote: true,
},
{
input: "./charts/myapp",
remote: false,
},
{
input: "charts/mysubsystem/myapp",
remote: false,
},
{
input: "./charts/mysubsystem/myapp",
remote: false,
},
{
input: "/charts/mysubsystem/myapp",
remote: false,
},
{
// Regression test case for:
// * https://github.com/roboll/helmfile/issues/675
// * https://github.com/roboll/helmfile/issues/687
input: "https://github.com/arangodb/kube-arangodb/releases/download/0.3.11/kube-arangodb-crd.tgz",
remote: false,
},
{
input: "https://example.com/bar.tgz",
remote: false,
},
}
for i := range testcases {
testcase := testcases[i]
repo, chart, actual := resolveRemoteChart(testcase.input)
if testcase.remote != actual {
t.Fatalf("unexpected result: reolveRemoteChart(\"%s\"): expected=%v, got=%v", testcase.input, testcase.remote, actual)
}
if testcase.repo != repo {
t.Errorf("unexpected repo: %s: expected=%v, got=%v", testcase.input, testcase.repo, repo)
}
if testcase.chart != chart {
t.Errorf("unexpected chart: %s: expected=%v, got=%v", testcase.input, testcase.chart, chart)
}
}
}
func TestNormalizeChart(t *testing.T) {
testcases := []struct {
input string
expected string
}{
{
input: "mychart",
expected: "/path/to/mychart",
},
{
input: "/charts/mychart",
expected: "/charts/mychart",
},
}
for i := range testcases {
testcase := testcases[i]
actual := normalizeChart("/path/to", testcase.input)
if testcase.expected != actual {
t.Fatalf("unexpected result: normalizeChart(\"/path/to\", \"%s\"): expected=%v, got=%v", testcase.input, testcase.expected, actual)
}
}
}
+137
View File
@@ -0,0 +1,137 @@
package testhelper
import (
"fmt"
"os"
"path/filepath"
"strings"
)
type TestFs struct {
Cwd string
dirs map[string]bool
files map[string]string
GlobFixtures map[string][]string
fileReaderCalls int
successfulReads []string
}
func NewTestFs(files map[string]string) *TestFs {
dirs := map[string]bool{}
for abs, _ := range files {
for d := filepath.Dir(abs); !dirs[d]; d = filepath.Dir(d) {
dirs[d] = true
fmt.Fprintf(os.Stderr, "testfs: recognized dir: %s\n", d)
}
}
return &TestFs{
Cwd: "/path/to",
dirs: dirs,
files: files,
successfulReads: []string{},
GlobFixtures: map[string][]string{},
}
}
func (f *TestFs) FileExistsAt(path string) bool {
var ok bool
if strings.Contains(path, "/") {
_, ok = f.files[path]
} else {
_, ok = f.files[filepath.Join(f.Cwd, path)]
}
return ok
}
func (f *TestFs) FileExists(path string) (bool, error) {
return f.FileExistsAt(path), nil
}
func (f *TestFs) DirectoryExistsAt(path string) bool {
var ok bool
if strings.Contains(path, "/") {
_, ok = f.dirs[path]
} else {
_, ok = f.dirs[filepath.Join(f.Cwd, path)]
}
return ok
}
func (f *TestFs) ReadFile(filename string) ([]byte, error) {
var str string
var ok bool
if filename[0] == '/' {
str, ok = f.files[filename]
} else {
str, ok = f.files[filepath.Join(f.Cwd, filename)]
}
if !ok {
return []byte(nil), fmt.Errorf("no registered file found: %s", filename)
}
f.fileReaderCalls += 1
f.successfulReads = append(f.successfulReads, filename)
return []byte(str), nil
}
func (f *TestFs) SuccessfulReads() []string {
return f.successfulReads
}
func (f *TestFs) FileReaderCalls() int {
return f.fileReaderCalls
}
func (f *TestFs) Glob(relPattern string) ([]string, error) {
var pattern string
if relPattern[0] == '/' {
pattern = relPattern
} else {
pattern = filepath.Join(f.Cwd, relPattern)
}
fixtures, ok := f.GlobFixtures[pattern]
if ok {
return fixtures, nil
}
matches := []string{}
for name, _ := range f.files {
matched, err := filepath.Match(pattern, name)
if err != nil {
return nil, err
}
if matched {
matches = append(matches, name)
}
}
return matches, nil
}
func (f *TestFs) Abs(path string) (string, error) {
var p string
if path[0] == '/' {
p = path
} else {
p = filepath.Join(f.Cwd, path)
}
return filepath.Clean(p), nil
}
func (f *TestFs) Getwd() (string, error) {
return f.Cwd, nil
}
func (f *TestFs) Chdir(dir string) error {
if _, ok := f.dirs[dir]; ok {
f.Cwd = dir
return nil
}
return fmt.Errorf("unexpected chdir \"%s\"", dir)
}
+7
View File
@@ -0,0 +1,7 @@
package tmpl
type Context struct {
preRender bool
basePath string
readFile func(string) ([]byte, error)
}
+208
View File
@@ -0,0 +1,208 @@
package tmpl
import (
"fmt"
"gopkg.in/yaml.v2"
"io"
"os"
"os/exec"
"path/filepath"
"reflect"
"strings"
"text/template"
)
type Values = map[string]interface{}
func (c *Context) createFuncMap() template.FuncMap {
funcMap := template.FuncMap{
"exec": c.Exec,
"readFile": c.ReadFile,
"toYaml": ToYaml,
"fromYaml": FromYaml,
"setValueAtPath": SetValueAtPath,
"requiredEnv": RequiredEnv,
"get": get,
"getOrNil": getOrNil,
"tpl": c.Tpl,
}
if c.preRender {
// disable potential side-effect template calls
funcMap["exec"] = func(string, []interface{}, ...string) (string, error) {
return "", nil
}
funcMap["readFile"] = func(string) (string, error) {
return "", nil
}
}
return funcMap
}
func (c *Context) Exec(command string, args []interface{}, inputs ...string) (string, error) {
var input string
if len(inputs) > 0 {
input = inputs[0]
}
strArgs := make([]string, len(args))
for i, a := range args {
switch a.(type) {
case string:
strArgs[i] = a.(string)
default:
return "", fmt.Errorf("unexpected type of arg \"%s\" in args %v at index %d", reflect.TypeOf(a), args, i)
}
}
cmd := exec.Command(command, strArgs...)
cmd.Dir = c.basePath
writeErrs := make(chan error)
cmdErrs := make(chan error)
cmdOuts := make(chan []byte)
if len(input) > 0 {
stdin, err := cmd.StdinPipe()
if err != nil {
return "", err
}
go func(input string, stdin io.WriteCloser) {
defer stdin.Close()
defer close(writeErrs)
size := len(input)
var n int
var err error
i := 0
for {
n, err = io.WriteString(stdin, input[i:])
if err != nil {
writeErrs <- fmt.Errorf("failed while writing %d bytes to stdin of \"%s\": %v", len(input), command, err)
break
}
i += n
if n == size {
break
}
}
}(input, stdin)
}
go func() {
defer close(cmdOuts)
defer close(cmdErrs)
bytes, err := cmd.Output()
if err != nil {
cmdErrs <- fmt.Errorf("exec cmd=%s args=[%s] failed: %v", command, strings.Join(strArgs, ", "), err)
} else {
cmdOuts <- bytes
}
}()
for {
select {
case bytes := <-cmdOuts:
return string(bytes), nil
case err := <-cmdErrs:
return "", err
case err := <-writeErrs:
return "", err
}
}
}
func (c *Context) ReadFile(filename string) (string, error) {
var path string
if filepath.IsAbs(filename) {
path = filename
} else {
path = filepath.Join(c.basePath, filename)
}
bytes, err := c.readFile(path)
if err != nil {
return "", err
}
return string(bytes), nil
}
func (c *Context) Tpl(text string, data interface{}) (string, error) {
buf, err := c.RenderTemplateToBuffer(text, data)
if err != nil {
return "", err
}
return buf.String(), nil
}
func ToYaml(v interface{}) (string, error) {
data, err := yaml.Marshal(v)
if err != nil {
return "", err
}
return string(data), nil
}
func FromYaml(str string) (Values, error) {
m := Values{}
if err := yaml.Unmarshal([]byte(str), &m); err != nil {
return nil, fmt.Errorf("%s, offending yaml: %s", err, str)
}
return m, nil
}
func SetValueAtPath(path string, value interface{}, values Values) (Values, error) {
var current interface{}
current = values
components := strings.Split(path, ".")
pathToMap := components[:len(components)-1]
key := components[len(components)-1]
for _, k := range pathToMap {
var elem interface{}
switch typedCurrent := current.(type) {
case map[string]interface{}:
v, exists := typedCurrent[k]
if !exists {
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" does not exist", path, k)
}
elem = v
case map[interface{}]interface{}:
v, exists := typedCurrent[k]
if !exists {
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" does not exist", path, k)
}
elem = v
default:
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" was not a map", path, k)
}
switch typedElem := elem.(type) {
case map[string]interface{}, map[interface{}]interface{}:
current = typedElem
default:
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" was not a map", path, k)
}
}
switch typedCurrent := current.(type) {
case map[string]interface{}:
typedCurrent[key] = value
case map[interface{}]interface{}:
typedCurrent[key] = value
default:
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" was not a map", path, key)
}
return values, nil
}
func RequiredEnv(name string) (string, error) {
if val, exists := os.LookupEnv(name); exists && len(val) > 0 {
return val, nil
}
return "", fmt.Errorf("required env var `%s` is not set", name)
}
+135
View File
@@ -0,0 +1,135 @@
package tmpl
import (
"fmt"
"path/filepath"
"reflect"
"testing"
)
func TestReadFile(t *testing.T) {
expected := `foo:
bar: BAR
`
expectedFilename := "values.yaml"
ctx := &Context{basePath: ".", readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(expected), nil
}}
actual, err := ctx.ReadFile(expectedFilename)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestReadFile_PassAbsPath(t *testing.T) {
expected := `foo:
bar: BAR
`
expectedFilename, _ := filepath.Abs("values.yaml")
ctx := &Context{basePath: ".", readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(expected), nil
}}
actual, err := ctx.ReadFile(expectedFilename)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestToYaml(t *testing.T) {
expected := `foo:
bar: BAR
`
vals := Values(map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "BAR",
},
})
actual, err := ToYaml(vals)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestFromYaml(t *testing.T) {
raw := `foo:
bar: BAR
`
expected := Values(map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "BAR",
},
})
actual, err := FromYaml(raw)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestSetValueAtPath_OneComponent(t *testing.T) {
input := map[string]interface{}{
"foo": "",
}
expected := map[string]interface{}{
"foo": "FOO",
}
actual, err := SetValueAtPath("foo", "FOO", input)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestSetValueAtPath_TwoComponents(t *testing.T) {
input := map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "",
},
}
expected := map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "FOO_BAR",
},
}
actual, err := SetValueAtPath("foo.bar", "FOO_BAR", input)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestTpl(t *testing.T) {
text := `foo: {{ .foo }}
`
expected := `foo: FOO
`
ctx := &Context{basePath: "."}
actual, err := ctx.Tpl(text, map[string]interface{}{"foo": "FOO"})
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
+41
View File
@@ -0,0 +1,41 @@
package tmpl
import (
"bytes"
"github.com/Masterminds/sprig"
"text/template"
)
func (c *Context) stringTemplate() *template.Template {
funcMap := sprig.TxtFuncMap()
for name, f := range c.createFuncMap() {
funcMap[name] = f
}
tmpl := template.New("stringTemplate").Funcs(funcMap)
if c.preRender {
tmpl = tmpl.Option("missingkey=zero")
} else {
tmpl = tmpl.Option("missingkey=error")
}
return tmpl
}
func (c *Context) RenderTemplateToBuffer(s string, data ...interface{}) (*bytes.Buffer, error) {
var t, parseErr = c.stringTemplate().Parse(s)
if parseErr != nil {
return nil, parseErr
}
var tplString bytes.Buffer
var d interface{}
if len(data) > 0 {
d = data[0]
}
var execErr = t.Execute(&tplString, d)
if execErr != nil {
return &tplString, execErr
}
return &tplString, nil
}
+252
View File
@@ -0,0 +1,252 @@
package tmpl
import (
"fmt"
"os"
"reflect"
"testing"
)
func TestRenderTemplate_Values(t *testing.T) {
valuesYamlContent := `foo:
bar: BAR
`
expected := `foo:
bar: FOO_BAR
`
expectedFilename := "values.yaml"
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(valuesYamlContent), nil
}}
buf, err := ctx.RenderTemplateToBuffer(`{{ readFile "values.yaml" | fromYaml | setValueAtPath "foo.bar" "FOO_BAR" | toYaml }}`)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := buf.String()
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestRenderTemplate_WithData(t *testing.T) {
valuesYamlContent := `foo:
bar: {{ .foo.bar }}
`
expected := `foo:
bar: FOO_BAR
`
expectedFilename := "values.yaml"
data := map[string]interface{}{
"foo": map[string]interface{}{
"bar": "FOO_BAR",
},
}
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(valuesYamlContent), nil
}}
buf, err := ctx.RenderTemplateToBuffer(valuesYamlContent, data)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := buf.String()
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestRenderTemplate_AccessingMissingKeyWithGetOrNil(t *testing.T) {
valuesYamlContent := `foo:
bar: {{ . | getOrNil "foo.bar" }}
`
expected := `foo:
bar: <no value>
`
expectedFilename := "values.yaml"
data := map[string]interface{}{}
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(valuesYamlContent), nil
}}
buf, err := ctx.RenderTemplateToBuffer(valuesYamlContent, data)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := buf.String()
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestRenderTemplate_Defaulting(t *testing.T) {
valuesYamlContent := `foo:
bar: {{ . | getOrNil "foo.bar" | default "DEFAULT" }}
`
expected := `foo:
bar: DEFAULT
`
expectedFilename := "values.yaml"
data := map[string]interface{}{}
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(valuesYamlContent), nil
}}
buf, err := ctx.RenderTemplateToBuffer(valuesYamlContent, data)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := buf.String()
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func renderTemplateToString(s string, data ...interface{}) (string, error) {
ctx := &Context{readFile: func(filename string) ([]byte, error) {
return nil, fmt.Errorf("unexpected call to readFile: filename=%s", filename)
}}
tplString, err := ctx.RenderTemplateToBuffer(s, data...)
if err != nil {
return "", err
}
return tplString.String(), nil
}
func Test_renderTemplateToString(t *testing.T) {
type args struct {
s string
envs map[string]string
data interface{}
}
tests := []struct {
name string
args args
want string
wantErr bool
}{
{
name: "simple replacement",
args: args{
s: "{{ env \"HF_TEST_VAR\" }}",
envs: map[string]string{
"HF_TEST_VAR": "content",
},
},
want: "content",
wantErr: false,
},
{
name: "two replacements",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" }}{{ env \"HF_TEST_BETA\" }}",
envs: map[string]string{
"HF_TEST_ALPHA": "first",
"HF_TEST_BETA": "second",
},
},
want: "firstsecond",
wantErr: false,
},
{
name: "replacement and comment",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" }}{{/* comment */}}",
envs: map[string]string{
"HF_TEST_ALPHA": "first",
},
},
want: "first",
wantErr: false,
},
{
name: "global template function",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" | len }}",
envs: map[string]string{
"HF_TEST_ALPHA": "abcdefg",
},
},
want: "7",
wantErr: false,
},
{
name: "get",
args: args{
s: `{{ . | get "Foo" }}, {{ . | get "Bar" "2" }}`,
envs: map[string]string{},
data: map[string]interface{}{
"Foo": "1",
},
},
want: "1, 2",
wantErr: false,
},
{
name: "env var not set",
args: args{
s: "{{ env \"HF_TEST_NONE\" }}",
envs: map[string]string{
"HF_TEST_THIS": "first",
},
},
want: "",
},
{
name: "undefined function",
args: args{
s: "{{ env foo }}",
envs: map[string]string{
"foo": "bar",
},
},
wantErr: true,
},
{
name: "required env var",
args: args{
s: "{{ requiredEnv \"HF_TEST\" }}",
envs: map[string]string{
"HF_TEST": "value",
},
},
want: "value",
wantErr: false,
},
{
name: "required env var not set",
args: args{
s: "{{ requiredEnv \"HF_TEST_NONE\" }}",
envs: map[string]string{},
},
wantErr: true,
},
}
for i := range tests {
tt := tests[i]
t.Run(tt.name, func(t *testing.T) {
for k, v := range tt.args.envs {
err := os.Setenv(k, v)
if err != nil {
t.Error("renderTemplateToString() could not set env var for testing")
}
}
got, err := renderTemplateToString(tt.args.s, tt.args.data)
if (err != nil) != tt.wantErr {
t.Errorf("renderTemplateToString() for %s error = %v, wantErr %v", tt.name, err, tt.wantErr)
return
}
if got != tt.want {
t.Errorf("renderTemplateToString() for %s = %v, want %v", tt.name, got, tt.want)
}
})
}
}
+80
View File
@@ -0,0 +1,80 @@
package tmpl
import (
"bytes"
"io/ioutil"
"fmt"
"strings"
)
type FileRenderer struct {
ReadFile func(string) ([]byte, error)
Context *Context
Data interface{}
}
func NewFileRenderer(readFile func(filename string) ([]byte, error), basePath string, data interface{}) *FileRenderer {
return &FileRenderer{
ReadFile: readFile,
Context: &Context{
basePath: basePath,
readFile: readFile,
},
Data: data,
}
}
func NewFirstPassRenderer(basePath string, data interface{}) *FileRenderer {
return &FileRenderer{
ReadFile: ioutil.ReadFile,
Context: &Context{
preRender: true,
basePath: basePath,
readFile: ioutil.ReadFile,
},
Data: data,
}
}
func (r *FileRenderer) RenderTemplateFileToBuffer(file string) (*bytes.Buffer, error) {
content, err := r.ReadFile(file)
if err != nil {
return nil, err
}
return r.RenderTemplateContentToBuffer(content)
}
// RenderToBytes loads the content of the file.
// If its extension is `gotmpl` it treats the content as a go template and renders it.
func (r *FileRenderer) RenderToBytes(path string) ([]byte, error) {
var yamlBytes []byte
splits := strings.Split(path, ".")
if len(splits) > 0 && splits[len(splits)-1] == "gotmpl" {
yamlBuf, err := r.RenderTemplateFileToBuffer(path)
if err != nil {
return nil, fmt.Errorf("failed to render [%s], because of %v", path, err)
}
yamlBytes = yamlBuf.Bytes()
} else {
var err error
yamlBytes, err = r.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("failed to load [%s]: %v", path, err)
}
}
return yamlBytes, nil
}
func (r *FileRenderer) RenderTemplateContentToBuffer(content []byte) (*bytes.Buffer, error) {
return r.Context.RenderTemplateToBuffer(string(content), r.Data)
}
func (r *FileRenderer) RenderTemplateContentToString(content []byte) (string, error) {
buf, err := r.Context.RenderTemplateToBuffer(string(content), r.Data)
if err != nil {
return "", err
}
return buf.String(), nil
}
+67
View File
@@ -0,0 +1,67 @@
package tmpl
import (
"fmt"
"github.com/roboll/helmfile/pkg/environment"
"reflect"
"testing"
)
var emptyEnvTmplData = map[string]interface{}{
"Environment": environment.EmptyEnvironment,
"Namespace": "",
}
func TestRenderToBytes_Gotmpl(t *testing.T) {
valuesYamlTmplContent := `foo:
bar: '{{ readFile "data.txt" }}'
`
dataFileContent := "FOO_BAR"
expected := `foo:
bar: 'FOO_BAR'
`
dataFile := "data.txt"
valuesTmplFile := "values.yaml.gotmpl"
r := NewFileRenderer(func(filename string) ([]byte, error) {
switch filename {
case valuesTmplFile:
return []byte(valuesYamlTmplContent), nil
case dataFile:
return []byte(dataFileContent), nil
}
return nil, fmt.Errorf("unexpected filename: expected=%v or %v, actual=%s", dataFile, valuesTmplFile, filename)
}, "", emptyEnvTmplData)
buf, err := r.RenderToBytes(valuesTmplFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := string(buf)
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestRenderToBytes_Yaml(t *testing.T) {
valuesYamlContent := `foo:
bar: '{{ readFile "data.txt" }}'
`
expected := `foo:
bar: '{{ readFile "data.txt" }}'
`
valuesFile := "values.yaml"
r := NewFileRenderer(func(filename string) ([]byte, error) {
switch filename {
case valuesFile:
return []byte(valuesYamlContent), nil
}
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", valuesFile, filename)
}, "", emptyEnvTmplData)
buf, err := r.RenderToBytes(valuesFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := string(buf)
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
+91
View File
@@ -0,0 +1,91 @@
package tmpl
import (
"fmt"
"reflect"
"strings"
)
type noValueError struct {
msg string
}
func (e *noValueError) Error() string {
return e.msg
}
func get(path string, varArgs ...interface{}) (interface{}, error) {
var defSet bool
var def interface{}
var obj interface{}
switch len(varArgs) {
case 1:
defSet = false
def = nil
obj = varArgs[0]
case 2:
defSet = true
def = varArgs[0]
obj = varArgs[1]
default:
return nil, fmt.Errorf("unexpected number of args pased to the template function get(path, [def, ]obj): expected 1 or 2, got %d, args was %v", len(varArgs), varArgs)
}
if path == "" {
return obj, nil
}
keys := strings.Split(path, ".")
var v interface{}
var ok bool
switch typedObj := obj.(type) {
case map[string]interface{}:
v, ok = typedObj[keys[0]]
if !ok {
if defSet {
return def, nil
}
return nil, &noValueError{fmt.Sprintf("no value exist for key \"%s\" in %v", keys[0], typedObj)}
}
case map[interface{}]interface{}:
v, ok = typedObj[keys[0]]
if !ok {
if defSet {
return def, nil
}
return nil, &noValueError{fmt.Sprintf("no value exist for key \"%s\" in %v", keys[0], typedObj)}
}
default:
maybeStruct := reflect.ValueOf(typedObj)
if maybeStruct.Kind() != reflect.Struct {
return nil, &noValueError{fmt.Sprintf("unexpected type(%v) of value for key \"%s\": it must be either map[string]interface{} or any struct", reflect.TypeOf(obj), keys[0])}
} else if maybeStruct.NumField() < 1 {
return nil, &noValueError{fmt.Sprintf("no accessible struct fields for key \"%s\"", keys[0])}
}
f := maybeStruct.FieldByName(keys[0])
if !f.IsValid() {
if defSet {
return def, nil
}
return nil, &noValueError{fmt.Sprintf("no field named \"%s\" exist in %v", keys[0], typedObj)}
}
v = f.Interface()
}
if defSet {
return get(strings.Join(keys[1:], "."), def, v)
}
return get(strings.Join(keys[1:], "."), v)
}
func getOrNil(path string, o interface{}) (interface{}, error) {
v, err := get(path, o)
if err != nil {
switch err.(type) {
case *noValueError:
return nil, nil
default:
return nil, err
}
}
return v, nil
}
+128
View File
@@ -0,0 +1,128 @@
package tmpl
import (
"testing"
)
type EmptyStruct struct {
}
func TestGetStruct(t *testing.T) {
type Foo struct{ Bar string }
obj := struct{ Foo }{Foo{Bar: "Bar"}}
v1, err := get("Foo.Bar", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v1 != "Bar" {
t.Errorf("unexpected value for path Foo.Bar in %v: expected=Bar, actual=%v", obj, v1)
}
_, err = get("Foo.baz", obj)
if err == nil {
t.Errorf("expected error but was not occurred")
}
_, err = get("foo", EmptyStruct{})
if err == nil {
t.Errorf("expected error but was not occurred")
}
}
func TestGetMap(t *testing.T) {
obj := map[string]interface{}{"Foo": map[string]interface{}{"Bar": "Bar"}}
v1, err := get("Foo.Bar", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v1 != "Bar" {
t.Errorf("unexpected value for path Foo.Bar in %v: expected=Bar, actual=%v", obj, v1)
}
_, err = get("Foo.baz", obj)
if err == nil {
t.Errorf("expected error but was not occurred")
}
}
func TestGet_Default(t *testing.T) {
obj := map[string]interface{}{"Foo": map[string]interface{}{}, "foo": 1}
v1, err := get("Foo.Bar", "Bar", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v1 != "Bar" {
t.Errorf("unexpected value for path Foo.Bar in %v: expected=Bar, actual=%v", obj, v1)
}
v2, err := get("Baz", "Baz", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v2 != "Baz" {
t.Errorf("unexpected value for path Baz in %v: expected=Baz, actual=%v", obj, v2)
}
_, err = get("foo.Bar", "fooBar", obj)
if err == nil {
t.Errorf("expected error but was not occurred")
}
}
func TestGetOrNilStruct(t *testing.T) {
type Foo struct{ Bar string }
obj := struct{ Foo }{Foo{Bar: "Bar"}}
v1, err := getOrNil("Foo.Bar", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v1 != "Bar" {
t.Errorf("unexpected value for path Foo.Bar in %v: expected=Bar, actual=%v", obj, v1)
}
v2, err := getOrNil("Foo.baz", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v2 != nil {
t.Errorf("unexpected value for path Foo.baz in %v: expected=nil, actual=%v", obj, v2)
}
}
func TestGetOrNilMap(t *testing.T) {
obj := map[string]interface{}{"Foo": map[string]interface{}{"Bar": "Bar"}}
v1, err := getOrNil("Foo.Bar", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v1 != "Bar" {
t.Errorf("unexpected value for path Foo.Bar in %v: expected=Bar, actual=%v", obj, v1)
}
v2, err := getOrNil("Foo.baz", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v2 != nil {
t.Errorf("unexpected value for path Foo.baz in %v: expected=nil, actual=%v", obj, v2)
}
}
+30
View File
@@ -0,0 +1,30 @@
package tmpl
type templateTextRenderer struct {
ReadText func(string) ([]byte, error)
Context *Context
Data interface{}
}
type TextRenderer interface {
RenderTemplateText(text string) (string, error)
}
func NewTextRenderer(readFile func(filename string) ([]byte, error), basePath string, data interface{}) *templateTextRenderer {
return &templateTextRenderer{
ReadText: readFile,
Context: &Context{
basePath: basePath,
readFile: readFile,
},
Data: data,
}
}
func (r *templateTextRenderer) RenderTemplateText(text string) (string, error) {
buf, err := r.Context.RenderTemplateToBuffer(text, r.Data)
if err != nil {
return "", err
}
return buf.String(), nil
}
-730
View File
@@ -1,730 +0,0 @@
package state
import (
"errors"
"fmt"
"io/ioutil"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"sync"
"text/template"
"github.com/Masterminds/sprig"
"github.com/roboll/helmfile/helmexec"
"bytes"
"regexp"
yaml "gopkg.in/yaml.v2"
)
// HelmState structure for the helmfile
type HelmState struct {
BaseChartPath string
HelmDefaults HelmSpec `yaml:"helmDefaults"`
Context string `yaml:"context"`
DeprecatedReleases []ReleaseSpec `yaml:"charts"`
Namespace string `yaml:"namespace"`
Repositories []RepositorySpec `yaml:"repositories"`
Releases []ReleaseSpec `yaml:"releases"`
}
// HelmSpec to defines helmDefault values
type HelmSpec struct {
Args []string `yaml:"args"`
}
// RepositorySpec that defines values for a helm repo
type RepositorySpec struct {
Name string `yaml:"name"`
URL string `yaml:"url"`
CertFile string `yaml:"certFile"`
KeyFile string `yaml:"keyFile"`
Username string `yaml:"username"`
Password string `yaml:"password"`
}
// ReleaseSpec defines the structure of a helm release
type ReleaseSpec struct {
// Chart is the name of the chart being installed to create this release
Chart string `yaml:"chart"`
Version string `yaml:"version"`
Verify bool `yaml:"verify"`
// Name is the name of this release
Name string `yaml:"name"`
Namespace string `yaml:"namespace"`
Labels map[string]string `yaml:"labels"`
Values []interface{} `yaml:"values"`
Secrets []string `yaml:"secrets"`
SetValues []SetValue `yaml:"set"`
// The 'env' section is not really necessary any longer, as 'set' would now provide the same functionality
EnvValues []SetValue `yaml:"env"`
// generatedValues are values that need cleaned up on exit
generatedValues []string
}
// SetValue are the key values to set on a helm release
type SetValue struct {
Name string `yaml:"name"`
Value string `yaml:"value"`
}
// ReadFromFile loads the helmfile from disk and processes the template
func ReadFromFile(file string) (*HelmState, error) {
content, err := ioutil.ReadFile(file)
if err != nil {
return nil, err
}
tpl, err := stringTemplate().Parse(string(content))
if err != nil {
return nil, err
}
var tplString bytes.Buffer
err = tpl.Execute(&tplString, nil)
if err != nil {
return nil, err
}
return readFromYaml(tplString.Bytes(), file)
}
func readFromYaml(content []byte, file string) (*HelmState, error) {
var state HelmState
state.BaseChartPath, _ = filepath.Abs(filepath.Dir(file))
if err := yaml.UnmarshalStrict(content, &state); err != nil {
return nil, err
}
if len(state.DeprecatedReleases) > 0 {
if len(state.Releases) > 0 {
return nil, fmt.Errorf("failed to parse %s: you can't specify both `charts` and `releases` sections", file)
}
state.Releases = state.DeprecatedReleases
state.DeprecatedReleases = []ReleaseSpec{}
}
return &state, nil
}
func stringTemplate() *template.Template {
funcMap := sprig.TxtFuncMap()
alterFuncMap(&funcMap)
return template.New("stringTemplate").Funcs(funcMap)
}
func alterFuncMap(funcMap *template.FuncMap) {
(*funcMap)["requiredEnv"] = getRequiredEnv
}
func getRequiredEnv(name string) (string, error) {
if val, exists := os.LookupEnv(name); exists && len(val) > 0 {
return val, nil
}
return "", fmt.Errorf("required env var `%s` is not set", name)
}
func renderTemplateString(s string) (string, error) {
var t, parseErr = stringTemplate().Parse(s)
if parseErr != nil {
return "", parseErr
}
var tplString bytes.Buffer
var execErr = t.Execute(&tplString, nil)
if execErr != nil {
return "", execErr
}
return tplString.String(), nil
}
func (state *HelmState) applyDefaultsTo(spec *ReleaseSpec) {
if state.Namespace != "" {
spec.Namespace = state.Namespace
}
}
// SyncRepos will update the given helm releases
func (state *HelmState) SyncRepos(helm helmexec.Interface) []error {
errs := []error{}
for _, repo := range state.Repositories {
if err := helm.AddRepo(repo.Name, repo.URL, repo.CertFile, repo.KeyFile, repo.Username, repo.Password); err != nil {
errs = append(errs, err)
}
}
if len(errs) != 0 {
return errs
}
if err := helm.UpdateRepo(); err != nil {
return []error{err}
}
return nil
}
// SyncReleases wrapper for executing helm upgrade on the releases
func (state *HelmState) SyncReleases(helm helmexec.Interface, additionalValues []string, workerLimit int) []error {
errs := []error{}
jobQueue := make(chan *ReleaseSpec)
doneQueue := make(chan bool)
errQueue := make(chan error)
if workerLimit < 1 {
workerLimit = len(state.Releases)
}
for w := 1; w <= workerLimit; w++ {
go func() {
for release := range jobQueue {
state.applyDefaultsTo(release)
flags, flagsErr := flagsForRelease(helm, state.BaseChartPath, release)
if flagsErr != nil {
errQueue <- flagsErr
doneQueue <- true
continue
}
haveValueErr := false
for _, value := range additionalValues {
valfile, err := filepath.Abs(value)
if err != nil {
errQueue <- err
haveValueErr = true
}
if _, err := os.Stat(valfile); os.IsNotExist(err) {
errQueue <- err
haveValueErr = true
}
flags = append(flags, "--values", valfile)
}
if haveValueErr {
doneQueue <- true
continue
}
chart := normalizeChart(state.BaseChartPath, release.Chart)
if err := helm.SyncRelease(release.Name, chart, flags...); err != nil {
errQueue <- err
}
doneQueue <- true
}
}()
}
go func() {
for i := 0; i < len(state.Releases); i++ {
jobQueue <- &state.Releases[i]
}
close(jobQueue)
}()
for i := 0; i < len(state.Releases); {
select {
case err := <-errQueue:
errs = append(errs, err)
case <-doneQueue:
i++
}
}
if len(errs) != 0 {
return errs
}
return nil
}
// DiffReleases wrapper for executing helm diff on the releases
func (state *HelmState) DiffReleases(helm helmexec.Interface, additionalValues []string, workerLimit int) []error {
var wgRelease sync.WaitGroup
var wgError sync.WaitGroup
errs := []error{}
jobQueue := make(chan *ReleaseSpec, len(state.Releases))
errQueue := make(chan error)
if workerLimit < 1 {
workerLimit = len(state.Releases)
}
wgRelease.Add(len(state.Releases))
for w := 1; w <= workerLimit; w++ {
go func() {
for release := range jobQueue {
errs := []error{}
// Plugin command doesn't support explicit namespace
release.Namespace = ""
flags, err := flagsForRelease(helm, state.BaseChartPath, release)
if err != nil {
errs = append(errs, err)
}
for _, value := range additionalValues {
valfile, err := filepath.Abs(value)
if err != nil {
errs = append(errs, err)
}
if _, err := os.Stat(valfile); os.IsNotExist(err) {
errs = append(errs, err)
}
flags = append(flags, "--values", valfile)
}
if len(errs) == 0 {
if err := helm.DiffRelease(release.Name, normalizeChart(state.BaseChartPath, release.Chart), flags...); err != nil {
errs = append(errs, err)
}
}
for _, err := range errs {
errQueue <- err
}
wgRelease.Done()
}
}()
}
wgError.Add(1)
go func() {
for err := range errQueue {
errs = append(errs, err)
}
wgError.Done()
}()
for i := 0; i < len(state.Releases); i++ {
jobQueue <- &state.Releases[i]
}
close(jobQueue)
wgRelease.Wait()
close(errQueue)
wgError.Wait()
if len(errs) != 0 {
return errs
}
return nil
}
// LintReleases wrapper for executing helm lint on the releases
func (state *HelmState) LintReleases(helm helmexec.Interface, additionalValues []string, workerLimit int) []error {
var wgRelease sync.WaitGroup
var wgError sync.WaitGroup
errs := []error{}
jobQueue := make(chan *ReleaseSpec, len(state.Releases))
errQueue := make(chan error)
if workerLimit < 1 {
workerLimit = len(state.Releases)
}
wgRelease.Add(len(state.Releases))
// Create tmp directory and bail immediately if it fails
dir, err := ioutil.TempDir("", "")
if err != nil {
errs = append(errs, err)
return errs
}
defer os.RemoveAll(dir)
for w := 1; w <= workerLimit; w++ {
go func() {
for release := range jobQueue {
errs := []error{}
flags, err := flagsForRelease(helm, state.BaseChartPath, release)
if err != nil {
errs = append(errs, err)
}
for _, value := range additionalValues {
valfile, err := filepath.Abs(value)
if err != nil {
errs = append(errs, err)
}
if _, err := os.Stat(valfile); os.IsNotExist(err) {
errs = append(errs, err)
}
flags = append(flags, "--values", valfile)
}
chartPath := ""
if isLocalChart(release.Chart) {
chartPath = normalizeChart(state.BaseChartPath, release.Chart)
} else {
fetchFlags := []string{}
if release.Version != "" {
chartPath = path.Join(dir, release.Name, release.Version, release.Chart)
fetchFlags = append(fetchFlags, "--version", release.Version)
} else {
chartPath = path.Join(dir, release.Name, "latest", release.Chart)
}
// only fetch chart if it is not already fetched
if _, err := os.Stat(chartPath); os.IsNotExist(err) {
fetchFlags = append(fetchFlags, "--untar", "--untardir", chartPath)
if err := helm.Fetch(release.Chart, fetchFlags...); err != nil {
errs = append(errs, err)
}
}
chartPath = path.Join(chartPath, chartNameWithoutRepository(release.Chart))
}
// strip version from the slice returned from flagsForRelease
realFlags := []string{}
isVersion := false
for _, v := range flags {
if v == "--version" {
isVersion = true
} else if isVersion {
isVersion = false
} else {
realFlags = append(realFlags, v)
}
}
if len(errs) == 0 {
if err := helm.Lint(chartPath, realFlags...); err != nil {
errs = append(errs, err)
}
}
for _, err := range errs {
errQueue <- err
}
wgRelease.Done()
}
}()
}
wgError.Add(1)
go func() {
for err := range errQueue {
errs = append(errs, err)
}
wgError.Done()
}()
for i := 0; i < len(state.Releases); i++ {
jobQueue <- &state.Releases[i]
}
close(jobQueue)
wgRelease.Wait()
close(errQueue)
wgError.Wait()
if len(errs) != 0 {
return errs
}
return nil
}
func (state *HelmState) ReleaseStatuses(helm helmexec.Interface, workerLimit int) []error {
var errs []error
jobQueue := make(chan ReleaseSpec)
doneQueue := make(chan bool)
errQueue := make(chan error)
if workerLimit < 1 {
workerLimit = len(state.Releases)
}
for w := 1; w <= workerLimit; w++ {
go func() {
for release := range jobQueue {
if err := helm.ReleaseStatus(release.Name); err != nil {
errQueue <- err
}
doneQueue <- true
}
}()
}
go func() {
for _, release := range state.Releases {
jobQueue <- release
}
close(jobQueue)
}()
for i := 0; i < len(state.Releases); {
select {
case err := <-errQueue:
errs = append(errs, err)
case <-doneQueue:
i++
}
}
if len(errs) != 0 {
return errs
}
return nil
}
// DeleteReleases wrapper for executing helm delete on the releases
func (state *HelmState) DeleteReleases(helm helmexec.Interface, purge bool) []error {
var wg sync.WaitGroup
errs := []error{}
for _, release := range state.Releases {
wg.Add(1)
go func(wg *sync.WaitGroup, release ReleaseSpec) {
flags := []string{}
if purge {
flags = append(flags, "--purge")
}
if err := helm.DeleteRelease(release.Name, flags...); err != nil {
errs = append(errs, err)
}
wg.Done()
}(&wg, release)
}
wg.Wait()
if len(errs) != 0 {
return errs
}
return nil
}
// TestReleases wrapper for executing helm test on the releases
func (state *HelmState) TestReleases(helm helmexec.Interface, cleanup bool, timeout int) []error {
var wg sync.WaitGroup
errs := []error{}
for _, release := range state.Releases {
wg.Add(1)
go func(wg *sync.WaitGroup, release ReleaseSpec) {
flags := []string{}
if cleanup {
flags = append(flags, "--cleanup")
}
flags = append(flags, "--timeout", strconv.Itoa(timeout))
if err := helm.TestRelease(release.Name, flags...); err != nil {
errs = append(errs, err)
}
wg.Done()
}(&wg, release)
}
wg.Wait()
if len(errs) != 0 {
return errs
}
return nil
}
// Clean will remove any generated secrets
func (state *HelmState) Clean() []error {
errs := []error{}
for _, release := range state.Releases {
for _, value := range release.generatedValues {
err := os.Remove(value)
if err != nil {
errs = append(errs, err)
}
}
}
if len(errs) != 0 {
return errs
}
return nil
}
// FilterReleases allows for the execution of helm commands against a subset of the releases in the helmfile.
func (state *HelmState) FilterReleases(labels []string) error {
var filteredReleases []ReleaseSpec
releaseSet := map[string]ReleaseSpec{}
filters := []ReleaseFilter{}
for _, label := range labels {
f, err := ParseLabels(label)
if err != nil {
return err
}
filters = append(filters, f)
}
for _, r := range state.Releases {
if r.Labels == nil {
r.Labels = map[string]string{}
}
// Let the release name be used as a tag
r.Labels["name"] = r.Name
for _, f := range filters {
if r.Labels == nil {
r.Labels = map[string]string{}
}
if f.Match(r) {
releaseSet[r.Name] = r
continue
}
}
}
for _, r := range releaseSet {
filteredReleases = append(filteredReleases, r)
}
if len(filteredReleases) == 0 {
return errors.New("Specified selector did not match any releases.\n")
}
state.Releases = filteredReleases
return nil
}
// UpdateDeps wrapper for updating dependencies on the releases
func (state *HelmState) UpdateDeps(helm helmexec.Interface) []error {
errs := []error{}
for _, release := range state.Releases {
if isLocalChart(release.Chart) {
if err := helm.UpdateDeps(normalizeChart(state.BaseChartPath, release.Chart)); err != nil {
errs = append(errs, err)
}
}
}
if len(errs) != 0 {
return errs
}
return nil
}
// normalizeChart allows for the distinction between a file path reference and repository references.
// - Any single (or double character) followed by a `/` will be considered a local file reference and
// be constructed relative to the `base path`.
// - Everything else is assumed to be an absolute path or an actual <repository>/<chart> reference.
func normalizeChart(basePath, chart string) string {
regex, _ := regexp.Compile("^[.]?./")
if !regex.MatchString(chart) {
return chart
}
return filepath.Join(basePath, chart)
}
func isLocalChart(chart string) bool {
_, err := os.Stat(chart)
return err == nil
}
func chartNameWithoutRepository(chart string) string {
chartSplit := strings.Split(chart, "/")
return chartSplit[len(chartSplit)-1]
}
func flagsForRelease(helm helmexec.Interface, basePath string, release *ReleaseSpec) ([]string, error) {
flags := []string{}
if release.Version != "" {
flags = append(flags, "--version", release.Version)
}
if release.Verify {
flags = append(flags, "--verify")
}
if release.Namespace != "" {
flags = append(flags, "--namespace", release.Namespace)
}
for _, value := range release.Values {
switch typedValue := value.(type) {
case string:
var path string
if filepath.IsAbs(typedValue) {
path = typedValue
} else {
path = filepath.Join(basePath, typedValue)
}
if _, err := os.Stat(path); os.IsNotExist(err) {
return nil, err
}
flags = append(flags, "--values", path)
case map[interface{}]interface{}:
valfile, err := ioutil.TempFile("", "values")
if err != nil {
return nil, err
}
defer valfile.Close()
encoder := yaml.NewEncoder(valfile)
defer encoder.Close()
if err := encoder.Encode(typedValue); err != nil {
return nil, err
}
release.generatedValues = append(release.generatedValues, valfile.Name())
flags = append(flags, "--values", valfile.Name())
}
}
for _, value := range release.Secrets {
path := filepath.Join(basePath, value)
if _, err := os.Stat(path); os.IsNotExist(err) {
return nil, err
}
valfile, err := helm.DecryptSecret(path)
if err != nil {
return nil, err
}
release.generatedValues = append(release.generatedValues, valfile)
flags = append(flags, "--values", valfile)
}
if len(release.SetValues) > 0 {
val := []string{}
for _, set := range release.SetValues {
val = append(val, fmt.Sprintf("%s=%s", escape(set.Name), escape(set.Value)))
}
flags = append(flags, "--set", strings.Join(val, ","))
}
/***********
* START 'env' section for backwards compatibility
***********/
// The 'env' section is not really necessary any longer, as 'set' would now provide the same functionality
if len(release.EnvValues) > 0 {
val := []string{}
envValErrs := []string{}
for _, set := range release.EnvValues {
value, isSet := os.LookupEnv(set.Value)
if isSet {
val = append(val, fmt.Sprintf("%s=%s", escape(set.Name), escape(value)))
} else {
errMsg := fmt.Sprintf("\t%s", set.Value)
envValErrs = append(envValErrs, errMsg)
}
}
if len(envValErrs) != 0 {
joinedEnvVals := strings.Join(envValErrs, "\n")
errMsg := fmt.Sprintf("Environment Variables not found. Please make sure they are set and try again:\n%s", joinedEnvVals)
return nil, errors.New(errMsg)
}
flags = append(flags, "--set", strings.Join(val, ","))
}
/**************
* END 'env' section for backwards compatibility
**************/
return flags, nil
}
func escape(value string) string {
intermediate := strings.Replace(value, "{", "\\{", -1)
intermediate = strings.Replace(intermediate, "}", "\\}", -1)
return strings.Replace(intermediate, ",", "\\,", -1)
}
-879
View File
@@ -1,879 +0,0 @@
package state
import (
"os"
"reflect"
"testing"
"errors"
"strings"
)
func TestReadFromYaml(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease
namespace: mynamespace
chart: mychart
`)
state, err := readFromYaml(yamlContent, yamlFile)
if err != nil {
t.Errorf("unxpected error: %v", err)
}
if state.Releases[0].Name != "myrelease" {
t.Errorf("unexpected release name: expected=myrelease actual=%s", state.Releases[0].Name)
}
if state.Releases[0].Namespace != "mynamespace" {
t.Errorf("unexpected chart namespace: expected=mynamespace actual=%s", state.Releases[0].Chart)
}
if state.Releases[0].Chart != "mychart" {
t.Errorf("unexpected chart name: expected=mychart actual=%s", state.Releases[0].Chart)
}
}
func TestReadFromYaml_StrictUnmarshalling(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease
namespace: mynamespace
releases: mychart
`)
_, err := readFromYaml(yamlContent, yamlFile)
if err == nil {
t.Error("expected an error for wrong key 'releases' which is not in struct")
}
}
func TestReadFromYaml_DeprecatedReleaseReferences(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`charts:
- name: myrelease
chart: mychart
`)
state, err := readFromYaml(yamlContent, yamlFile)
if err != nil {
t.Errorf("unxpected error: %v", err)
}
if state.Releases[0].Name != "myrelease" {
t.Errorf("unexpected release name: expected=myrelease actual=%s", state.Releases[0].Name)
}
if state.Releases[0].Chart != "mychart" {
t.Errorf("unexpected chart name: expected=mychart actual=%s", state.Releases[0].Chart)
}
}
func TestReadFromYaml_ConflictingReleasesConfig(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`charts:
- name: myrelease1
chart: mychart1
releases:
- name: myrelease2
chart: mychart2
`)
_, err := readFromYaml(yamlContent, yamlFile)
if err == nil {
t.Error("expected error")
}
}
func TestReadFromYaml_FilterReleasesOnLabels(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease1
chart: mychart1
labels:
tier: frontend
foo: bar
- name: myrelease2
chart: mychart2
labels:
tier: frontend
- name: myrelease3
chart: mychart3
labels:
tier: backend
`)
cases := []struct {
filter LabelFilter
results []bool
}{
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}}},
[]bool{true, true, false}},
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}, []string{"foo", "bar"}}},
[]bool{true, false, false}},
{LabelFilter{negativeLabels: [][]string{[]string{"tier", "frontend"}}},
[]bool{false, false, true}},
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}}, negativeLabels: [][]string{[]string{"foo", "bar"}}},
[]bool{false, true, false}},
}
state, err := readFromYaml(yamlContent, yamlFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
for idx, c := range cases {
for idx2, expected := range c.results {
if f := c.filter.Match(state.Releases[idx2]); f != expected {
t.Errorf("[case: %d][outcome: %d] Unexpected outcome wanted %t, got %t", idx, idx2, expected, f)
}
}
}
}
func TestReadFromYaml_FilterNegatives(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease1
chart: mychart1
labels:
stage: pre
foo: bar
- name: myrelease2
chart: mychart2
labels:
stage: post
- name: myrelease3
chart: mychart3
`)
cases := []struct {
filter LabelFilter
results []bool
}{
{LabelFilter{positiveLabels: [][]string{[]string{"stage", "pre"}}},
[]bool{true, false, false}},
{LabelFilter{positiveLabels: [][]string{[]string{"stage", "post"}}},
[]bool{false, true, false}},
{LabelFilter{negativeLabels: [][]string{[]string{"stage", "pre"}, []string{"stage", "post"}}},
[]bool{false, false, true}},
}
state, err := readFromYaml(yamlContent, yamlFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
for idx, c := range cases {
for idx2, expected := range c.results {
if f := c.filter.Match(state.Releases[idx2]); f != expected {
t.Errorf("[case: %d][outcome: %d] Unexpected outcome wanted %t, got %t", idx, idx2, expected, f)
}
}
}
}
func TestLabelParsing(t *testing.T) {
cases := []struct {
labelString string
expectedFilter LabelFilter
errorExected bool
}{
{"foo=bar", LabelFilter{positiveLabels: [][]string{[]string{"foo", "bar"}}, negativeLabels: [][]string{}}, false},
{"foo!=bar", LabelFilter{positiveLabels: [][]string{}, negativeLabels: [][]string{[]string{"foo", "bar"}}}, false},
{"foo!=bar,baz=bat", LabelFilter{positiveLabels: [][]string{[]string{"baz", "bat"}}, negativeLabels: [][]string{[]string{"foo", "bar"}}}, false},
{"foo", LabelFilter{positiveLabels: [][]string{}, negativeLabels: [][]string{}}, true},
{"foo!=bar=baz", LabelFilter{positiveLabels: [][]string{}, negativeLabels: [][]string{}}, true},
{"=bar", LabelFilter{positiveLabels: [][]string{}, negativeLabels: [][]string{}}, true},
}
for idx, c := range cases {
filter, err := ParseLabels(c.labelString)
if err != nil && !c.errorExected {
t.Errorf("[%d] Didn't expect an error parsing labels: %s", idx, err)
} else if err == nil && c.errorExected {
t.Errorf("[%d] Expected %s to result in an error but got none", idx, c.labelString)
} else if !reflect.DeepEqual(filter, c.expectedFilter) {
t.Errorf("[%d] parsed label did not result in expected filter: %v, expected: %v", idx, filter, c.expectedFilter)
}
}
}
func TestHelmState_applyDefaultsTo(t *testing.T) {
type fields struct {
BaseChartPath string
Context string
DeprecatedReleases []ReleaseSpec
Namespace string
Repositories []RepositorySpec
Releases []ReleaseSpec
}
type args struct {
spec ReleaseSpec
}
specWithNamespace := ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Verify: false,
Name: "test-charts",
Namespace: "test-namespace",
Values: nil,
SetValues: nil,
EnvValues: nil,
}
specWithoutNamespace := specWithNamespace
specWithoutNamespace.Namespace = ""
specWithNamespaceFromFields := specWithNamespace
specWithNamespaceFromFields.Namespace = "test-namespace-field"
fieldsWithNamespace := fields{
BaseChartPath: ".",
Context: "test_context",
DeprecatedReleases: nil,
Namespace: specWithNamespaceFromFields.Namespace,
Repositories: nil,
Releases: []ReleaseSpec{
specWithNamespace,
},
}
fieldsWithoutNamespace := fieldsWithNamespace
fieldsWithoutNamespace.Namespace = ""
tests := []struct {
name string
fields fields
args args
want ReleaseSpec
}{
{
name: "Has a namespace from spec",
fields: fieldsWithoutNamespace,
args: args{
spec: specWithNamespace,
},
want: specWithNamespace,
},
{
name: "Has a namespace from flags",
fields: fieldsWithoutNamespace,
args: args{
spec: specWithNamespace,
},
want: specWithNamespace,
},
{
name: "Has a namespace from flags and from spec",
fields: fieldsWithNamespace,
args: args{
spec: specWithNamespace,
},
want: specWithNamespaceFromFields,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
state := &HelmState{
BaseChartPath: tt.fields.BaseChartPath,
Context: tt.fields.Context,
DeprecatedReleases: tt.fields.DeprecatedReleases,
Namespace: tt.fields.Namespace,
Repositories: tt.fields.Repositories,
Releases: tt.fields.Releases,
}
if state.applyDefaultsTo(&tt.args.spec); !reflect.DeepEqual(tt.args.spec, tt.want) {
t.Errorf("HelmState.applyDefaultsTo() = %v, want %v", tt.args.spec, tt.want)
}
})
}
}
func Test_renderTemplateString(t *testing.T) {
type args struct {
s string
envs map[string]string
}
tests := []struct {
name string
args args
want string
wantErr bool
}{
{
name: "simple replacement",
args: args{
s: "{{ env \"HF_TEST_VAR\" }}",
envs: map[string]string{
"HF_TEST_VAR": "content",
},
},
want: "content",
wantErr: false,
},
{
name: "two replacements",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" }}{{ env \"HF_TEST_BETA\" }}",
envs: map[string]string{
"HF_TEST_ALPHA": "first",
"HF_TEST_BETA": "second",
},
},
want: "firstsecond",
wantErr: false,
},
{
name: "replacement and comment",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" }}{{/* comment */}}",
envs: map[string]string{
"HF_TEST_ALPHA": "first",
},
},
want: "first",
wantErr: false,
},
{
name: "global template function",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" | len }}",
envs: map[string]string{
"HF_TEST_ALPHA": "abcdefg",
},
},
want: "7",
wantErr: false,
},
{
name: "env var not set",
args: args{
s: "{{ env \"HF_TEST_NONE\" }}",
envs: map[string]string{
"HF_TEST_THIS": "first",
},
},
want: "",
},
{
name: "undefined function",
args: args{
s: "{{ env foo }}",
envs: map[string]string{
"foo": "bar",
},
},
wantErr: true,
},
{
name: "required env var",
args: args{
s: "{{ requiredEnv \"HF_TEST\" }}",
envs: map[string]string{
"HF_TEST": "value",
},
},
want: "value",
wantErr: false,
},
{
name: "required env var not set",
args: args{
s: "{{ requiredEnv \"HF_TEST_NONE\" }}",
envs: map[string]string{},
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
for k, v := range tt.args.envs {
err := os.Setenv(k, v)
if err != nil {
t.Error("renderTemplateString() could not set env var for testing")
}
}
got, err := renderTemplateString(tt.args.s)
if (err != nil) != tt.wantErr {
t.Errorf("renderTemplateString() for %s error = %v, wantErr %v", tt.name, err, tt.wantErr)
return
}
if got != tt.want {
t.Errorf("renderTemplateString() for %s = %v, want %v", tt.name, got, tt.want)
}
})
}
}
func Test_isLocalChart(t *testing.T) {
type args struct {
chart string
}
tests := []struct {
name string
args args
want bool
}{
{
name: "local chart",
args: args{
chart: "./",
},
want: true,
},
{
name: "repo chart",
args: args{
chart: "stable/genius",
},
want: false,
},
{
name: "empty",
args: args{
chart: "",
},
want: false,
},
{
name: "parent local path",
args: args{
chart: "../examples",
},
want: true,
},
{
name: "parent-parent local path",
args: args{
chart: "../../",
},
want: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isLocalChart(tt.args.chart); got != tt.want {
t.Errorf("isLocalChart() = %v, want %v", got, tt.want)
}
})
}
}
func Test_normalizeChart(t *testing.T) {
type args struct {
basePath string
chart string
}
tests := []struct {
name string
args args
want string
}{
{
name: "construct local chart path",
args: args{
basePath: "/Users/jane/code/deploy/charts",
chart: "./app",
},
want: "/Users/jane/code/deploy/charts/app",
},
{
name: "repo path",
args: args{
basePath: "/Users/jane/code/deploy/charts",
chart: "remote/app",
},
want: "remote/app",
},
{
name: "construct local chart path, parent dir",
args: args{
basePath: "/Users/jane/code/deploy/charts",
chart: "../app",
},
want: "/Users/jane/code/deploy/app",
},
{
name: "too much parent levels",
args: args{
basePath: "/src",
chart: "../../app",
},
want: "/app",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := normalizeChart(tt.args.basePath, tt.args.chart); got != tt.want {
t.Errorf("normalizeChart() = %v, want %v", got, tt.want)
}
})
}
}
// mocking helmexec.Interface
type mockHelmExec struct {
charts []string
repo []string
releases []mockRelease
}
type mockRelease struct {
name string
flags []string
}
func (helm *mockHelmExec) UpdateDeps(chart string) error {
if strings.Contains(chart, "error") {
return errors.New("error")
}
helm.charts = append(helm.charts, chart)
return nil
}
func (helm *mockHelmExec) SetExtraArgs(args ...string) {
return
}
func (helm *mockHelmExec) AddRepo(name, repository, certfile, keyfile, username, password string) error {
helm.repo = []string{name, repository, certfile, keyfile, username, password}
return nil
}
func (helm *mockHelmExec) UpdateRepo() error {
return nil
}
func (helm *mockHelmExec) SyncRelease(name, chart string, flags ...string) error {
if strings.Contains(name, "error") {
return errors.New("error")
}
helm.releases = append(helm.releases, mockRelease{name: name, flags: flags})
helm.charts = append(helm.charts, chart)
return nil
}
func (helm *mockHelmExec) DiffRelease(name, chart string, flags ...string) error {
return nil
}
func (helm *mockHelmExec) ReleaseStatus(release string) error {
if strings.Contains(release, "error") {
return errors.New("error")
}
helm.releases = append(helm.releases, mockRelease{name: release, flags: []string{}})
return nil
}
func (helm *mockHelmExec) DeleteRelease(name string, flags ...string) error {
return nil
}
func (helm *mockHelmExec) DecryptSecret(name string) (string, error) {
return "", nil
}
func (helm *mockHelmExec) TestRelease(name string, flags ...string) error {
if strings.Contains(name, "error") {
return errors.New("error")
}
helm.releases = append(helm.releases, mockRelease{name: name, flags: flags})
return nil
}
func (helm *mockHelmExec) Fetch(chart string, flags ...string) error {
return nil
}
func (helm *mockHelmExec) Lint(chart string, flags ...string) error {
return nil
}
func TestHelmState_SyncRepos(t *testing.T) {
tests := []struct {
name string
repos []RepositorySpec
helm *mockHelmExec
envs map[string]string
want []string
}{
{
name: "normal repository",
repos: []RepositorySpec{
{
Name: "name",
URL: "http://example.com/",
CertFile: "",
KeyFile: "",
Username: "",
Password: "",
},
},
helm: &mockHelmExec{},
want: []string{"name", "http://example.com/", "", "", "", ""},
},
{
name: "repository with cert and key",
repos: []RepositorySpec{
{
Name: "name",
URL: "http://example.com/",
CertFile: "certfile",
KeyFile: "keyfile",
Username: "",
Password: "",
},
},
helm: &mockHelmExec{},
want: []string{"name", "http://example.com/", "certfile", "keyfile", "", ""},
},
{
name: "repository with username and password",
repos: []RepositorySpec{
{
Name: "name",
URL: "http://example.com/",
CertFile: "",
KeyFile: "",
Username: "example_user",
Password: "example_password",
},
},
helm: &mockHelmExec{},
want: []string{"name", "http://example.com/", "", "", "example_user", "example_password"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
for k, v := range tt.envs {
err := os.Setenv(k, v)
if err != nil {
t.Error("HelmState.SyncRepos() could not set env var for testing")
}
}
state := &HelmState{
Repositories: tt.repos,
}
if _ = state.SyncRepos(tt.helm); !reflect.DeepEqual(tt.helm.repo, tt.want) {
t.Errorf("HelmState.SyncRepos() for [%s] = %v, want %v", tt.name, tt.helm.repo, tt.want)
}
})
}
}
func TestHelmState_SyncReleases(t *testing.T) {
tests := []struct {
name string
releases []ReleaseSpec
helm *mockHelmExec
wantReleases []mockRelease
}{
{
name: "normal release",
releases: []ReleaseSpec{
{
Name: "releaseName",
Chart: "foo",
},
},
helm: &mockHelmExec{},
wantReleases: []mockRelease{{"releaseName", []string{}}},
},
{
name: "escaped values",
releases: []ReleaseSpec{
{
Name: "releaseName",
Chart: "foo",
SetValues: []SetValue{
{
Name: "someList",
Value: "a,b,c",
},
{
Name: "json",
Value: "{\"name\": \"john\"}",
},
},
},
},
helm: &mockHelmExec{},
wantReleases: []mockRelease{{"releaseName", []string{"--set", "someList=a\\,b\\,c,json=\\{\"name\": \"john\"\\}"}}},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
state := &HelmState{
Releases: tt.releases,
}
if _ = state.SyncReleases(tt.helm, []string{}, 1); !reflect.DeepEqual(tt.helm.releases, tt.wantReleases) {
t.Errorf("HelmState.SyncReleases() for [%s] = %v, want %v", tt.name, tt.helm.releases, tt.wantReleases)
}
})
}
}
func TestHelmState_UpdateDeps(t *testing.T) {
state := &HelmState{
BaseChartPath: "/src",
Releases: []ReleaseSpec{
{
Chart: "./..",
},
{
Chart: "../examples",
},
{
Chart: "../../helmfile",
},
{
Chart: "published",
},
{
Chart: "published/deeper",
},
{
Chart: "./error",
},
},
}
want := []string{"/", "/examples", "/helmfile"}
helm := &mockHelmExec{}
errs := state.UpdateDeps(helm)
if !reflect.DeepEqual(helm.charts, want) {
t.Errorf("HelmState.UpdateDeps() = %v, want %v", helm.charts, want)
}
if len(errs) != 0 {
t.Errorf("HelmState.UpdateDeps() - no errors, but got: %v", len(errs))
}
}
func TestHelmState_ReleaseStatuses(t *testing.T) {
tests := []struct {
name string
releases []ReleaseSpec
helm *mockHelmExec
want []mockRelease
wantErr bool
}{
{
name: "happy path",
releases: []ReleaseSpec{
{
Name: "releaseA",
},
},
helm: &mockHelmExec{},
want: []mockRelease{{"releaseA", []string{}}},
},
{
name: "happy path",
releases: []ReleaseSpec{
{
Name: "error",
},
},
helm: &mockHelmExec{},
wantErr: true,
},
}
for _, tt := range tests {
i := func(t *testing.T) {
state := &HelmState{
Releases: tt.releases,
}
errs := state.ReleaseStatuses(tt.helm, 1)
if (errs != nil) != tt.wantErr {
t.Errorf("ReleaseStatuses() for %s error = %v, wantErr %v", tt.name, errs, tt.wantErr)
return
}
if !reflect.DeepEqual(tt.helm.releases, tt.want) {
t.Errorf("HelmState.ReleaseStatuses() for [%s] = %v, want %v", tt.name, tt.helm.releases, tt.want)
}
}
t.Run(tt.name, i)
}
}
func TestHelmState_TestReleasesNoCleanUp(t *testing.T) {
tests := []struct {
name string
cleanup bool
releases []ReleaseSpec
helm *mockHelmExec
want []mockRelease
wantErr bool
}{
{
name: "happy path",
releases: []ReleaseSpec{
{
Name: "releaseA",
},
},
helm: &mockHelmExec{},
want: []mockRelease{{"releaseA", []string{"--timeout", "1"}}},
},
{
name: "do cleanup",
cleanup: true,
releases: []ReleaseSpec{
{
Name: "releaseB",
},
},
helm: &mockHelmExec{},
want: []mockRelease{{"releaseB", []string{"--cleanup", "--timeout", "1"}}},
},
{
name: "happy path",
releases: []ReleaseSpec{
{
Name: "error",
},
},
helm: &mockHelmExec{},
wantErr: true,
},
}
for _, tt := range tests {
i := func(t *testing.T) {
state := &HelmState{
Releases: tt.releases,
}
errs := state.TestReleases(tt.helm, tt.cleanup, 1)
if (errs != nil) != tt.wantErr {
t.Errorf("TestReleases() for %s error = %v, wantErr %v", tt.name, errs, tt.wantErr)
return
}
if !reflect.DeepEqual(tt.helm.releases, tt.want) {
t.Errorf("HelmState.TestReleases() for [%s] = %v, want %v", tt.name, tt.helm.releases, tt.want)
}
}
t.Run(tt.name, i)
}
}
func TestHelmState_NoReleaseMatched(t *testing.T) {
releases := []ReleaseSpec{
{
Name: "releaseA",
Labels: map[string]string{
"foo": "bar",
},
},
}
tests := []struct {
name string
labels string
wantErr bool
}{
{
name: "happy path",
labels: "foo=bar",
wantErr: false,
},
{
name: "name does not exist",
labels: "name=releaseB",
wantErr: true,
},
{
name: "label does not match anything",
labels: "foo=notbar",
wantErr: true,
},
}
for _, tt := range tests {
i := func(t *testing.T) {
state := &HelmState{
Releases: releases,
}
errs := state.FilterReleases([]string{tt.labels})
if (errs != nil) != tt.wantErr {
t.Errorf("ReleaseStatuses() for %s error = %v, wantErr %v", tt.name, errs, tt.wantErr)
return
}
}
t.Run(tt.name, i)
}
}
+1
View File
@@ -0,0 +1 @@
mysecret: MYSECRET
+15 -1
View File
@@ -1,8 +1,16 @@
environments:
default:
values:
- environment.values.yaml
---
repositories:
- name: stable
url: https://kubernetes-charts.storage.googleapis.com/
- name: incubator
url: https://kubernetes-charts-incubator.storage.googleapis.com
context: minikube
helmDefaults:
kubeContext: minikube
releases:
@@ -11,3 +19,9 @@ releases:
set:
- name: ingress.enabled
value: false
- name: raw
chart: incubator/raw
values:
- mysecret: {{ .Environment.Values.mysecret }}
- values.yaml
+43 -3
View File
@@ -36,7 +36,8 @@ function retry() {
${command} && break # substitute your command here
retry_result=$?
n=$[$n+1]
sleep 1
# approximated binary exponential backoff to reduce flakiness
sleep $((n ** 2))
done
}
@@ -46,6 +47,7 @@ set -e
info "Using namespace: ${test_ns}"
info "Using Helm version: $(helm version --short --client | grep -o v.*$)"
${helm} init --wait --override spec.template.spec.automountServiceAccountToken=true
${helm} plugin install https://github.com/databus23/helm-diff --version master
${kubectl} get namespace ${test_ns} &> /dev/null && warn "Namespace ${test_ns} exists, from a previous test run?"
$kubectl create namespace ${test_ns} || fail "Could not create namespace ${test_ns}"
trap "{ $kubectl delete namespace ${test_ns}; }" EXIT # remove namespace whenever we exit this script
@@ -54,17 +56,55 @@ trap "{ $kubectl delete namespace ${test_ns}; }" EXIT # remove namespace wheneve
# TEST CASES----------------------------------------------------------------------------------------------------------
test_start "happypath - simple rollout of httpbin chart"
info "Diffing ${dir}/happypath.yaml"
bash -c "${helmfile} -f ${dir}/happypath.yaml diff --detailed-exitcode; code="'$?'"; [ "'${code}'" -eq 2 ]" || fail "unexpected exit code returned by helmfile diff"
info "Diffing ${dir}/happypath.yaml without color"
bash -c "${helmfile} -f ${dir}/happypath.yaml --no-color diff --detailed-exitcode; code="'$?'"; [ "'${code}'" -eq 2 ]" || fail "unexpected exit code returned by helmfile diff"
info "Diffing ${dir}/happypath.yaml with limited context"
bash -c "${helmfile} -f ${dir}/happypath.yaml diff --context 3 --detailed-exitcode; code="'$?'"; [ "'${code}'" -eq 2 ]" || fail "unexpected exit code returned by helmfile diff"
info "Templating ${dir}/happypath.yaml"
${helmfile} -f ${dir}/happypath.yaml template
code=$?
[ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile template: ${code}"
info "Syncing ${dir}/happypath.yaml"
${helmfile} -f ${dir}/happypath.yaml sync
wait_deploy_ready httpbin-httpbin
retry 3 "curl --fail $(minikube service --url --namespace=${test_ns} httpbin-httpbin)/status/200"
retry 5 "curl --fail $(minikube service --url --namespace=${test_ns} httpbin-httpbin)/status/200"
[ ${retry_result} -eq 0 ] || fail "httpbin failed to return 200 OK"
info "Applying ${dir}/happypath.yaml"
${helmfile} -f ${dir}/happypath.yaml apply
code=$?
[ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile apply: ${code}"
info "Locking dependencies"
${helmfile} -f ${dir}/happypath.yaml deps
code=$?
[ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile deps: ${code}"
info "Applying ${dir}/happypath.yaml with locked dependencies"
${helmfile} -f ${dir}/happypath.yaml apply
code=$?
[ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile apply: ${code}"
info "Deleting release"
${helmfile} -f ${dir}/happypath.yaml delete
${helm} status --namespace=${test_ns} httpbin &> /dev/null && fail "release should not exist anymore after a delete"
info "Ensuring \"helmfile delete\" doesn't fail when no releases installed"
${helmfile} -f ${dir}/happypath.yaml delete || fail "\"helmfile delete\" shouldn't fail when there are no installed releases"
info "Ensuring \"helmfile template\" output does contain only YAML docs"
(${helmfile} -f ${dir}/happypath.yaml template | kubectl apply -f -) || fail "\"helmfile template | kubectl apply -f -\" shouldn't fail"
test_pass "happypath"
# ALL DONE -----------------------------------------------------------------------------------------------------------
all_tests_passed
all_tests_passed
+9
View File
@@ -0,0 +1,9 @@
resources: []
templates:
- |
apiVersion: v1
kind: Secret
metadata:
name: common-secret
stringData:
mykey: {{ .Values.mysecret }}
-27
View File
@@ -1,27 +0,0 @@
language: go
go:
- 1.6.x
- 1.7.x
- 1.8.x
- 1.9.x
- 1.10.x
- tip
# Setting sudo access to false will let Travis CI use containers rather than
# VMs to run the tests. For more details see:
# - http://docs.travis-ci.com/user/workers/container-based-infrastructure/
# - http://docs.travis-ci.com/user/workers/standard-infrastructure/
sudo: false
script:
- make setup
- make test
notifications:
webhooks:
urls:
- https://webhooks.gitter.im/e/06e3328629952dabe3e0
on_success: change # options: [always|never|change] default: always
on_failure: always # options: [always|never|change] default: always
on_start: never # options: [always|never|change] default: always
-77
View File
@@ -1,77 +0,0 @@
# 1.4.1 (2018-04-02)
## Fixed
- Fixed #64: Fix pre-release precedence issue (thanks @uudashr)
# 1.4.0 (2017-10-04)
## Changed
- #61: Update NewVersion to parse ints with a 64bit int size (thanks @zknill)
# 1.3.1 (2017-07-10)
## Fixed
- Fixed #57: number comparisons in prerelease sometimes inaccurate
# 1.3.0 (2017-05-02)
## Added
- #45: Added json (un)marshaling support (thanks @mh-cbon)
- Stability marker. See https://masterminds.github.io/stability/
## Fixed
- #51: Fix handling of single digit tilde constraint (thanks @dgodd)
## Changed
- #55: The godoc icon moved from png to svg
# 1.2.3 (2017-04-03)
## Fixed
- #46: Fixed 0.x.x and 0.0.x in constraints being treated as *
# Release 1.2.2 (2016-12-13)
## Fixed
- #34: Fixed issue where hyphen range was not working with pre-release parsing.
# Release 1.2.1 (2016-11-28)
## Fixed
- #24: Fixed edge case issue where constraint "> 0" does not handle "0.0.1-alpha"
properly.
# Release 1.2.0 (2016-11-04)
## Added
- #20: Added MustParse function for versions (thanks @adamreese)
- #15: Added increment methods on versions (thanks @mh-cbon)
## Fixed
- Issue #21: Per the SemVer spec (section 9) a pre-release is unstable and
might not satisfy the intended compatibility. The change here ignores pre-releases
on constraint checks (e.g., ~ or ^) when a pre-release is not part of the
constraint. For example, `^1.2.3` will ignore pre-releases while
`^1.2.3-alpha` will include them.
# Release 1.1.1 (2016-06-30)
## Changed
- Issue #9: Speed up version comparison performance (thanks @sdboyer)
- Issue #8: Added benchmarks (thanks @sdboyer)
- Updated Go Report Card URL to new location
- Updated Readme to add code snippet formatting (thanks @mh-cbon)
- Updating tagging to v[SemVer] structure for compatibility with other tools.
# Release 1.1.0 (2016-03-11)
- Issue #2: Implemented validation to provide reasons a versions failed a
constraint.
# Release 1.0.1 (2015-12-31)
- Fixed #1: * constraint failing on valid versions.
# Release 1.0.0 (2015-10-20)
- Initial release
-20
View File
@@ -1,20 +0,0 @@
The Masterminds
Copyright (C) 2014-2015, Matt Butcher and Matt Farina
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
-36
View File
@@ -1,36 +0,0 @@
.PHONY: setup
setup:
go get -u gopkg.in/alecthomas/gometalinter.v1
gometalinter.v1 --install
.PHONY: test
test: validate lint
@echo "==> Running tests"
go test -v
.PHONY: validate
validate:
@echo "==> Running static validations"
@gometalinter.v1 \
--disable-all \
--enable deadcode \
--severity deadcode:error \
--enable gofmt \
--enable gosimple \
--enable ineffassign \
--enable misspell \
--enable vet \
--tests \
--vendor \
--deadline 60s \
./... || exit_code=1
.PHONY: lint
lint:
@echo "==> Running linters"
@gometalinter.v1 \
--disable-all \
--enable golint \
--vendor \
--deadline 60s \
./... || :
-165
View File
@@ -1,165 +0,0 @@
# SemVer
The `semver` package provides the ability to work with [Semantic Versions](http://semver.org) in Go. Specifically it provides the ability to:
* Parse semantic versions
* Sort semantic versions
* Check if a semantic version fits within a set of constraints
* Optionally work with a `v` prefix
[![Stability:
Active](https://masterminds.github.io/stability/active.svg)](https://masterminds.github.io/stability/active.html)
[![Build Status](https://travis-ci.org/Masterminds/semver.svg)](https://travis-ci.org/Masterminds/semver) [![Build status](https://ci.appveyor.com/api/projects/status/jfk66lib7hb985k8/branch/master?svg=true&passingText=windows%20build%20passing&failingText=windows%20build%20failing)](https://ci.appveyor.com/project/mattfarina/semver/branch/master) [![GoDoc](https://godoc.org/github.com/Masterminds/semver?status.svg)](https://godoc.org/github.com/Masterminds/semver) [![Go Report Card](https://goreportcard.com/badge/github.com/Masterminds/semver)](https://goreportcard.com/report/github.com/Masterminds/semver)
## Parsing Semantic Versions
To parse a semantic version use the `NewVersion` function. For example,
```go
v, err := semver.NewVersion("1.2.3-beta.1+build345")
```
If there is an error the version wasn't parseable. The version object has methods
to get the parts of the version, compare it to other versions, convert the
version back into a string, and get the original string. For more details
please see the [documentation](https://godoc.org/github.com/Masterminds/semver).
## Sorting Semantic Versions
A set of versions can be sorted using the [`sort`](https://golang.org/pkg/sort/)
package from the standard library. For example,
```go
raw := []string{"1.2.3", "1.0", "1.3", "2", "0.4.2",}
vs := make([]*semver.Version, len(raw))
for i, r := range raw {
v, err := semver.NewVersion(r)
if err != nil {
t.Errorf("Error parsing version: %s", err)
}
vs[i] = v
}
sort.Sort(semver.Collection(vs))
```
## Checking Version Constraints
Checking a version against version constraints is one of the most featureful
parts of the package.
```go
c, err := semver.NewConstraint(">= 1.2.3")
if err != nil {
// Handle constraint not being parseable.
}
v, _ := semver.NewVersion("1.3")
if err != nil {
// Handle version not being parseable.
}
// Check if the version meets the constraints. The a variable will be true.
a := c.Check(v)
```
## Basic Comparisons
There are two elements to the comparisons. First, a comparison string is a list
of comma separated and comparisons. These are then separated by || separated or
comparisons. For example, `">= 1.2, < 3.0.0 || >= 4.2.3"` is looking for a
comparison that's greater than or equal to 1.2 and less than 3.0.0 or is
greater than or equal to 4.2.3.
The basic comparisons are:
* `=`: equal (aliased to no operator)
* `!=`: not equal
* `>`: greater than
* `<`: less than
* `>=`: greater than or equal to
* `<=`: less than or equal to
_Note, according to the Semantic Version specification pre-releases may not be
API compliant with their release counterpart. It says,_
> _A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version._
_SemVer comparisons without a pre-release value will skip pre-release versions.
For example, `>1.2.3` will skip pre-releases when looking at a list of values
while `>1.2.3-alpha.1` will evaluate pre-releases._
## Hyphen Range Comparisons
There are multiple methods to handle ranges and the first is hyphens ranges.
These look like:
* `1.2 - 1.4.5` which is equivalent to `>= 1.2, <= 1.4.5`
* `2.3.4 - 4.5` which is equivalent to `>= 2.3.4, <= 4.5`
## Wildcards In Comparisons
The `x`, `X`, and `*` characters can be used as a wildcard character. This works
for all comparison operators. When used on the `=` operator it falls
back to the pack level comparison (see tilde below). For example,
* `1.2.x` is equivalent to `>= 1.2.0, < 1.3.0`
* `>= 1.2.x` is equivalent to `>= 1.2.0`
* `<= 2.x` is equivalent to `<= 3`
* `*` is equivalent to `>= 0.0.0`
## Tilde Range Comparisons (Patch)
The tilde (`~`) comparison operator is for patch level ranges when a minor
version is specified and major level changes when the minor number is missing.
For example,
* `~1.2.3` is equivalent to `>= 1.2.3, < 1.3.0`
* `~1` is equivalent to `>= 1, < 2`
* `~2.3` is equivalent to `>= 2.3, < 2.4`
* `~1.2.x` is equivalent to `>= 1.2.0, < 1.3.0`
* `~1.x` is equivalent to `>= 1, < 2`
## Caret Range Comparisons (Major)
The caret (`^`) comparison operator is for major level changes. This is useful
when comparisons of API versions as a major change is API breaking. For example,
* `^1.2.3` is equivalent to `>= 1.2.3, < 2.0.0`
* `^1.2.x` is equivalent to `>= 1.2.0, < 2.0.0`
* `^2.3` is equivalent to `>= 2.3, < 3`
* `^2.x` is equivalent to `>= 2.0.0, < 3`
# Validation
In addition to testing a version against a constraint, a version can be validated
against a constraint. When validation fails a slice of errors containing why a
version didn't meet the constraint is returned. For example,
```go
c, err := semver.NewConstraint("<= 1.2.3, >= 1.4")
if err != nil {
// Handle constraint not being parseable.
}
v, _ := semver.NewVersion("1.3")
if err != nil {
// Handle version not being parseable.
}
// Validate a version against a constraint.
a, msgs := c.Validate(v)
// a is false
for _, m := range msgs {
fmt.Println(m)
// Loops over the errors which would read
// "1.3 is greater than 1.2.3"
// "1.3 is less than 1.4"
}
```
# Contribute
If you find an issue or want to contribute please file an [issue](https://github.com/Masterminds/semver/issues)
or [create a pull request](https://github.com/Masterminds/semver/pulls).
-44
View File
@@ -1,44 +0,0 @@
version: build-{build}.{branch}
clone_folder: C:\gopath\src\github.com\Masterminds\semver
shallow_clone: true
environment:
GOPATH: C:\gopath
platform:
- x64
install:
- go version
- go env
- go get -u gopkg.in/alecthomas/gometalinter.v1
- set PATH=%PATH%;%GOPATH%\bin
- gometalinter.v1.exe --install
build_script:
- go install -v ./...
test_script:
- "gometalinter.v1 \
--disable-all \
--enable deadcode \
--severity deadcode:error \
--enable gofmt \
--enable gosimple \
--enable ineffassign \
--enable misspell \
--enable vet \
--tests \
--vendor \
--deadline 60s \
./... || exit_code=1"
- "gometalinter.v1 \
--disable-all \
--enable golint \
--vendor \
--deadline 60s \
./... || :"
- go test -v
deploy: off
-24
View File
@@ -1,24 +0,0 @@
package semver
// Collection is a collection of Version instances and implements the sort
// interface. See the sort package for more details.
// https://golang.org/pkg/sort/
type Collection []*Version
// Len returns the length of a collection. The number of Version instances
// on the slice.
func (c Collection) Len() int {
return len(c)
}
// Less is needed for the sort interface to compare two Version objects on the
// slice. If checks if one is less than the other.
func (c Collection) Less(i, j int) bool {
return c[i].LessThan(c[j])
}
// Swap is needed for the sort interface to replace the Version objects
// at two different positions in the slice.
func (c Collection) Swap(i, j int) {
c[i], c[j] = c[j], c[i]
}
-426
View File
@@ -1,426 +0,0 @@
package semver
import (
"errors"
"fmt"
"regexp"
"strings"
)
// Constraints is one or more constraint that a semantic version can be
// checked against.
type Constraints struct {
constraints [][]*constraint
}
// NewConstraint returns a Constraints instance that a Version instance can
// be checked against. If there is a parse error it will be returned.
func NewConstraint(c string) (*Constraints, error) {
// Rewrite - ranges into a comparison operation.
c = rewriteRange(c)
ors := strings.Split(c, "||")
or := make([][]*constraint, len(ors))
for k, v := range ors {
cs := strings.Split(v, ",")
result := make([]*constraint, len(cs))
for i, s := range cs {
pc, err := parseConstraint(s)
if err != nil {
return nil, err
}
result[i] = pc
}
or[k] = result
}
o := &Constraints{constraints: or}
return o, nil
}
// Check tests if a version satisfies the constraints.
func (cs Constraints) Check(v *Version) bool {
// loop over the ORs and check the inner ANDs
for _, o := range cs.constraints {
joy := true
for _, c := range o {
if !c.check(v) {
joy = false
break
}
}
if joy {
return true
}
}
return false
}
// Validate checks if a version satisfies a constraint. If not a slice of
// reasons for the failure are returned in addition to a bool.
func (cs Constraints) Validate(v *Version) (bool, []error) {
// loop over the ORs and check the inner ANDs
var e []error
for _, o := range cs.constraints {
joy := true
for _, c := range o {
if !c.check(v) {
em := fmt.Errorf(c.msg, v, c.orig)
e = append(e, em)
joy = false
}
}
if joy {
return true, []error{}
}
}
return false, e
}
var constraintOps map[string]cfunc
var constraintMsg map[string]string
var constraintRegex *regexp.Regexp
func init() {
constraintOps = map[string]cfunc{
"": constraintTildeOrEqual,
"=": constraintTildeOrEqual,
"!=": constraintNotEqual,
">": constraintGreaterThan,
"<": constraintLessThan,
">=": constraintGreaterThanEqual,
"=>": constraintGreaterThanEqual,
"<=": constraintLessThanEqual,
"=<": constraintLessThanEqual,
"~": constraintTilde,
"~>": constraintTilde,
"^": constraintCaret,
}
constraintMsg = map[string]string{
"": "%s is not equal to %s",
"=": "%s is not equal to %s",
"!=": "%s is equal to %s",
">": "%s is less than or equal to %s",
"<": "%s is greater than or equal to %s",
">=": "%s is less than %s",
"=>": "%s is less than %s",
"<=": "%s is greater than %s",
"=<": "%s is greater than %s",
"~": "%s does not have same major and minor version as %s",
"~>": "%s does not have same major and minor version as %s",
"^": "%s does not have same major version as %s",
}
ops := make([]string, 0, len(constraintOps))
for k := range constraintOps {
ops = append(ops, regexp.QuoteMeta(k))
}
constraintRegex = regexp.MustCompile(fmt.Sprintf(
`^\s*(%s)\s*(%s)\s*$`,
strings.Join(ops, "|"),
cvRegex))
constraintRangeRegex = regexp.MustCompile(fmt.Sprintf(
`\s*(%s)\s+-\s+(%s)\s*`,
cvRegex, cvRegex))
}
// An individual constraint
type constraint struct {
// The callback function for the restraint. It performs the logic for
// the constraint.
function cfunc
msg string
// The version used in the constraint check. For example, if a constraint
// is '<= 2.0.0' the con a version instance representing 2.0.0.
con *Version
// The original parsed version (e.g., 4.x from != 4.x)
orig string
// When an x is used as part of the version (e.g., 1.x)
minorDirty bool
dirty bool
patchDirty bool
}
// Check if a version meets the constraint
func (c *constraint) check(v *Version) bool {
return c.function(v, c)
}
type cfunc func(v *Version, c *constraint) bool
func parseConstraint(c string) (*constraint, error) {
m := constraintRegex.FindStringSubmatch(c)
if m == nil {
return nil, fmt.Errorf("improper constraint: %s", c)
}
ver := m[2]
orig := ver
minorDirty := false
patchDirty := false
dirty := false
if isX(m[3]) {
ver = "0.0.0"
dirty = true
} else if isX(strings.TrimPrefix(m[4], ".")) || m[4] == "" {
minorDirty = true
dirty = true
ver = fmt.Sprintf("%s.0.0%s", m[3], m[6])
} else if isX(strings.TrimPrefix(m[5], ".")) {
dirty = true
patchDirty = true
ver = fmt.Sprintf("%s%s.0%s", m[3], m[4], m[6])
}
con, err := NewVersion(ver)
if err != nil {
// The constraintRegex should catch any regex parsing errors. So,
// we should never get here.
return nil, errors.New("constraint Parser Error")
}
cs := &constraint{
function: constraintOps[m[1]],
msg: constraintMsg[m[1]],
con: con,
orig: orig,
minorDirty: minorDirty,
patchDirty: patchDirty,
dirty: dirty,
}
return cs, nil
}
// Constraint functions
func constraintNotEqual(v *Version, c *constraint) bool {
if c.dirty {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if c.con.Major() != v.Major() {
return true
}
if c.con.Minor() != v.Minor() && !c.minorDirty {
return true
} else if c.minorDirty {
return false
}
return false
}
return !v.Equal(c.con)
}
func constraintGreaterThan(v *Version, c *constraint) bool {
// An edge case the constraint is 0.0.0 and the version is 0.0.0-someprerelease
// exists. This that case.
if !isNonZero(c.con) && isNonZero(v) {
return true
}
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
return v.Compare(c.con) == 1
}
func constraintLessThan(v *Version, c *constraint) bool {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if !c.dirty {
return v.Compare(c.con) < 0
}
if v.Major() > c.con.Major() {
return false
} else if v.Minor() > c.con.Minor() && !c.minorDirty {
return false
}
return true
}
func constraintGreaterThanEqual(v *Version, c *constraint) bool {
// An edge case the constraint is 0.0.0 and the version is 0.0.0-someprerelease
// exists. This that case.
if !isNonZero(c.con) && isNonZero(v) {
return true
}
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
return v.Compare(c.con) >= 0
}
func constraintLessThanEqual(v *Version, c *constraint) bool {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if !c.dirty {
return v.Compare(c.con) <= 0
}
if v.Major() > c.con.Major() {
return false
} else if v.Minor() > c.con.Minor() && !c.minorDirty {
return false
}
return true
}
// ~*, ~>* --> >= 0.0.0 (any)
// ~2, ~2.x, ~2.x.x, ~>2, ~>2.x ~>2.x.x --> >=2.0.0, <3.0.0
// ~2.0, ~2.0.x, ~>2.0, ~>2.0.x --> >=2.0.0, <2.1.0
// ~1.2, ~1.2.x, ~>1.2, ~>1.2.x --> >=1.2.0, <1.3.0
// ~1.2.3, ~>1.2.3 --> >=1.2.3, <1.3.0
// ~1.2.0, ~>1.2.0 --> >=1.2.0, <1.3.0
func constraintTilde(v *Version, c *constraint) bool {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if v.LessThan(c.con) {
return false
}
// ~0.0.0 is a special case where all constraints are accepted. It's
// equivalent to >= 0.0.0.
if c.con.Major() == 0 && c.con.Minor() == 0 && c.con.Patch() == 0 &&
!c.minorDirty && !c.patchDirty {
return true
}
if v.Major() != c.con.Major() {
return false
}
if v.Minor() != c.con.Minor() && !c.minorDirty {
return false
}
return true
}
// When there is a .x (dirty) status it automatically opts in to ~. Otherwise
// it's a straight =
func constraintTildeOrEqual(v *Version, c *constraint) bool {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if c.dirty {
c.msg = constraintMsg["~"]
return constraintTilde(v, c)
}
return v.Equal(c.con)
}
// ^* --> (any)
// ^2, ^2.x, ^2.x.x --> >=2.0.0, <3.0.0
// ^2.0, ^2.0.x --> >=2.0.0, <3.0.0
// ^1.2, ^1.2.x --> >=1.2.0, <2.0.0
// ^1.2.3 --> >=1.2.3, <2.0.0
// ^1.2.0 --> >=1.2.0, <2.0.0
func constraintCaret(v *Version, c *constraint) bool {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if v.LessThan(c.con) {
return false
}
if v.Major() != c.con.Major() {
return false
}
return true
}
var constraintRangeRegex *regexp.Regexp
const cvRegex string = `v?([0-9|x|X|\*]+)(\.[0-9|x|X|\*]+)?(\.[0-9|x|X|\*]+)?` +
`(-([0-9A-Za-z\-]+(\.[0-9A-Za-z\-]+)*))?` +
`(\+([0-9A-Za-z\-]+(\.[0-9A-Za-z\-]+)*))?`
func isX(x string) bool {
switch x {
case "x", "*", "X":
return true
default:
return false
}
}
func rewriteRange(i string) string {
m := constraintRangeRegex.FindAllStringSubmatch(i, -1)
if m == nil {
return i
}
o := i
for _, v := range m {
t := fmt.Sprintf(">= %s, <= %s", v[1], v[11])
o = strings.Replace(o, v[0], t, 1)
}
return o
}
// Detect if a version is not zero (0.0.0)
func isNonZero(v *Version) bool {
if v.Major() != 0 || v.Minor() != 0 || v.Patch() != 0 || v.Prerelease() != "" {
return true
}
return false
}
-115
View File
@@ -1,115 +0,0 @@
/*
Package semver provides the ability to work with Semantic Versions (http://semver.org) in Go.
Specifically it provides the ability to:
* Parse semantic versions
* Sort semantic versions
* Check if a semantic version fits within a set of constraints
* Optionally work with a `v` prefix
Parsing Semantic Versions
To parse a semantic version use the `NewVersion` function. For example,
v, err := semver.NewVersion("1.2.3-beta.1+build345")
If there is an error the version wasn't parseable. The version object has methods
to get the parts of the version, compare it to other versions, convert the
version back into a string, and get the original string. For more details
please see the documentation at https://godoc.org/github.com/Masterminds/semver.
Sorting Semantic Versions
A set of versions can be sorted using the `sort` package from the standard library.
For example,
raw := []string{"1.2.3", "1.0", "1.3", "2", "0.4.2",}
vs := make([]*semver.Version, len(raw))
for i, r := range raw {
v, err := semver.NewVersion(r)
if err != nil {
t.Errorf("Error parsing version: %s", err)
}
vs[i] = v
}
sort.Sort(semver.Collection(vs))
Checking Version Constraints
Checking a version against version constraints is one of the most featureful
parts of the package.
c, err := semver.NewConstraint(">= 1.2.3")
if err != nil {
// Handle constraint not being parseable.
}
v, _ := semver.NewVersion("1.3")
if err != nil {
// Handle version not being parseable.
}
// Check if the version meets the constraints. The a variable will be true.
a := c.Check(v)
Basic Comparisons
There are two elements to the comparisons. First, a comparison string is a list
of comma separated and comparisons. These are then separated by || separated or
comparisons. For example, `">= 1.2, < 3.0.0 || >= 4.2.3"` is looking for a
comparison that's greater than or equal to 1.2 and less than 3.0.0 or is
greater than or equal to 4.2.3.
The basic comparisons are:
* `=`: equal (aliased to no operator)
* `!=`: not equal
* `>`: greater than
* `<`: less than
* `>=`: greater than or equal to
* `<=`: less than or equal to
Hyphen Range Comparisons
There are multiple methods to handle ranges and the first is hyphens ranges.
These look like:
* `1.2 - 1.4.5` which is equivalent to `>= 1.2, <= 1.4.5`
* `2.3.4 - 4.5` which is equivalent to `>= 2.3.4, <= 4.5`
Wildcards In Comparisons
The `x`, `X`, and `*` characters can be used as a wildcard character. This works
for all comparison operators. When used on the `=` operator it falls
back to the pack level comparison (see tilde below). For example,
* `1.2.x` is equivalent to `>= 1.2.0, < 1.3.0`
* `>= 1.2.x` is equivalent to `>= 1.2.0`
* `<= 2.x` is equivalent to `<= 3`
* `*` is equivalent to `>= 0.0.0`
Tilde Range Comparisons (Patch)
The tilde (`~`) comparison operator is for patch level ranges when a minor
version is specified and major level changes when the minor number is missing.
For example,
* `~1.2.3` is equivalent to `>= 1.2.3, < 1.3.0`
* `~1` is equivalent to `>= 1, < 2`
* `~2.3` is equivalent to `>= 2.3, < 2.4`
* `~1.2.x` is equivalent to `>= 1.2.0, < 1.3.0`
* `~1.x` is equivalent to `>= 1, < 2`
Caret Range Comparisons (Major)
The caret (`^`) comparison operator is for major level changes. This is useful
when comparisons of API versions as a major change is API breaking. For example,
* `^1.2.3` is equivalent to `>= 1.2.3, < 2.0.0`
* `^1.2.x` is equivalent to `>= 1.2.0, < 2.0.0`
* `^2.3` is equivalent to `>= 2.3, < 3`
* `^2.x` is equivalent to `>= 2.0.0, < 3`
*/
package semver
-421
View File
@@ -1,421 +0,0 @@
package semver
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"regexp"
"strconv"
"strings"
)
// The compiled version of the regex created at init() is cached here so it
// only needs to be created once.
var versionRegex *regexp.Regexp
var validPrereleaseRegex *regexp.Regexp
var (
// ErrInvalidSemVer is returned a version is found to be invalid when
// being parsed.
ErrInvalidSemVer = errors.New("Invalid Semantic Version")
// ErrInvalidMetadata is returned when the metadata is an invalid format
ErrInvalidMetadata = errors.New("Invalid Metadata string")
// ErrInvalidPrerelease is returned when the pre-release is an invalid format
ErrInvalidPrerelease = errors.New("Invalid Prerelease string")
)
// SemVerRegex is the regular expression used to parse a semantic version.
const SemVerRegex string = `v?([0-9]+)(\.[0-9]+)?(\.[0-9]+)?` +
`(-([0-9A-Za-z\-]+(\.[0-9A-Za-z\-]+)*))?` +
`(\+([0-9A-Za-z\-]+(\.[0-9A-Za-z\-]+)*))?`
// ValidPrerelease is the regular expression which validates
// both prerelease and metadata values.
const ValidPrerelease string = `^([0-9A-Za-z\-]+(\.[0-9A-Za-z\-]+)*)`
// Version represents a single semantic version.
type Version struct {
major, minor, patch int64
pre string
metadata string
original string
}
func init() {
versionRegex = regexp.MustCompile("^" + SemVerRegex + "$")
validPrereleaseRegex = regexp.MustCompile(ValidPrerelease)
}
// NewVersion parses a given version and returns an instance of Version or
// an error if unable to parse the version.
func NewVersion(v string) (*Version, error) {
m := versionRegex.FindStringSubmatch(v)
if m == nil {
return nil, ErrInvalidSemVer
}
sv := &Version{
metadata: m[8],
pre: m[5],
original: v,
}
var temp int64
temp, err := strconv.ParseInt(m[1], 10, 64)
if err != nil {
return nil, fmt.Errorf("Error parsing version segment: %s", err)
}
sv.major = temp
if m[2] != "" {
temp, err = strconv.ParseInt(strings.TrimPrefix(m[2], "."), 10, 64)
if err != nil {
return nil, fmt.Errorf("Error parsing version segment: %s", err)
}
sv.minor = temp
} else {
sv.minor = 0
}
if m[3] != "" {
temp, err = strconv.ParseInt(strings.TrimPrefix(m[3], "."), 10, 64)
if err != nil {
return nil, fmt.Errorf("Error parsing version segment: %s", err)
}
sv.patch = temp
} else {
sv.patch = 0
}
return sv, nil
}
// MustParse parses a given version and panics on error.
func MustParse(v string) *Version {
sv, err := NewVersion(v)
if err != nil {
panic(err)
}
return sv
}
// String converts a Version object to a string.
// Note, if the original version contained a leading v this version will not.
// See the Original() method to retrieve the original value. Semantic Versions
// don't contain a leading v per the spec. Instead it's optional on
// impelementation.
func (v *Version) String() string {
var buf bytes.Buffer
fmt.Fprintf(&buf, "%d.%d.%d", v.major, v.minor, v.patch)
if v.pre != "" {
fmt.Fprintf(&buf, "-%s", v.pre)
}
if v.metadata != "" {
fmt.Fprintf(&buf, "+%s", v.metadata)
}
return buf.String()
}
// Original returns the original value passed in to be parsed.
func (v *Version) Original() string {
return v.original
}
// Major returns the major version.
func (v *Version) Major() int64 {
return v.major
}
// Minor returns the minor version.
func (v *Version) Minor() int64 {
return v.minor
}
// Patch returns the patch version.
func (v *Version) Patch() int64 {
return v.patch
}
// Prerelease returns the pre-release version.
func (v *Version) Prerelease() string {
return v.pre
}
// Metadata returns the metadata on the version.
func (v *Version) Metadata() string {
return v.metadata
}
// originalVPrefix returns the original 'v' prefix if any.
func (v *Version) originalVPrefix() string {
// Note, only lowercase v is supported as a prefix by the parser.
if v.original != "" && v.original[:1] == "v" {
return v.original[:1]
}
return ""
}
// IncPatch produces the next patch version.
// If the current version does not have prerelease/metadata information,
// it unsets metadata and prerelease values, increments patch number.
// If the current version has any of prerelease or metadata information,
// it unsets both values and keeps curent patch value
func (v Version) IncPatch() Version {
vNext := v
// according to http://semver.org/#spec-item-9
// Pre-release versions have a lower precedence than the associated normal version.
// according to http://semver.org/#spec-item-10
// Build metadata SHOULD be ignored when determining version precedence.
if v.pre != "" {
vNext.metadata = ""
vNext.pre = ""
} else {
vNext.metadata = ""
vNext.pre = ""
vNext.patch = v.patch + 1
}
vNext.original = v.originalVPrefix() + "" + vNext.String()
return vNext
}
// IncMinor produces the next minor version.
// Sets patch to 0.
// Increments minor number.
// Unsets metadata.
// Unsets prerelease status.
func (v Version) IncMinor() Version {
vNext := v
vNext.metadata = ""
vNext.pre = ""
vNext.patch = 0
vNext.minor = v.minor + 1
vNext.original = v.originalVPrefix() + "" + vNext.String()
return vNext
}
// IncMajor produces the next major version.
// Sets patch to 0.
// Sets minor to 0.
// Increments major number.
// Unsets metadata.
// Unsets prerelease status.
func (v Version) IncMajor() Version {
vNext := v
vNext.metadata = ""
vNext.pre = ""
vNext.patch = 0
vNext.minor = 0
vNext.major = v.major + 1
vNext.original = v.originalVPrefix() + "" + vNext.String()
return vNext
}
// SetPrerelease defines the prerelease value.
// Value must not include the required 'hypen' prefix.
func (v Version) SetPrerelease(prerelease string) (Version, error) {
vNext := v
if len(prerelease) > 0 && !validPrereleaseRegex.MatchString(prerelease) {
return vNext, ErrInvalidPrerelease
}
vNext.pre = prerelease
vNext.original = v.originalVPrefix() + "" + vNext.String()
return vNext, nil
}
// SetMetadata defines metadata value.
// Value must not include the required 'plus' prefix.
func (v Version) SetMetadata(metadata string) (Version, error) {
vNext := v
if len(metadata) > 0 && !validPrereleaseRegex.MatchString(metadata) {
return vNext, ErrInvalidMetadata
}
vNext.metadata = metadata
vNext.original = v.originalVPrefix() + "" + vNext.String()
return vNext, nil
}
// LessThan tests if one version is less than another one.
func (v *Version) LessThan(o *Version) bool {
return v.Compare(o) < 0
}
// GreaterThan tests if one version is greater than another one.
func (v *Version) GreaterThan(o *Version) bool {
return v.Compare(o) > 0
}
// Equal tests if two versions are equal to each other.
// Note, versions can be equal with different metadata since metadata
// is not considered part of the comparable version.
func (v *Version) Equal(o *Version) bool {
return v.Compare(o) == 0
}
// Compare compares this version to another one. It returns -1, 0, or 1 if
// the version smaller, equal, or larger than the other version.
//
// Versions are compared by X.Y.Z. Build metadata is ignored. Prerelease is
// lower than the version without a prerelease.
func (v *Version) Compare(o *Version) int {
// Compare the major, minor, and patch version for differences. If a
// difference is found return the comparison.
if d := compareSegment(v.Major(), o.Major()); d != 0 {
return d
}
if d := compareSegment(v.Minor(), o.Minor()); d != 0 {
return d
}
if d := compareSegment(v.Patch(), o.Patch()); d != 0 {
return d
}
// At this point the major, minor, and patch versions are the same.
ps := v.pre
po := o.Prerelease()
if ps == "" && po == "" {
return 0
}
if ps == "" {
return 1
}
if po == "" {
return -1
}
return comparePrerelease(ps, po)
}
// UnmarshalJSON implements JSON.Unmarshaler interface.
func (v *Version) UnmarshalJSON(b []byte) error {
var s string
if err := json.Unmarshal(b, &s); err != nil {
return err
}
temp, err := NewVersion(s)
if err != nil {
return err
}
v.major = temp.major
v.minor = temp.minor
v.patch = temp.patch
v.pre = temp.pre
v.metadata = temp.metadata
v.original = temp.original
temp = nil
return nil
}
// MarshalJSON implements JSON.Marshaler interface.
func (v *Version) MarshalJSON() ([]byte, error) {
return json.Marshal(v.String())
}
func compareSegment(v, o int64) int {
if v < o {
return -1
}
if v > o {
return 1
}
return 0
}
func comparePrerelease(v, o string) int {
// split the prelease versions by their part. The separator, per the spec,
// is a .
sparts := strings.Split(v, ".")
oparts := strings.Split(o, ".")
// Find the longer length of the parts to know how many loop iterations to
// go through.
slen := len(sparts)
olen := len(oparts)
l := slen
if olen > slen {
l = olen
}
// Iterate over each part of the prereleases to compare the differences.
for i := 0; i < l; i++ {
// Since the lentgh of the parts can be different we need to create
// a placeholder. This is to avoid out of bounds issues.
stemp := ""
if i < slen {
stemp = sparts[i]
}
otemp := ""
if i < olen {
otemp = oparts[i]
}
d := comparePrePart(stemp, otemp)
if d != 0 {
return d
}
}
// Reaching here means two versions are of equal value but have different
// metadata (the part following a +). They are not identical in string form
// but the version comparison finds them to be equal.
return 0
}
func comparePrePart(s, o string) int {
// Fastpath if they are equal
if s == o {
return 0
}
// When s or o are empty we can use the other in an attempt to determine
// the response.
if s == "" {
if o != "" {
return -1
}
return 1
}
if o == "" {
if s != "" {
return 1
}
return -1
}
// When comparing strings "99" is greater than "103". To handle
// cases like this we need to detect numbers and compare them.
oi, n1 := strconv.ParseInt(o, 10, 64)
si, n2 := strconv.ParseInt(s, 10, 64)
// The case where both are strings compare the strings
if n1 != nil && n2 != nil {
if s > o {
return 1
}
return -1
} else if n1 != nil {
// o is a string and s is a number
return -1
} else if n2 != nil {
// s is a string and o is a number
return 1
}
// Both are numbers
if si > oi {
return 1
}
return -1
}
-2
View File
@@ -1,2 +0,0 @@
vendor/
/.glide
-23
View File
@@ -1,23 +0,0 @@
language: go
go:
- 1.9.x
- 1.10.x
- tip
# Setting sudo access to false will let Travis CI use containers rather than
# VMs to run the tests. For more details see:
# - http://docs.travis-ci.com/user/workers/container-based-infrastructure/
# - http://docs.travis-ci.com/user/workers/standard-infrastructure/
sudo: false
script:
- make setup test
notifications:
webhooks:
urls:
- https://webhooks.gitter.im/e/06e3328629952dabe3e0
on_success: change # options: [always|never|change] default: always
on_failure: always # options: [always|never|change] default: always
on_start: never # options: [always|never|change] default: always
-153
View File
@@ -1,153 +0,0 @@
# Changelog
## Release 2.15.0 (2018-04-02)
### Added
- #68 and #69: Add json helpers to docs (thanks @arunvelsriram)
- #66: Add ternary function (thanks @binoculars)
- #67: Allow keys function to take multiple dicts (thanks @binoculars)
- #89: Added sha1sum to crypto function (thanks @benkeil)
- #81: Allow customizing Root CA that used by genSignedCert (thanks @chenzhiwei)
- #92: Add travis testing for go 1.10
- #93: Adding appveyor config for windows testing
### Changed
- #90: Updating to more recent dependencies
- #73: replace satori/go.uuid with google/uuid (thanks @petterw)
### Fixed
- #76: Fixed documentation typos (thanks @Thiht)
- Fixed rounding issue on the `ago` function. Note, the removes support for Go 1.8 and older
## Release 2.14.1 (2017-12-01)
### Fixed
- #60: Fix typo in function name documentation (thanks @neil-ca-moore)
- #61: Removing line with {{ due to blocking github pages genertion
- #64: Update the list functions to handle int, string, and other slices for compatibility
## Release 2.14.0 (2017-10-06)
This new version of Sprig adds a set of functions for generating and working with SSL certificates.
- `genCA` generates an SSL Certificate Authority
- `genSelfSignedCert` generates an SSL self-signed certificate
- `genSignedCert` generates an SSL certificate and key based on a given CA
## Release 2.13.0 (2017-09-18)
This release adds new functions, including:
- `regexMatch`, `regexFindAll`, `regexFind`, `regexReplaceAll`, `regexReplaceAllLiteral`, and `regexSplit` to work with regular expressions
- `floor`, `ceil`, and `round` math functions
- `toDate` converts a string to a date
- `nindent` is just like `indent` but also prepends a new line
- `ago` returns the time from `time.Now`
### Added
- #40: Added basic regex functionality (thanks @alanquillin)
- #41: Added ceil floor and round functions (thanks @alanquillin)
- #48: Added toDate function (thanks @andreynering)
- #50: Added nindent function (thanks @binoculars)
- #46: Added ago function (thanks @slayer)
### Changed
- #51: Updated godocs to include new string functions (thanks @curtisallen)
- #49: Added ability to merge multiple dicts (thanks @binoculars)
## Release 2.12.0 (2017-05-17)
- `snakecase`, `camelcase`, and `shuffle` are three new string functions
- `fail` allows you to bail out of a template render when conditions are not met
## Release 2.11.0 (2017-05-02)
- Added `toJson` and `toPrettyJson`
- Added `merge`
- Refactored documentation
## Release 2.10.0 (2017-03-15)
- Added `semver` and `semverCompare` for Semantic Versions
- `list` replaces `tuple`
- Fixed issue with `join`
- Added `first`, `last`, `intial`, `rest`, `prepend`, `append`, `toString`, `toStrings`, `sortAlpha`, `reverse`, `coalesce`, `pluck`, `pick`, `compact`, `keys`, `omit`, `uniq`, `has`, `without`
## Release 2.9.0 (2017-02-23)
- Added `splitList` to split a list
- Added crypto functions of `genPrivateKey` and `derivePassword`
## Release 2.8.0 (2016-12-21)
- Added access to several path functions (`base`, `dir`, `clean`, `ext`, and `abs`)
- Added functions for _mutating_ dictionaries (`set`, `unset`, `hasKey`)
## Release 2.7.0 (2016-12-01)
- Added `sha256sum` to generate a hash of an input
- Added functions to convert a numeric or string to `int`, `int64`, `float64`
## Release 2.6.0 (2016-10-03)
- Added a `uuidv4` template function for generating UUIDs inside of a template.
## Release 2.5.0 (2016-08-19)
- New `trimSuffix`, `trimPrefix`, `hasSuffix`, and `hasPrefix` functions
- New aliases have been added for a few functions that didn't follow the naming conventions (`trimAll` and `abbrevBoth`)
- `trimall` and `abbrevboth` (notice the case) are deprecated and will be removed in 3.0.0
## Release 2.4.0 (2016-08-16)
- Adds two functions: `until` and `untilStep`
## Release 2.3.0 (2016-06-21)
- cat: Concatenate strings with whitespace separators.
- replace: Replace parts of a string: `replace " " "-" "Me First"` renders "Me-First"
- plural: Format plurals: `len "foo" | plural "one foo" "many foos"` renders "many foos"
- indent: Indent blocks of text in a way that is sensitive to "\n" characters.
## Release 2.2.0 (2016-04-21)
- Added a `genPrivateKey` function (Thanks @bacongobbler)
## Release 2.1.0 (2016-03-30)
- `default` now prints the default value when it does not receive a value down the pipeline. It is much safer now to do `{{.Foo | default "bar"}}`.
- Added accessors for "hermetic" functions. These return only functions that, when given the same input, produce the same output.
## Release 2.0.0 (2016-03-29)
Because we switched from `int` to `int64` as the return value for all integer math functions, the library's major version number has been incremented.
- `min` complements `max` (formerly `biggest`)
- `empty` indicates that a value is the empty value for its type
- `tuple` creates a tuple inside of a template: `{{$t := tuple "a", "b" "c"}}`
- `dict` creates a dictionary inside of a template `{{$d := dict "key1" "val1" "key2" "val2"}}`
- Date formatters have been added for HTML dates (as used in `date` input fields)
- Integer math functions can convert from a number of types, including `string` (via `strconv.ParseInt`).
## Release 1.2.0 (2016-02-01)
- Added quote and squote
- Added b32enc and b32dec
- add now takes varargs
- biggest now takes varargs
## Release 1.1.0 (2015-12-29)
- Added #4: Added contains function. strings.Contains, but with the arguments
switched to simplify common pipelines. (thanks krancour)
- Added Travis-CI testing support
## Release 1.0.0 (2015-12-23)
- Initial release
-20
View File
@@ -1,20 +0,0 @@
Sprig
Copyright (C) 2013 Masterminds
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
-13
View File
@@ -1,13 +0,0 @@
HAS_GLIDE := $(shell command -v glide;)
.PHONY: test
test:
go test -v .
.PHONY: setup
setup:
ifndef HAS_GLIDE
go get -u github.com/Masterminds/glide
endif
glide install
-81
View File
@@ -1,81 +0,0 @@
# Sprig: Template functions for Go templates
[![Stability: Sustained](https://masterminds.github.io/stability/sustained.svg)](https://masterminds.github.io/stability/sustained.html)
[![Build Status](https://travis-ci.org/Masterminds/sprig.svg?branch=master)](https://travis-ci.org/Masterminds/sprig)
The Go language comes with a [built-in template
language](http://golang.org/pkg/text/template/), but not
very many template functions. This library provides a group of commonly
used template functions.
It is inspired by the template functions found in
[Twig](http://twig.sensiolabs.org/documentation) and also in various
JavaScript libraries, such as [underscore.js](http://underscorejs.org/).
## Usage
Template developers can read the [Sprig function documentation](http://masterminds.github.io/sprig/) to
learn about the >100 template functions available.
For Go developers wishing to include Sprig as a library in their programs,
API documentation is available [at GoDoc.org](http://godoc.org/github.com/Masterminds/sprig), but
read on for standard usage.
### Load the Sprig library
To load the Sprig `FuncMap`:
```go
import (
"github.com/Masterminds/sprig"
"html/template"
)
// This example illustrates that the FuncMap *must* be set before the
// templates themselves are loaded.
tpl := template.Must(
template.New("base").Funcs(sprig.FuncMap()).ParseGlob("*.html")
)
```
### Call the functions inside of templates
By convention, all functions are lowercase. This seems to follow the Go
idiom for template functions (as opposed to template methods, which are
TitleCase).
Example:
```
{{ "hello!" | upper | repeat 5 }}
```
Produces:
```
HELLO!HELLO!HELLO!HELLO!HELLO!
```
## Principles:
The following principles were used in deciding on which functions to add, and
determining how to implement them.
- Template functions should be used to build layout. Therefore, the following
types of operations are within the domain of template functions:
- Formatting
- Layout
- Simple type conversions
- Utilities that assist in handling common formatting and layout needs (e.g. arithmetic)
- Template functions should not return errors unless there is no way to print
a sensible value. For example, converting a string to an integer should not
produce an error if conversion fails. Instead, it should display a default
value that can be displayed.
- Simple math is necessary for grid layouts, pagers, and so on. Complex math
(anything other than arithmetic) should be done outside of templates.
- Template functions only deal with the data passed into them. They never retrieve
data from a source.
- Finally, do not override core Go template functions.
-26
View File
@@ -1,26 +0,0 @@
version: build-{build}.{branch}
clone_folder: C:\gopath\src\github.com\Masterminds\sprig
shallow_clone: true
environment:
GOPATH: C:\gopath
platform:
- x64
install:
- go get -u github.com/Masterminds/glide
- set PATH=%GOPATH%\bin;%PATH%
- go version
- go env
build_script:
- glide install
- go install ./...
test_script:
- go test -v
deploy: off
-430
View File
@@ -1,430 +0,0 @@
package sprig
import (
"bytes"
"crypto/dsa"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/hmac"
"crypto/rand"
"crypto/rsa"
"crypto/sha1"
"crypto/sha256"
"crypto/x509"
"crypto/x509/pkix"
"encoding/asn1"
"encoding/base64"
"encoding/binary"
"encoding/hex"
"encoding/pem"
"errors"
"fmt"
"math/big"
"net"
"time"
"github.com/google/uuid"
"golang.org/x/crypto/scrypt"
)
func sha256sum(input string) string {
hash := sha256.Sum256([]byte(input))
return hex.EncodeToString(hash[:])
}
func sha1sum(input string) string {
hash := sha1.Sum([]byte(input))
return hex.EncodeToString(hash[:])
}
// uuidv4 provides a safe and secure UUID v4 implementation
func uuidv4() string {
return fmt.Sprintf("%s", uuid.New())
}
var master_password_seed = "com.lyndir.masterpassword"
var password_type_templates = map[string][][]byte{
"maximum": {[]byte("anoxxxxxxxxxxxxxxxxx"), []byte("axxxxxxxxxxxxxxxxxno")},
"long": {[]byte("CvcvnoCvcvCvcv"), []byte("CvcvCvcvnoCvcv"), []byte("CvcvCvcvCvcvno"), []byte("CvccnoCvcvCvcv"), []byte("CvccCvcvnoCvcv"),
[]byte("CvccCvcvCvcvno"), []byte("CvcvnoCvccCvcv"), []byte("CvcvCvccnoCvcv"), []byte("CvcvCvccCvcvno"), []byte("CvcvnoCvcvCvcc"),
[]byte("CvcvCvcvnoCvcc"), []byte("CvcvCvcvCvccno"), []byte("CvccnoCvccCvcv"), []byte("CvccCvccnoCvcv"), []byte("CvccCvccCvcvno"),
[]byte("CvcvnoCvccCvcc"), []byte("CvcvCvccnoCvcc"), []byte("CvcvCvccCvccno"), []byte("CvccnoCvcvCvcc"), []byte("CvccCvcvnoCvcc"),
[]byte("CvccCvcvCvccno")},
"medium": {[]byte("CvcnoCvc"), []byte("CvcCvcno")},
"short": {[]byte("Cvcn")},
"basic": {[]byte("aaanaaan"), []byte("aannaaan"), []byte("aaannaaa")},
"pin": {[]byte("nnnn")},
}
var template_characters = map[byte]string{
'V': "AEIOU",
'C': "BCDFGHJKLMNPQRSTVWXYZ",
'v': "aeiou",
'c': "bcdfghjklmnpqrstvwxyz",
'A': "AEIOUBCDFGHJKLMNPQRSTVWXYZ",
'a': "AEIOUaeiouBCDFGHJKLMNPQRSTVWXYZbcdfghjklmnpqrstvwxyz",
'n': "0123456789",
'o': "@&%?,=[]_:-+*$#!'^~;()/.",
'x': "AEIOUaeiouBCDFGHJKLMNPQRSTVWXYZbcdfghjklmnpqrstvwxyz0123456789!@#$%^&*()",
}
func derivePassword(counter uint32, password_type, password, user, site string) string {
var templates = password_type_templates[password_type]
if templates == nil {
return fmt.Sprintf("cannot find password template %s", password_type)
}
var buffer bytes.Buffer
buffer.WriteString(master_password_seed)
binary.Write(&buffer, binary.BigEndian, uint32(len(user)))
buffer.WriteString(user)
salt := buffer.Bytes()
key, err := scrypt.Key([]byte(password), salt, 32768, 8, 2, 64)
if err != nil {
return fmt.Sprintf("failed to derive password: %s", err)
}
buffer.Truncate(len(master_password_seed))
binary.Write(&buffer, binary.BigEndian, uint32(len(site)))
buffer.WriteString(site)
binary.Write(&buffer, binary.BigEndian, counter)
var hmacv = hmac.New(sha256.New, key)
hmacv.Write(buffer.Bytes())
var seed = hmacv.Sum(nil)
var temp = templates[int(seed[0])%len(templates)]
buffer.Truncate(0)
for i, element := range temp {
pass_chars := template_characters[element]
pass_char := pass_chars[int(seed[i+1])%len(pass_chars)]
buffer.WriteByte(pass_char)
}
return buffer.String()
}
func generatePrivateKey(typ string) string {
var priv interface{}
var err error
switch typ {
case "", "rsa":
// good enough for government work
priv, err = rsa.GenerateKey(rand.Reader, 4096)
case "dsa":
key := new(dsa.PrivateKey)
// again, good enough for government work
if err = dsa.GenerateParameters(&key.Parameters, rand.Reader, dsa.L2048N256); err != nil {
return fmt.Sprintf("failed to generate dsa params: %s", err)
}
err = dsa.GenerateKey(key, rand.Reader)
priv = key
case "ecdsa":
// again, good enough for government work
priv, err = ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
default:
return "Unknown type " + typ
}
if err != nil {
return fmt.Sprintf("failed to generate private key: %s", err)
}
return string(pem.EncodeToMemory(pemBlockForKey(priv)))
}
type DSAKeyFormat struct {
Version int
P, Q, G, Y, X *big.Int
}
func pemBlockForKey(priv interface{}) *pem.Block {
switch k := priv.(type) {
case *rsa.PrivateKey:
return &pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(k)}
case *dsa.PrivateKey:
val := DSAKeyFormat{
P: k.P, Q: k.Q, G: k.G,
Y: k.Y, X: k.X,
}
bytes, _ := asn1.Marshal(val)
return &pem.Block{Type: "DSA PRIVATE KEY", Bytes: bytes}
case *ecdsa.PrivateKey:
b, _ := x509.MarshalECPrivateKey(k)
return &pem.Block{Type: "EC PRIVATE KEY", Bytes: b}
default:
return nil
}
}
type certificate struct {
Cert string
Key string
}
func buildCustomCertificate(b64cert string, b64key string) (certificate, error) {
crt := certificate{}
cert, err := base64.StdEncoding.DecodeString(b64cert)
if err != nil {
return crt, errors.New("unable to decode base64 certificate")
}
key, err := base64.StdEncoding.DecodeString(b64key)
if err != nil {
return crt, errors.New("unable to decode base64 private key")
}
decodedCert, _ := pem.Decode(cert)
if decodedCert == nil {
return crt, errors.New("unable to decode certificate")
}
_, err = x509.ParseCertificate(decodedCert.Bytes)
if err != nil {
return crt, fmt.Errorf(
"error parsing certificate: decodedCert.Bytes: %s",
err,
)
}
decodedKey, _ := pem.Decode(key)
if decodedKey == nil {
return crt, errors.New("unable to decode key")
}
_, err = x509.ParsePKCS1PrivateKey(decodedKey.Bytes)
if err != nil {
return crt, fmt.Errorf(
"error parsing prive key: decodedKey.Bytes: %s",
err,
)
}
crt.Cert = string(cert)
crt.Key = string(key)
return crt, nil
}
func generateCertificateAuthority(
cn string,
daysValid int,
) (certificate, error) {
ca := certificate{}
template, err := getBaseCertTemplate(cn, nil, nil, daysValid)
if err != nil {
return ca, err
}
// Override KeyUsage and IsCA
template.KeyUsage = x509.KeyUsageKeyEncipherment |
x509.KeyUsageDigitalSignature |
x509.KeyUsageCertSign
template.IsCA = true
priv, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return ca, fmt.Errorf("error generating rsa key: %s", err)
}
ca.Cert, ca.Key, err = getCertAndKey(template, priv, template, priv)
if err != nil {
return ca, err
}
return ca, nil
}
func generateSelfSignedCertificate(
cn string,
ips []interface{},
alternateDNS []interface{},
daysValid int,
) (certificate, error) {
cert := certificate{}
template, err := getBaseCertTemplate(cn, ips, alternateDNS, daysValid)
if err != nil {
return cert, err
}
priv, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return cert, fmt.Errorf("error generating rsa key: %s", err)
}
cert.Cert, cert.Key, err = getCertAndKey(template, priv, template, priv)
if err != nil {
return cert, err
}
return cert, nil
}
func generateSignedCertificate(
cn string,
ips []interface{},
alternateDNS []interface{},
daysValid int,
ca certificate,
) (certificate, error) {
cert := certificate{}
decodedSignerCert, _ := pem.Decode([]byte(ca.Cert))
if decodedSignerCert == nil {
return cert, errors.New("unable to decode certificate")
}
signerCert, err := x509.ParseCertificate(decodedSignerCert.Bytes)
if err != nil {
return cert, fmt.Errorf(
"error parsing certificate: decodedSignerCert.Bytes: %s",
err,
)
}
decodedSignerKey, _ := pem.Decode([]byte(ca.Key))
if decodedSignerKey == nil {
return cert, errors.New("unable to decode key")
}
signerKey, err := x509.ParsePKCS1PrivateKey(decodedSignerKey.Bytes)
if err != nil {
return cert, fmt.Errorf(
"error parsing prive key: decodedSignerKey.Bytes: %s",
err,
)
}
template, err := getBaseCertTemplate(cn, ips, alternateDNS, daysValid)
if err != nil {
return cert, err
}
priv, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return cert, fmt.Errorf("error generating rsa key: %s", err)
}
cert.Cert, cert.Key, err = getCertAndKey(
template,
priv,
signerCert,
signerKey,
)
if err != nil {
return cert, err
}
return cert, nil
}
func getCertAndKey(
template *x509.Certificate,
signeeKey *rsa.PrivateKey,
parent *x509.Certificate,
signingKey *rsa.PrivateKey,
) (string, string, error) {
derBytes, err := x509.CreateCertificate(
rand.Reader,
template,
parent,
&signeeKey.PublicKey,
signingKey,
)
if err != nil {
return "", "", fmt.Errorf("error creating certificate: %s", err)
}
certBuffer := bytes.Buffer{}
if err := pem.Encode(
&certBuffer,
&pem.Block{Type: "CERTIFICATE", Bytes: derBytes},
); err != nil {
return "", "", fmt.Errorf("error pem-encoding certificate: %s", err)
}
keyBuffer := bytes.Buffer{}
if err := pem.Encode(
&keyBuffer,
&pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(signeeKey),
},
); err != nil {
return "", "", fmt.Errorf("error pem-encoding key: %s", err)
}
return string(certBuffer.Bytes()), string(keyBuffer.Bytes()), nil
}
func getBaseCertTemplate(
cn string,
ips []interface{},
alternateDNS []interface{},
daysValid int,
) (*x509.Certificate, error) {
ipAddresses, err := getNetIPs(ips)
if err != nil {
return nil, err
}
dnsNames, err := getAlternateDNSStrs(alternateDNS)
if err != nil {
return nil, err
}
return &x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{
CommonName: cn,
},
IPAddresses: ipAddresses,
DNSNames: dnsNames,
NotBefore: time.Now(),
NotAfter: time.Now().Add(time.Hour * 24 * time.Duration(daysValid)),
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{
x509.ExtKeyUsageServerAuth,
x509.ExtKeyUsageClientAuth,
},
BasicConstraintsValid: true,
}, nil
}
func getNetIPs(ips []interface{}) ([]net.IP, error) {
if ips == nil {
return []net.IP{}, nil
}
var ipStr string
var ok bool
var netIP net.IP
netIPs := make([]net.IP, len(ips))
for i, ip := range ips {
ipStr, ok = ip.(string)
if !ok {
return nil, fmt.Errorf("error parsing ip: %v is not a string", ip)
}
netIP = net.ParseIP(ipStr)
if netIP == nil {
return nil, fmt.Errorf("error parsing ip: %s", ipStr)
}
netIPs[i] = netIP
}
return netIPs, nil
}
func getAlternateDNSStrs(alternateDNS []interface{}) ([]string, error) {
if alternateDNS == nil {
return []string{}, nil
}
var dnsStr string
var ok bool
alternateDNSStrs := make([]string, len(alternateDNS))
for i, dns := range alternateDNS {
dnsStr, ok = dns.(string)
if !ok {
return nil, fmt.Errorf(
"error processing alternate dns name: %v is not a string",
dns,
)
}
alternateDNSStrs[i] = dnsStr
}
return alternateDNSStrs, nil
}

Some files were not shown because too many files have changed in this diff Show More