Compare commits

...
200 Commits
Author SHA1 Message Date
KUOKA Yusuke 7c793fdb88 feat: helmfiles: <ordered glob patterns of helmfiles> configuration (#266)
Resolves #247
2018-08-31 12:03:18 +09:00
Dan Wendorf b70956b5d8 Bump Helm 2.9.1 -> 2.10.0 (#265) 2018-08-31 10:17:51 +09:00
KUOKA Yusuke 3840605e04 feat: helmfile apply [--auto-approve] (#263)
This command syncs releases only if there is any difference between the desired and the current state. It asks for an confirmation by default. Provide `--auto-approve` flag after the `apply` command to skip it.

Resolves #205
2018-08-31 10:15:02 +09:00
KUOKA Yusuke bb3b44e511 fix: follow up for the relative paths improvement (#262)
Fixes for the bugs that are introduced by #261, that is values.yaml files specified in `values:` have redundant base path in their prefixes, and remaining .dec files after secrets decryption(https://github.com/roboll/helmfile/issues/251#issuecomment-417285854)
2018-08-30 21:59:59 +09:00
KUOKA Yusuke 421299c883 feat: All the paths are relative to helmfile.yaml (#261)
`helmfile lint` works with relative chart reference (#252)
The tempalte function `readFile` accepts the path relative to helmfile.yaml

Resolves #246
Fixes #252
2018-08-30 19:47:52 +09:00
KUOKA Yusuke 79f0e70ce8 fix: avoid "cross-device link" errors while decrypting secrets (#260)
ref https://github.com/roboll/helmfile/issues/251#issuecomment-417166296
2018-08-30 17:20:38 +09:00
KUOKA Yusuke 0c8a89cbaf fix: coordinate secret decryption (#259)
Fixes #258
2018-08-30 17:03:45 +09:00
KUOKA Yusuke b3ebd4cdd0 fix: Keep backward-compatibility broken after introduction of values file template (#257)
Fixes #249
2018-08-30 16:57:37 +09:00
Shane Starcher 9b71c64ef2 fix: use temporary file for secret file to prevent deletion collision (#250)
Fixes #167
2018-08-30 09:28:26 +09:00
KUOKA Yusuke 822cc13e72 feat: Template functions to replace sets (#242)
Resolves #227
2018-08-28 22:00:51 +09:00
KUOKA Yusuke 0ac8401d1e feat: feat: Configurable default values for --verify, --force, --recreate-pods, and --timeout (#241)
Resolves #230
2018-08-28 15:11:05 +09:00
KUOKA Yusuke 815ee1f85b feat: --force, --recreate-pods, --timeout as first-class citizens in helmfile.yaml (#239)
Resolves #229
2018-08-27 23:06:16 +09:00
KUOKA Yusuke 2c36640ad2 To reduce integration test flakiness 2018-08-27 23:04:16 +09:00
KUOKA Yusuke faaf172615 fix: panic on flag parsing error (#238)
This is a follow-up for #235.

We setup the default logger only after the flags are parsed successfully.
That's because we init the logger according to the --log-level flag value.
2018-08-27 22:58:25 +09:00
KUOKA Yusuke fcbd6aff29 fix: Relative path in KUBECONFIG envvar (#237)
Resolves #173
2018-08-27 22:53:17 +09:00
Shane Starcher 543a653864 improve error handling for value rendering (#235)
Fixes #233

Output on values render error:

```
err: failed to render [/Users/sstarcher/xxx/xxx/values.yaml], because of template: stringTemplate:10:18: executing "stringTemplate" at <requiredEnv "HELM_AC...>: error calling requiredEnv: required env var `HELM_ACCOUNT` is not set
```

Also removes panic and sets the output as `apps.Run()` can and will return errors.  Panic makes no sense.

Changelog:

* improve error handling for value rendering

* only output if error exists

* add exit status
2018-08-27 12:53:25 +09:00
rmartinez3 fa69ac2cfa fix: Args being reordered (#224)
Fixes #212
2018-08-27 10:41:00 +09:00
KUOKA Yusuke 2121fe57b2 Merge pull request #232 from mumoshu/set-array
feat: set array
2018-08-24 17:12:17 +09:00
Yusuke KUOKA cfcafa2e46 feat: set array
Fixes #148
2018-08-24 17:01:34 +09:00
KUOKA Yusuke b6d2ee9d43 Merge pull request #231 from mumoshu/partly-allow-duplicate-release-name
fix: helmfile should fail on duplicate release name after filtered by labels
2018-08-24 16:43:38 +09:00
Yusuke KUOKA 35542b5516 fix: helmfile should fail on duplicate release name after filtered by labels
This is a follow-up for #218, fixes the unintentional degradation that broken the use-case described in https://github.com/roboll/helmfile/issues/193#issuecomment-415434408
2018-08-24 16:18:08 +09:00
KUOKA Yusuke 45d0f1c8e4 Merge pull request #228 from mumoshu/set-file
feat: Support for helm's --set-file
2018-08-24 13:44:07 +09:00
KUOKA Yusuke 96d0781b85 Merge pull request #226 from mumoshu/usage-without-internet
doc: Using helmfile without an Internet connection
2018-08-24 13:43:53 +09:00
Yusuke KUOKA 48e65791c0 feat: Support for helm's --set-file
Resolves #207
2018-08-24 13:06:31 +09:00
Yusuke KUOKA 2398ad2f35 doc: Using helmfile without an Internet connection
Resolves #155
2018-08-24 12:10:18 +09:00
KUOKA Yusuke 013276f576 Merge pull request #225 from mumoshu/unmatched-selector-exit-code
feat: exit code 2 on unmatched selectors
2018-08-24 11:47:10 +09:00
Yusuke KUOKA 3a16b7ed91 feat: exit code 2 on unmatched selectors
Along with the fix for the bug introduced via #217

Resolves #219
2018-08-24 11:40:30 +09:00
KUOKA Yusuke a5d42e741b Merge pull request #223 from mumoshu/show-which-release-failed
feat: emit error message containing which release in which helmfile an upgrade failed
2018-08-23 12:00:15 +09:00
Yusuke KUOKA 313b5de378 feat: emit error message containing which release in which helmfile an upgrade failed
Resolves #66
2018-08-23 11:45:43 +09:00
KUOKA Yusuke 0541731350 Merge pull request #222 from mumoshu/wait-for-release
feat: `wait: true` in each release
2018-08-23 10:43:01 +09:00
Yusuke KUOKA b4623f0515 feat: wait: true in each release
To instruct `helm upgrade` to wait for resources via `--wait`

Resolves #65
2018-08-23 10:37:15 +09:00
KUOKA Yusuke ffa40beff9 Merge pull request #221 from mumoshu/env-files
doc: Instruction to use env files
2018-08-23 09:47:01 +09:00
Yusuke KUOKA 4fae044f0e doc: Instruction to use env files
Closes #203
2018-08-23 09:44:42 +09:00
KUOKA Yusuke d8aa9f305b Merge pull request #218 from mumoshu/should-fail-on-duplicate-release-name
fix: helmfile should fail on duplicate release name
2018-08-23 09:32:03 +09:00
KUOKA Yusuke c402759363 Merge pull request #217 from mumoshu/less-verbose-unmatched-selector-logging
fix: Make logging on unmatched selectors less verbose
2018-08-23 09:27:32 +09:00
Yusuke KUOKA e84ff05a63 fix: helmfile should fail on duplicate release name
Fixes #193
2018-08-23 09:26:39 +09:00
Yusuke KUOKA fcb75fa851 fix: Make logging on unmatched selectors less verbose
Fixes #200
2018-08-23 09:23:55 +09:00
KUOKA Yusuke d609dab848 feat: Template values files (#216)
Resolves #97
2018-08-22 18:53:32 +09:00
KUOKA Yusuke 6c89fac8e5 Merge pull request #215 from mumoshu/diff-target-namespace
feat(diff): Pass namespace to helm-diff
2018-08-22 18:51:10 +09:00
KUOKA Yusuke 3550e5f335 Merge pull request #213 from chadasapp/master
Typofix: required_env -> requiredEnv
2018-08-22 15:05:29 +09:00
Yusuke KUOKA 127dd857df feat(diff): Pass namespace to helm-diff
Tested manually by running `helmfile diff` with debug logging:

```yaml
helmDefaults:
  tillerNamespace: foo

charts:
  - name: grafana
    namespace: grafana
    chart: stable/grafana
```

```console
$ ./helmfile --log-level debug diff
Comparing grafana stable/grafana
exec: helm diff upgrade --allow-unreleased grafana stable/grafana --namespace grafana --tiller-namespace=foo
```

Resolves #179
2018-08-22 11:20:55 +09:00
Chad Horohoe 2cf2dbb868 Typofix: required_env -> requiredEnv
The name is `requiredEnv` and I cannot find any references to `required_env` in the repository anywhere.
2018-08-20 11:10:01 -07:00
KUOKA Yusuke 545a4a3efa Merge pull request #208 from cloudposse/add-examples
Add link to more examples
2018-08-20 21:32:26 +09:00
KUOKA Yusuke 0ed1355573 Merge pull request #206 from InTheCloudDan/patch-1
remove outdated warning.
2018-08-07 15:28:05 +09:00
Erik Osterman 9fef3f7d0f Add link to more examples 2018-08-06 23:20:49 -07:00
KUOKA Yusuke 0d457c4df0 Merge pull request #201 from maver1ck/helmfile-111
Add --args to delete
2018-08-03 09:36:48 +09:00
KUOKA Yusuke 6be52bfbbb Merge pull request #199 from mumoshu/optional-detailed-exitcode-for-diff
feat: Optional detailed exitcodes for `helmfile diff`
2018-08-03 09:36:08 +09:00
Dan O'Brien 089c158084 remove outdated warning. 2018-08-02 20:19:29 -04:00
Maciej Bryński d67573299c Add --args to delete 2018-07-31 10:18:31 +02:00
Yusuke KUOKA 3b4ce90a5a feat: Optional detailed exitcodes for helmfile diff
Adds the `--detailed-exitcode` to the `helmfile diff` command to return `1` on failure, and `2` when no error but diff is seen.

This feature requires the latest `helm-diff` containing https://github.com/databus23/helm-diff/pull/78, and `helm` containing https://github.com/helm/helm/pull/4367.

This is verified to work by manually running commands like the followings:

```bash
./helmfile --helm-binary helm211dev -f ./examples/helmfile.d diff --detailed-exitcode; echo $?
./helmfile --helm-binary helm211dev -f ./examples/helmfile.d diff; echo $?
```

Note that, in above example commands, `helm211dev` is a custom `helm` binary that is built from helm's master branch containing [the necessary enhancement to allow propagate non-zero plugin exit code](https://github.com/helm/helm/pull/4367).
2018-07-26 00:16:47 +09:00
KUOKA Yusuke 92d09a3e92 Merge pull request #198 from mumoshu/fix-ci-flakiness
fix(ci): Reduce flakiness
2018-07-25 23:24:44 +09:00
Yusuke KUOKA 1a5558a306 fix(ci): Reduce flakiness
Fix #103
2018-07-25 11:21:01 +09:00
KUOKA Yusuke ee3632cebc Merge pull request #191 from mumoshu/fix-cmd-descrs
Fix command descriptions
2018-07-19 12:02:07 +09:00
KUOKA Yusuke 2297093f84 Merge pull request #190 from mumoshu/fix-helmfile.d-defaulting
fix: `helmfile sync` doesn't recognize the default `helmfile.d/` directory
2018-07-19 12:01:49 +09:00
KUOKA Yusuke 506de12fcd fix: helmfile -f dir/ -l k=v sync fails on first helmfile w/ no match (#188)
Fixes #187

Changelog:

* fix: `helmfile -f dir/ -l k=v sync` fails on first helmfile w/ no match

* fix(ux): present the filename if the selector did not match any releases

```console
$ helmfile -f helmfile.d/ -l foo=bar sync
2018/07/12 23:37:29 specified selector did not match any releases in helmfile.d/00-backend.yaml
2018/07/12 23:37:29 specified selector did not match any releases in helmfile.d/01-frontend.yaml
2018/07/12 23:37:29 err: specified selector did not match any releases in any helmfile
```
2018-07-19 12:01:14 +09:00
rmartinez3 01fe5b4213 Adding merging for helmDefaults args and --args, adding tillerNamspace and kubeContext to helmDefaults (#186)
Add tillerNamespace and kubeContext to helmDefaults. Also add merging of args capability. 

People can start adding additional fields under helmDefaults on top of this. The additional fields are merged to args as long as they are implemented using the framework introduced by this commit.

Changelog:

* adding merging for helm defaults args and --args, adding tillernamspace, kubecontext to helmdefaults, adding getargs to lint command

* adding docs for helmdefaults

* checking if empty default values
2018-07-19 11:53:09 +09:00
Simon Li 06a1d245ae Multi-level logging (#185)
Closes #93

Summary of changes:

- Adds the [uber-go-/zap](https://github.com/uber-go/zap) library for leveled logging (69040996e0bcbd2f211d0d8d576147027e928e3b)
- A new CLI arg `--log-level` which can take the values `debug` `info` `warn` `error` `fatal` `panic`. The default is `info`, and the existing `--quiet` flag is treated as `--log-level warn`
- All `helm.exec` calls are preceded by a one-line summary. The current information on the full helm command line arguments `exec: helm exec ... ` is only output at `debug` level. This means sensitive command line arguments which may include passwords and `--set` arguments should be hidden by default.
2018-07-19 11:33:57 +09:00
Yusuke KUOKA 438d19852a Fix command descriptions
Resolves #170
2018-07-12 16:09:46 +09:00
Yusuke KUOKA c74db5d281 fix: helmfile sync doesn't recognize the default helmfile.d/ directory
Fixes #189
2018-07-12 14:13:56 +09:00
KUOKA Yusuke 35732f3a93 Bugfix for helm-binary flag. to global (#184)
Two follow-up commits for #181 

* Make the --helm-binary flag global
* Add helm-binary support to the delete command
2018-07-05 11:56:09 +09:00
hori-ryota 7f37c6b39f Add helm-binary opt to delete command 2018-07-04 22:10:17 +09:00
hori-ryota fa0ff32b00 Bugfix for helm-binary flag. to global 2018-07-04 19:01:46 +09:00
Hori Ryota 9f57f8be7d Add the option to specify an alternative helm binary (#181)
`helm` may not be version compatible. Add an option to specify an alternative `helm` command on runtime, possibly according to the version of tiller running on your cluster.
2018-07-03 22:19:02 +09:00
KUOKA Yusuke 88e84e5ae8 Merge pull request #177 from amnk/amnk/issue-176
Add helm-secrets plugin to Dockerfile
2018-06-26 22:37:44 +09:00
KUOKA Yusuke c52504cf8e Merge pull request #178 from Scukerman/master
Fix typo in README.md
2018-06-26 22:33:37 +09:00
Scukerman a8212587b8 Fix typo in README.md 2018-06-26 16:25:42 +03:00
amnk e1040c2aa1 Add helm-secrets plugin to Dockerfile
Closes: #176
2018-06-26 10:42:47 +02:00
rmartinez3 58bc2a2997 adding args to be added in helmfile along with context to be set in helmspec (#171)
Ref #114 

Changelog:

* adding args to be added in helmfile along with context to be set in helmspec

* making command args take precedence
2018-06-26 10:12:38 +09:00
KUOKA Yusuke af1914c575 Merge pull request #172 from mumoshu/diff-against-new-release
feat: `helmfile diff` also works against non-existent releases
2018-06-25 16:15:09 +09:00
Yusuke KUOKA da337e8724 feat: helmfile diff also works against non-existent releases
Resolves #127
2018-06-14 22:45:26 +09:00
Johan Lyheden 6856c6e979 Add helmfile lint support (#162)
The use case is to have a list of helmfile releases version controlled together with all settings and have a CI pipeline that will lint all releases with settings before running sync. The new functionality was mostly copy pasted from the Diff implementation with some extra handling for fetching remote charts.

Notes:

* Added release name to chart path to avoid potential race condition when fetching the chart
2018-06-14 22:35:09 +09:00
KUOKA Yusuke 2fba241122 Ordered helmfile(s) from a directory (#160)
Resolves #151 #137

Notes:

- You can have either helmfile.d or helmfile.yaml/charts.yaml, not both
2018-06-14 22:20:15 +09:00
KUOKA Yusuke df264e2736 Merge pull request #165 from cmeury/no-match-error
fail when no releases match selector
2018-06-12 23:58:33 +09:00
Cedric Meury 575198424a fail when no releases match selector 2018-06-12 14:19:06 +02:00
KUOKA Yusuke afa45930c1 Merge pull request #159 from wendorf/dockerfile-helm-diff
Dockerfile installs helm-diff
2018-06-01 12:06:06 +09:00
KUOKA Yusuke 3044255f7a Merge pull request #156 from wendorf/dockerfile-helm-2.9.1
Docker image uses Helm v2.9.1 (from v2.8.2)
2018-06-01 12:04:35 +09:00
Dan Wendorf 98d3897262 Dockerfile installs helm-diff
Enables the `helmfile diff` command

- git and bash are required to install helm-diff
2018-05-31 15:52:33 -07:00
Dan Wendorf a9a8fb8388 Docker image uses Helm v2.9.1 (from v2.8.2) 2018-05-31 13:43:50 -07:00
Dan Wendorf fb6ac7bba2 Dockerfile uses ARG instead of ENV
Allows customization at image build time
2018-05-31 13:41:36 -07:00
Dan O'Brien 1a4f342f25 Add basic repository authentication (#154)
* basic repository authentication via new `username` and `password` keys

* add warning to readme
2018-05-31 10:42:38 +09:00
KUOKA Yusuke 1768e5dea7 Add tests for helmfile test (#152)
Follow up for #150
2018-05-22 17:19:10 +09:00
Mike Eves 37f6ae8557 Add helmfile test sub-command (#150)
**Feature**
An additional sub-command to the helmfile binary; helmfile test

**Why**
Helm provides helm test (https://github.com/kubernetes/helm/blob/master/docs/chart_tests.md) as a method to run automated tests post chart install to ensure things are working as they should be.

It would be nice to be able to run something like helmfile test against a particular helmfile in order to run helm tests against all charts/releases defined in the file. Either as part of the sync (i.e. helmfile sync --test) to be ran immediately after the corresponding chart is installed, or as a separate command ran after a sync (i.e. helmfile test).

A chart without tests will exit with a 0 status, so it can be safely ran against any charts.

**Notes**

`--cleanup` (bool) & `--timeout` (default: 300) are available as first class arguments. Additional arguments can be passed to the helm binary as with other sub commands using `--args=`

Resolves #144
2018-05-22 17:12:48 +09:00
KUOKA Yusuke 5735efa8c0 Merge pull request #147 from mumoshu/fix-static-linking-for-linux-binary-on-circle
Fix static linking for linux binaries built on CircleCI
2018-05-17 16:22:49 +09:00
Yusuke KUOKA f08b981814 Fix static linking for linux binaries built on CircleCI
Fixes #146
2018-05-16 22:46:35 +09:00
KUOKA Yusuke a7e35b7316 Merge pull request #142 from mumoshu/make-purge-optional
Make purge optional when running "helmfile delete"
2018-05-16 08:59:35 +09:00
KUOKA Yusuke f288032d60 Merge pull request #145 from ConnectedHomes/absolute-value-path
Don't prefix base path to absolute path
2018-05-15 22:36:40 +09:00
David Bliss 4fe8d988d5 Don't prefix base path to absolute path
If an entry in the `values` section of a helmfile is an absolute
path, then it would be erroneously prefixed with the base path.

This would cause `helmfile sync` to fail because the value.yaml file
would not exist at the incorrectly prefixed path.
2018-05-15 11:22:38 +01:00
KUOKA Yusuke 09f19047bd Merge pull request #143 from mumoshu/fix-helm-diff-deprecation-warning
Fix the helm-diff deprecation warning
2018-05-11 21:56:15 +09:00
Yusuke KUOKA 52507de39b Mention about the minimum ver. of helm-diff 2018-05-11 21:52:08 +09:00
Yusuke KUOKA ac70e45bff Fix the helm-diff deprecation warning
Resolves #130
2018-05-11 16:51:44 +09:00
Yusuke KUOKA 3979c18ad3 Make purge optional when running "helmfile delete"
`helmfile delete` has been implying `--purge` but it is not the case since this change.

The new behavior is `helmfile delete --purge` to actually purge releases.
Run just `helmfile delete` to delete releases but not purge them.

Resolves #71
2018-05-11 16:41:02 +09:00
KUOKA Yusuke fb781be560 Merge pull request #139 from int128/inline-values
Inline values support
2018-05-11 14:34:25 +09:00
Hidetake Iwata 5db3ba2112 Add inline values support
This allows `values` accepts the following types:
- string type for values file path
- map type for inline values
2018-05-11 11:55:01 +09:00
KUOKA Yusuke 568d750f75 Merge pull request #141 from mumoshu/always-reset-values
Force `--reset-values` on sync
2018-05-11 03:05:45 +09:00
Yusuke KUOKA c41dbc4c9f Force --reset-values on sync
Resolves #63
2018-05-11 03:04:36 +09:00
KUOKA Yusuke 26ffdc2f24 Merge pull request #136 from AXOOM/feat/arg_escaping
Escape names and values before passing to Helm
2018-05-10 09:27:45 +09:00
Bastian Eicher eea4746ae2 Escape names and values before passing to Helm
This is required to handle values with commas and/or curly braces.
2018-05-08 14:43:58 +02:00
KUOKA Yusuke c5d530632b Merge pull request #135 from AXOOM/feat/required_envs
Introduce template function `required_env`
2018-05-06 19:35:04 +09:00
Andreas Bieber 77204d9a8c Introduce template function requiredEnv
The new template function `requiredEnv` ensures that the given env var is set and not empty.
2018-05-05 21:36:59 +02:00
KUOKA Yusuke 67ddd7f1ae Merge pull request #128 from cmeury/retry-curl
retry curl to reduce risk of race condition
2018-05-01 12:08:42 +09:00
Cedric Meury 10255aecc7 retry curl to reduce risk of race condition 2018-04-29 22:13:16 +02:00
KUOKA Yusuke 89ca84fb21 Merge pull request #129 from cmeury/install-ghr
install ghr for releasing
2018-04-28 23:33:15 +09:00
KUOKA Yusuke bdf7209434 Merge pull request #126 from cmeury/chart-missing-error
fail fast on syntax error by strictly unmarshalling
2018-04-28 23:31:34 +09:00
Cedric Meury 763c5424b6 install ghr for releasing 2018-04-28 11:40:23 +02:00
Cedric Meury e4c016142d add unit tests and upgrade minikube, helm and k8s; set automountServiceAccountToken for 2.9.0 compatibility 2018-04-28 00:19:01 +02:00
Cedric Meury 2a1cdb0253 upgrade gopkg.in/yaml from v1 to v2 to strictly unmarshal 2018-04-28 00:17:56 +02:00
Cedric Meury 8c6137ee62 adding simple unit test for sync releases 2018-04-28 00:17:56 +02:00
KUOKA Yusuke e4b3ba444c Merge pull request #125 from cmeury/users
add users file to track production use
2018-04-27 22:16:41 +09:00
KUOKA Yusuke ad34be579d Merge pull request #124 from cmeury/build-container
build consistent statically linked binary for docker image
2018-04-27 22:14:45 +09:00
KUOKA Yusuke 44241f675f Merge pull request #70 from cmeury/status-command
status command retrieves release status
2018-04-27 22:09:12 +09:00
Cedric Meury 10cdae5dab table format 2018-04-27 13:39:33 +02:00
Cedric Meury 930ea98245 add users file to track production use 2018-04-27 13:38:17 +02:00
Cedric Meury 151fe0716e keep it simple 2018-04-27 11:21:43 +02:00
Cedric Meury 2af039dcfc don't use gox for single build 2018-04-27 11:20:18 +02:00
Cedric Meury 5f9d3f818c build consistent statically linked binary for docker image
* use golang 1.10.1 images everywhere
* introduce a `static-linux` target to build a statically linked 64-bit binary only
* use that binary for the official image and the circle build and tests
* use a multi-stage dockerfile for a consistent build environment
2018-04-27 11:11:35 +02:00
Cedric Meury 579fa4c765 status command retrieves release status 2018-04-27 10:31:29 +02:00
KUOKA Yusuke efe5a2e56b Merge pull request #121 from roboll/up-to-date-release-note
Ensure the release note to be up-to-date
2018-04-26 20:01:10 +09:00
KUOKA Yusuke b564c4ad4c Ensure the release note to be up-to-date 2018-04-26 20:00:59 +09:00
KUOKA Yusuke ae2777f6ad Merge pull request #120 from roboll/fix-release
Fix failing automated release
2018-04-26 19:51:14 +09:00
KUOKA Yusuke 652e001d45 Fix failing automated release
Follow up for 71fa693 and #98
2018-04-26 19:51:01 +09:00
KUOKA Yusuke 9b0d0d4cba Merge pull request #115 from sstarcher/chart_detection
support relative path for is local check
2018-04-26 06:13:22 +09:00
KUOKA Yusuke d8465c4ca5 Merge pull request #116 from AXOOM/master
Fix resolving relative paths on windows
2018-04-25 22:31:57 +09:00
Andreas Bieber 46f119b224 Fix resolving relative paths on windows
path.Dir() does not support Windows-style directory separators (backslashes). This caused invocations such as "helmfile -f subdir\helmfile.yml" to fail. Fixed by using filepath.Dir() instead.
2018-04-25 14:40:45 +02:00
Shane Starcher df5d2e01d5 updating tests to use local files 2018-04-24 14:10:46 -04:00
Shane Starcher cf01221f3a support relative path for is local check 2018-04-24 09:01:42 -04:00
KUOKA Yusuke ca26d757f7 Merge pull request #113 from gavvvr/patch-1
Add root CAs to docker image
2018-04-23 10:46:14 +09:00
Kirill Gavrilov a28279233e Add root CAs to docker image 2018-04-22 01:09:54 +03:00
KUOKA Yusuke e85b87828a Merge pull request #110 from sstarcher/empty_filter
support selector that is empty
2018-04-18 15:49:25 +09:00
Shane Starcher 789362dfd4 support the same selector repeated 2018-04-17 11:10:46 -04:00
KUOKA Yusuke 79c7d1501a Merge pull request #106 from roboll/update-readme-for-breaking-change
Update README not to confuse new-comers due to breaking change in master
2018-04-12 10:46:03 +09:00
KUOKA Yusuke 883cd1e3f3 Update README not to confuse new-comers due to breaking change in master 2018-04-11 21:15:49 +09:00
Shane Starcher 71fa693997 templating the entire helmfile and adding some documentation (#98)
* templating the entire helmfile and adding some documentation

* updating vendor and using dep

* update to golang 1.10

* updating docs
2018-04-10 20:19:43 +09:00
KUOKA Yusuke a5706e9951 Introducing semver (#102)
Resolves #101
2018-04-10 14:20:05 +09:00
KUOKA Yusuke eaee6ef6a9 Merge pull request #95 from danielcb/fix-race-condition
Fix race condition. Add concurrency option similiar to DiffReleases
2018-04-09 23:50:49 +09:00
KUOKA Yusuke 69c3e5513a Merge pull request #100 from mumoshu/redo-release-automation
Redo release automation via CircleCI
2018-04-09 21:45:36 +09:00
Yusuke KUOKA 8c4ada4457 Allow overriding docker repository ORG name 2018-04-09 20:28:11 +09:00
Yusuke KUOKA 35cb4d4032 Redo release automation via CircleCI
Resolves #91
2018-04-09 20:27:50 +09:00
Daniel 2b07b9715e Fix race condition. Add concurrency option similiar to DiffReleases 2018-04-06 16:05:32 +02:00
KUOKA Yusuke d380b8d550 Merge pull request #87 from danielcb/feature-envvar-support-for-names
Add envvar interpolation for names and namespaces
2018-04-06 19:03:08 +09:00
KUOKA Yusuke 24dfc477ca Merge pull request #89 from danielcb/fix-readme
Add quotes to envvar example in readme
2018-04-06 18:54:42 +09:00
KUOKA Yusuke 2caf1a15ad Merge pull request #90 from danielcb/add-envvar-examples
Add explanation for envvar interpolation and examples to readme
2018-04-06 18:54:02 +09:00
KUOKA Yusuke 635046fa23 Merge pull request #81 from cmeury/integration-test
Run simple integration test against minikube (local & CI)
2018-04-06 18:43:00 +09:00
Daniel 3dd49cac66 Add explanation for envvar interpolation and examples to readme 2018-04-06 09:06:14 +02:00
Daniel a3d6cce636 Add quotes to envvar example in readme 2018-04-06 08:50:09 +02:00
Daniel 191d9ef1a2 Add envvar interpolation for names and namespaces 2018-04-05 19:05:37 +02:00
Cedric Meury 627a09d272 clean out test dependency steps 2018-04-05 04:55:29 +02:00
Cedric Meury b485abf818 keep one single workflow 2018-04-05 04:50:10 +02:00
Cedric Meury 8912eef468 run integration tests against minikube on circleci 2018-04-05 04:44:35 +02:00
Cedric Meury 3545c12ee4 migrate to circle config v2 2018-04-05 04:44:35 +02:00
Cedric Meury 8bcdeed432 run an integration test script against minikube 2018-04-05 04:44:21 +02:00
KUOKA Yusuke 283848c594 Merge pull request #83 from sstarcher/file_exists
checks for file existence
2018-04-04 16:48:29 +09:00
KUOKA Yusuke b75c1d8436 Merge pull request #85 from fsilberstein/sync-args-without-update
Sync command: change the way we apply args
2018-04-04 16:41:35 +09:00
Shane Starcher b4f9c1ce79 Do delete decrypted secret on sync command (#82)
Due to not passing around the pointer into channels and through the functions, we were not getting back the generatedValues. This fixes that.
2018-04-04 10:22:17 +09:00
Fabien Silberstein caf784699d Apply args flag only to sync command and not updatedeps 2018-04-03 17:46:57 +02:00
Shane Starcher 9ee0f967c8 checks for file existance 2018-04-03 10:03:42 -04:00
Cedric Meury 813fd4bfc7 render environment variables in repository URLs (#78)
* unit test for template rendering

* render env vars in repository urls
2018-04-02 19:16:34 +09:00
KUOKA Yusuke 1af8743bb6 Merge pull request #77 from cmeury/sync-update-deps
Update dependencies of local charts when running 'sync'
2018-04-02 18:29:11 +09:00
KUOKA Yusuke e6216564b2 Merge pull request #76 from cmeury/update-dockerfile
Upgrade container to alpine 3.7 and Helm 2.8.2 & check sha256
2018-04-02 18:22:31 +09:00
KUOKA Yusuke 70b1e8ae1f Merge pull request #74 from mumoshu/oneshot-job-example
doc: Add a basic example for managing oneshot jobs
2018-04-02 18:21:47 +09:00
Cedric Meury 96ae2a9827 remove unnecessary print in test 2018-04-02 00:02:45 +02:00
Cedric Meury 1b302db7f8 running update dependencies for local charts 2018-04-01 21:23:31 +02:00
Cedric Meury 963c84e07c use wget to download file 2018-03-30 11:41:47 +02:00
Cedric Meury cef2d26c06 extract method for duplicated code 2018-03-28 01:56:43 +02:00
Cedric Meury 174e5557c8 alpine 3.7; helm 2.8.2; check sha256 sum; add run target 2018-03-28 01:15:19 +02:00
Yusuke KUOKA 9d7c036753 doc: Add a basic example for managfing oneshot jobs
Resolves #49
2018-03-27 17:21:01 +09:00
KUOKA Yusuke a33d111ecb Merge pull request #72 from cmeury/add-tests
describing current functionality of helmexec as tests
2018-03-27 16:34:43 +09:00
KUOKA Yusuke da2686eaad Merge pull request #75 from mumoshu/note-version-range
doc: Note about specifying chart semver
2018-03-27 16:19:36 +09:00
Cedric Meury baba83b124 renamed New() returns execer instead of interface 2018-03-27 00:03:04 +02:00
Yusuke KUOKA 0fc53a43cf doc: Note about specifying chart semver
Ref #61
2018-03-26 13:29:53 +09:00
KUOKA Yusuke 49223eb968 Merge pull request #69 from cmeury/charts-help-text
correct help text for sync, fixes #62
2018-03-26 11:45:14 +09:00
Cedric Meury 7f90415689 code style 2018-03-26 00:17:12 +02:00
Cedric Meury 915c190901 a tab snuck in 2018-03-26 00:11:10 +02:00
Cedric Meury 6c6f300423 describing current functionality as tests 2018-03-26 00:06:05 +02:00
Cedric Meury 1a0d1192ce correct help text for sync, fixes #62 2018-03-25 00:13:43 +01:00
KUOKA Yusuke 5aa7da98af Merge pull request #60 from mxey/sync-args
Add --args option to sync command
2018-03-24 22:57:26 +09:00
Alex Withrow 4b08ea9292 Allow running helmfile against a subset of releases (#30)
This adds `releases[].labels` in which you can set arbitrary number of key-value pairs, so that commands like `helmfile sync --selector key=value` can be used to run the helmfile subcommand against a subnet of declared releases.

`labels` and `selector` are named as such on purpose of being consistent with terminology of Kubernetes and other tools in the K8S ecosystem, including kubectl, stern, helm, and so on.

Resolves #8
2018-03-24 01:05:19 +09:00
Maximilian Gaß 03d1671527 Add --args option to sync command 2018-03-22 13:51:06 +01:00
KUOKA Yusuke e81541d65c Merge pull request #57 from mumoshu/improve-readme
Improve readme for first-time users
2018-03-21 23:22:25 +09:00
Yusuke KUOKA 1a197e214a Improve readme for first-time users
So that our user can start leveraging helmfile without too much confusion.

Includes two things:

- A "getting started" guide
- A basic description of the `sync` subcommand
2018-03-18 10:26:31 +09:00
Shane Starcher ffa39af554 Add secrets support via helm-secrets plugin (#53)
Resolves #41
2018-03-16 22:43:02 +09:00
rob boll c88d6c431c circle 2018-03-16 09:24:42 -04:00
rob boll f4560f9d18 update ghr 2018-03-16 09:13:01 -04:00
rob boll 29a559b256 add docker to circleci 2018-03-16 09:04:44 -04:00
Greg Burton 1ad426b338 Support client certs when accessing remote repos (#47)
This is important for deployments where the chart repo is protected by client cert validation.

helmfile.yaml is extended to support `certFile` and `keyFile`:

```
 repositories:
   - name: roboll
     url: http://roboll.io/charts
    certFile: optional_client_cert
    keyFile: optional_client_key
```

Everything works the same if you don't provide values for them.

Regarding the implementation, the "helm repo add" command already supports the cert-file and key-file values, so we just have to pass them through.
2018-03-10 22:16:31 +09:00
KUOKA Yusuke e4961a8c34 Merge pull request #46 from mseashor/master
Synchronous add repository calls avoid helm add repo race conditions
2018-03-06 11:23:35 +09:00
mseashor 412d9244c8 Remove inadvertantly added extra line 2018-03-05 16:15:59 -08:00
mseashor 72eaa2fb62 Add helm add repo synchonously to avoid race conditions with multiple repositories 2018-03-05 16:06:49 -08:00
KUOKA Yusuke 8f607b0da7 Merge pull request #44 from nyodas/fix_issue_with_namespaces_from_spec
[state] - Fix issue where spec[].Namespace were ignored
2018-03-05 21:35:02 +09:00
bob 76dcc6b182 [state] - Fix issue where spec[].Namespace were ignored 2018-03-05 11:05:58 +01:00
KUOKA Yusuke c990d19cb0 Merge pull request #43 from roboll/fix-err-on-charts-yaml-fallback
Fix error on charts.yml fallback
2018-03-04 22:15:37 +09:00
KUOKA Yusuke e128f149bd Fix error on charts.yml fallback
Fixes #42
2018-03-03 13:32:16 +09:00
KUOKA Yusuke 0fc74ea771 Fix terminology (#35)
In a backward compatible manner, so that (I believe) we could move the discussion made in #25 forward.

Fixes #25
2018-03-03 00:14:43 +09:00
KUOKA Yusuke efdede1658 Allow specifying a default namespace used across all the releases (#39)
* Allow specifying a default namespace used across all the releases

Omit `namespace` in `charts[].namespace` of charts.yaml and then run `helmfile sync -n yourns` to install your charts into `yourns`.

Resolves #31
2018-03-01 23:09:02 +09:00
KUOKA Yusuke f02790d566 Fix possible race while templating (#38)
Fixes #32
Fixes #37
2018-03-01 22:54:31 +09:00
KUOKA Yusuke bb67abffb4 Merge pull request #34 from mumoshu/purge-on-delete
Purge releases on deletion
2018-03-01 22:49:17 +09:00
Thomas O'Donnell effc747081 Add option to limit concurrent helm calls (#24)
* Add option to throttle concurrent `helm repo update` calls

Have added a new flag `--concurrency N` to `helmfile sync charts` that can be used to set a
limit on the number of concurrent calls to helm. 

Implementation details:

Switched `SyncCharts` from using a WaitGroup to using a pool
of workers and a queue of jobs. To ensure that this is thread safe and
an attempt is made to sync each chart at the end.

Fixes #23

* Fix formatting and update CI to catch these errors

Have fixed the formatting so that `make pristine` now passes. Have also
added this to the Circle CI config to catch these errors in the future.
2018-03-01 22:39:23 +09:00
Yusuke KUOKA a31757ae07 Purge releases on deletion
So that we can just reuse release names after `helmfile delete`.

This is verified to work by confirming that `helmfile delete` does add `--purge` flag to `helm delete`:

```
$ helmfile delete
exec: helm delete --purge brigade-project
release "brigade-project" deleted
```

And by confirming that I can now re-install releases from the same charts.yaml without any modification of release names:

```
PROJECT=deis/empty-testbed ENV=staging IMAGE=mumoshu/golang-k8s-aws:1.9.1 COMMAND='go' SERVICE_ACCOUNT=default helmfile sync
Release "brigade-project" does not exist. Installing it now.
NAME:   brigade-project
LAST DEPLOYED: Tue Feb 27 15:59:38 2018
NAMESPACE: kube-system
STATUS: DEPLOYED

RESOURCES:
*snip*
```

Fixes #33
2018-02-27 16:01:06 +09:00
jayste c00b869045 Added ability to set context in charts.yaml (#21)
* Added ability to set context in charts.yaml

* Added context to the documentation
Added error if —kube-context and context are both used.
2018-01-02 16:49:02 -05:00
Karl Fischer 5da5cd1c39 Allow env var interpolation in set and values (#20)
* Allow env var interpolation in `set` and `values`

* Use go templating instead of regex

* Re-add 'env' section for backwards compatibility

* Fix typo
2018-01-02 16:48:45 -05:00
rob boll 12b8b7237b Create LICENSE 2017-11-10 16:07:38 -05:00
223 changed files with 22737 additions and 1663 deletions
+105
View File
@@ -0,0 +1,105 @@
version: 2
jobs:
build:
docker:
- image: circleci/golang:1.10.1
working_directory: /go/src/github.com/roboll/helmfile
steps:
- checkout
- run: make build
- persist_to_workspace:
root: /go/src/github.com/roboll/helmfile
paths:
- .
test:
docker:
- image: circleci/golang:1.10.1
working_directory: /go/src/github.com/roboll/helmfile
steps:
- checkout
- run: make check
- run: make pristine
- run: make test
# thanks to https://raw.githubusercontent.com/weaveworks/launcher/master/.circleci/config.yml
integration_tests:
machine: true
environment:
CHANGE_MINIKUBE_NONE_USER: true
steps:
- checkout
- run: mkdir ~/build
- attach_workspace:
at: ~/build
- run: cp ~/build/helmfile ~/project/helmfile
- run:
name: Install helm
environment:
HELM_VERSION: v2.9.0
command: |
HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
curl -Lo ${HELM_FILENAME} "https://kubernetes-helm.storage.googleapis.com/${HELM_FILENAME}"
tar zxf ${HELM_FILENAME} linux-amd64/helm
chmod +x linux-amd64/helm
sudo mv linux-amd64/helm /usr/local/bin/
- run:
name: Deploy minikube
environment:
CHANGE_MINIKUBE_NONE_USER: true
K8S_VERSION: v1.9.0
MINIKUBE_VERSION: v0.25.2
command: |
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/linux/amd64/kubectl
chmod +x kubectl && sudo mv kubectl /usr/local/bin/
curl -Lo minikube https://storage.googleapis.com/minikube/releases/${MINIKUBE_VERSION}/minikube-linux-amd64
chmod +x minikube && sudo mv minikube /usr/local/bin/
sudo minikube config set WantReportErrorPrompt false
sudo -E minikube start --vm-driver=none --kubernetes-version=${K8S_VERSION}
sudo -E minikube update-context
- run:
name: Wait for nodes to become ready
command: JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.type}={@.status};{end}{end}'; until kubectl get nodes -o jsonpath="$JSONPATH" 2>&1 | grep -q "Ready=True"; do sleep 1; done
- run:
name: Execute integration tests
command: |
export TERM=xterm
make integration
# GITHUB_TOKEN env var must be setup in circleci console
release:
docker:
- image: circleci/golang:1.10.1
working_directory: /go/src/github.com/roboll/helmfile
steps:
- checkout
- setup_remote_docker
- run:
command: |
docker login -u="$DOCKER_USER" -p="$DOCKER_PASS" quay.io
make tools
BUILD_URL="$CIRCLE_BUILD_URL" make push release
workflows:
version: 2
build_and_test:
jobs:
- build
- test
- integration_tests:
requires:
- build
filters:
branches:
only:
- master
- /pull.*/
- release:
filters:
branches:
ignore: /.*/
tags:
only: /v.*/
+1
View File
@@ -1,2 +1,3 @@
dist/
.idea/
helmfile
+25 -5
View File
@@ -1,9 +1,29 @@
FROM alpine:3.4
FROM golang:1.10.1-alpine3.7 as builder
RUN apk add --update curl && \
curl -L https://kubernetes-helm.storage.googleapis.com/helm-v2.0.0-linux-amd64.tar.gz | \
tar zxf - && mv /linux-amd64/helm /usr/local/bin/ && rm -r /linux-amd64 && apk del curl
RUN apk add --no-cache make git
WORKDIR /go/src/github.com/roboll/helmfile/
COPY . /go/src/github.com/roboll/helmfile/
RUN make static-linux
COPY dist/helmfile_linux_amd64 /usr/local/bin/helmfile
# -----------------------------------------------------------------------------
FROM alpine:3.7
RUN apk add --no-cache ca-certificates git bash curl
ARG HELM_VERSION=v2.10.0
ARG HELM_LOCATION="https://kubernetes-helm.storage.googleapis.com"
ARG HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
ARG HELM_SHA256="0fa2ed4983b1e4a3f90f776d08b88b0c73fd83f305b5b634175cb15e61342ffe"
RUN wget ${HELM_LOCATION}/${HELM_FILENAME} && \
sha256sum ${HELM_FILENAME} | grep -q "${HELM_SHA256}" && \
tar zxf ${HELM_FILENAME} && mv /linux-amd64/helm /usr/local/bin/ && \
rm ${HELM_FILENAME} && rm -r /linux-amd64
RUN mkdir -p "$(helm home)/plugins"
RUN helm plugin install https://github.com/databus23/helm-diff && \
helm plugin install https://github.com/futuresimple/helm-secrets
COPY --from=builder /go/src/github.com/roboll/helmfile/dist/helmfile_linux_amd64 /usr/local/bin/helmfile
CMD ["/usr/local/bin/helmfile"]
Generated
+90
View File
@@ -0,0 +1,90 @@
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
name = "github.com/Masterminds/semver"
packages = ["."]
revision = "8d82589cda2d7b5b9167396841975c4535c9bec6"
version = "v1.4.1"
[[projects]]
name = "github.com/Masterminds/sprig"
packages = ["."]
revision = "6b2a58267f6a8b1dc8e2eb5519b984008fa85e8c"
version = "v2.15.0"
[[projects]]
name = "github.com/aokoli/goutils"
packages = ["."]
revision = "3391d3790d23d03408670993e957e8f408993c34"
version = "v1.0.1"
[[projects]]
name = "github.com/google/uuid"
packages = ["."]
revision = "064e2069ce9c359c118179501254f67d7d37ba24"
version = "0.2"
[[projects]]
name = "github.com/huandu/xstrings"
packages = ["."]
revision = "2bf18b218c51864a87384c06996e40ff9dcff8e1"
version = "v1.0.0"
[[projects]]
name = "github.com/imdario/mergo"
packages = ["."]
revision = "9d5f1277e9a8ed20c3684bda8fde67c05628518c"
version = "v0.3.4"
[[projects]]
name = "github.com/urfave/cli"
packages = ["."]
revision = "6011f165dc288c72abd8acd7722f837c5c64198d"
[[projects]]
name = "go.uber.org/atomic"
packages = ["."]
revision = "1ea20fb1cbb1cc08cbd0d913a96dead89aa18289"
version = "v1.3.2"
[[projects]]
name = "go.uber.org/multierr"
packages = ["."]
revision = "3c4937480c32f4c13a875a1829af76c98ca3d40a"
version = "v1.1.0"
[[projects]]
name = "go.uber.org/zap"
packages = [
".",
"buffer",
"internal/bufferpool",
"internal/color",
"internal/exit",
"zapcore"
]
revision = "eeedf312bc6c57391d84767a4cd413f02a917974"
version = "v1.8.0"
[[projects]]
branch = "master"
name = "golang.org/x/crypto"
packages = [
"pbkdf2",
"scrypt"
]
revision = "b2aa35443fbc700ab74c586ae79b81c171851023"
[[projects]]
name = "gopkg.in/yaml.v2"
packages = ["."]
revision = "5420a8b6744d3b0345ab293f6fcba19c978f1183"
version = "v2.2.1"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "57e868f6ae57c81a07ee682742f3b71bf5c7956311a3bb8ea76459677fc104c7"
solver-name = "gps-cdcl"
solver-version = 1
+11
View File
@@ -0,0 +1,11 @@
[[constraint]]
name = "github.com/Masterminds/sprig"
version = "2.15.0"
[[constraint]]
branch = "v2"
name = "gopkg.in/yaml.v2"
[prune]
go-tests = true
unused-packages = true
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2017 rob boll
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+22 -7
View File
@@ -1,4 +1,4 @@
ORG := $(shell basename $(realpath ..))
ORG ?= $(shell basename $(realpath ..))
PKGS := $(shell go list ./... | grep -v /vendor/)
build:
@@ -21,26 +21,41 @@ test:
go test -v ${PKGS} -cover -race -p=1
.PHONY: test
integration:
bash test/integration/run.sh
.PHONY: integration
cross:
gox -os '!freebsd !netbsd' -arch '!arm' -output "dist/{{.Dir}}_{{.OS}}_{{.Arch}}" -ldflags '-X main.Version=${TAG}' ${TARGETS}
env CGO_ENABLED=0 gox -os '!freebsd !netbsd' -arch '!arm' -output "dist/{{.Dir}}_{{.OS}}_{{.Arch}}" -ldflags '-X main.Version=${TAG}' ${TARGETS}
.PHONY: cross
static-linux:
env CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o "dist/helmfile_linux_amd64" -ldflags '-X main.Version=${TAG}' ${TARGETS}
.PHONY: linux
clean:
rm dist/helmfile_*
.PHONY: clean
pristine: generate fmt
git ls-files --exclude-standard --modified --deleted --others | diff /dev/null -
.PHONY: pristine
release: pristine cross
@ghr -b ${BODY} -t ${GITHUB_TOKEN} -u ${ORG} -replace ${TAG} dist
@ghr -b ${BODY} -t ${GITHUB_TOKEN} -u ${ORG} -recreate ${TAG} dist
.PHONY: release
image: cross
docker build -t quay.io/roboll/helmfile:${TAG} .
image:
docker build -t quay.io/${ORG}/helmfile:${TAG} .
run: image
docker run --rm -it -t quay.io/${ORG}/helmfile:${TAG} sh
push: image
docker push quay.io/roboll/helmfile:${TAG}
docker push quay.io/${ORG}/helmfile:${TAG}
tools:
go get -u github.com/roboll/ghr github.com/mitchellh/gox
go get -u github.com/tcnksm/ghr github.com/mitchellh/gox
.PHONY: tools
TAG = $(shell git describe --tags --abbrev=0 HEAD)
+402 -33
View File
@@ -4,6 +4,14 @@ Deploy Kubernetes Helm Charts
[![Docker Repository on Quay](https://quay.io/repository/roboll/helmfile/status "Docker Repository on Quay")](https://quay.io/repository/roboll/helmfile)
## status
Even though Helmfile is used in production environments [across multiple organizations](USERS.md), it is still in its early stage of development, hence versioned 0.x.
Helmfile complies to Semantic Versioning 2.0.0 in which v0.x means that there could be backward-incompatible changes for every release.
Note that we will try our best to document any backward incompatibility.
## about
Helmfile is a declarative spec for deploying helm charts. It lets you...
@@ -14,81 +22,442 @@ Helmfile is a declarative spec for deploying helm charts. It lets you...
To avoid upgrades for each iteration of `helm`, the `helmfile` executable delegates to `helm` - as a result, `helm` must be installed.
The default helmfile is `charts.yaml`:
## configuration syntax
```
**CAUTION**: This documentation is for the development version of helmfile. If you are looking for the documentation for any of releases, please switch to the corresponding branch like [v0.12.0](https://github.com/roboll/helmfile/tree/v0.12.0).
The default helmfile is `helmfile.yaml`:
```yaml
repositories:
- name: roboll
url: http://roboll.io/charts
certFile: optional_client_cert
keyFile: optional_client_key
username: optional_username
password: optional_password
charts:
context: kube-context # kube-context (--kube-context)
#default values to set for args along with dedicated keys that can be set by contributers, cli args take precedence overe these
helmDefaults:
tillerNamespace: tiller-namespace #dedicated default key for tiller-namespace
kubeContext: kube-context #dedicated default key for kube-context
# additional and global args passed to helm
args:
- "--set k=v"
# defaults for verify, wait, force, timeout and recreatePods under releases[]
verify: true
wait: true
timeout: 600
recreatePods: true
force: true
releases:
# Published chart example
- name: vault # helm deployment name
namespace: vault # target namespace
chart: roboll/vault-secret-manager # chart reference (repository)
values: [ vault.yaml ] # value files (--values)
set: # values (--set)
- name: address
value: https://vault.example.com
env: # values (--set) but value will be pulled from environment variables. Will throw an error if the environment variable is not set.
- name: db.password
value: DB_PASSWORD # $DB_PASSOWRD needs to be set in the calling environment ex: export DB_PASSWORD='password1'
- name: vault # name of this release
namespace: vault # target namespace
labels: # Arbitrary key value pairs for filtering releases
foo: bar
chart: roboll/vault-secret-manager # the chart being installed to create this release, referenced by `repository/chart` syntax
version: ~1.24.1 # the semver of the chart. range constraint is supported
values:
# value files passed via --values
- vault.yaml
# inline values, passed via a temporary values file and --values
- address: https://vault.example.com
db:
username: {{ requiredEnv "DB_USERNAME" }}
# value taken from environment variable. Quotes are necessary. Will throw an error if the environment variable is not set. $DB_PASSWORD needs to be set in the calling environment ex: export DB_PASSWORD='password1'
password: {{ requiredEnv "DB_PASSWORD" }}
proxy:
# Interpolate environment variable with a fixed string
domain: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
scheme: {{ env "SCHEME" | default "https" }}
set:
# single value loaded from a local file, translates to --set-file foo.config=path/to/file
- name: foo.config
file: path/to/file
# set a single array value in an array, translates to --set bar[0]={1,2}
- name: bar[0]
values:
- 1
- 2
# will attempt to decrypt it using helm-secrets plugin
secrets:
- vault_secret.yaml
# wait for k8s resources via --wait. Defaults to `false`
wait: true
# time in seconds to wait for any individual Kubernetes operation (like Jobs for hooks, and waits on pod/pvc/svc/deployment readiness) (default 300)
timeout: 60
# performs pods restart for the resource if applicable
recreatePods: true
# forces resource update through delete/recreate if needed
force: true
# Local chart example
- name: grafana # helm deployment name
namespace: another # target namespace
chart: ../my-charts/grafana # chart reference (relative path to manifest)
- name: grafana # name of this release
namespace: another # target namespace
chart: ../my-charts/grafana # the chart being installed to create this release, referenced by relative path to local chart
values:
- ../../my-values/grafana/values.yaml # Values file (relative path to manifest)
- "../../my-values/grafana/values.yaml" # Values file (relative path to manifest)
- ./values/{{ requiredEnv "PLATFORM_ENV" }}/config.yaml # Values file taken from path with environment variable. $PLATFORM_ENV must be set in the calling environment.
wait: true
```
## install
## Templating
`go get github.com/roboll/helmfile` or [releases](https://github.com/roboll/helmfile/releases) or [container](https://quay.io/roboll/helmfile)
Helmfile uses [Go templates](https://godoc.org/text/template) for templating your helmfile.yaml. While go ships several built-in functions, we have added all of the functions in the [Sprig library](https://godoc.org/github.com/Masterminds/sprig).
We also added one special template function: `requiredEnv`.
The `requiredEnv` function allows you to declare a particular environment variable as required for template rendering.
If the environment variable is unset or empty, the template rendering will fail with an error message.
## usage
## Using environment variables
Environment variables can be used in most places for templating the helmfile. Currently this is supported for `name`, `namespace`, `value` (in set), `values` and `url` (in repositories).
Examples:
```yaml
respositories:
- name: your-private-git-repo-hosted-charts
url: https://{{ requiredEnv "GITHUB_TOKEN"}}@raw.githubusercontent.com/kmzfs/helm-repo-in-github/master/
```
```yaml
releases:
- name: {{ requiredEnv "NAME" }}-vault
namespace: {{ requiredEnv "NAME" }}
chart: roboll/vault-secret-manager
values:
- db:
username: {{ requiredEnv "DB_USERNAME" }}
password: {{ requiredEnv "DB_PASSWORD" }}
set:
- name: proxy.domain
value: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
- name: proxy.scheme
value: {{ env "SCHEME" | default "https" }}
```
## installation
- `go get github.com/roboll/helmfile` or
- download one of [releases](https://github.com/roboll/helmfile/releases) or
- run as a [container](https://quay.io/roboll/helmfile)
## getting started
Let's start with a simple `helmfile` and gradually improve it to fit your use-case!
Suppose the `helmfile.yaml` representing the desired state of your helm releases looks like:
```yaml
releases:
- name: prom-norbac-ubuntu
namespace: prometheus
chart: stable/prometheus
set:
- name: rbac.create
value: false
```
Sync your Kubernetes cluster state to the desired one by running:
```console
helmfile sync
```
Congratulations! You now have your first Prometheus deployment running inside your cluster.
Iterate on the `helmfile.yaml` by referencing the [configuration syntax](#configuration-syntax) and the [cli reference](#cli-reference).
## cli reference
```
NAME:
helmfile -
USAGE:
main [global options] command [command options] [arguments...]
helmfile [global options] command [command options] [arguments...]
COMMANDS:
repos sync repositories from state file (helm repo add && helm repo update)
charts sync charts from state file (helm repo upgrade --install)
diff diff charts from state file against env (helm diff)
sync sync all resources from state file (repos && charts)
delete delete charts from state file (helm delete)
charts sync releases from state file (helm upgrade --install)
diff diff releases from state file against env (helm diff)
lint lint charts from state file (helm lint)
sync sync all resources from state file (repos, releases and chart deps)
apply apply all resources from state file only when there are changes
status retrieve status of releases in state file
delete delete releases from state file (helm delete)
test test releases from state file (helm test)
GLOBAL OPTIONS:
--file FILE, -f FILE load config from FILE (default: "charts.yaml")
--quiet, -q silence output
--kube-context value Set kubectl context. Uses current context by default
--help, -h show help
--version, -v print the version
--helm-binary value, -b value path to helm binary
--file helmfile.yaml, -f helmfile.yaml load config from file or directory. defaults to helmfile.yaml or `helmfile.d`(means `helmfile.d/*.yaml`) in this preference
--quiet, -q Silence output. Equivalent to log-level warn
--kube-context value Set kubectl context. Uses current context by default
--log-level value Set log level, default info
--namespace value, -n value Set namespace. Uses the namespace set in the context by default
--selector value, -l value Only run using the releases that match labels. Labels can take the form of foo=bar or foo!=bar.
A release must match all labels in a group in order to be used. Multiple groups can be specified at once.
--selector tier=frontend,tier!=proxy --selector tier=backend. Will match all frontend, non-proxy releases AND all backend releases.
The name of a release can be used as a label. --selector name=myrelease
--help, -h show help
--version, -v print the version
```
### sync
The `helmfile sync` sub-command sync your cluster state as described in your `helmfile`. The default helmfile is `helmfile.yaml`, but any yaml file can be passed by specifying a `--file path/to/your/yaml/file` flag.
Under the covers, Helmfile executes `helm upgrade --install` for each `release` declared in the manifest, by optionally decrypting [secrets](#secrets) to be consumed as helm chart values. It also updates specified chart repositories and updates the
dependencies of any referenced local charts.
For Helm 2.9+ you can use a username and password to authenticate to a remote repository.
### diff
The `helmfile diff` sub-command executes the [helm-diff](https://github.com/databus23/helm-diff) plugin across all of
the charts/releases defined in the manifest.
Under the covers Helmfile is simply using the `helm diff` plugin, so that needs to be installed prior. For Helm 2.3+
To supply the diff functionality Helmfile needs the [helm-diff](https://github.com/databus23/helm-diff) plugin v2.9.0+1 or greater installed. For Helm 2.3+
you should be able to simply execute `helm plugin install https://github.com/databus23/helm-diff`. For more details
please look at their [documentation](https://github.com/databus23/helm-diff#helm-diff-plugin).
### apply
The `helmfile apply` sub-command begins by executing `diff`. If `diff` finds that there is any changes, `sync` is executed after prompting you for an confirmation. `--auto-approve` skips the confirmation.
An expected use-case of `apply` is to schedule it to run periodically, so that you can auto-fix skews between the desired and the current state of your apps running on Kubernetes clusters.
### delete
The `helmfile delete` sub-command deletes all the releases defined in the manfiests
Note that `delete` doesn't purge releases. So `helmfile delete && helmfile sync` results in sync failed due to that releases names are not deleted but preserved for future references. If you really want to remove releases for reuse, add `--purge` flag to run it like `helmfile delete --purge`.
### secrets
The `secrets` parameter in a `helmfile.yaml` causes the [helm-secrets](https://github.com/futuresimple/helm-secrets) plugin to be executed to decrypt the file.
To supply the secret functionality Helmfile needs the `helm secrets` plugin installed. For Helm 2.3+
you should be able to simply execute `helm plugin install https://github.com/futuresimple/helm-secrets
`.
### test
The `helmfile test` sub-command runs a `helm test` against specified releases in the manifest, default to all
Use `--cleanup` to delete pods upon completion.
### lint
The `helmfile lint` sub-command runs a `helm lint` across all of the charts/releases defined in the manifest. Non local charts will be fetched into a temporary folder which will be deleted once the task is completed.
## Paths Overview
Using manifest files in conjunction with command line argument can be a bit confusing.
Using manifest files in conjunction with command line argument can be a bit confusing.
A few rules to clear up this ambiguity:
A few rules to clear up this ambiguity:
- Absolute paths are always resolved as absolute paths
- Relative paths referenced *in* the helmfile manifest itself are relative to that manifest
- Relative paths referenced on the command line are relative to the current working directory the user is in
For additional context, take a look at [paths examples](PATHS.md)
## Labels Overview
A selector can be used to only target a subset of releases when running helmfile. This is useful for large helmfiles with releases that are logically grouped together.
Labels are simple key value pairs that are an optional field of the release spec. When selecting by label, the search can be inverted. `tier!=backend` would match all releases that do NOT have the `tier: backend` label. `tier=fronted` would only match releases with the `tier: frontend` label.
Multiple labels can be specified using `,` as a separator. A release must match all selectors in order to be selected for the final helm command.
The `selector` parameter can be specified multiple times. Each parameter is resolved independently so a release that matches any parameter will be used.
`--selector tier=frontend --selector tier=backend` will select all the charts
## Templates
You can use go's text/template expressions in `helmfile.yaml` and `values.yaml`(helm values files).
In addition to built-in ones, the following custom template functions are available:
- `readFile` reads the specified local file and generate a golang string
- `fromYaml` reads a golang string and generates a map
- `setValuleAtPath PATH NEW_VALUE` traverses a golang map, replaces the value at the PATH with NEW_VALUE
- `toYaml` marshals a map into a string
### Values Files Templates
You can reference a template of values file in your `helmfile.yaml` like below:
```yaml
releases
- name: myapp
chart: mychart
values:
- values.yaml.gotmpl
```
Every values file whose file extension is `.gotmpl` is considered as a tempalte file.
Suppose `values.yaml.gotmpl` was something like:
```yaml
{{ readFile "values.yaml" | fromYaml | setValueAtPath "foo.bar" "FOO_BAR" | toYaml }}
```
And `values.yaml` was:
```yaml
foo:
bar: ""
```
The resulting, temporary values.yaml that is generated from `values.yaml.tpl` would become:
```yaml
foo:
# Notice `setValueAtPath "foo.bar" "FOO_BAR"` in the template above
bar: FOO_BAR
```
## Refactoring `helmfile.yaml` with values files templates
One of expected use-cases of values files templates is to keep `helmfile.yaml` small and concise.
See the example `helmfile.yaml` below:
```yaml
releases:
- name: {{ requiredEnv "NAME" }}-vault
namespace: {{ requiredEnv "NAME" }}
chart: roboll/vault-secret-manager
values:
- db:
username: {{ requiredEnv "DB_USERNAME" }}
password: {{ requiredEnv "DB_PASSWORD" }}
set:
- name: proxy.domain
value: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
- name: proxy.scheme
value: {{ env "SCHEME" | default "https" }}
```
The `values` and `set` sections of the config file can be separated out into a template:
`helmfile.yaml`:
```yaml
releases:
- name: {{ requiredEnv "NAME" }}-vault
namespace: {{ requiredEnv "NAME" }}
chart: roboll/vault-secret-manager
values:
- values.yaml.tmpl
```
`values.yaml.tmpl`:
```yaml
db:
username: {{ requiredEnv "DB_USERNAME" }}
password: {{ requiredEnv "DB_PASSWORD" }}
proxy:
domain: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
scheme: {{ env "SCHEME" | default "https" }}
```
## Separating helmfile.yaml into multiple independent files
Once your `helmfile.yaml` got to contain too many releases,
split it into multiple yaml files.
Recommended granularity of helmfile.yaml files is "per microservice" or "per team".
And there are two ways to organize your files.
- Single directory
- Glob patterns
### Single directory
`helmfile -f path/to/directory` loads and runs all the yaml files under the specified directory, each file as an independent helmfile.yaml.
The default helmfile directory is `helmfile.d`, that is,
in case helmfile is unable to locate `helmfile.yaml`, it tries to locate `helmfile.d/*.yaml`.
All the yaml files under the specified directory are processed in the alphabetical order. For example, you can use a `<two digit number>-<microservice>.yaml` naming convention to control the sync order.
- `helmfile.d`/
- `00-database.yaml`
- `00-backend.yaml`
- `01-frontend.yaml`
### Glob patterns
In case you want more control over how multiple `helmfile.yaml` files are organized, use `helmfiles:` configuration key in the `helmfile.yaml`:
Suppose you have multiple microservices organized in a Git reposistory that looks like:
- `myteam/` (sometimes it is equivalent to a k8s ns, that is `kube-system` for `clusterops` team)
- `apps/`
- `filebeat/`
- `helmfile.yaml` (no `charts/` exists, because it depends on the stable/filebeat chart hosted on the official helm charts repository)
- `README.md` (each app managed by my team has a dedicated README maintained by the owners of the app)
- `metricbeat/`
- `helmfile.yaml`
- `README.md`
- `elastalert-operator/`
- `helmfile.yaml`
- `README.md`
- `charts/`
- `elastalert-operator/`
- `<the content of the local helm chart>`
The benefits of this structure is that you can run `git diff` to locate in which directory=microservice a git commit has changes.
It allows your CI system to run a workflow for the changed microservice only.
A downside of this is that you don't have an obvious way to sync all microservices at once. That is, you have to run:
```bash
for d in apps/*; do helmfile -f $d diff; if [ $? -eq 2 ]; then helmfile -f $d sync; fi; done
```
At this point, you'll start writing a `Makefile` under `myteam/` so that `make sync-all` will do the job.
It does work, but you can rely on the helmfile's feature instead.
Put `myteam/helmfile.yaml` that looks like:
```yaml
helmfiles:
- apps/*/helmfile.yaml
```
So that you can get rid of the `Makefile` and the bash snippet.
Just run `helmfile sync` inside `myteam/`, and you are done.
All the files are sorted alphabetically per group = array item inside `helmfiles:`, so that you have granular control over ordering, too.
## Using env files
helmfile itself doesn't have an ability to load env files. But you can write some bash script to achieve the goal:
```console
set -a; . .env; set +a; helmfile sync
```
Please see #203 for more context.
## Running helmfile without an Internet connection
Once you download all required charts into your machine, you can run `helmfile charts` to deploy your apps.
It basically run only `helm upgrade --install` with your already-downloaded charts, hence no Internet connection is required.
See #155 for more information on this topic.
## Examples
For more examples, see the [examples/README.md](https://github.com/roboll/helmfile/blob/master/examples/README.md) or the [`helmfile.d`](https://github.com/cloudposse/helmfiles/tree/master/helmfile.d) distribution of helmfiles by [Cloud Posse](https://github.com/cloudposse/).
+18
View File
@@ -0,0 +1,18 @@
# Helmfile Users
The following table is compiled from public information or from survey responses. If you are using Helmfile in your
company or organization, we would like to invite you to [participate in our user survey](https://docs.google.com/forms/d/e/1FAIpQLSfrF5kR9nXsr5yWqNZGgN-KquM1mBTuYNn5lRmEVXEvhq0VIQ/viewform)
to help us extend this list.
<!-- TABLE_START -->
| Organization | Workload | More Info | Location |
|---|---|---|---|
| [reddit.com](https://www.reddit.com/) | production | [GitHub issue](https://github.com/roboll/helmfile/issues/96#issue-312012249) | San Francisco, CA, US |
| [ricardo.ch](https://www.ricardo.ch/) | proof-of-concept | | Zug, Switzerland |
<!-- TABLE_END -->
## Contact
Please fill out [the Google form](https://docs.google.com/forms/d/e/1FAIpQLSfrF5kR9nXsr5yWqNZGgN-KquM1mBTuYNn5lRmEVXEvhq0VIQ/viewform) to add your organization instead of opening a PR.
You can also [reach the maintainer on Twitter](https://twitter.com/DerCed).
+100
View File
@@ -0,0 +1,100 @@
package args
import (
"fmt"
"strings"
"github.com/roboll/helmfile/state"
)
type argMap struct {
m map[string]string
flags []string
}
func (a *argMap) SetArg(flag, arg string) {
if _, exists := a.m[flag]; !exists {
a.m[flag] = arg
a.flags = append(a.flags, flag)
}
}
func newArgMap() *argMap {
return &argMap{m: map[string]string{}}
}
func GetArgs(args string, state *state.HelmState) []string {
//args := c.String("args")
argsMap := newArgMap()
spaceflagArg := map[string]bool{}
if len(args) > 0 {
argsVals := strings.Split(args, " ")
prevFlag := ""
for _, arg := range argsVals {
if strings.HasPrefix(arg, "--") {
argVal := strings.SplitN(arg, "=", 2)
if len(argVal) > 1 {
arg := argVal[0]
value := argVal[1]
argsMap.SetArg(arg, value)
} else {
argsMap.SetArg(arg, "")
}
prevFlag = arg
} else {
spaceflagArg[prevFlag] = true
argsMap.m[prevFlag] = arg
}
}
}
if len(state.HelmDefaults.Args) > 0 {
for _, arg := range state.HelmDefaults.Args {
var flag string
var val string
argsNum, _ := fmt.Sscanf(arg, "--%s %s", &flag, &val)
if argsNum == 2 {
argsMap.SetArg(flag, arg)
} else {
argVal := strings.SplitN(arg, "=", 2)
argFirst := argVal[0]
if len(argVal) > 1 {
val = argVal[1]
argsMap.SetArg(argFirst, val)
} else {
argsMap.SetArg(argFirst, "")
}
}
}
}
if state.HelmDefaults.TillerNamespace != "" {
argsMap.SetArg("--tiller-namespace", state.HelmDefaults.TillerNamespace)
}
if state.HelmDefaults.KubeContext != "" {
argsMap.SetArg("--kube-context", state.HelmDefaults.KubeContext)
}
var argArr []string
for _, flag := range argsMap.flags {
val := argsMap.m[flag]
if val != "" {
if spaceflagArg[flag] {
argArr = append(argArr, fmt.Sprintf("%s %s", flag, val))
} else {
argArr = append(argArr, fmt.Sprintf("%s=%s", flag, val))
}
} else {
argArr = append(argArr, fmt.Sprintf("%s", flag))
}
}
state.HelmDefaults.Args = argArr
return state.HelmDefaults.Args
}
+34
View File
@@ -0,0 +1,34 @@
package args
import (
"fmt"
"strings"
"testing"
"github.com/roboll/helmfile/state"
)
func TestGetArgs(t *testing.T) {
args := "--timeout=3600 --set app1.bootstrap=true,app2.bootstrap=false --tiller-namespace ns"
defaultArgs := []string{"--recreate-pods", "--force"}
fmt.Println(defaultArgs)
fmt.Println(len(defaultArgs))
Helmdefaults := state.HelmSpec{KubeContext: "test", TillerNamespace: "test-namespace", Args: defaultArgs}
testState := &state.HelmState{HelmDefaults: Helmdefaults}
receivedArgs := GetArgs(args, testState)
expectedOutput := "--timeout=3600 --set app1.bootstrap=true,app2.bootstrap=false --tiller-namespace ns --recreate-pods --force --kube-context=test"
if compareArgs(expectedOutput, receivedArgs) == false {
t.Errorf("expected %s, got %s", expectedOutput, strings.Join(receivedArgs, " "))
}
}
func compareArgs(expectedArgs string, args []string) bool {
if strings.Compare(strings.Join(args, " "), expectedArgs) != 0 {
return false
}
return true
}
+33
View File
@@ -0,0 +1,33 @@
package main
import (
"bufio"
"fmt"
"log"
"os"
"strings"
)
// Copyright (c) 2017 Roland Singer [roland.singer@desertbit.com]
//
// Shamelessly borrowed from @r0l1's awesome work that is available at https://gist.github.com/r0l1/3dcbb0c8f6cfe9c66ab8008f55f8f28b
func askForConfirmation(s string) bool {
reader := bufio.NewReader(os.Stdin)
for {
fmt.Printf("%s [y/n]: ", s)
response, err := reader.ReadString('\n')
if err != nil {
log.Fatal(err)
}
response = strings.ToLower(strings.TrimSpace(response))
if response == "y" || response == "yes" {
return true
} else if response == "n" || response == "no" {
return false
}
}
}
-32
View File
@@ -1,32 +0,0 @@
machine:
post:
- mkdir -p download
- test -e download/$GODIST || curl -o download/$GODIST https://storage.googleapis.com/golang/$GODIST
- sudo rm -rf /usr/local/go
- sudo tar -C /usr/local -xzf download/$GODIST
environment:
GODIST: "go1.7.linux-amd64.tar.gz"
GOPATH: "$HOME/go"
PATH: "$PATH:$GOPATH/bin"
WORK: "$GOPATH/src/github.com/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME/"
dependencies:
override:
- mkdir -p "$WORK"
- rsync -az --delete ./ "$WORK"
test:
pre:
- cd "$WORK" && make check
override:
- cd "$WORK" && make test
# GITHUB_TOKEN env var must be setup in circleci console
deployment:
release:
tag: /v.*/
commands:
- docker login -e="." -u="$DOCKER_USER" -p="$DOCKER_PASS" quay.io
- cd "$WORK" && make tools
- cd "$WORK" && BUILD_URL="$CIRCLE_BUILD_URL" make push release
+35
View File
@@ -0,0 +1,35 @@
# Helmfile practical examples and advanced usages
## Managing oneshot-jobs with Helmfile
In case you want to manage oneshot-jobs like a report generation or a database migration and so on, add a dedicated release spec for the job in `helmfile.yaml` like:
```yaml
repositories:
- name: yourorg
url: https://yourorg.example.com/charts
releases:
- name: dbmigrator
labels:
job: dbmigrator
chart: ./dbmigrator
# DB host, port, and connection opts for the environment
values:
- "deploy/environments/{{ env "RAILS_ENV" }}/values.yaml"
# DB username and password encrypted with helm-secrets(mozilla/sops)
secrets:
- "deploy/environments/{{ env "RAILS_ENV" }}/secrets.yaml"
```
You would then start a database migration job by executing:
```console
# Start a database migration for the prod environment
$ RAILS_ENV=prod helmfile sync --selector job=dbmigrator
# Tail log until you are satisfied
$ kubectl logs -l job=dbmigrator
```
For more context, see [this issue](https://github.com/roboll/helmfile/issues/49).
+1 -1
View File
@@ -1,5 +1,5 @@
charts:
# Published chart example
- name: grafana # helm deployment name
- name: grafana # helm release name
namespace: grafana # target namespace
chart: stable/grafana # chart reference (repository)
+143 -40
View File
@@ -1,12 +1,14 @@
package helmexec
import (
"fmt"
"io"
"io/ioutil"
"os"
"os/exec"
"strings"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
"sync"
)
const (
@@ -14,15 +16,38 @@ const (
)
type execer struct {
writer io.Writer
kubeContext string
extra []string
helmBinary string
runner Runner
logger *zap.SugaredLogger
kubeContext string
extra []string
decryptionMutex sync.Mutex
}
func NewHelmExec(writer io.Writer, kubeContext string) Interface {
func NewLogger(writer io.Writer, logLevel string) *zap.SugaredLogger {
var cfg zapcore.EncoderConfig
cfg.MessageKey = "message"
out := zapcore.AddSync(writer)
var level zapcore.Level
err := level.Set(logLevel)
if err != nil {
panic(err)
}
core := zapcore.NewCore(
zapcore.NewConsoleEncoder(cfg),
out,
level,
)
return zap.New(core).Sugar()
}
// New for running helm commands
func New(logger *zap.SugaredLogger, kubeContext string) *execer {
return &execer{
writer: writer,
helmBinary: command,
logger: logger,
kubeContext: kubeContext,
runner: &ShellRunner{},
}
}
@@ -30,53 +55,130 @@ func (helm *execer) SetExtraArgs(args ...string) {
helm.extra = args
}
func (helm *execer) AddRepo(name, repository string) error {
out, err := helm.exec("repo", "add", name, repository)
if helm.writer != nil {
helm.writer.Write(out)
func (helm *execer) SetHelmBinary(bin string) {
helm.helmBinary = bin
}
func (helm *execer) AddRepo(name, repository, certfile, keyfile, username, password string) error {
var args []string
args = append(args, "repo", "add", name, repository)
if certfile != "" && keyfile != "" {
args = append(args, "--cert-file", certfile, "--key-file", keyfile)
}
if username != "" && password != "" {
args = append(args, "--username", username, "--password", password)
}
helm.logger.Infof("Adding repo %v %v", name, repository)
out, err := helm.exec(args...)
helm.write(out)
return err
}
func (helm *execer) UpdateRepo() error {
helm.logger.Info("Updating repo")
out, err := helm.exec("repo", "update")
if helm.writer != nil {
helm.writer.Write(out)
}
helm.write(out)
return err
}
func (helm *execer) SyncChart(name, chart string, flags ...string) error {
out, err := helm.exec(append([]string{"upgrade", "--install", name, chart}, flags...)...)
if helm.writer != nil {
helm.writer.Write(out)
}
func (helm *execer) UpdateDeps(chart string) error {
helm.logger.Infof("Updating dependency %v", chart)
out, err := helm.exec("dependency", "update", chart)
helm.write(out)
return err
}
func (helm *execer) DiffChart(name, chart string, flags ...string) error {
out, err := helm.exec(append([]string{"diff", name, chart}, flags...)...)
if helm.writer != nil {
helm.writer.Write(out)
}
func (helm *execer) SyncRelease(name, chart string, flags ...string) error {
helm.logger.Infof("Upgrading %v", chart)
out, err := helm.exec(append([]string{"upgrade", "--install", "--reset-values", name, chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) DeleteChart(name string) error {
out, err := helm.exec("delete", name)
if helm.writer != nil {
helm.writer.Write(out)
func (helm *execer) ReleaseStatus(name string) error {
helm.logger.Infof("Getting status %v", name)
out, err := helm.exec(append([]string{"status", name})...)
helm.write(out)
return err
}
func (helm *execer) DecryptSecret(name string) (string, error) {
// Prevents https://github.com/roboll/helmfile/issues/258
helm.decryptionMutex.Lock()
defer helm.decryptionMutex.Unlock()
helm.logger.Infof("Decrypting secret %v", name)
out, err := helm.exec(append([]string{"secrets", "dec", name})...)
helm.write(out)
tmpFile, err := ioutil.TempFile("", "secret")
if err != nil {
return "", err
}
defer tmpFile.Close()
// os.Rename seems to results in "cross-device link` errors in some cases
// Instead of moving, copy it to the destination temp file as a work-around
// See https://github.com/roboll/helmfile/issues/251#issuecomment-417166296f
decFilename := name + ".dec"
decFile, err := os.Open(decFilename)
if err != nil {
return "", err
}
defer decFile.Close()
_, err = io.Copy(tmpFile, decFile)
if err != nil {
return "", err
}
if err := decFile.Close(); err != nil {
return "", err
}
if err := os.Remove(decFilename); err != nil {
return "", err
}
return tmpFile.Name(), err
}
func (helm *execer) DiffRelease(name, chart string, flags ...string) error {
helm.logger.Infof("Comparing %v %v", name, chart)
out, err := helm.exec(append([]string{"diff", "upgrade", "--allow-unreleased", name, chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) Lint(chart string, flags ...string) error {
helm.logger.Infof("Linting %v", chart)
out, err := helm.exec(append([]string{"lint", chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) Fetch(chart string, flags ...string) error {
helm.logger.Infof("Fetching %v", chart)
out, err := helm.exec(append([]string{"fetch", chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) DeleteRelease(name string, flags ...string) error {
helm.logger.Infof("Deleting %v", name)
out, err := helm.exec(append([]string{"delete", name}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) TestRelease(name string, flags ...string) error {
helm.logger.Infof("Testing %v", name)
out, err := helm.exec(append([]string{"test", name}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) exec(args ...string) ([]byte, error) {
dir, err := ioutil.TempDir("", "helmfile-exec")
if err != nil {
return nil, err
}
defer os.RemoveAll(dir)
cmdargs := args
if len(helm.extra) > 0 {
cmdargs = append(cmdargs, helm.extra...)
@@ -84,11 +186,12 @@ func (helm *execer) exec(args ...string) ([]byte, error) {
if helm.kubeContext != "" {
cmdargs = append(cmdargs, "--kube-context", helm.kubeContext)
}
if helm.writer != nil {
helm.writer.Write([]byte(fmt.Sprintf("exec: helm %s\n", strings.Join(cmdargs, " "))))
helm.logger.Debugf("exec: %s %s", helm.helmBinary, strings.Join(cmdargs, " "))
return helm.runner.Execute(helm.helmBinary, cmdargs)
}
func (helm *execer) write(out []byte) {
if len(out) > 0 {
helm.logger.Infof("%s", out)
}
cmd := exec.Command(command, cmdargs...)
cmd.Dir = dir
return cmd.CombinedOutput()
}
+355
View File
@@ -0,0 +1,355 @@
package helmexec
import (
"bytes"
"os"
"reflect"
"testing"
"go.uber.org/zap"
)
// Mocking the command-line runner
type mockRunner struct {
output []byte
err error
}
func (mock *mockRunner) Execute(cmd string, args []string) ([]byte, error) {
return []byte{}, nil
}
func MockExecer(logger *zap.SugaredLogger, kubeContext string) *execer {
execer := New(logger, kubeContext)
execer.runner = &mockRunner{}
return execer
}
// Test methods
func TestNewHelmExec(t *testing.T) {
buffer := bytes.NewBufferString("something")
logger := NewLogger(buffer, "debug")
helm := New(logger, "dev")
if helm.kubeContext != "dev" {
t.Error("helmexec.New() - kubeContext")
}
if buffer.String() != "something" {
t.Error("helmexec.New() - changed buffer")
}
if len(helm.extra) != 0 {
t.Error("helmexec.New() - extra args not empty")
}
}
func Test_SetExtraArgs(t *testing.T) {
helm := New(NewLogger(os.Stdout, "info"), "dev")
helm.SetExtraArgs()
if len(helm.extra) != 0 {
t.Error("helmexec.SetExtraArgs() - passing no arguments should not change extra field")
}
helm.SetExtraArgs("foo")
if !reflect.DeepEqual(helm.extra, []string{"foo"}) {
t.Error("helmexec.SetExtraArgs() - one extra argument missing")
}
helm.SetExtraArgs("alpha", "beta")
if !reflect.DeepEqual(helm.extra, []string{"alpha", "beta"}) {
t.Error("helmexec.SetExtraArgs() - two extra arguments missing (overwriting the previous value)")
}
}
func Test_SetHelmBinary(t *testing.T) {
helm := New(NewLogger(os.Stdout, "info"), "dev")
if helm.helmBinary != "helm" {
t.Error("helmexec.command - default command is not helm")
}
helm.SetHelmBinary("foo")
if helm.helmBinary != "foo" {
t.Errorf("helmexec.SetHelmBinary() - actual = %s expect = foo", helm.helmBinary)
}
}
func Test_AddRepo(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.AddRepo("myRepo", "https://repo.example.com/", "cert.pem", "key.pem", "", "")
expected := `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --cert-file cert.pem --key-file key.pem --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "", "")
expected = `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "example_user", "example_password")
expected = `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_UpdateRepo(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.UpdateRepo()
expected := `Updating repo
exec: helm repo update --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.UpdateRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_SyncRelease(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.SyncRelease("release", "chart", "--timeout 10", "--wait")
expected := `Upgrading chart
exec: helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SyncRelease("release", "chart")
expected = `Upgrading chart
exec: helm upgrade --install --reset-values release chart --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_UpdateDeps(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.UpdateDeps("./chart/foo")
expected := `Updating dependency ./chart/foo
exec: helm dependency update ./chart/foo --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.UpdateDeps()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SetExtraArgs("--verify")
helm.UpdateDeps("./chart/foo")
expected = `Updating dependency ./chart/foo
exec: helm dependency update ./chart/foo --verify --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DecryptSecret(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DecryptSecret("secretName")
expected := `Decrypting secret secretName
exec: helm secrets dec secretName --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.DecryptSecret()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DiffRelease(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DiffRelease("release", "chart", "--timeout 10", "--wait")
expected := `Comparing release chart
exec: helm diff upgrade --allow-unreleased release chart --timeout 10 --wait --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.DiffRelease("release", "chart")
expected = `Comparing release chart
exec: helm diff upgrade --allow-unreleased release chart --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DeleteRelease(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DeleteRelease("release")
expected := `Deleting release
exec: helm delete release --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.DeleteRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DeleteRelease_Flags(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DeleteRelease("release", "--purge")
expected := `Deleting release
exec: helm delete release --purge --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.DeleteRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_TestRelease(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.TestRelease("release")
expected := `Testing release
exec: helm test release --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.TestRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_TestRelease_Flags(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.TestRelease("release", "--cleanup", "--timeout", "60")
expected := `Testing release
exec: helm test release --cleanup --timeout 60 --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.TestRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_ReleaseStatus(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.ReleaseStatus("myRelease")
expected := `Getting status myRelease
exec: helm status myRelease --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.ReleaseStatus()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_exec(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "")
helm.exec("version")
expected := "exec: helm version\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
helm = MockExecer(logger, "dev")
ret, _ := helm.exec("diff")
if len(ret) != 0 {
t.Error("helmexec.exec() - expected empty return value")
}
buffer.Reset()
helm = MockExecer(logger, "dev")
helm.exec("diff", "release", "chart", "--timeout 10", "--wait")
expected = "exec: helm diff release chart --timeout 10 --wait --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.exec("version")
expected = "exec: helm version --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SetExtraArgs("foo")
helm.exec("version")
expected = "exec: helm version foo --kube-context dev\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm = MockExecer(logger, "")
helm.SetHelmBinary("overwritten")
helm.exec("version")
expected = "exec: overwritten version\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_Lint(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.Lint("path/to/chart", "--values", "file.yml")
expected := `Linting path/to/chart
exec: helm lint path/to/chart --values file.yml --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.Lint()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_Fetch(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.Fetch("chart", "--version", "1.2.3", "--untar", "--untardir", "/tmp/dir")
expected := `Fetching chart
exec: helm fetch chart --version 1.2.3 --untar --untardir /tmp/dir --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.Lint()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
var logLevelTests = map[string]string{
"debug": `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password
`,
"info": `Adding repo myRepo https://repo.example.com/
`,
"warn": ``,
}
func Test_LogLevels(t *testing.T) {
var buffer bytes.Buffer
for logLevel, expected := range logLevelTests {
buffer.Reset()
logger := NewLogger(&buffer, logLevel)
helm := MockExecer(logger, "")
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "example_user", "example_password")
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
}
+12 -4
View File
@@ -1,12 +1,20 @@
package helmexec
// Interface for executing helm commands
type Interface interface {
SetExtraArgs(args ...string)
SetHelmBinary(bin string)
AddRepo(name, repository string) error
AddRepo(name, repository, certfile, keyfile, username, password string) error
UpdateRepo() error
UpdateDeps(chart string) error
SyncRelease(name, chart string, flags ...string) error
DiffRelease(name, chart string, flags ...string) error
Fetch(chart string, flags ...string) error
Lint(chart string, flags ...string) error
ReleaseStatus(name string) error
DeleteRelease(name string, flags ...string) error
TestRelease(name string, flags ...string) error
SyncChart(name, chart string, flags ...string) error
DiffChart(name, chart string, flags ...string) error
DeleteChart(name string) error
DecryptSecret(name string) (string, error)
}
+24
View File
@@ -0,0 +1,24 @@
package helmexec
import (
"os/exec"
)
const (
tmpPrefix = "helmfile-"
tmpSuffix = "-exec"
)
// Runner interface for shell commands
type Runner interface {
Execute(cmd string, args []string) ([]byte, error)
}
// ShellRunner implemention for shell commands
type ShellRunner struct{}
// Execute a shell command
func (shell ShellRunner) Execute(cmd string, args []string) ([]byte, error) {
preparedCmd := exec.Command(cmd, args...)
return preparedCmd.CombinedOutput()
}
+535 -107
View File
@@ -2,22 +2,57 @@ package main
import (
"fmt"
"io"
"log"
"os"
"strings"
"os/signal"
"path/filepath"
"sort"
"syscall"
"os/exec"
"github.com/roboll/helmfile/args"
"github.com/roboll/helmfile/helmexec"
"github.com/roboll/helmfile/state"
"github.com/roboll/helmfile/tmpl"
"github.com/urfave/cli"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
"io/ioutil"
)
const (
helmfile = "charts.yaml"
DefaultHelmfile = "helmfile.yaml"
DeprecatedHelmfile = "charts.yaml"
DefaultHelmfileDirectory = "helmfile.d"
)
var Version string
var logger *zap.SugaredLogger
func configureLogging(c *cli.Context) error {
// Valid levels:
// https://github.com/uber-go/zap/blob/7e7e266a8dbce911a49554b945538c5b950196b8/zapcore/level.go#L126
logLevel := c.GlobalString("log-level")
if c.GlobalBool("quiet") {
logLevel = "warn"
}
var level zapcore.Level
err := level.Set(logLevel)
if err != nil {
return err
}
logger = helmexec.NewLogger(os.Stdout, logLevel)
if c.App.Metadata == nil {
// Auto-initialised in 1.19.0
// https://github.com/urfave/cli/blob/master/CHANGELOG.md#1190---2016-11-19
c.App.Metadata = make(map[string]interface{})
}
c.App.Metadata["logger"] = logger
return nil
}
func main() {
app := cli.NewApp()
@@ -25,21 +60,40 @@ func main() {
app.Usage = ""
app.Version = Version
app.Flags = []cli.Flag{
cli.StringFlag{
Name: "helm-binary, b",
Usage: "path to helm binary",
},
cli.StringFlag{
Name: "file, f",
Value: helmfile,
Usage: "load config from `FILE`",
Usage: "load config from file or directory. defaults to `helmfile.yaml` or `helmfile.d`(means `helmfile.d/*.yaml`) in this preference",
},
cli.BoolFlag{
Name: "quiet, q",
Usage: "silence output",
Usage: "Silence output. Equivalent to log-level warn",
},
cli.StringFlag{
Name: "kube-context",
Usage: "Set kubectl context. Uses current context by default",
},
cli.StringFlag{
Name: "log-level",
Usage: "Set log level, default info",
},
cli.StringFlag{
Name: "namespace, n",
Usage: "Set namespace. Uses the namespace set in the context by default",
},
cli.StringSliceFlag{
Name: "selector, l",
Usage: `Only run using the releases that match labels. Labels can take the form of foo=bar or foo!=bar.
A release must match all labels in a group in order to be used. Multiple groups can be specified at once.
--selector tier=frontend,tier!=proxy --selector tier=backend. Will match all frontend, non-proxy releases AND all backend releases.
The name of a release can be used as a label. --selector name=myrelease`,
},
}
app.Before = configureLogging
app.Commands = []cli.Command{
{
Name: "repos",
@@ -52,28 +106,22 @@ func main() {
},
},
Action: func(c *cli.Context) error {
state, helm, err := before(c)
if err != nil {
return err
}
args := c.String("args")
if len(args) > 0 {
helm.SetExtraArgs(strings.Split(args, " ")...)
}
if errs := state.SyncRepos(helm); errs != nil && len(errs) > 0 {
for _, err := range errs {
fmt.Printf("err: %s\n", err.Error())
return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface) []error {
args := args.GetArgs(c.String("args"), state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
os.Exit(1)
}
return nil
if c.GlobalString("helm-binary") != "" {
helm.SetHelmBinary(c.GlobalString("helm-binary"))
}
return state.SyncRepos(helm)
})
},
},
{
Name: "charts",
Usage: "sync charts from state file (helm repo upgrade --install)",
Usage: "sync releases from state file (helm upgrade --install)",
Flags: []cli.Flag{
cli.StringFlag{
Name: "args",
@@ -84,32 +132,32 @@ func main() {
Name: "values",
Usage: "additional value files to be merged into the command",
},
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
},
Action: func(c *cli.Context) error {
state, helm, err := before(c)
if err != nil {
return err
}
args := c.String("args")
if len(args) > 0 {
helm.SetExtraArgs(strings.Split(args, " ")...)
}
values := c.StringSlice("values")
if errs := state.SyncCharts(helm, values); errs != nil && len(errs) > 0 {
for _, err := range errs {
fmt.Printf("err: %s\n", err.Error())
return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface) []error {
args := args.GetArgs(c.String("args"), state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
os.Exit(1)
}
return nil
if c.GlobalString("helm-binary") != "" {
helm.SetHelmBinary(c.GlobalString("helm-binary"))
}
values := c.StringSlice("values")
workers := c.Int("concurrency")
return state.SyncReleases(helm, values, workers)
})
},
},
{
Name: "diff",
Usage: "diff charts from state file against env (helm diff)",
Usage: "diff releases from state file against env (helm diff)",
Flags: []cli.Flag{
cli.StringFlag{
Name: "args",
@@ -124,112 +172,492 @@ func main() {
Name: "sync-repos",
Usage: "enable a repo sync prior to diffing",
},
cli.BoolFlag{
Name: "detailed-exitcode",
Usage: "return a non-zero exit code when there are changes",
},
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
},
Action: func(c *cli.Context) error {
state, helm, err := before(c)
if err != nil {
return err
}
args := c.String("args")
if len(args) > 0 {
helm.SetExtraArgs(strings.Split(args, " ")...)
}
if c.Bool("sync-repos") {
if errs := state.SyncRepos(helm); errs != nil && len(errs) > 0 {
for _, err := range errs {
fmt.Printf("err: %s\n", err.Error())
}
os.Exit(1)
return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface) []error {
return executeDiffCommand(c, state, helm, c.Bool("detailed-exitcode"))
})
},
},
{
Name: "lint",
Usage: "lint charts from state file (helm lint)",
Flags: []cli.Flag{
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
cli.StringSliceFlag{
Name: "values",
Usage: "additional value files to be merged into the command",
},
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
},
Action: func(c *cli.Context) error {
return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface) []error {
args := args.GetArgs(c.String("args"), state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
}
values := c.StringSlice("values")
if errs := state.DiffCharts(helm, values); errs != nil && len(errs) > 0 {
for _, err := range errs {
fmt.Printf("err: %s\n", err.Error())
if c.GlobalString("helm-binary") != "" {
helm.SetHelmBinary(c.GlobalString("helm-binary"))
}
os.Exit(1)
}
return nil
values := c.StringSlice("values")
workers := c.Int("concurrency")
return state.LintReleases(helm, values, workers)
})
},
},
{
Name: "sync",
Usage: "sync all resources from state file (repos && charts)",
Usage: "sync all resources from state file (repos, releases and chart deps)",
Flags: []cli.Flag{
cli.StringSliceFlag{
Name: "values",
Usage: "additional value files to be merged into the command",
},
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
},
Action: func(c *cli.Context) error {
state, helm, err := before(c)
if err != nil {
return err
}
return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface) []error {
return executeSyncCommand(c, state, helm)
})
},
},
{
Name: "apply",
Usage: "apply all resources from state file only when there are changes",
Flags: []cli.Flag{
cli.StringSliceFlag{
Name: "values",
Usage: "additional value files to be merged into the command",
},
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
cli.BoolFlag{
Name: "auto-approve",
Usage: "Skip interactive approval before applying",
},
},
Action: func(c *cli.Context) error {
return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface) []error {
errs := executeDiffCommand(c, state, helm, true)
if errs := state.SyncRepos(helm); errs != nil && len(errs) > 0 {
for _, err := range errs {
fmt.Printf("err: %s\n", err.Error())
// sync only when there are changes
if len(errs) > 0 {
allErrsIndicateChanges := true
for _, err := range errs {
switch e := err.(type) {
case *exec.ExitError:
status := e.Sys().(syscall.WaitStatus)
// `helm diff --detailed-exitcode` returns 2 when there are changes
allErrsIndicateChanges = allErrsIndicateChanges && status.ExitStatus() == 2
default:
allErrsIndicateChanges = false
}
}
msg := `Do you really want to apply?
Helmfile will apply all your changes, as shown above.
`
if allErrsIndicateChanges {
autoApprove := c.Bool("auto-approve")
if autoApprove || !autoApprove && askForConfirmation(msg) {
return executeSyncCommand(c, state, helm)
}
}
}
os.Exit(1)
}
values := c.StringSlice("values")
return errs
})
},
},
{
Name: "status",
Usage: "retrieve status of releases in state file",
Flags: []cli.Flag{
cli.IntFlag{
Name: "concurrency",
Value: 0,
Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
},
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
},
Action: func(c *cli.Context) error {
return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface) []error {
workers := c.Int("concurrency")
if errs := state.SyncCharts(helm, values); errs != nil && len(errs) > 0 {
for _, err := range errs {
fmt.Printf("err: %s\n", err.Error())
args := args.GetArgs(c.String("args"), state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
os.Exit(1)
}
return nil
if c.GlobalString("helm-binary") != "" {
helm.SetHelmBinary(c.GlobalString("helm-binary"))
}
return state.ReleaseStatuses(helm, workers)
})
},
},
{
Name: "delete",
Usage: "delete charts from state file (helm delete)",
Usage: "delete releases from state file (helm delete)",
Flags: []cli.Flag{
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass args to helm exec",
},
cli.BoolFlag{
Name: "purge",
Usage: "purge releases i.e. free release names and histories",
},
},
Action: func(c *cli.Context) error {
state, helm, err := before(c)
if err != nil {
return err
}
return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface) []error {
purge := c.Bool("purge")
if errs := state.DeleteCharts(helm); errs != nil && len(errs) > 0 {
for _, err := range errs {
fmt.Printf("err: %s\n", err.Error())
args := args.GetArgs(c.String("args"), state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
os.Exit(1)
}
return nil
if c.GlobalString("helm-binary") != "" {
helm.SetHelmBinary(c.GlobalString("helm-binary"))
}
return state.DeleteReleases(helm, purge)
})
},
},
{
Name: "test",
Usage: "test releases from state file (helm test)",
Flags: []cli.Flag{
cli.BoolFlag{
Name: "cleanup",
Usage: "delete test pods upon completion",
},
cli.StringFlag{
Name: "args",
Value: "",
Usage: "pass additional args to helm exec",
},
cli.IntFlag{
Name: "timeout",
Value: 300,
Usage: "maximum time for tests to run before being considered failed",
},
},
Action: func(c *cli.Context) error {
return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface) []error {
cleanup := c.Bool("cleanup")
timeout := c.Int("timeout")
args := args.GetArgs(c.String("args"), state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
if c.GlobalString("helm-binary") != "" {
helm.SetHelmBinary(c.GlobalString("helm-binary"))
}
return state.TestReleases(helm, cleanup, timeout)
})
},
},
}
err := app.Run(os.Args)
if err != nil {
log.Printf("err: %s", err.Error())
os.Exit(1)
fmt.Fprintf(os.Stderr, "%v\n", err)
os.Exit(3)
}
}
func before(c *cli.Context) (*state.HelmState, helmexec.Interface, error) {
file := c.GlobalString("file")
quiet := c.GlobalBool("quiet")
func executeSyncCommand(c *cli.Context, state *state.HelmState, helm helmexec.Interface) []error {
if errs := state.SyncRepos(helm); errs != nil && len(errs) > 0 {
return errs
}
if errs := state.UpdateDeps(helm); errs != nil && len(errs) > 0 {
return errs
}
args := args.GetArgs(c.String("args"), state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
if c.GlobalString("helm-binary") != "" {
helm.SetHelmBinary(c.GlobalString("helm-binary"))
}
values := c.StringSlice("values")
workers := c.Int("concurrency")
return state.SyncReleases(helm, values, workers)
}
func executeDiffCommand(c *cli.Context, state *state.HelmState, helm helmexec.Interface, detailedExitCode bool) []error {
args := args.GetArgs(c.String("args"), state)
if len(args) > 0 {
helm.SetExtraArgs(args...)
}
if c.GlobalString("helm-binary") != "" {
helm.SetHelmBinary(c.GlobalString("helm-binary"))
}
if c.Bool("sync-repos") {
if errs := state.SyncRepos(helm); errs != nil && len(errs) > 0 {
return errs
}
}
values := c.StringSlice("values")
workers := c.Int("concurrency")
return state.DiffReleases(helm, values, workers, detailedExitCode)
}
func findAndIterateOverDesiredStatesUsingFlags(c *cli.Context, converge func(*state.HelmState, helmexec.Interface) []error) error {
fileOrDir := c.GlobalString("file")
kubeContext := c.GlobalString("kube-context")
namespace := c.GlobalString("namespace")
selectors := c.GlobalStringSlice("selector")
logger := c.App.Metadata["logger"].(*zap.SugaredLogger)
return findAndIterateOverDesiredStates(fileOrDir, converge, kubeContext, namespace, selectors, logger)
}
state, err := state.ReadFromFile(file)
func findAndIterateOverDesiredStates(fileOrDir string, converge func(*state.HelmState, helmexec.Interface) []error, kubeContext, namespace string, selectors []string, logger *zap.SugaredLogger) error {
desiredStateFiles, err := findDesiredStateFiles(fileOrDir)
if err != nil {
return nil, nil, err
return err
}
allSelectorNotMatched := true
for _, f := range desiredStateFiles {
logger.Debugf("Processing %s", f)
yamlBuf, err := tmpl.NewFileRenderer(ioutil.ReadFile, "").RenderTemplateFileToBuffer(f)
if err != nil {
return err
}
state, helm, noReleases, err := loadDesiredStateFromFile(
yamlBuf.Bytes(),
f,
kubeContext,
namespace,
selectors,
logger,
)
if err != nil {
return err
}
var writer io.Writer
if !quiet {
writer = os.Stdout
if len(state.Helmfiles) > 0 {
for _, globPattern := range state.Helmfiles {
matches, err := filepath.Glob(globPattern)
if err != nil {
return fmt.Errorf("failed processing %s: %v", globPattern, err)
}
sort.Strings(matches)
for _, m := range matches {
if err := findAndIterateOverDesiredStates(m, converge, kubeContext, namespace, selectors, logger); err != nil {
return fmt.Errorf("failed processing %s: %v", globPattern, err)
}
}
}
return nil
}
allSelectorNotMatched = allSelectorNotMatched && noReleases
if noReleases {
continue
}
errs := converge(state, helm)
if err := clean(state, errs); err != nil {
return err
}
}
if allSelectorNotMatched {
logger.Error("specified selector did not match any releases in any helmfile")
os.Exit(2)
}
return nil
}
func findDesiredStateFiles(specifiedPath string) ([]string, error) {
var helmfileDir string
if specifiedPath != "" {
if fileExistsAt(specifiedPath) {
return []string{specifiedPath}, nil
} else if directoryExistsAt(specifiedPath) {
helmfileDir = specifiedPath
} else {
return []string{}, fmt.Errorf("specified state file %s is not found", specifiedPath)
}
} else {
var defaultFile string
if fileExistsAt(DefaultHelmfile) {
defaultFile = DefaultHelmfile
} else if fileExistsAt(DeprecatedHelmfile) {
log.Printf(
"warn: %s is being loaded: %s is deprecated in favor of %s. See https://github.com/roboll/helmfile/issues/25 for more information",
DeprecatedHelmfile,
DeprecatedHelmfile,
DefaultHelmfile,
)
defaultFile = DeprecatedHelmfile
}
if directoryExistsAt(DefaultHelmfileDirectory) {
if defaultFile != "" {
return []string{}, fmt.Errorf("configuration conlict error: you can have either %s or %s, but not both", defaultFile, DefaultHelmfileDirectory)
}
helmfileDir = DefaultHelmfileDirectory
} else if defaultFile != "" {
return []string{defaultFile}, nil
} else {
return []string{}, fmt.Errorf("no state file found. It must be named %s/*.yaml, %s, or %s, or otherwise specified with the --file flag", DefaultHelmfileDirectory, DefaultHelmfile, DeprecatedHelmfile)
}
}
files, err := filepath.Glob(filepath.Join(helmfileDir, "*.yaml"))
if err != nil {
return []string{}, err
}
sort.Slice(files, func(i, j int) bool {
return files[i] < files[j]
})
return files, nil
}
func fileExistsAt(path string) bool {
fileInfo, err := os.Stat(path)
return err == nil && fileInfo.Mode().IsRegular()
}
func directoryExistsAt(path string) bool {
fileInfo, err := os.Stat(path)
return err == nil && fileInfo.Mode().IsDir()
}
func loadDesiredStateFromFile(yaml []byte, file string, kubeContext, namespace string, labels []string, logger *zap.SugaredLogger) (*state.HelmState, helmexec.Interface, bool, error) {
st, err := state.CreateFromYaml(yaml, file, logger)
if err != nil {
return nil, nil, false, fmt.Errorf("failed to read %s: %v", file, err)
}
return state, helmexec.NewHelmExec(writer, kubeContext), nil
if st.Context != "" {
if kubeContext != "" {
log.Printf("err: Cannot use option --kube-context and set attribute context.")
os.Exit(1)
}
kubeContext = st.Context
}
if namespace != "" {
if st.Namespace != "" {
log.Printf("err: Cannot use option --namespace and set attribute namespace.")
os.Exit(1)
}
st.Namespace = namespace
}
if len(labels) > 0 {
err = st.FilterReleases(labels)
if err != nil {
log.Print(err)
return nil, nil, true, nil
}
}
releaseNameCounts := map[string]int{}
for _, r := range st.Releases {
releaseNameCounts[r.Name] += 1
}
for name, c := range releaseNameCounts {
if c > 1 {
return nil, nil, false, fmt.Errorf("duplicate release \"%s\" found: there were %d releases named \"%s\" matching specified selector", name, c, name)
}
}
sigs := make(chan os.Signal, 1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
go func() {
sig := <-sigs
errs := []error{fmt.Errorf("Received [%s] to shutdown ", sig)}
clean(st, errs)
}()
return st, helmexec.New(logger, kubeContext), len(st.Releases) == 0, nil
}
func clean(st *state.HelmState, errs []error) error {
if errs == nil {
errs = []error{}
}
cleanErrs := st.Clean()
if cleanErrs != nil {
errs = append(errs, cleanErrs...)
}
if errs != nil && len(errs) > 0 {
for _, err := range errs {
switch e := err.(type) {
case *state.ReleaseError:
fmt.Printf("err: release \"%s\" in \"%s\" failed: %v\n", e.Name, st.FilePath, e)
default:
fmt.Printf("err: %v\n", e)
}
}
switch e := errs[0].(type) {
case *exec.ExitError:
// Propagate any non-zero exit status from the external command like `helm` that is failed under the hood
status := e.Sys().(syscall.WaitStatus)
os.Exit(status.ExitStatus())
default:
os.Exit(1)
}
}
return nil
}
+26
View File
@@ -0,0 +1,26 @@
package main
import "testing"
// See https://github.com/roboll/helmfile/issues/193
func TestReadFromYaml_DuplicateReleaseName(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease1
chart: mychart1
labels:
stage: pre
foo: bar
- name: myrelease1
chart: mychart2
labels:
stage: post
`)
_, _, _, err := loadDesiredStateFromFile(yamlContent, yamlFile, "default", "default", []string{}, logger)
if err == nil {
t.Error("error expected but not happened")
}
if err.Error() != "duplicate release \"myrelease1\" found: there were 2 releases named \"myrelease1\" matching specified selector" {
t.Errorf("unexpected error happened: %v", err)
}
}
+69
View File
@@ -0,0 +1,69 @@
package state
import (
"fmt"
"regexp"
"strings"
)
// ReleaseFilter is used to determine if a given release should be used during helmfile execution
type ReleaseFilter interface {
// Match returns true if the ReleaseSpec matches the Filter
Match(r ReleaseSpec) bool
}
// LabelFilter matches a release with the given positive lables. Negative labels
// invert the match for cases such as tier!=backend
type LabelFilter struct {
positiveLabels [][]string
negativeLabels [][]string
}
// Match will match a release that has the same labels as the filter
func (l LabelFilter) Match(r ReleaseSpec) bool {
if len(l.positiveLabels) > 0 {
for _, element := range l.positiveLabels {
k := element[0]
v := element[1]
if rVal, ok := r.Labels[k]; !ok {
return false
} else if rVal != v {
return false
}
}
}
if len(l.negativeLabels) > 0 {
for _, element := range l.negativeLabels {
k := element[0]
v := element[1]
if rVal, ok := r.Labels[k]; !ok {
return true
} else if rVal == v {
return false
}
}
}
return true
}
// ParseLabels takes a label in the form foo=bar,baz!=bat and returns a LabelFilter that will match the labels
func ParseLabels(l string) (LabelFilter, error) {
lf := LabelFilter{}
lf.positiveLabels = [][]string{}
lf.negativeLabels = [][]string{}
var err error
labels := strings.Split(l, ",")
for _, label := range labels {
if match, _ := regexp.MatchString("^[a-zA-Z0-9_-]+!=[a-zA-Z0-9_-]+$", label); match == true { // k!=v case
kv := strings.Split(label, "!=")
lf.negativeLabels = append(lf.negativeLabels, kv)
} else if match, _ := regexp.MatchString("^[a-zA-Z0-9_-]+=[a-zA-Z0-9_-]+$", label); match == true { // k=v case
kv := strings.Split(label, "=")
lf.positiveLabels = append(lf.positiveLabels, kv)
} else { // malformed case
err = fmt.Errorf("Malformed label: %s. Expected label in form k=v or k!=v", label)
}
}
return lf, err
}
+645 -100
View File
@@ -3,76 +3,138 @@ package state
import (
"errors"
"fmt"
"github.com/roboll/helmfile/helmexec"
"io/ioutil"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"sync"
"github.com/roboll/helmfile/helmexec"
yaml "gopkg.in/yaml.v1"
"path"
"regexp"
"github.com/roboll/helmfile/valuesfile"
"go.uber.org/zap"
"gopkg.in/yaml.v2"
)
// HelmState structure for the helmfile
type HelmState struct {
BaseChartPath string
Repositories []RepositorySpec `yaml:"repositories"`
Charts []ChartSpec `yaml:"charts"`
BaseChartPath string
FilePath string
HelmDefaults HelmSpec `yaml:"helmDefaults"`
Helmfiles []string `yaml:"helmfiles"`
Context string `yaml:"context"`
DeprecatedReleases []ReleaseSpec `yaml:"charts"`
Namespace string `yaml:"namespace"`
Repositories []RepositorySpec `yaml:"repositories"`
Releases []ReleaseSpec `yaml:"releases"`
logger *zap.SugaredLogger
}
// HelmSpec to defines helmDefault values
type HelmSpec struct {
KubeContext string `yaml:"kubeContext"`
TillerNamespace string `yaml:"tillerNamespace"`
Args []string `yaml:"args"`
Verify bool `yaml:"verify"`
// Wait, if set to true, will wait until all Pods, PVCs, Services, and minimum number of Pods of a Deployment are in a ready state before marking the release as successful
Wait bool `yaml:"wait"`
// Timeout is the time in seconds to wait for any individual Kubernetes operation (like Jobs for hooks, and waits on pod/pvc/svc/deployment readiness) (default 300)
Timeout int `yaml:"timeout"`
// RecreatePods, when set to true, instruct helmfile to perform pods restart for the resource if applicable
RecreatePods bool `yaml:"recreatePods"`
// Force, when set to true, forces resource update through delete/recreate if needed
Force bool `yaml:"force"`
}
// RepositorySpec that defines values for a helm repo
type RepositorySpec struct {
Name string `yaml:"name"`
URL string `yaml:"url"`
Name string `yaml:"name"`
URL string `yaml:"url"`
CertFile string `yaml:"certFile"`
KeyFile string `yaml:"keyFile"`
Username string `yaml:"username"`
Password string `yaml:"password"`
}
type ChartSpec struct {
// ReleaseSpec defines the structure of a helm release
type ReleaseSpec struct {
// Chart is the name of the chart being installed to create this release
Chart string `yaml:"chart"`
Version string `yaml:"version"`
Verify bool `yaml:"verify"`
Verify *bool `yaml:"verify"`
// Wait, if set to true, will wait until all Pods, PVCs, Services, and minimum number of Pods of a Deployment are in a ready state before marking the release as successful
Wait *bool `yaml:"wait"`
// Timeout is the time in seconds to wait for any individual Kubernetes operation (like Jobs for hooks, and waits on pod/pvc/svc/deployment readiness) (default 300)
Timeout *int `yaml:"timeout"`
// RecreatePods, when set to true, instruct helmfile to perform pods restart for the resource if applicable
RecreatePods *bool `yaml:"recreatePods"`
// Force, when set to true, forces resource update through delete/recreate if needed
Force *bool `yaml:"force"`
Name string `yaml:"name"`
Namespace string `yaml:"namespace"`
Values []string `yaml:"values"`
SetValues []SetValue `yaml:"set"`
// Name is the name of this release
Name string `yaml:"name"`
Namespace string `yaml:"namespace"`
Labels map[string]string `yaml:"labels"`
Values []interface{} `yaml:"values"`
Secrets []string `yaml:"secrets"`
SetValues []SetValue `yaml:"set"`
// The 'env' section is not really necessary any longer, as 'set' would now provide the same functionality
EnvValues []SetValue `yaml:"env"`
// generatedValues are values that need cleaned up on exit
generatedValues []string
}
// SetValue are the key values to set on a helm release
type SetValue struct {
Name string `yaml:"name"`
Value string `yaml:"value"`
Name string `yaml:"name"`
Value string `yaml:"value"`
File string `yaml:"file"`
Values []string `yaml:"values"`
}
func ReadFromFile(file string) (*HelmState, error) {
content, err := ioutil.ReadFile(file)
if err != nil {
return nil, err
}
func CreateFromYaml(content []byte, file string, logger *zap.SugaredLogger) (*HelmState, error) {
var state HelmState
state.BaseChartPath, _ = filepath.Abs(path.Dir(file))
if err := yaml.Unmarshal(content, &state); err != nil {
state.BaseChartPath, _ = filepath.Abs(filepath.Dir(file))
if err := yaml.UnmarshalStrict(content, &state); err != nil {
return nil, err
}
state.FilePath = file
if len(state.DeprecatedReleases) > 0 {
if len(state.Releases) > 0 {
return nil, fmt.Errorf("failed to parse %s: you can't specify both `charts` and `releases` sections", file)
}
state.Releases = state.DeprecatedReleases
state.DeprecatedReleases = []ReleaseSpec{}
}
state.logger = logger
return &state, nil
}
func (state *HelmState) applyDefaultsTo(spec *ReleaseSpec) {
if state.Namespace != "" {
spec.Namespace = state.Namespace
}
}
// SyncRepos will update the given helm releases
func (state *HelmState) SyncRepos(helm helmexec.Interface) []error {
var wg sync.WaitGroup
errs := []error{}
for _, repo := range state.Repositories {
wg.Add(1)
go func(wg *sync.WaitGroup, name, url string) {
if err := helm.AddRepo(name, url); err != nil {
errs = append(errs, err)
}
wg.Done()
}(&wg, repo.Name, repo.URL)
if err := helm.AddRepo(repo.Name, repo.URL, repo.CertFile, repo.KeyFile, repo.Username, repo.Password); err != nil {
errs = append(errs, err)
}
}
wg.Wait()
if len(errs) != 0 {
return errs
@@ -84,33 +146,80 @@ func (state *HelmState) SyncRepos(helm helmexec.Interface) []error {
return nil
}
func (state *HelmState) SyncCharts(helm helmexec.Interface, additonalValues []string) []error {
var wg sync.WaitGroup
errs := []error{}
type ReleaseError struct {
*ReleaseSpec
underlying error
}
for _, chart := range state.Charts {
wg.Add(1)
go func(wg *sync.WaitGroup, chart ChartSpec) {
flags, flagsErr := flagsForChart(state.BaseChartPath, &chart)
if flagsErr != nil {
errs = append(errs, flagsErr)
}
for _, value := range additonalValues {
valfile, err := filepath.Abs(value)
if err != nil {
errs = append(errs, err)
func (e *ReleaseError) Error() string {
return e.underlying.Error()
}
// SyncReleases wrapper for executing helm upgrade on the releases
func (state *HelmState) SyncReleases(helm helmexec.Interface, additionalValues []string, workerLimit int) []error {
errs := []error{}
jobQueue := make(chan *ReleaseSpec)
doneQueue := make(chan bool)
errQueue := make(chan *ReleaseError)
if workerLimit < 1 {
workerLimit = len(state.Releases)
}
for w := 1; w <= workerLimit; w++ {
go func() {
for release := range jobQueue {
state.applyDefaultsTo(release)
flags, flagsErr := state.flagsForUpgrade(helm, release)
if flagsErr != nil {
errQueue <- &ReleaseError{release, flagsErr}
doneQueue <- true
continue
}
haveValueErr := false
for _, value := range additionalValues {
valfile, err := filepath.Abs(value)
if err != nil {
errQueue <- &ReleaseError{release, err}
haveValueErr = true
}
if _, err := os.Stat(valfile); os.IsNotExist(err) {
errQueue <- &ReleaseError{release, err}
haveValueErr = true
}
flags = append(flags, "--values", valfile)
}
if haveValueErr {
doneQueue <- true
continue
}
flags = append(flags, "--values", valfile)
}
if len(errs) == 0 {
if err := helm.SyncChart(chart.Name, normalizeChart(state.BaseChartPath, chart.Chart), flags...); err != nil {
errs = append(errs, err)
chart := normalizeChart(state.BaseChartPath, release.Chart)
if err := helm.SyncRelease(release.Name, chart, flags...); err != nil {
errQueue <- &ReleaseError{release, err}
}
doneQueue <- true
}
wg.Done()
}(&wg, chart)
}()
}
go func() {
for i := 0; i < len(state.Releases); i++ {
jobQueue <- &state.Releases[i]
}
close(jobQueue)
}()
for i := 0; i < len(state.Releases); {
select {
case err := <-errQueue:
errs = append(errs, err)
case <-doneQueue:
i++
}
}
wg.Wait()
if len(errs) != 0 {
return errs
@@ -119,35 +228,77 @@ func (state *HelmState) SyncCharts(helm helmexec.Interface, additonalValues []st
return nil
}
func (state *HelmState) DiffCharts(helm helmexec.Interface, additonalValues []string) []error {
var wg sync.WaitGroup
// DiffReleases wrapper for executing helm diff on the releases
func (state *HelmState) DiffReleases(helm helmexec.Interface, additionalValues []string, workerLimit int, detailedExitCode bool) []error {
var wgRelease sync.WaitGroup
var wgError sync.WaitGroup
errs := []error{}
jobQueue := make(chan *ReleaseSpec, len(state.Releases))
errQueue := make(chan error)
for _, chart := range state.Charts {
wg.Add(1)
go func(wg *sync.WaitGroup, chart ChartSpec) {
// Plugin command doesn't support explicit namespace
chart.Namespace = ""
flags, flagsErr := flagsForChart(state.BaseChartPath, &chart)
if flagsErr != nil {
errs = append(errs, flagsErr)
}
for _, value := range additonalValues {
valfile, err := filepath.Abs(value)
if workerLimit < 1 {
workerLimit = len(state.Releases)
}
wgRelease.Add(len(state.Releases))
for w := 1; w <= workerLimit; w++ {
go func() {
for release := range jobQueue {
errs := []error{}
state.applyDefaultsTo(release)
flags, err := state.flagsForDiff(helm, state.BaseChartPath, release)
if err != nil {
errs = append(errs, err)
}
flags = append(flags, "--values", valfile)
}
if len(errs) == 0 {
if err := helm.DiffChart(chart.Name, normalizeChart(state.BaseChartPath, chart.Chart), flags...); err != nil {
errs = append(errs, err)
for _, value := range additionalValues {
valfile, err := filepath.Abs(value)
if err != nil {
errs = append(errs, err)
}
if _, err := os.Stat(valfile); os.IsNotExist(err) {
errs = append(errs, err)
}
flags = append(flags, "--values", valfile)
}
if detailedExitCode {
flags = append(flags, "--detailed-exitcode")
}
if len(errs) == 0 {
if err := helm.DiffRelease(release.Name, normalizeChart(state.BaseChartPath, release.Chart), flags...); err != nil {
errs = append(errs, err)
}
}
for _, err := range errs {
errQueue <- err
}
wgRelease.Done()
}
wg.Done()
}(&wg, chart)
}()
}
wg.Wait()
wgError.Add(1)
go func() {
for err := range errQueue {
errs = append(errs, err)
}
wgError.Done()
}()
for i := 0; i < len(state.Releases); i++ {
jobQueue <- &state.Releases[i]
}
close(jobQueue)
wgRelease.Wait()
close(errQueue)
wgError.Wait()
if len(errs) != 0 {
return errs
@@ -156,18 +307,167 @@ func (state *HelmState) DiffCharts(helm helmexec.Interface, additonalValues []st
return nil
}
func (state *HelmState) DeleteCharts(helm helmexec.Interface) []error {
// LintReleases wrapper for executing helm lint on the releases
func (state *HelmState) LintReleases(helm helmexec.Interface, additionalValues []string, workerLimit int) []error {
var wgRelease sync.WaitGroup
var wgError sync.WaitGroup
errs := []error{}
jobQueue := make(chan *ReleaseSpec, len(state.Releases))
errQueue := make(chan error)
if workerLimit < 1 {
workerLimit = len(state.Releases)
}
wgRelease.Add(len(state.Releases))
// Create tmp directory and bail immediately if it fails
dir, err := ioutil.TempDir("", "")
if err != nil {
errs = append(errs, err)
return errs
}
defer os.RemoveAll(dir)
for w := 1; w <= workerLimit; w++ {
go func() {
for release := range jobQueue {
errs := []error{}
flags, err := state.flagsForLint(helm, state.BaseChartPath, release)
if err != nil {
errs = append(errs, err)
}
for _, value := range additionalValues {
valfile, err := filepath.Abs(value)
if err != nil {
errs = append(errs, err)
}
if _, err := os.Stat(valfile); os.IsNotExist(err) {
errs = append(errs, err)
}
flags = append(flags, "--values", valfile)
}
chartPath := ""
if pathExists(normalizeChart(state.BaseChartPath, release.Chart)) {
chartPath = normalizeChart(state.BaseChartPath, release.Chart)
} else {
fetchFlags := []string{}
if release.Version != "" {
chartPath = path.Join(dir, release.Name, release.Version, release.Chart)
fetchFlags = append(fetchFlags, "--version", release.Version)
} else {
chartPath = path.Join(dir, release.Name, "latest", release.Chart)
}
// only fetch chart if it is not already fetched
if _, err := os.Stat(chartPath); os.IsNotExist(err) {
fetchFlags = append(fetchFlags, "--untar", "--untardir", chartPath)
if err := helm.Fetch(release.Chart, fetchFlags...); err != nil {
errs = append(errs, err)
}
}
chartPath = path.Join(chartPath, chartNameWithoutRepository(release.Chart))
}
if len(errs) == 0 {
if err := helm.Lint(chartPath, flags...); err != nil {
errs = append(errs, err)
}
}
for _, err := range errs {
errQueue <- err
}
wgRelease.Done()
}
}()
}
wgError.Add(1)
go func() {
for err := range errQueue {
errs = append(errs, err)
}
wgError.Done()
}()
for i := 0; i < len(state.Releases); i++ {
jobQueue <- &state.Releases[i]
}
close(jobQueue)
wgRelease.Wait()
close(errQueue)
wgError.Wait()
if len(errs) != 0 {
return errs
}
return nil
}
func (state *HelmState) ReleaseStatuses(helm helmexec.Interface, workerLimit int) []error {
var errs []error
jobQueue := make(chan ReleaseSpec)
doneQueue := make(chan bool)
errQueue := make(chan error)
if workerLimit < 1 {
workerLimit = len(state.Releases)
}
for w := 1; w <= workerLimit; w++ {
go func() {
for release := range jobQueue {
if err := helm.ReleaseStatus(release.Name); err != nil {
errQueue <- err
}
doneQueue <- true
}
}()
}
go func() {
for _, release := range state.Releases {
jobQueue <- release
}
close(jobQueue)
}()
for i := 0; i < len(state.Releases); {
select {
case err := <-errQueue:
errs = append(errs, err)
case <-doneQueue:
i++
}
}
if len(errs) != 0 {
return errs
}
return nil
}
// DeleteReleases wrapper for executing helm delete on the releases
func (state *HelmState) DeleteReleases(helm helmexec.Interface, purge bool) []error {
var wg sync.WaitGroup
errs := []error{}
for _, chart := range state.Charts {
for _, release := range state.Releases {
wg.Add(1)
go func(wg *sync.WaitGroup, chart ChartSpec) {
if err := helm.DeleteChart(chart.Name); err != nil {
go func(wg *sync.WaitGroup, release ReleaseSpec) {
flags := []string{}
if purge {
flags = append(flags, "--purge")
}
if err := helm.DeleteRelease(release.Name, flags...); err != nil {
errs = append(errs, err)
}
wg.Done()
}(&wg, chart)
}(&wg, release)
}
wg.Wait()
@@ -178,47 +478,282 @@ func (state *HelmState) DeleteCharts(helm helmexec.Interface) []error {
return nil
}
// TestReleases wrapper for executing helm test on the releases
func (state *HelmState) TestReleases(helm helmexec.Interface, cleanup bool, timeout int) []error {
var wg sync.WaitGroup
errs := []error{}
for _, release := range state.Releases {
wg.Add(1)
go func(wg *sync.WaitGroup, release ReleaseSpec) {
flags := []string{}
if cleanup {
flags = append(flags, "--cleanup")
}
flags = append(flags, "--timeout", strconv.Itoa(timeout))
if err := helm.TestRelease(release.Name, flags...); err != nil {
errs = append(errs, err)
}
wg.Done()
}(&wg, release)
}
wg.Wait()
if len(errs) != 0 {
return errs
}
return nil
}
// Clean will remove any generated secrets
func (state *HelmState) Clean() []error {
errs := []error{}
for _, release := range state.Releases {
for _, value := range release.generatedValues {
err := os.Remove(value)
if err != nil {
errs = append(errs, err)
}
}
}
if len(errs) != 0 {
return errs
}
return nil
}
// FilterReleases allows for the execution of helm commands against a subset of the releases in the helmfile.
func (state *HelmState) FilterReleases(labels []string) error {
var filteredReleases []ReleaseSpec
releaseSet := map[string]ReleaseSpec{}
filters := []ReleaseFilter{}
for _, label := range labels {
f, err := ParseLabels(label)
if err != nil {
return err
}
filters = append(filters, f)
}
for _, r := range state.Releases {
if r.Labels == nil {
r.Labels = map[string]string{}
}
// Let the release name be used as a tag
r.Labels["name"] = r.Name
for _, f := range filters {
if r.Labels == nil {
r.Labels = map[string]string{}
}
if f.Match(r) {
releaseSet[r.Name] = r
continue
}
}
}
for _, r := range releaseSet {
filteredReleases = append(filteredReleases, r)
}
state.Releases = filteredReleases
if len(filteredReleases) == 0 {
state.logger.Debugf("specified selector did not match any releases in %s\n", state.FilePath)
return nil
}
return nil
}
// UpdateDeps wrapper for updating dependencies on the releases
func (state *HelmState) UpdateDeps(helm helmexec.Interface) []error {
errs := []error{}
for _, release := range state.Releases {
if isLocalChart(release.Chart) {
if err := helm.UpdateDeps(normalizeChart(state.BaseChartPath, release.Chart)); err != nil {
errs = append(errs, err)
}
}
}
if len(errs) != 0 {
return errs
}
return nil
}
// normalizeChart allows for the distinction between a file path reference and repository references.
// - Any single (or double character) followed by a `/` will be considered a local file reference and
// be constructed relative to the `base path`.
// - Everything else is assumed to be an absolute path or an actual <repository>/<chart> reference.
func normalizeChart(basePath, chart string) string {
regex, _ := regexp.Compile("^[.]?./")
if !regex.MatchString(chart) {
if !isLocalChart(chart) {
return chart
}
return filepath.Join(basePath, chart)
}
func flagsForChart(basePath string, chart *ChartSpec) ([]string, error) {
func isLocalChart(chart string) bool {
regex, _ := regexp.Compile("^[.]?./")
return regex.MatchString(chart)
}
func pathExists(chart string) bool {
_, err := os.Stat(chart)
return err == nil
}
func chartNameWithoutRepository(chart string) string {
chartSplit := strings.Split(chart, "/")
return chartSplit[len(chartSplit)-1]
}
func (state *HelmState) flagsForUpgrade(helm helmexec.Interface, release *ReleaseSpec) ([]string, error) {
flags := []string{}
if chart.Version != "" {
flags = append(flags, "--version", chart.Version)
if release.Version != "" {
flags = append(flags, "--version", release.Version)
}
if chart.Verify {
if release.Verify != nil && *release.Verify || state.HelmDefaults.Verify {
flags = append(flags, "--verify")
}
if chart.Namespace != "" {
flags = append(flags, "--namespace", chart.Namespace)
if release.Wait != nil && *release.Wait || state.HelmDefaults.Wait {
flags = append(flags, "--wait")
}
for _, value := range chart.Values {
valfile := filepath.Join(basePath, value)
timeout := state.HelmDefaults.Timeout
if release.Timeout != nil {
timeout = *release.Timeout
}
if timeout != 0 {
flags = append(flags, "--timeout", fmt.Sprintf("%d", timeout))
}
if release.Force != nil && *release.Force || state.HelmDefaults.Force {
flags = append(flags, "--force")
}
if release.RecreatePods != nil && *release.RecreatePods || state.HelmDefaults.RecreatePods {
flags = append(flags, "--recreate-pods")
}
common, err := state.namespaceAndValuesFlags(helm, state.BaseChartPath, release)
if err != nil {
return nil, err
}
return append(flags, common...), nil
}
func (state *HelmState) flagsForDiff(helm helmexec.Interface, basePath string, release *ReleaseSpec) ([]string, error) {
flags := []string{}
if release.Version != "" {
flags = append(flags, "--version", release.Version)
}
common, err := state.namespaceAndValuesFlags(helm, basePath, release)
if err != nil {
return nil, err
}
return append(flags, common...), nil
}
func (state *HelmState) flagsForLint(helm helmexec.Interface, basePath string, release *ReleaseSpec) ([]string, error) {
return state.namespaceAndValuesFlags(helm, basePath, release)
}
func (state *HelmState) namespaceAndValuesFlags(helm helmexec.Interface, basePath string, release *ReleaseSpec) ([]string, error) {
flags := []string{}
if release.Namespace != "" {
flags = append(flags, "--namespace", release.Namespace)
}
for _, value := range release.Values {
switch typedValue := value.(type) {
case string:
var path string
if filepath.IsAbs(typedValue) {
path = typedValue
} else {
path = filepath.Join(basePath, typedValue)
}
if _, err := os.Stat(path); os.IsNotExist(err) {
return nil, err
}
valfile, err := ioutil.TempFile("", "values")
if err != nil {
return nil, err
}
defer valfile.Close()
r := valuesfile.NewRenderer(ioutil.ReadFile, state.BaseChartPath)
yamlBytes, err := r.RenderToBytes(path)
if err != nil {
return nil, err
}
if _, err := valfile.Write(yamlBytes); err != nil {
return nil, fmt.Errorf("failed to write %s: %v", valfile.Name(), err)
}
state.logger.Debugf("successfully generated the value file at %s. produced:\n%s", path, string(yamlBytes))
flags = append(flags, "--values", valfile.Name())
case map[interface{}]interface{}:
valfile, err := ioutil.TempFile("", "values")
if err != nil {
return nil, err
}
defer valfile.Close()
encoder := yaml.NewEncoder(valfile)
defer encoder.Close()
if err := encoder.Encode(typedValue); err != nil {
return nil, err
}
release.generatedValues = append(release.generatedValues, valfile.Name())
flags = append(flags, "--values", valfile.Name())
}
}
for _, value := range release.Secrets {
path := filepath.Join(basePath, value)
if _, err := os.Stat(path); os.IsNotExist(err) {
return nil, err
}
valfile, err := helm.DecryptSecret(path)
if err != nil {
return nil, err
}
release.generatedValues = append(release.generatedValues, valfile)
flags = append(flags, "--values", valfile)
}
if len(chart.SetValues) > 0 {
val := []string{}
for _, set := range chart.SetValues {
val = append(val, fmt.Sprintf("%s=%s", set.Name, set.Value))
if len(release.SetValues) > 0 {
for _, set := range release.SetValues {
if set.Value != "" {
flags = append(flags, "--set", fmt.Sprintf("%s=%s", escape(set.Name), escape(set.Value)))
} else if set.File != "" {
flags = append(flags, "--set-file", fmt.Sprintf("%s=%s", escape(set.Name), set.File))
} else if len(set.Values) > 0 {
items := make([]string, len(set.Values))
for i, raw := range set.Values {
items[i] = escape(raw)
}
v := strings.Join(items, ",")
flags = append(flags, "--set", fmt.Sprintf("%s={%s}", escape(set.Name), v))
}
}
flags = append(flags, "--set", strings.Join(val, ","))
}
if len(chart.EnvValues) > 0 {
/***********
* START 'env' section for backwards compatibility
***********/
// The 'env' section is not really necessary any longer, as 'set' would now provide the same functionality
if len(release.EnvValues) > 0 {
val := []string{}
envValErrs := []string{}
for _, set := range chart.EnvValues {
for _, set := range release.EnvValues {
value, isSet := os.LookupEnv(set.Value)
if isSet {
val = append(val, fmt.Sprintf("%s=%s", set.Name, value))
val = append(val, fmt.Sprintf("%s=%s", escape(set.Name), escape(value)))
} else {
errMsg := fmt.Sprintf("\t%s", set.Value)
envValErrs = append(envValErrs, errMsg)
@@ -231,5 +766,15 @@ func flagsForChart(basePath string, chart *ChartSpec) ([]string, error) {
}
flags = append(flags, "--set", strings.Join(val, ","))
}
/**************
* END 'env' section for backwards compatibility
**************/
return flags, nil
}
func escape(value string) string {
intermediate := strings.Replace(value, "{", "\\{", -1)
intermediate = strings.Replace(intermediate, "}", "\\}", -1)
return strings.Replace(intermediate, ",", "\\,", -1)
}
+1049
View File
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,21 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*~
# Various IDEs
.project
.idea/
*.tmproj
@@ -0,0 +1,5 @@
apiVersion: v1
description: HTTP Request & Response Service, written in Python + Flask. https://httpbin.org
name: httpbin
version: 0.1.0
@@ -0,0 +1,16 @@
{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "httpbin.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
*/}}
{{- define "httpbin.fullname" -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
@@ -0,0 +1,38 @@
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
metadata:
name: {{ template "httpbin.fullname" . }}
labels:
app: {{ template "httpbin.name" . }}
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
replicas: 1
strategy: {}
template:
metadata:
labels:
app: {{ template "httpbin.name" . }}
release: {{ .Release.Name }}
spec:
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
livenessProbe:
httpGet:
path: /
port: {{ .Values.service.internalPort }}
readinessProbe:
httpGet:
path: /
port: {{ .Values.service.internalPort }}
ports:
- containerPort: {{ .Values.service.internalPort }}
resources:
{{ toYaml .Values.resources | indent 10 }}
status: {}
---
@@ -0,0 +1,33 @@
{{- if .Values.ingress.enabled -}}
{{- $serviceName := include "httpbin.fullname" . -}}
{{- $servicePort := .Values.service.externalPort -}}
{{- $prefix := .Values.ingress.prefix -}}
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: {{ template "httpbin.fullname" . }}
labels:
app: {{ template "httpbin.name" . }}
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
annotations:
{{- range $key, $value := .Values.ingress.annotations }}
{{ $key }}: {{ $value | quote }}
{{- end }}
spec:
rules:
{{- range $host := .Values.ingress.hosts }}
- host: {{ $host }}
http:
paths:
- path: /
backend:
serviceName: {{ $serviceName }}
servicePort: {{ $servicePort }}
{{- end -}}
{{- if .Values.ingress.tls }}
tls:
{{ toYaml .Values.ingress.tls | indent 4 }}
{{- end -}}
{{- end -}}
@@ -0,0 +1,19 @@
apiVersion: v1
kind: Service
metadata:
name: {{ template "httpbin.fullname" . }}
labels:
app: {{ template "httpbin.name" . }}
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.externalPort }}
targetPort: {{ .Values.service.internalPort }}
protocol: TCP
name: {{ .Values.service.name }}
selector:
app: {{ template "httpbin.name" . }}
release: {{ .Release.Name }}
@@ -0,0 +1,14 @@
image:
repository: docker.io/citizenstig/httpbin
tag: latest
pullPolicy: IfNotPresent
service:
name: httpbin
type: LoadBalancer
externalPort: 8000
internalPort: 8000
ingress:
enabled: false
hosts:
- httpbin.local
resources: {}
+13
View File
@@ -0,0 +1,13 @@
repositories:
- name: stable
url: https://kubernetes-charts.storage.googleapis.com/
context: minikube
releases:
- name: httpbin
chart: ./charts/httpbin
set:
- name: ingress.enabled
value: false
+28
View File
@@ -0,0 +1,28 @@
#!/usr/bin/env bash
# Check environment is correctly setup
if ! hash minikube 2>/dev/null; then
fail "Minikube needs to be installed."
fi
if [ ! $(minikube status --format '{{.MinikubeStatus}}') == "Running" ]; then
fail "Minikube is not running."
fi
if [ ! $(minikube status --format '{{.ClusterStatus}}') == "Running" ]; then
fail "Minikube Cluster is not running."
fi
if ! kubectl version --short 1> /dev/null; then
fail "Could not connect to minikube apiserver"
fi
if ! hash curl 1>/dev/null; then
fail "curl needs to be installed."
fi
if ! hash docker 1>/dev/null; then
fail "Docker needs to be installed."
fi
if ! docker version 1> /dev/null; then
fail "Could not connect to Docker daemon"
fi
if ! hash helm 1>/dev/null; then
fail "Helm needs to be installed."
fi
+24
View File
@@ -0,0 +1,24 @@
#!/usr/bin/env bash
declare -i tests_total=0
function info () {
tput bold; tput setaf 4; echo -n "INFO: "; tput sgr0; echo "${@}"
}
function warn () {
tput bold; tput setaf 3; echo -n "WARN: "; tput sgr0; echo "${@}"
}
function fail () {
tput bold; tput setaf 1; echo -n "FAIL: "; tput sgr0; echo "${@}"
exit 1
}
function test_start () {
tput bold; tput setaf 6; echo -n "TEST: "; tput sgr0; echo "${@}"
}
function test_pass () {
tests_total=$((tests_total+1))
tput bold; tput setaf 2; echo -n "PASS: "; tput sgr0; echo "${@}"
}
function all_tests_passed () {
tput bold; tput setaf 2; echo -n "PASS: "; tput sgr0; echo "${tests_total} tests passed"
}
+71
View File
@@ -0,0 +1,71 @@
#!/usr/bin/env bash
# IMPORTS -----------------------------------------------------------------------------------------------------------
# determine working directory to use to relative paths irrespective of starting directory
dir="${BASH_SOURCE%/*}"
if [[ ! -d "${dir}" ]]; then dir="${PWD}"; fi
. "${dir}/lib/output.sh"
. "${dir}/lib/ensure.sh"
# GLOBALS -----------------------------------------------------------------------------------------------------------
test_ns="helmfile-tests-$(date +"%Y%m%d-%H%M%S")"
helmfile="./helmfile --namespace=${test_ns}"
helm="helm --kube-context=minikube"
kubectl="kubectl --context=minikube --namespace=${test_ns}"
# FUNCTIONS ----------------------------------------------------------------------------------------------------------
function wait_deploy_ready() {
${kubectl} rollout status deployment ${1}
while [ "$(${kubectl} get deploy ${1} -o=jsonpath='{.status.readyReplicas}')" == "0" ]; do
info "Waiting for deployment ${1} to be ready"
sleep 1
done
}
function retry() {
local -r max=${1}
local -r command=${2}
n=0
retry_result=0
until [ ${n} -ge ${max} ]; do
info "Executing: ${command} (attempt $((n+1)))"
${command} && break # substitute your command here
retry_result=$?
n=$[$n+1]
# approximated binary exponential backoff to reduce flakiness
sleep $((n ** 2))
done
}
# SETUP --------------------------------------------------------------------------------------------------------------
set -e
info "Using namespace: ${test_ns}"
info "Using Helm version: $(helm version --short --client | grep -o v.*$)"
${helm} init --wait --override spec.template.spec.automountServiceAccountToken=true
${kubectl} get namespace ${test_ns} &> /dev/null && warn "Namespace ${test_ns} exists, from a previous test run?"
$kubectl create namespace ${test_ns} || fail "Could not create namespace ${test_ns}"
trap "{ $kubectl delete namespace ${test_ns}; }" EXIT # remove namespace whenever we exit this script
# TEST CASES----------------------------------------------------------------------------------------------------------
test_start "happypath - simple rollout of httpbin chart"
info "Syncing ${dir}/happypath.yaml"
${helmfile} -f ${dir}/happypath.yaml sync
wait_deploy_ready httpbin-httpbin
retry 5 "curl --fail $(minikube service --url --namespace=${test_ns} httpbin-httpbin)/status/200"
[ ${retry_result} -eq 0 ] || fail "httpbin failed to return 200 OK"
info "Deleting release"
${helmfile} -f ${dir}/happypath.yaml delete
${helm} status --namespace=${test_ns} httpbin &> /dev/null && fail "release should not exist anymore after a delete"
test_pass "happypath"
# ALL DONE -----------------------------------------------------------------------------------------------------------
all_tests_passed
+6
View File
@@ -0,0 +1,6 @@
package tmpl
type Context struct {
basePath string
readFile func(string) ([]byte, error)
}
+33
View File
@@ -0,0 +1,33 @@
package tmpl
import (
"bytes"
)
type templateFileRenderer struct {
ReadFile func(string) ([]byte, error)
Context *Context
}
type FileRenderer interface {
RenderTemplateFileToBuffer(file string) (*bytes.Buffer, error)
}
func NewFileRenderer(readFile func(filename string) ([]byte, error), basePath string) *templateFileRenderer {
return &templateFileRenderer{
ReadFile: readFile,
Context: &Context{
basePath: basePath,
readFile: readFile,
},
}
}
func (r *templateFileRenderer) RenderTemplateFileToBuffer(file string) (*bytes.Buffer, error) {
content, err := r.ReadFile(file)
if err != nil {
return nil, err
}
return r.Context.RenderTemplateToBuffer(string(content))
}
+102
View File
@@ -0,0 +1,102 @@
package tmpl
import (
"fmt"
"gopkg.in/yaml.v2"
"os"
"path/filepath"
"strings"
"text/template"
)
type Values = map[string]interface{}
func (c *Context) createFuncMap() template.FuncMap {
return template.FuncMap{
"readFile": c.ReadFile,
"toYaml": ToYaml,
"fromYaml": FromYaml,
"setValueAtPath": SetValueAtPath,
"requiredEnv": RequiredEnv,
}
}
func (c *Context) ReadFile(filename string) (string, error) {
path := filepath.Join(c.basePath, filename)
bytes, err := c.readFile(path)
if err != nil {
return "", err
}
return string(bytes), nil
}
func ToYaml(v interface{}) (string, error) {
data, err := yaml.Marshal(v)
if err != nil {
return "", err
}
return string(data), nil
}
func FromYaml(str string) (Values, error) {
m := Values{}
if err := yaml.Unmarshal([]byte(str), &m); err != nil {
return nil, err
}
return m, nil
}
func SetValueAtPath(path string, value interface{}, values Values) (Values, error) {
var current interface{}
current = values
components := strings.Split(path, ".")
pathToMap := components[:len(components)-1]
key := components[len(components)-1]
for _, k := range pathToMap {
var elem interface{}
switch typedCurrent := current.(type) {
case map[string]interface{}:
v, exists := typedCurrent[k]
if !exists {
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" does not exist", path, k)
}
elem = v
case map[interface{}]interface{}:
v, exists := typedCurrent[k]
if !exists {
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" does not exist", path, k)
}
elem = v
default:
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" was not a map", path, k)
}
switch typedElem := elem.(type) {
case map[string]interface{}, map[interface{}]interface{}:
current = typedElem
default:
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" was not a map", path, k)
}
}
switch typedCurrent := current.(type) {
case map[string]interface{}:
typedCurrent[key] = value
case map[interface{}]interface{}:
typedCurrent[key] = value
default:
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" was not a map", path, key)
}
return values, nil
}
func RequiredEnv(name string) (string, error) {
if val, exists := os.LookupEnv(name); exists && len(val) > 0 {
return val, nil
}
return "", fmt.Errorf("required env var `%s` is not set", name)
}
+99
View File
@@ -0,0 +1,99 @@
package tmpl
import (
"fmt"
"reflect"
"testing"
)
func TestReadFile(t *testing.T) {
expected := `foo:
bar: BAR
`
expectedFilename := "values.yaml"
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(expected), nil
}}
actual, err := ctx.ReadFile(expectedFilename)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestToYaml(t *testing.T) {
expected := `foo:
bar: BAR
`
vals := Values(map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "BAR",
},
})
actual, err := ToYaml(vals)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestFromYaml(t *testing.T) {
raw := `foo:
bar: BAR
`
expected := Values(map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "BAR",
},
})
actual, err := FromYaml(raw)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestSetValueAtPath_OneComponent(t *testing.T) {
input := map[string]interface{}{
"foo": "",
}
expected := map[string]interface{}{
"foo": "FOO",
}
actual, err := SetValueAtPath("foo", "FOO", input)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestSetValueAtPath_TwoComponents(t *testing.T) {
input := map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "",
},
}
expected := map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "FOO_BAR",
},
}
actual, err := SetValueAtPath("foo.bar", "FOO_BAR", input)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
+31
View File
@@ -0,0 +1,31 @@
package tmpl
import (
"bytes"
"github.com/Masterminds/sprig"
"text/template"
)
func (c *Context) stringTemplate() *template.Template {
funcMap := sprig.TxtFuncMap()
for name, f := range c.createFuncMap() {
funcMap[name] = f
}
return template.New("stringTemplate").Funcs(funcMap)
}
func (c *Context) RenderTemplateToBuffer(s string) (*bytes.Buffer, error) {
var t, parseErr = c.stringTemplate().Parse(s)
if parseErr != nil {
return nil, parseErr
}
var tplString bytes.Buffer
var execErr = t.Execute(&tplString, nil)
if execErr != nil {
return nil, execErr
}
return &tplString, nil
}
+159
View File
@@ -0,0 +1,159 @@
package tmpl
import (
"fmt"
"os"
"reflect"
"testing"
)
func TestRenderTemplate_Values(t *testing.T) {
valuesYamlContent := `foo:
bar: BAR
`
expected := `foo:
bar: FOO_BAR
`
expectedFilename := "values.yaml"
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(valuesYamlContent), nil
}}
buf, err := ctx.RenderTemplateToBuffer(`{{ readFile "values.yaml" | fromYaml | setValueAtPath "foo.bar" "FOO_BAR" | toYaml }}`)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := buf.String()
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func renderTemplateToString(s string) (string, error) {
ctx := &Context{readFile: func(filename string) ([]byte, error) {
return nil, fmt.Errorf("unexpected call to readFile: filename=%s", filename)
}}
tplString, err := ctx.RenderTemplateToBuffer(s)
if err != nil {
return "", err
}
return tplString.String(), nil
}
func Test_renderTemplateToString(t *testing.T) {
type args struct {
s string
envs map[string]string
}
tests := []struct {
name string
args args
want string
wantErr bool
}{
{
name: "simple replacement",
args: args{
s: "{{ env \"HF_TEST_VAR\" }}",
envs: map[string]string{
"HF_TEST_VAR": "content",
},
},
want: "content",
wantErr: false,
},
{
name: "two replacements",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" }}{{ env \"HF_TEST_BETA\" }}",
envs: map[string]string{
"HF_TEST_ALPHA": "first",
"HF_TEST_BETA": "second",
},
},
want: "firstsecond",
wantErr: false,
},
{
name: "replacement and comment",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" }}{{/* comment */}}",
envs: map[string]string{
"HF_TEST_ALPHA": "first",
},
},
want: "first",
wantErr: false,
},
{
name: "global template function",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" | len }}",
envs: map[string]string{
"HF_TEST_ALPHA": "abcdefg",
},
},
want: "7",
wantErr: false,
},
{
name: "env var not set",
args: args{
s: "{{ env \"HF_TEST_NONE\" }}",
envs: map[string]string{
"HF_TEST_THIS": "first",
},
},
want: "",
},
{
name: "undefined function",
args: args{
s: "{{ env foo }}",
envs: map[string]string{
"foo": "bar",
},
},
wantErr: true,
},
{
name: "required env var",
args: args{
s: "{{ requiredEnv \"HF_TEST\" }}",
envs: map[string]string{
"HF_TEST": "value",
},
},
want: "value",
wantErr: false,
},
{
name: "required env var not set",
args: args{
s: "{{ requiredEnv \"HF_TEST_NONE\" }}",
envs: map[string]string{},
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
for k, v := range tt.args.envs {
err := os.Setenv(k, v)
if err != nil {
t.Error("renderTemplateToString() could not set env var for testing")
}
}
got, err := renderTemplateToString(tt.args.s)
if (err != nil) != tt.wantErr {
t.Errorf("renderTemplateToString() for %s error = %v, wantErr %v", tt.name, err, tt.wantErr)
return
}
if got != tt.want {
t.Errorf("renderTemplateToString() for %s = %v, want %v", tt.name, got, tt.want)
}
})
}
}
+39
View File
@@ -0,0 +1,39 @@
package valuesfile
import (
"fmt"
"github.com/roboll/helmfile/tmpl"
"strings"
)
type renderer struct {
readFile func(string) ([]byte, error)
tmplFileRenderer tmpl.FileRenderer
}
func NewRenderer(readFile func(filename string) ([]byte, error), basePath string) *renderer {
return &renderer{
readFile: readFile,
tmplFileRenderer: tmpl.NewFileRenderer(readFile, basePath),
}
}
func (r *renderer) RenderToBytes(path string) ([]byte, error) {
var yamlBytes []byte
splits := strings.Split(path, ".")
if len(splits) > 0 && splits[len(splits)-1] == "gotmpl" {
yamlBuf, err := r.tmplFileRenderer.RenderTemplateFileToBuffer(path)
if err != nil {
return nil, fmt.Errorf("failed to render [%s], because of %v", path, err)
}
yamlBytes = yamlBuf.Bytes()
} else {
var err error
yamlBytes, err = r.readFile(path)
if err != nil {
return nil, fmt.Errorf("failed to load [%s]: %v", path, err)
}
}
return yamlBytes, nil
}
+61
View File
@@ -0,0 +1,61 @@
package valuesfile
import (
"fmt"
"reflect"
"testing"
)
func TestRenderToBytes_Gotmpl(t *testing.T) {
valuesYamlTmplContent := `foo:
bar: '{{ readFile "data.txt" }}'
`
dataFileContent := "FOO_BAR"
expected := `foo:
bar: 'FOO_BAR'
`
dataFile := "data.txt"
valuesTmplFile := "values.yaml.gotmpl"
r := NewRenderer(func(filename string) ([]byte, error) {
switch filename {
case valuesTmplFile:
return []byte(valuesYamlTmplContent), nil
case dataFile:
return []byte(dataFileContent), nil
}
return nil, fmt.Errorf("unexpected filename: expected=%v or %v, actual=%s", dataFile, valuesTmplFile, filename)
}, "")
buf, err := r.RenderToBytes(valuesTmplFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := string(buf)
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestRenderToBytes_Yaml(t *testing.T) {
valuesYamlContent := `foo:
bar: '{{ readFile "data.txt" }}'
`
expected := `foo:
bar: '{{ readFile "data.txt" }}'
`
valuesFile := "values.yaml"
r := NewRenderer(func(filename string) ([]byte, error) {
switch filename {
case valuesFile:
return []byte(valuesYamlContent), nil
}
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", valuesFile, filename)
}, "")
buf, err := r.RenderToBytes(valuesFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := string(buf)
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
+27
View File
@@ -0,0 +1,27 @@
language: go
go:
- 1.6.x
- 1.7.x
- 1.8.x
- 1.9.x
- 1.10.x
- tip
# Setting sudo access to false will let Travis CI use containers rather than
# VMs to run the tests. For more details see:
# - http://docs.travis-ci.com/user/workers/container-based-infrastructure/
# - http://docs.travis-ci.com/user/workers/standard-infrastructure/
sudo: false
script:
- make setup
- make test
notifications:
webhooks:
urls:
- https://webhooks.gitter.im/e/06e3328629952dabe3e0
on_success: change # options: [always|never|change] default: always
on_failure: always # options: [always|never|change] default: always
on_start: never # options: [always|never|change] default: always
+77
View File
@@ -0,0 +1,77 @@
# 1.4.1 (2018-04-02)
## Fixed
- Fixed #64: Fix pre-release precedence issue (thanks @uudashr)
# 1.4.0 (2017-10-04)
## Changed
- #61: Update NewVersion to parse ints with a 64bit int size (thanks @zknill)
# 1.3.1 (2017-07-10)
## Fixed
- Fixed #57: number comparisons in prerelease sometimes inaccurate
# 1.3.0 (2017-05-02)
## Added
- #45: Added json (un)marshaling support (thanks @mh-cbon)
- Stability marker. See https://masterminds.github.io/stability/
## Fixed
- #51: Fix handling of single digit tilde constraint (thanks @dgodd)
## Changed
- #55: The godoc icon moved from png to svg
# 1.2.3 (2017-04-03)
## Fixed
- #46: Fixed 0.x.x and 0.0.x in constraints being treated as *
# Release 1.2.2 (2016-12-13)
## Fixed
- #34: Fixed issue where hyphen range was not working with pre-release parsing.
# Release 1.2.1 (2016-11-28)
## Fixed
- #24: Fixed edge case issue where constraint "> 0" does not handle "0.0.1-alpha"
properly.
# Release 1.2.0 (2016-11-04)
## Added
- #20: Added MustParse function for versions (thanks @adamreese)
- #15: Added increment methods on versions (thanks @mh-cbon)
## Fixed
- Issue #21: Per the SemVer spec (section 9) a pre-release is unstable and
might not satisfy the intended compatibility. The change here ignores pre-releases
on constraint checks (e.g., ~ or ^) when a pre-release is not part of the
constraint. For example, `^1.2.3` will ignore pre-releases while
`^1.2.3-alpha` will include them.
# Release 1.1.1 (2016-06-30)
## Changed
- Issue #9: Speed up version comparison performance (thanks @sdboyer)
- Issue #8: Added benchmarks (thanks @sdboyer)
- Updated Go Report Card URL to new location
- Updated Readme to add code snippet formatting (thanks @mh-cbon)
- Updating tagging to v[SemVer] structure for compatibility with other tools.
# Release 1.1.0 (2016-03-11)
- Issue #2: Implemented validation to provide reasons a versions failed a
constraint.
# Release 1.0.1 (2015-12-31)
- Fixed #1: * constraint failing on valid versions.
# Release 1.0.0 (2015-10-20)
- Initial release
+20
View File
@@ -0,0 +1,20 @@
The Masterminds
Copyright (C) 2014-2015, Matt Butcher and Matt Farina
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
+36
View File
@@ -0,0 +1,36 @@
.PHONY: setup
setup:
go get -u gopkg.in/alecthomas/gometalinter.v1
gometalinter.v1 --install
.PHONY: test
test: validate lint
@echo "==> Running tests"
go test -v
.PHONY: validate
validate:
@echo "==> Running static validations"
@gometalinter.v1 \
--disable-all \
--enable deadcode \
--severity deadcode:error \
--enable gofmt \
--enable gosimple \
--enable ineffassign \
--enable misspell \
--enable vet \
--tests \
--vendor \
--deadline 60s \
./... || exit_code=1
.PHONY: lint
lint:
@echo "==> Running linters"
@gometalinter.v1 \
--disable-all \
--enable golint \
--vendor \
--deadline 60s \
./... || :
+165
View File
@@ -0,0 +1,165 @@
# SemVer
The `semver` package provides the ability to work with [Semantic Versions](http://semver.org) in Go. Specifically it provides the ability to:
* Parse semantic versions
* Sort semantic versions
* Check if a semantic version fits within a set of constraints
* Optionally work with a `v` prefix
[![Stability:
Active](https://masterminds.github.io/stability/active.svg)](https://masterminds.github.io/stability/active.html)
[![Build Status](https://travis-ci.org/Masterminds/semver.svg)](https://travis-ci.org/Masterminds/semver) [![Build status](https://ci.appveyor.com/api/projects/status/jfk66lib7hb985k8/branch/master?svg=true&passingText=windows%20build%20passing&failingText=windows%20build%20failing)](https://ci.appveyor.com/project/mattfarina/semver/branch/master) [![GoDoc](https://godoc.org/github.com/Masterminds/semver?status.svg)](https://godoc.org/github.com/Masterminds/semver) [![Go Report Card](https://goreportcard.com/badge/github.com/Masterminds/semver)](https://goreportcard.com/report/github.com/Masterminds/semver)
## Parsing Semantic Versions
To parse a semantic version use the `NewVersion` function. For example,
```go
v, err := semver.NewVersion("1.2.3-beta.1+build345")
```
If there is an error the version wasn't parseable. The version object has methods
to get the parts of the version, compare it to other versions, convert the
version back into a string, and get the original string. For more details
please see the [documentation](https://godoc.org/github.com/Masterminds/semver).
## Sorting Semantic Versions
A set of versions can be sorted using the [`sort`](https://golang.org/pkg/sort/)
package from the standard library. For example,
```go
raw := []string{"1.2.3", "1.0", "1.3", "2", "0.4.2",}
vs := make([]*semver.Version, len(raw))
for i, r := range raw {
v, err := semver.NewVersion(r)
if err != nil {
t.Errorf("Error parsing version: %s", err)
}
vs[i] = v
}
sort.Sort(semver.Collection(vs))
```
## Checking Version Constraints
Checking a version against version constraints is one of the most featureful
parts of the package.
```go
c, err := semver.NewConstraint(">= 1.2.3")
if err != nil {
// Handle constraint not being parseable.
}
v, _ := semver.NewVersion("1.3")
if err != nil {
// Handle version not being parseable.
}
// Check if the version meets the constraints. The a variable will be true.
a := c.Check(v)
```
## Basic Comparisons
There are two elements to the comparisons. First, a comparison string is a list
of comma separated and comparisons. These are then separated by || separated or
comparisons. For example, `">= 1.2, < 3.0.0 || >= 4.2.3"` is looking for a
comparison that's greater than or equal to 1.2 and less than 3.0.0 or is
greater than or equal to 4.2.3.
The basic comparisons are:
* `=`: equal (aliased to no operator)
* `!=`: not equal
* `>`: greater than
* `<`: less than
* `>=`: greater than or equal to
* `<=`: less than or equal to
_Note, according to the Semantic Version specification pre-releases may not be
API compliant with their release counterpart. It says,_
> _A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version._
_SemVer comparisons without a pre-release value will skip pre-release versions.
For example, `>1.2.3` will skip pre-releases when looking at a list of values
while `>1.2.3-alpha.1` will evaluate pre-releases._
## Hyphen Range Comparisons
There are multiple methods to handle ranges and the first is hyphens ranges.
These look like:
* `1.2 - 1.4.5` which is equivalent to `>= 1.2, <= 1.4.5`
* `2.3.4 - 4.5` which is equivalent to `>= 2.3.4, <= 4.5`
## Wildcards In Comparisons
The `x`, `X`, and `*` characters can be used as a wildcard character. This works
for all comparison operators. When used on the `=` operator it falls
back to the pack level comparison (see tilde below). For example,
* `1.2.x` is equivalent to `>= 1.2.0, < 1.3.0`
* `>= 1.2.x` is equivalent to `>= 1.2.0`
* `<= 2.x` is equivalent to `<= 3`
* `*` is equivalent to `>= 0.0.0`
## Tilde Range Comparisons (Patch)
The tilde (`~`) comparison operator is for patch level ranges when a minor
version is specified and major level changes when the minor number is missing.
For example,
* `~1.2.3` is equivalent to `>= 1.2.3, < 1.3.0`
* `~1` is equivalent to `>= 1, < 2`
* `~2.3` is equivalent to `>= 2.3, < 2.4`
* `~1.2.x` is equivalent to `>= 1.2.0, < 1.3.0`
* `~1.x` is equivalent to `>= 1, < 2`
## Caret Range Comparisons (Major)
The caret (`^`) comparison operator is for major level changes. This is useful
when comparisons of API versions as a major change is API breaking. For example,
* `^1.2.3` is equivalent to `>= 1.2.3, < 2.0.0`
* `^1.2.x` is equivalent to `>= 1.2.0, < 2.0.0`
* `^2.3` is equivalent to `>= 2.3, < 3`
* `^2.x` is equivalent to `>= 2.0.0, < 3`
# Validation
In addition to testing a version against a constraint, a version can be validated
against a constraint. When validation fails a slice of errors containing why a
version didn't meet the constraint is returned. For example,
```go
c, err := semver.NewConstraint("<= 1.2.3, >= 1.4")
if err != nil {
// Handle constraint not being parseable.
}
v, _ := semver.NewVersion("1.3")
if err != nil {
// Handle version not being parseable.
}
// Validate a version against a constraint.
a, msgs := c.Validate(v)
// a is false
for _, m := range msgs {
fmt.Println(m)
// Loops over the errors which would read
// "1.3 is greater than 1.2.3"
// "1.3 is less than 1.4"
}
```
# Contribute
If you find an issue or want to contribute please file an [issue](https://github.com/Masterminds/semver/issues)
or [create a pull request](https://github.com/Masterminds/semver/pulls).
+44
View File
@@ -0,0 +1,44 @@
version: build-{build}.{branch}
clone_folder: C:\gopath\src\github.com\Masterminds\semver
shallow_clone: true
environment:
GOPATH: C:\gopath
platform:
- x64
install:
- go version
- go env
- go get -u gopkg.in/alecthomas/gometalinter.v1
- set PATH=%PATH%;%GOPATH%\bin
- gometalinter.v1.exe --install
build_script:
- go install -v ./...
test_script:
- "gometalinter.v1 \
--disable-all \
--enable deadcode \
--severity deadcode:error \
--enable gofmt \
--enable gosimple \
--enable ineffassign \
--enable misspell \
--enable vet \
--tests \
--vendor \
--deadline 60s \
./... || exit_code=1"
- "gometalinter.v1 \
--disable-all \
--enable golint \
--vendor \
--deadline 60s \
./... || :"
- go test -v
deploy: off
+24
View File
@@ -0,0 +1,24 @@
package semver
// Collection is a collection of Version instances and implements the sort
// interface. See the sort package for more details.
// https://golang.org/pkg/sort/
type Collection []*Version
// Len returns the length of a collection. The number of Version instances
// on the slice.
func (c Collection) Len() int {
return len(c)
}
// Less is needed for the sort interface to compare two Version objects on the
// slice. If checks if one is less than the other.
func (c Collection) Less(i, j int) bool {
return c[i].LessThan(c[j])
}
// Swap is needed for the sort interface to replace the Version objects
// at two different positions in the slice.
func (c Collection) Swap(i, j int) {
c[i], c[j] = c[j], c[i]
}
+426
View File
@@ -0,0 +1,426 @@
package semver
import (
"errors"
"fmt"
"regexp"
"strings"
)
// Constraints is one or more constraint that a semantic version can be
// checked against.
type Constraints struct {
constraints [][]*constraint
}
// NewConstraint returns a Constraints instance that a Version instance can
// be checked against. If there is a parse error it will be returned.
func NewConstraint(c string) (*Constraints, error) {
// Rewrite - ranges into a comparison operation.
c = rewriteRange(c)
ors := strings.Split(c, "||")
or := make([][]*constraint, len(ors))
for k, v := range ors {
cs := strings.Split(v, ",")
result := make([]*constraint, len(cs))
for i, s := range cs {
pc, err := parseConstraint(s)
if err != nil {
return nil, err
}
result[i] = pc
}
or[k] = result
}
o := &Constraints{constraints: or}
return o, nil
}
// Check tests if a version satisfies the constraints.
func (cs Constraints) Check(v *Version) bool {
// loop over the ORs and check the inner ANDs
for _, o := range cs.constraints {
joy := true
for _, c := range o {
if !c.check(v) {
joy = false
break
}
}
if joy {
return true
}
}
return false
}
// Validate checks if a version satisfies a constraint. If not a slice of
// reasons for the failure are returned in addition to a bool.
func (cs Constraints) Validate(v *Version) (bool, []error) {
// loop over the ORs and check the inner ANDs
var e []error
for _, o := range cs.constraints {
joy := true
for _, c := range o {
if !c.check(v) {
em := fmt.Errorf(c.msg, v, c.orig)
e = append(e, em)
joy = false
}
}
if joy {
return true, []error{}
}
}
return false, e
}
var constraintOps map[string]cfunc
var constraintMsg map[string]string
var constraintRegex *regexp.Regexp
func init() {
constraintOps = map[string]cfunc{
"": constraintTildeOrEqual,
"=": constraintTildeOrEqual,
"!=": constraintNotEqual,
">": constraintGreaterThan,
"<": constraintLessThan,
">=": constraintGreaterThanEqual,
"=>": constraintGreaterThanEqual,
"<=": constraintLessThanEqual,
"=<": constraintLessThanEqual,
"~": constraintTilde,
"~>": constraintTilde,
"^": constraintCaret,
}
constraintMsg = map[string]string{
"": "%s is not equal to %s",
"=": "%s is not equal to %s",
"!=": "%s is equal to %s",
">": "%s is less than or equal to %s",
"<": "%s is greater than or equal to %s",
">=": "%s is less than %s",
"=>": "%s is less than %s",
"<=": "%s is greater than %s",
"=<": "%s is greater than %s",
"~": "%s does not have same major and minor version as %s",
"~>": "%s does not have same major and minor version as %s",
"^": "%s does not have same major version as %s",
}
ops := make([]string, 0, len(constraintOps))
for k := range constraintOps {
ops = append(ops, regexp.QuoteMeta(k))
}
constraintRegex = regexp.MustCompile(fmt.Sprintf(
`^\s*(%s)\s*(%s)\s*$`,
strings.Join(ops, "|"),
cvRegex))
constraintRangeRegex = regexp.MustCompile(fmt.Sprintf(
`\s*(%s)\s+-\s+(%s)\s*`,
cvRegex, cvRegex))
}
// An individual constraint
type constraint struct {
// The callback function for the restraint. It performs the logic for
// the constraint.
function cfunc
msg string
// The version used in the constraint check. For example, if a constraint
// is '<= 2.0.0' the con a version instance representing 2.0.0.
con *Version
// The original parsed version (e.g., 4.x from != 4.x)
orig string
// When an x is used as part of the version (e.g., 1.x)
minorDirty bool
dirty bool
patchDirty bool
}
// Check if a version meets the constraint
func (c *constraint) check(v *Version) bool {
return c.function(v, c)
}
type cfunc func(v *Version, c *constraint) bool
func parseConstraint(c string) (*constraint, error) {
m := constraintRegex.FindStringSubmatch(c)
if m == nil {
return nil, fmt.Errorf("improper constraint: %s", c)
}
ver := m[2]
orig := ver
minorDirty := false
patchDirty := false
dirty := false
if isX(m[3]) {
ver = "0.0.0"
dirty = true
} else if isX(strings.TrimPrefix(m[4], ".")) || m[4] == "" {
minorDirty = true
dirty = true
ver = fmt.Sprintf("%s.0.0%s", m[3], m[6])
} else if isX(strings.TrimPrefix(m[5], ".")) {
dirty = true
patchDirty = true
ver = fmt.Sprintf("%s%s.0%s", m[3], m[4], m[6])
}
con, err := NewVersion(ver)
if err != nil {
// The constraintRegex should catch any regex parsing errors. So,
// we should never get here.
return nil, errors.New("constraint Parser Error")
}
cs := &constraint{
function: constraintOps[m[1]],
msg: constraintMsg[m[1]],
con: con,
orig: orig,
minorDirty: minorDirty,
patchDirty: patchDirty,
dirty: dirty,
}
return cs, nil
}
// Constraint functions
func constraintNotEqual(v *Version, c *constraint) bool {
if c.dirty {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if c.con.Major() != v.Major() {
return true
}
if c.con.Minor() != v.Minor() && !c.minorDirty {
return true
} else if c.minorDirty {
return false
}
return false
}
return !v.Equal(c.con)
}
func constraintGreaterThan(v *Version, c *constraint) bool {
// An edge case the constraint is 0.0.0 and the version is 0.0.0-someprerelease
// exists. This that case.
if !isNonZero(c.con) && isNonZero(v) {
return true
}
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
return v.Compare(c.con) == 1
}
func constraintLessThan(v *Version, c *constraint) bool {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if !c.dirty {
return v.Compare(c.con) < 0
}
if v.Major() > c.con.Major() {
return false
} else if v.Minor() > c.con.Minor() && !c.minorDirty {
return false
}
return true
}
func constraintGreaterThanEqual(v *Version, c *constraint) bool {
// An edge case the constraint is 0.0.0 and the version is 0.0.0-someprerelease
// exists. This that case.
if !isNonZero(c.con) && isNonZero(v) {
return true
}
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
return v.Compare(c.con) >= 0
}
func constraintLessThanEqual(v *Version, c *constraint) bool {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if !c.dirty {
return v.Compare(c.con) <= 0
}
if v.Major() > c.con.Major() {
return false
} else if v.Minor() > c.con.Minor() && !c.minorDirty {
return false
}
return true
}
// ~*, ~>* --> >= 0.0.0 (any)
// ~2, ~2.x, ~2.x.x, ~>2, ~>2.x ~>2.x.x --> >=2.0.0, <3.0.0
// ~2.0, ~2.0.x, ~>2.0, ~>2.0.x --> >=2.0.0, <2.1.0
// ~1.2, ~1.2.x, ~>1.2, ~>1.2.x --> >=1.2.0, <1.3.0
// ~1.2.3, ~>1.2.3 --> >=1.2.3, <1.3.0
// ~1.2.0, ~>1.2.0 --> >=1.2.0, <1.3.0
func constraintTilde(v *Version, c *constraint) bool {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if v.LessThan(c.con) {
return false
}
// ~0.0.0 is a special case where all constraints are accepted. It's
// equivalent to >= 0.0.0.
if c.con.Major() == 0 && c.con.Minor() == 0 && c.con.Patch() == 0 &&
!c.minorDirty && !c.patchDirty {
return true
}
if v.Major() != c.con.Major() {
return false
}
if v.Minor() != c.con.Minor() && !c.minorDirty {
return false
}
return true
}
// When there is a .x (dirty) status it automatically opts in to ~. Otherwise
// it's a straight =
func constraintTildeOrEqual(v *Version, c *constraint) bool {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if c.dirty {
c.msg = constraintMsg["~"]
return constraintTilde(v, c)
}
return v.Equal(c.con)
}
// ^* --> (any)
// ^2, ^2.x, ^2.x.x --> >=2.0.0, <3.0.0
// ^2.0, ^2.0.x --> >=2.0.0, <3.0.0
// ^1.2, ^1.2.x --> >=1.2.0, <2.0.0
// ^1.2.3 --> >=1.2.3, <2.0.0
// ^1.2.0 --> >=1.2.0, <2.0.0
func constraintCaret(v *Version, c *constraint) bool {
// If there is a pre-release on the version but the constraint isn't looking
// for them assume that pre-releases are not compatible. See issue 21 for
// more details.
if v.Prerelease() != "" && c.con.Prerelease() == "" {
return false
}
if v.LessThan(c.con) {
return false
}
if v.Major() != c.con.Major() {
return false
}
return true
}
var constraintRangeRegex *regexp.Regexp
const cvRegex string = `v?([0-9|x|X|\*]+)(\.[0-9|x|X|\*]+)?(\.[0-9|x|X|\*]+)?` +
`(-([0-9A-Za-z\-]+(\.[0-9A-Za-z\-]+)*))?` +
`(\+([0-9A-Za-z\-]+(\.[0-9A-Za-z\-]+)*))?`
func isX(x string) bool {
switch x {
case "x", "*", "X":
return true
default:
return false
}
}
func rewriteRange(i string) string {
m := constraintRangeRegex.FindAllStringSubmatch(i, -1)
if m == nil {
return i
}
o := i
for _, v := range m {
t := fmt.Sprintf(">= %s, <= %s", v[1], v[11])
o = strings.Replace(o, v[0], t, 1)
}
return o
}
// Detect if a version is not zero (0.0.0)
func isNonZero(v *Version) bool {
if v.Major() != 0 || v.Minor() != 0 || v.Patch() != 0 || v.Prerelease() != "" {
return true
}
return false
}
+115
View File
@@ -0,0 +1,115 @@
/*
Package semver provides the ability to work with Semantic Versions (http://semver.org) in Go.
Specifically it provides the ability to:
* Parse semantic versions
* Sort semantic versions
* Check if a semantic version fits within a set of constraints
* Optionally work with a `v` prefix
Parsing Semantic Versions
To parse a semantic version use the `NewVersion` function. For example,
v, err := semver.NewVersion("1.2.3-beta.1+build345")
If there is an error the version wasn't parseable. The version object has methods
to get the parts of the version, compare it to other versions, convert the
version back into a string, and get the original string. For more details
please see the documentation at https://godoc.org/github.com/Masterminds/semver.
Sorting Semantic Versions
A set of versions can be sorted using the `sort` package from the standard library.
For example,
raw := []string{"1.2.3", "1.0", "1.3", "2", "0.4.2",}
vs := make([]*semver.Version, len(raw))
for i, r := range raw {
v, err := semver.NewVersion(r)
if err != nil {
t.Errorf("Error parsing version: %s", err)
}
vs[i] = v
}
sort.Sort(semver.Collection(vs))
Checking Version Constraints
Checking a version against version constraints is one of the most featureful
parts of the package.
c, err := semver.NewConstraint(">= 1.2.3")
if err != nil {
// Handle constraint not being parseable.
}
v, _ := semver.NewVersion("1.3")
if err != nil {
// Handle version not being parseable.
}
// Check if the version meets the constraints. The a variable will be true.
a := c.Check(v)
Basic Comparisons
There are two elements to the comparisons. First, a comparison string is a list
of comma separated and comparisons. These are then separated by || separated or
comparisons. For example, `">= 1.2, < 3.0.0 || >= 4.2.3"` is looking for a
comparison that's greater than or equal to 1.2 and less than 3.0.0 or is
greater than or equal to 4.2.3.
The basic comparisons are:
* `=`: equal (aliased to no operator)
* `!=`: not equal
* `>`: greater than
* `<`: less than
* `>=`: greater than or equal to
* `<=`: less than or equal to
Hyphen Range Comparisons
There are multiple methods to handle ranges and the first is hyphens ranges.
These look like:
* `1.2 - 1.4.5` which is equivalent to `>= 1.2, <= 1.4.5`
* `2.3.4 - 4.5` which is equivalent to `>= 2.3.4, <= 4.5`
Wildcards In Comparisons
The `x`, `X`, and `*` characters can be used as a wildcard character. This works
for all comparison operators. When used on the `=` operator it falls
back to the pack level comparison (see tilde below). For example,
* `1.2.x` is equivalent to `>= 1.2.0, < 1.3.0`
* `>= 1.2.x` is equivalent to `>= 1.2.0`
* `<= 2.x` is equivalent to `<= 3`
* `*` is equivalent to `>= 0.0.0`
Tilde Range Comparisons (Patch)
The tilde (`~`) comparison operator is for patch level ranges when a minor
version is specified and major level changes when the minor number is missing.
For example,
* `~1.2.3` is equivalent to `>= 1.2.3, < 1.3.0`
* `~1` is equivalent to `>= 1, < 2`
* `~2.3` is equivalent to `>= 2.3, < 2.4`
* `~1.2.x` is equivalent to `>= 1.2.0, < 1.3.0`
* `~1.x` is equivalent to `>= 1, < 2`
Caret Range Comparisons (Major)
The caret (`^`) comparison operator is for major level changes. This is useful
when comparisons of API versions as a major change is API breaking. For example,
* `^1.2.3` is equivalent to `>= 1.2.3, < 2.0.0`
* `^1.2.x` is equivalent to `>= 1.2.0, < 2.0.0`
* `^2.3` is equivalent to `>= 2.3, < 3`
* `^2.x` is equivalent to `>= 2.0.0, < 3`
*/
package semver
+421
View File
@@ -0,0 +1,421 @@
package semver
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"regexp"
"strconv"
"strings"
)
// The compiled version of the regex created at init() is cached here so it
// only needs to be created once.
var versionRegex *regexp.Regexp
var validPrereleaseRegex *regexp.Regexp
var (
// ErrInvalidSemVer is returned a version is found to be invalid when
// being parsed.
ErrInvalidSemVer = errors.New("Invalid Semantic Version")
// ErrInvalidMetadata is returned when the metadata is an invalid format
ErrInvalidMetadata = errors.New("Invalid Metadata string")
// ErrInvalidPrerelease is returned when the pre-release is an invalid format
ErrInvalidPrerelease = errors.New("Invalid Prerelease string")
)
// SemVerRegex is the regular expression used to parse a semantic version.
const SemVerRegex string = `v?([0-9]+)(\.[0-9]+)?(\.[0-9]+)?` +
`(-([0-9A-Za-z\-]+(\.[0-9A-Za-z\-]+)*))?` +
`(\+([0-9A-Za-z\-]+(\.[0-9A-Za-z\-]+)*))?`
// ValidPrerelease is the regular expression which validates
// both prerelease and metadata values.
const ValidPrerelease string = `^([0-9A-Za-z\-]+(\.[0-9A-Za-z\-]+)*)`
// Version represents a single semantic version.
type Version struct {
major, minor, patch int64
pre string
metadata string
original string
}
func init() {
versionRegex = regexp.MustCompile("^" + SemVerRegex + "$")
validPrereleaseRegex = regexp.MustCompile(ValidPrerelease)
}
// NewVersion parses a given version and returns an instance of Version or
// an error if unable to parse the version.
func NewVersion(v string) (*Version, error) {
m := versionRegex.FindStringSubmatch(v)
if m == nil {
return nil, ErrInvalidSemVer
}
sv := &Version{
metadata: m[8],
pre: m[5],
original: v,
}
var temp int64
temp, err := strconv.ParseInt(m[1], 10, 64)
if err != nil {
return nil, fmt.Errorf("Error parsing version segment: %s", err)
}
sv.major = temp
if m[2] != "" {
temp, err = strconv.ParseInt(strings.TrimPrefix(m[2], "."), 10, 64)
if err != nil {
return nil, fmt.Errorf("Error parsing version segment: %s", err)
}
sv.minor = temp
} else {
sv.minor = 0
}
if m[3] != "" {
temp, err = strconv.ParseInt(strings.TrimPrefix(m[3], "."), 10, 64)
if err != nil {
return nil, fmt.Errorf("Error parsing version segment: %s", err)
}
sv.patch = temp
} else {
sv.patch = 0
}
return sv, nil
}
// MustParse parses a given version and panics on error.
func MustParse(v string) *Version {
sv, err := NewVersion(v)
if err != nil {
panic(err)
}
return sv
}
// String converts a Version object to a string.
// Note, if the original version contained a leading v this version will not.
// See the Original() method to retrieve the original value. Semantic Versions
// don't contain a leading v per the spec. Instead it's optional on
// impelementation.
func (v *Version) String() string {
var buf bytes.Buffer
fmt.Fprintf(&buf, "%d.%d.%d", v.major, v.minor, v.patch)
if v.pre != "" {
fmt.Fprintf(&buf, "-%s", v.pre)
}
if v.metadata != "" {
fmt.Fprintf(&buf, "+%s", v.metadata)
}
return buf.String()
}
// Original returns the original value passed in to be parsed.
func (v *Version) Original() string {
return v.original
}
// Major returns the major version.
func (v *Version) Major() int64 {
return v.major
}
// Minor returns the minor version.
func (v *Version) Minor() int64 {
return v.minor
}
// Patch returns the patch version.
func (v *Version) Patch() int64 {
return v.patch
}
// Prerelease returns the pre-release version.
func (v *Version) Prerelease() string {
return v.pre
}
// Metadata returns the metadata on the version.
func (v *Version) Metadata() string {
return v.metadata
}
// originalVPrefix returns the original 'v' prefix if any.
func (v *Version) originalVPrefix() string {
// Note, only lowercase v is supported as a prefix by the parser.
if v.original != "" && v.original[:1] == "v" {
return v.original[:1]
}
return ""
}
// IncPatch produces the next patch version.
// If the current version does not have prerelease/metadata information,
// it unsets metadata and prerelease values, increments patch number.
// If the current version has any of prerelease or metadata information,
// it unsets both values and keeps curent patch value
func (v Version) IncPatch() Version {
vNext := v
// according to http://semver.org/#spec-item-9
// Pre-release versions have a lower precedence than the associated normal version.
// according to http://semver.org/#spec-item-10
// Build metadata SHOULD be ignored when determining version precedence.
if v.pre != "" {
vNext.metadata = ""
vNext.pre = ""
} else {
vNext.metadata = ""
vNext.pre = ""
vNext.patch = v.patch + 1
}
vNext.original = v.originalVPrefix() + "" + vNext.String()
return vNext
}
// IncMinor produces the next minor version.
// Sets patch to 0.
// Increments minor number.
// Unsets metadata.
// Unsets prerelease status.
func (v Version) IncMinor() Version {
vNext := v
vNext.metadata = ""
vNext.pre = ""
vNext.patch = 0
vNext.minor = v.minor + 1
vNext.original = v.originalVPrefix() + "" + vNext.String()
return vNext
}
// IncMajor produces the next major version.
// Sets patch to 0.
// Sets minor to 0.
// Increments major number.
// Unsets metadata.
// Unsets prerelease status.
func (v Version) IncMajor() Version {
vNext := v
vNext.metadata = ""
vNext.pre = ""
vNext.patch = 0
vNext.minor = 0
vNext.major = v.major + 1
vNext.original = v.originalVPrefix() + "" + vNext.String()
return vNext
}
// SetPrerelease defines the prerelease value.
// Value must not include the required 'hypen' prefix.
func (v Version) SetPrerelease(prerelease string) (Version, error) {
vNext := v
if len(prerelease) > 0 && !validPrereleaseRegex.MatchString(prerelease) {
return vNext, ErrInvalidPrerelease
}
vNext.pre = prerelease
vNext.original = v.originalVPrefix() + "" + vNext.String()
return vNext, nil
}
// SetMetadata defines metadata value.
// Value must not include the required 'plus' prefix.
func (v Version) SetMetadata(metadata string) (Version, error) {
vNext := v
if len(metadata) > 0 && !validPrereleaseRegex.MatchString(metadata) {
return vNext, ErrInvalidMetadata
}
vNext.metadata = metadata
vNext.original = v.originalVPrefix() + "" + vNext.String()
return vNext, nil
}
// LessThan tests if one version is less than another one.
func (v *Version) LessThan(o *Version) bool {
return v.Compare(o) < 0
}
// GreaterThan tests if one version is greater than another one.
func (v *Version) GreaterThan(o *Version) bool {
return v.Compare(o) > 0
}
// Equal tests if two versions are equal to each other.
// Note, versions can be equal with different metadata since metadata
// is not considered part of the comparable version.
func (v *Version) Equal(o *Version) bool {
return v.Compare(o) == 0
}
// Compare compares this version to another one. It returns -1, 0, or 1 if
// the version smaller, equal, or larger than the other version.
//
// Versions are compared by X.Y.Z. Build metadata is ignored. Prerelease is
// lower than the version without a prerelease.
func (v *Version) Compare(o *Version) int {
// Compare the major, minor, and patch version for differences. If a
// difference is found return the comparison.
if d := compareSegment(v.Major(), o.Major()); d != 0 {
return d
}
if d := compareSegment(v.Minor(), o.Minor()); d != 0 {
return d
}
if d := compareSegment(v.Patch(), o.Patch()); d != 0 {
return d
}
// At this point the major, minor, and patch versions are the same.
ps := v.pre
po := o.Prerelease()
if ps == "" && po == "" {
return 0
}
if ps == "" {
return 1
}
if po == "" {
return -1
}
return comparePrerelease(ps, po)
}
// UnmarshalJSON implements JSON.Unmarshaler interface.
func (v *Version) UnmarshalJSON(b []byte) error {
var s string
if err := json.Unmarshal(b, &s); err != nil {
return err
}
temp, err := NewVersion(s)
if err != nil {
return err
}
v.major = temp.major
v.minor = temp.minor
v.patch = temp.patch
v.pre = temp.pre
v.metadata = temp.metadata
v.original = temp.original
temp = nil
return nil
}
// MarshalJSON implements JSON.Marshaler interface.
func (v *Version) MarshalJSON() ([]byte, error) {
return json.Marshal(v.String())
}
func compareSegment(v, o int64) int {
if v < o {
return -1
}
if v > o {
return 1
}
return 0
}
func comparePrerelease(v, o string) int {
// split the prelease versions by their part. The separator, per the spec,
// is a .
sparts := strings.Split(v, ".")
oparts := strings.Split(o, ".")
// Find the longer length of the parts to know how many loop iterations to
// go through.
slen := len(sparts)
olen := len(oparts)
l := slen
if olen > slen {
l = olen
}
// Iterate over each part of the prereleases to compare the differences.
for i := 0; i < l; i++ {
// Since the lentgh of the parts can be different we need to create
// a placeholder. This is to avoid out of bounds issues.
stemp := ""
if i < slen {
stemp = sparts[i]
}
otemp := ""
if i < olen {
otemp = oparts[i]
}
d := comparePrePart(stemp, otemp)
if d != 0 {
return d
}
}
// Reaching here means two versions are of equal value but have different
// metadata (the part following a +). They are not identical in string form
// but the version comparison finds them to be equal.
return 0
}
func comparePrePart(s, o string) int {
// Fastpath if they are equal
if s == o {
return 0
}
// When s or o are empty we can use the other in an attempt to determine
// the response.
if s == "" {
if o != "" {
return -1
}
return 1
}
if o == "" {
if s != "" {
return 1
}
return -1
}
// When comparing strings "99" is greater than "103". To handle
// cases like this we need to detect numbers and compare them.
oi, n1 := strconv.ParseInt(o, 10, 64)
si, n2 := strconv.ParseInt(s, 10, 64)
// The case where both are strings compare the strings
if n1 != nil && n2 != nil {
if s > o {
return 1
}
return -1
} else if n1 != nil {
// o is a string and s is a number
return -1
} else if n2 != nil {
// s is a string and o is a number
return 1
}
// Both are numbers
if si > oi {
return 1
}
return -1
}
+2
View File
@@ -0,0 +1,2 @@
vendor/
/.glide
+23
View File
@@ -0,0 +1,23 @@
language: go
go:
- 1.9.x
- 1.10.x
- tip
# Setting sudo access to false will let Travis CI use containers rather than
# VMs to run the tests. For more details see:
# - http://docs.travis-ci.com/user/workers/container-based-infrastructure/
# - http://docs.travis-ci.com/user/workers/standard-infrastructure/
sudo: false
script:
- make setup test
notifications:
webhooks:
urls:
- https://webhooks.gitter.im/e/06e3328629952dabe3e0
on_success: change # options: [always|never|change] default: always
on_failure: always # options: [always|never|change] default: always
on_start: never # options: [always|never|change] default: always
+153
View File
@@ -0,0 +1,153 @@
# Changelog
## Release 2.15.0 (2018-04-02)
### Added
- #68 and #69: Add json helpers to docs (thanks @arunvelsriram)
- #66: Add ternary function (thanks @binoculars)
- #67: Allow keys function to take multiple dicts (thanks @binoculars)
- #89: Added sha1sum to crypto function (thanks @benkeil)
- #81: Allow customizing Root CA that used by genSignedCert (thanks @chenzhiwei)
- #92: Add travis testing for go 1.10
- #93: Adding appveyor config for windows testing
### Changed
- #90: Updating to more recent dependencies
- #73: replace satori/go.uuid with google/uuid (thanks @petterw)
### Fixed
- #76: Fixed documentation typos (thanks @Thiht)
- Fixed rounding issue on the `ago` function. Note, the removes support for Go 1.8 and older
## Release 2.14.1 (2017-12-01)
### Fixed
- #60: Fix typo in function name documentation (thanks @neil-ca-moore)
- #61: Removing line with {{ due to blocking github pages genertion
- #64: Update the list functions to handle int, string, and other slices for compatibility
## Release 2.14.0 (2017-10-06)
This new version of Sprig adds a set of functions for generating and working with SSL certificates.
- `genCA` generates an SSL Certificate Authority
- `genSelfSignedCert` generates an SSL self-signed certificate
- `genSignedCert` generates an SSL certificate and key based on a given CA
## Release 2.13.0 (2017-09-18)
This release adds new functions, including:
- `regexMatch`, `regexFindAll`, `regexFind`, `regexReplaceAll`, `regexReplaceAllLiteral`, and `regexSplit` to work with regular expressions
- `floor`, `ceil`, and `round` math functions
- `toDate` converts a string to a date
- `nindent` is just like `indent` but also prepends a new line
- `ago` returns the time from `time.Now`
### Added
- #40: Added basic regex functionality (thanks @alanquillin)
- #41: Added ceil floor and round functions (thanks @alanquillin)
- #48: Added toDate function (thanks @andreynering)
- #50: Added nindent function (thanks @binoculars)
- #46: Added ago function (thanks @slayer)
### Changed
- #51: Updated godocs to include new string functions (thanks @curtisallen)
- #49: Added ability to merge multiple dicts (thanks @binoculars)
## Release 2.12.0 (2017-05-17)
- `snakecase`, `camelcase`, and `shuffle` are three new string functions
- `fail` allows you to bail out of a template render when conditions are not met
## Release 2.11.0 (2017-05-02)
- Added `toJson` and `toPrettyJson`
- Added `merge`
- Refactored documentation
## Release 2.10.0 (2017-03-15)
- Added `semver` and `semverCompare` for Semantic Versions
- `list` replaces `tuple`
- Fixed issue with `join`
- Added `first`, `last`, `intial`, `rest`, `prepend`, `append`, `toString`, `toStrings`, `sortAlpha`, `reverse`, `coalesce`, `pluck`, `pick`, `compact`, `keys`, `omit`, `uniq`, `has`, `without`
## Release 2.9.0 (2017-02-23)
- Added `splitList` to split a list
- Added crypto functions of `genPrivateKey` and `derivePassword`
## Release 2.8.0 (2016-12-21)
- Added access to several path functions (`base`, `dir`, `clean`, `ext`, and `abs`)
- Added functions for _mutating_ dictionaries (`set`, `unset`, `hasKey`)
## Release 2.7.0 (2016-12-01)
- Added `sha256sum` to generate a hash of an input
- Added functions to convert a numeric or string to `int`, `int64`, `float64`
## Release 2.6.0 (2016-10-03)
- Added a `uuidv4` template function for generating UUIDs inside of a template.
## Release 2.5.0 (2016-08-19)
- New `trimSuffix`, `trimPrefix`, `hasSuffix`, and `hasPrefix` functions
- New aliases have been added for a few functions that didn't follow the naming conventions (`trimAll` and `abbrevBoth`)
- `trimall` and `abbrevboth` (notice the case) are deprecated and will be removed in 3.0.0
## Release 2.4.0 (2016-08-16)
- Adds two functions: `until` and `untilStep`
## Release 2.3.0 (2016-06-21)
- cat: Concatenate strings with whitespace separators.
- replace: Replace parts of a string: `replace " " "-" "Me First"` renders "Me-First"
- plural: Format plurals: `len "foo" | plural "one foo" "many foos"` renders "many foos"
- indent: Indent blocks of text in a way that is sensitive to "\n" characters.
## Release 2.2.0 (2016-04-21)
- Added a `genPrivateKey` function (Thanks @bacongobbler)
## Release 2.1.0 (2016-03-30)
- `default` now prints the default value when it does not receive a value down the pipeline. It is much safer now to do `{{.Foo | default "bar"}}`.
- Added accessors for "hermetic" functions. These return only functions that, when given the same input, produce the same output.
## Release 2.0.0 (2016-03-29)
Because we switched from `int` to `int64` as the return value for all integer math functions, the library's major version number has been incremented.
- `min` complements `max` (formerly `biggest`)
- `empty` indicates that a value is the empty value for its type
- `tuple` creates a tuple inside of a template: `{{$t := tuple "a", "b" "c"}}`
- `dict` creates a dictionary inside of a template `{{$d := dict "key1" "val1" "key2" "val2"}}`
- Date formatters have been added for HTML dates (as used in `date` input fields)
- Integer math functions can convert from a number of types, including `string` (via `strconv.ParseInt`).
## Release 1.2.0 (2016-02-01)
- Added quote and squote
- Added b32enc and b32dec
- add now takes varargs
- biggest now takes varargs
## Release 1.1.0 (2015-12-29)
- Added #4: Added contains function. strings.Contains, but with the arguments
switched to simplify common pipelines. (thanks krancour)
- Added Travis-CI testing support
## Release 1.0.0 (2015-12-23)
- Initial release
+20
View File
@@ -0,0 +1,20 @@
Sprig
Copyright (C) 2013 Masterminds
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
+13
View File
@@ -0,0 +1,13 @@
HAS_GLIDE := $(shell command -v glide;)
.PHONY: test
test:
go test -v .
.PHONY: setup
setup:
ifndef HAS_GLIDE
go get -u github.com/Masterminds/glide
endif
glide install
+81
View File
@@ -0,0 +1,81 @@
# Sprig: Template functions for Go templates
[![Stability: Sustained](https://masterminds.github.io/stability/sustained.svg)](https://masterminds.github.io/stability/sustained.html)
[![Build Status](https://travis-ci.org/Masterminds/sprig.svg?branch=master)](https://travis-ci.org/Masterminds/sprig)
The Go language comes with a [built-in template
language](http://golang.org/pkg/text/template/), but not
very many template functions. This library provides a group of commonly
used template functions.
It is inspired by the template functions found in
[Twig](http://twig.sensiolabs.org/documentation) and also in various
JavaScript libraries, such as [underscore.js](http://underscorejs.org/).
## Usage
Template developers can read the [Sprig function documentation](http://masterminds.github.io/sprig/) to
learn about the >100 template functions available.
For Go developers wishing to include Sprig as a library in their programs,
API documentation is available [at GoDoc.org](http://godoc.org/github.com/Masterminds/sprig), but
read on for standard usage.
### Load the Sprig library
To load the Sprig `FuncMap`:
```go
import (
"github.com/Masterminds/sprig"
"html/template"
)
// This example illustrates that the FuncMap *must* be set before the
// templates themselves are loaded.
tpl := template.Must(
template.New("base").Funcs(sprig.FuncMap()).ParseGlob("*.html")
)
```
### Call the functions inside of templates
By convention, all functions are lowercase. This seems to follow the Go
idiom for template functions (as opposed to template methods, which are
TitleCase).
Example:
```
{{ "hello!" | upper | repeat 5 }}
```
Produces:
```
HELLO!HELLO!HELLO!HELLO!HELLO!
```
## Principles:
The following principles were used in deciding on which functions to add, and
determining how to implement them.
- Template functions should be used to build layout. Therefore, the following
types of operations are within the domain of template functions:
- Formatting
- Layout
- Simple type conversions
- Utilities that assist in handling common formatting and layout needs (e.g. arithmetic)
- Template functions should not return errors unless there is no way to print
a sensible value. For example, converting a string to an integer should not
produce an error if conversion fails. Instead, it should display a default
value that can be displayed.
- Simple math is necessary for grid layouts, pagers, and so on. Complex math
(anything other than arithmetic) should be done outside of templates.
- Template functions only deal with the data passed into them. They never retrieve
data from a source.
- Finally, do not override core Go template functions.
+26
View File
@@ -0,0 +1,26 @@
version: build-{build}.{branch}
clone_folder: C:\gopath\src\github.com\Masterminds\sprig
shallow_clone: true
environment:
GOPATH: C:\gopath
platform:
- x64
install:
- go get -u github.com/Masterminds/glide
- set PATH=%GOPATH%\bin;%PATH%
- go version
- go env
build_script:
- glide install
- go install ./...
test_script:
- go test -v
deploy: off
+430
View File
@@ -0,0 +1,430 @@
package sprig
import (
"bytes"
"crypto/dsa"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/hmac"
"crypto/rand"
"crypto/rsa"
"crypto/sha1"
"crypto/sha256"
"crypto/x509"
"crypto/x509/pkix"
"encoding/asn1"
"encoding/base64"
"encoding/binary"
"encoding/hex"
"encoding/pem"
"errors"
"fmt"
"math/big"
"net"
"time"
"github.com/google/uuid"
"golang.org/x/crypto/scrypt"
)
func sha256sum(input string) string {
hash := sha256.Sum256([]byte(input))
return hex.EncodeToString(hash[:])
}
func sha1sum(input string) string {
hash := sha1.Sum([]byte(input))
return hex.EncodeToString(hash[:])
}
// uuidv4 provides a safe and secure UUID v4 implementation
func uuidv4() string {
return fmt.Sprintf("%s", uuid.New())
}
var master_password_seed = "com.lyndir.masterpassword"
var password_type_templates = map[string][][]byte{
"maximum": {[]byte("anoxxxxxxxxxxxxxxxxx"), []byte("axxxxxxxxxxxxxxxxxno")},
"long": {[]byte("CvcvnoCvcvCvcv"), []byte("CvcvCvcvnoCvcv"), []byte("CvcvCvcvCvcvno"), []byte("CvccnoCvcvCvcv"), []byte("CvccCvcvnoCvcv"),
[]byte("CvccCvcvCvcvno"), []byte("CvcvnoCvccCvcv"), []byte("CvcvCvccnoCvcv"), []byte("CvcvCvccCvcvno"), []byte("CvcvnoCvcvCvcc"),
[]byte("CvcvCvcvnoCvcc"), []byte("CvcvCvcvCvccno"), []byte("CvccnoCvccCvcv"), []byte("CvccCvccnoCvcv"), []byte("CvccCvccCvcvno"),
[]byte("CvcvnoCvccCvcc"), []byte("CvcvCvccnoCvcc"), []byte("CvcvCvccCvccno"), []byte("CvccnoCvcvCvcc"), []byte("CvccCvcvnoCvcc"),
[]byte("CvccCvcvCvccno")},
"medium": {[]byte("CvcnoCvc"), []byte("CvcCvcno")},
"short": {[]byte("Cvcn")},
"basic": {[]byte("aaanaaan"), []byte("aannaaan"), []byte("aaannaaa")},
"pin": {[]byte("nnnn")},
}
var template_characters = map[byte]string{
'V': "AEIOU",
'C': "BCDFGHJKLMNPQRSTVWXYZ",
'v': "aeiou",
'c': "bcdfghjklmnpqrstvwxyz",
'A': "AEIOUBCDFGHJKLMNPQRSTVWXYZ",
'a': "AEIOUaeiouBCDFGHJKLMNPQRSTVWXYZbcdfghjklmnpqrstvwxyz",
'n': "0123456789",
'o': "@&%?,=[]_:-+*$#!'^~;()/.",
'x': "AEIOUaeiouBCDFGHJKLMNPQRSTVWXYZbcdfghjklmnpqrstvwxyz0123456789!@#$%^&*()",
}
func derivePassword(counter uint32, password_type, password, user, site string) string {
var templates = password_type_templates[password_type]
if templates == nil {
return fmt.Sprintf("cannot find password template %s", password_type)
}
var buffer bytes.Buffer
buffer.WriteString(master_password_seed)
binary.Write(&buffer, binary.BigEndian, uint32(len(user)))
buffer.WriteString(user)
salt := buffer.Bytes()
key, err := scrypt.Key([]byte(password), salt, 32768, 8, 2, 64)
if err != nil {
return fmt.Sprintf("failed to derive password: %s", err)
}
buffer.Truncate(len(master_password_seed))
binary.Write(&buffer, binary.BigEndian, uint32(len(site)))
buffer.WriteString(site)
binary.Write(&buffer, binary.BigEndian, counter)
var hmacv = hmac.New(sha256.New, key)
hmacv.Write(buffer.Bytes())
var seed = hmacv.Sum(nil)
var temp = templates[int(seed[0])%len(templates)]
buffer.Truncate(0)
for i, element := range temp {
pass_chars := template_characters[element]
pass_char := pass_chars[int(seed[i+1])%len(pass_chars)]
buffer.WriteByte(pass_char)
}
return buffer.String()
}
func generatePrivateKey(typ string) string {
var priv interface{}
var err error
switch typ {
case "", "rsa":
// good enough for government work
priv, err = rsa.GenerateKey(rand.Reader, 4096)
case "dsa":
key := new(dsa.PrivateKey)
// again, good enough for government work
if err = dsa.GenerateParameters(&key.Parameters, rand.Reader, dsa.L2048N256); err != nil {
return fmt.Sprintf("failed to generate dsa params: %s", err)
}
err = dsa.GenerateKey(key, rand.Reader)
priv = key
case "ecdsa":
// again, good enough for government work
priv, err = ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
default:
return "Unknown type " + typ
}
if err != nil {
return fmt.Sprintf("failed to generate private key: %s", err)
}
return string(pem.EncodeToMemory(pemBlockForKey(priv)))
}
type DSAKeyFormat struct {
Version int
P, Q, G, Y, X *big.Int
}
func pemBlockForKey(priv interface{}) *pem.Block {
switch k := priv.(type) {
case *rsa.PrivateKey:
return &pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(k)}
case *dsa.PrivateKey:
val := DSAKeyFormat{
P: k.P, Q: k.Q, G: k.G,
Y: k.Y, X: k.X,
}
bytes, _ := asn1.Marshal(val)
return &pem.Block{Type: "DSA PRIVATE KEY", Bytes: bytes}
case *ecdsa.PrivateKey:
b, _ := x509.MarshalECPrivateKey(k)
return &pem.Block{Type: "EC PRIVATE KEY", Bytes: b}
default:
return nil
}
}
type certificate struct {
Cert string
Key string
}
func buildCustomCertificate(b64cert string, b64key string) (certificate, error) {
crt := certificate{}
cert, err := base64.StdEncoding.DecodeString(b64cert)
if err != nil {
return crt, errors.New("unable to decode base64 certificate")
}
key, err := base64.StdEncoding.DecodeString(b64key)
if err != nil {
return crt, errors.New("unable to decode base64 private key")
}
decodedCert, _ := pem.Decode(cert)
if decodedCert == nil {
return crt, errors.New("unable to decode certificate")
}
_, err = x509.ParseCertificate(decodedCert.Bytes)
if err != nil {
return crt, fmt.Errorf(
"error parsing certificate: decodedCert.Bytes: %s",
err,
)
}
decodedKey, _ := pem.Decode(key)
if decodedKey == nil {
return crt, errors.New("unable to decode key")
}
_, err = x509.ParsePKCS1PrivateKey(decodedKey.Bytes)
if err != nil {
return crt, fmt.Errorf(
"error parsing prive key: decodedKey.Bytes: %s",
err,
)
}
crt.Cert = string(cert)
crt.Key = string(key)
return crt, nil
}
func generateCertificateAuthority(
cn string,
daysValid int,
) (certificate, error) {
ca := certificate{}
template, err := getBaseCertTemplate(cn, nil, nil, daysValid)
if err != nil {
return ca, err
}
// Override KeyUsage and IsCA
template.KeyUsage = x509.KeyUsageKeyEncipherment |
x509.KeyUsageDigitalSignature |
x509.KeyUsageCertSign
template.IsCA = true
priv, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return ca, fmt.Errorf("error generating rsa key: %s", err)
}
ca.Cert, ca.Key, err = getCertAndKey(template, priv, template, priv)
if err != nil {
return ca, err
}
return ca, nil
}
func generateSelfSignedCertificate(
cn string,
ips []interface{},
alternateDNS []interface{},
daysValid int,
) (certificate, error) {
cert := certificate{}
template, err := getBaseCertTemplate(cn, ips, alternateDNS, daysValid)
if err != nil {
return cert, err
}
priv, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return cert, fmt.Errorf("error generating rsa key: %s", err)
}
cert.Cert, cert.Key, err = getCertAndKey(template, priv, template, priv)
if err != nil {
return cert, err
}
return cert, nil
}
func generateSignedCertificate(
cn string,
ips []interface{},
alternateDNS []interface{},
daysValid int,
ca certificate,
) (certificate, error) {
cert := certificate{}
decodedSignerCert, _ := pem.Decode([]byte(ca.Cert))
if decodedSignerCert == nil {
return cert, errors.New("unable to decode certificate")
}
signerCert, err := x509.ParseCertificate(decodedSignerCert.Bytes)
if err != nil {
return cert, fmt.Errorf(
"error parsing certificate: decodedSignerCert.Bytes: %s",
err,
)
}
decodedSignerKey, _ := pem.Decode([]byte(ca.Key))
if decodedSignerKey == nil {
return cert, errors.New("unable to decode key")
}
signerKey, err := x509.ParsePKCS1PrivateKey(decodedSignerKey.Bytes)
if err != nil {
return cert, fmt.Errorf(
"error parsing prive key: decodedSignerKey.Bytes: %s",
err,
)
}
template, err := getBaseCertTemplate(cn, ips, alternateDNS, daysValid)
if err != nil {
return cert, err
}
priv, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return cert, fmt.Errorf("error generating rsa key: %s", err)
}
cert.Cert, cert.Key, err = getCertAndKey(
template,
priv,
signerCert,
signerKey,
)
if err != nil {
return cert, err
}
return cert, nil
}
func getCertAndKey(
template *x509.Certificate,
signeeKey *rsa.PrivateKey,
parent *x509.Certificate,
signingKey *rsa.PrivateKey,
) (string, string, error) {
derBytes, err := x509.CreateCertificate(
rand.Reader,
template,
parent,
&signeeKey.PublicKey,
signingKey,
)
if err != nil {
return "", "", fmt.Errorf("error creating certificate: %s", err)
}
certBuffer := bytes.Buffer{}
if err := pem.Encode(
&certBuffer,
&pem.Block{Type: "CERTIFICATE", Bytes: derBytes},
); err != nil {
return "", "", fmt.Errorf("error pem-encoding certificate: %s", err)
}
keyBuffer := bytes.Buffer{}
if err := pem.Encode(
&keyBuffer,
&pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(signeeKey),
},
); err != nil {
return "", "", fmt.Errorf("error pem-encoding key: %s", err)
}
return string(certBuffer.Bytes()), string(keyBuffer.Bytes()), nil
}
func getBaseCertTemplate(
cn string,
ips []interface{},
alternateDNS []interface{},
daysValid int,
) (*x509.Certificate, error) {
ipAddresses, err := getNetIPs(ips)
if err != nil {
return nil, err
}
dnsNames, err := getAlternateDNSStrs(alternateDNS)
if err != nil {
return nil, err
}
return &x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{
CommonName: cn,
},
IPAddresses: ipAddresses,
DNSNames: dnsNames,
NotBefore: time.Now(),
NotAfter: time.Now().Add(time.Hour * 24 * time.Duration(daysValid)),
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{
x509.ExtKeyUsageServerAuth,
x509.ExtKeyUsageClientAuth,
},
BasicConstraintsValid: true,
}, nil
}
func getNetIPs(ips []interface{}) ([]net.IP, error) {
if ips == nil {
return []net.IP{}, nil
}
var ipStr string
var ok bool
var netIP net.IP
netIPs := make([]net.IP, len(ips))
for i, ip := range ips {
ipStr, ok = ip.(string)
if !ok {
return nil, fmt.Errorf("error parsing ip: %v is not a string", ip)
}
netIP = net.ParseIP(ipStr)
if netIP == nil {
return nil, fmt.Errorf("error parsing ip: %s", ipStr)
}
netIPs[i] = netIP
}
return netIPs, nil
}
func getAlternateDNSStrs(alternateDNS []interface{}) ([]string, error) {
if alternateDNS == nil {
return []string{}, nil
}
var dnsStr string
var ok bool
alternateDNSStrs := make([]string, len(alternateDNS))
for i, dns := range alternateDNS {
dnsStr, ok = dns.(string)
if !ok {
return nil, fmt.Errorf(
"error processing alternate dns name: %v is not a string",
dns,
)
}
alternateDNSStrs[i] = dnsStr
}
return alternateDNSStrs, nil
}
+76
View File
@@ -0,0 +1,76 @@
package sprig
import (
"time"
)
// Given a format and a date, format the date string.
//
// Date can be a `time.Time` or an `int, int32, int64`.
// In the later case, it is treated as seconds since UNIX
// epoch.
func date(fmt string, date interface{}) string {
return dateInZone(fmt, date, "Local")
}
func htmlDate(date interface{}) string {
return dateInZone("2006-01-02", date, "Local")
}
func htmlDateInZone(date interface{}, zone string) string {
return dateInZone("2006-01-02", date, zone)
}
func dateInZone(fmt string, date interface{}, zone string) string {
var t time.Time
switch date := date.(type) {
default:
t = time.Now()
case time.Time:
t = date
case int64:
t = time.Unix(date, 0)
case int:
t = time.Unix(int64(date), 0)
case int32:
t = time.Unix(int64(date), 0)
}
loc, err := time.LoadLocation(zone)
if err != nil {
loc, _ = time.LoadLocation("UTC")
}
return t.In(loc).Format(fmt)
}
func dateModify(fmt string, date time.Time) time.Time {
d, err := time.ParseDuration(fmt)
if err != nil {
return date
}
return date.Add(d)
}
func dateAgo(date interface{}) string {
var t time.Time
switch date := date.(type) {
default:
t = time.Now()
case time.Time:
t = date
case int64:
t = time.Unix(date, 0)
case int:
t = time.Unix(int64(date), 0)
}
// Drop resolution to seconds
duration := time.Since(t).Round(time.Second)
return duration.String()
}
func toDate(fmt, str string) time.Time {
t, _ := time.ParseInLocation(fmt, str, time.Local)
return t
}
+84
View File
@@ -0,0 +1,84 @@
package sprig
import (
"encoding/json"
"reflect"
)
// dfault checks whether `given` is set, and returns default if not set.
//
// This returns `d` if `given` appears not to be set, and `given` otherwise.
//
// For numeric types 0 is unset.
// For strings, maps, arrays, and slices, len() = 0 is considered unset.
// For bool, false is unset.
// Structs are never considered unset.
//
// For everything else, including pointers, a nil value is unset.
func dfault(d interface{}, given ...interface{}) interface{} {
if empty(given) || empty(given[0]) {
return d
}
return given[0]
}
// empty returns true if the given value has the zero value for its type.
func empty(given interface{}) bool {
g := reflect.ValueOf(given)
if !g.IsValid() {
return true
}
// Basically adapted from text/template.isTrue
switch g.Kind() {
default:
return g.IsNil()
case reflect.Array, reflect.Slice, reflect.Map, reflect.String:
return g.Len() == 0
case reflect.Bool:
return g.Bool() == false
case reflect.Complex64, reflect.Complex128:
return g.Complex() == 0
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return g.Int() == 0
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return g.Uint() == 0
case reflect.Float32, reflect.Float64:
return g.Float() == 0
case reflect.Struct:
return false
}
return true
}
// coalesce returns the first non-empty value.
func coalesce(v ...interface{}) interface{} {
for _, val := range v {
if !empty(val) {
return val
}
}
return nil
}
// toJson encodes an item into a JSON string
func toJson(v interface{}) string {
output, _ := json.Marshal(v)
return string(output)
}
// toPrettyJson encodes an item into a pretty (indented) JSON string
func toPrettyJson(v interface{}) string {
output, _ := json.MarshalIndent(v, "", " ")
return string(output)
}
// ternary returns the first value if the last value is true, otherwise returns the second value.
func ternary(vt interface{}, vf interface{}, v bool) interface{} {
if v {
return vt
}
return vf
}
+88
View File
@@ -0,0 +1,88 @@
package sprig
import "github.com/imdario/mergo"
func set(d map[string]interface{}, key string, value interface{}) map[string]interface{} {
d[key] = value
return d
}
func unset(d map[string]interface{}, key string) map[string]interface{} {
delete(d, key)
return d
}
func hasKey(d map[string]interface{}, key string) bool {
_, ok := d[key]
return ok
}
func pluck(key string, d ...map[string]interface{}) []interface{} {
res := []interface{}{}
for _, dict := range d {
if val, ok := dict[key]; ok {
res = append(res, val)
}
}
return res
}
func keys(dicts ...map[string]interface{}) []string {
k := []string{}
for _, dict := range dicts {
for key := range dict {
k = append(k, key)
}
}
return k
}
func pick(dict map[string]interface{}, keys ...string) map[string]interface{} {
res := map[string]interface{}{}
for _, k := range keys {
if v, ok := dict[k]; ok {
res[k] = v
}
}
return res
}
func omit(dict map[string]interface{}, keys ...string) map[string]interface{} {
res := map[string]interface{}{}
omit := make(map[string]bool, len(keys))
for _, k := range keys {
omit[k] = true
}
for k, v := range dict {
if _, ok := omit[k]; !ok {
res[k] = v
}
}
return res
}
func dict(v ...interface{}) map[string]interface{} {
dict := map[string]interface{}{}
lenv := len(v)
for i := 0; i < lenv; i += 2 {
key := strval(v[i])
if i+1 >= lenv {
dict[key] = ""
continue
}
dict[key] = v[i+1]
}
return dict
}
func merge(dst map[string]interface{}, srcs ...map[string]interface{}) interface{} {
for _, src := range srcs {
if err := mergo.Merge(&dst, src); err != nil {
// Swallow errors inside of a template.
return ""
}
}
return dst
}
+233
View File
@@ -0,0 +1,233 @@
/*
Sprig: Template functions for Go.
This package contains a number of utility functions for working with data
inside of Go `html/template` and `text/template` files.
To add these functions, use the `template.Funcs()` method:
t := templates.New("foo").Funcs(sprig.FuncMap())
Note that you should add the function map before you parse any template files.
In several cases, Sprig reverses the order of arguments from the way they
appear in the standard library. This is to make it easier to pipe
arguments into functions.
Date Functions
- date FORMAT TIME: Format a date, where a date is an integer type or a time.Time type, and
format is a time.Format formatting string.
- dateModify: Given a date, modify it with a duration: `date_modify "-1.5h" now`. If the duration doesn't
parse, it returns the time unaltered. See `time.ParseDuration` for info on duration strings.
- now: Current time.Time, for feeding into date-related functions.
- htmlDate TIME: Format a date for use in the value field of an HTML "date" form element.
- dateInZone FORMAT TIME TZ: Like date, but takes three arguments: format, timestamp,
timezone.
- htmlDateInZone TIME TZ: Like htmlDate, but takes two arguments: timestamp,
timezone.
String Functions
- abbrev: Truncate a string with ellipses. `abbrev 5 "hello world"` yields "he..."
- abbrevboth: Abbreviate from both sides, yielding "...lo wo..."
- trunc: Truncate a string (no suffix). `trunc 5 "Hello World"` yields "hello".
- trim: strings.TrimSpace
- trimAll: strings.Trim, but with the argument order reversed `trimAll "$" "$5.00"` or `"$5.00 | trimAll "$"`
- trimSuffix: strings.TrimSuffix, but with the argument order reversed: `trimSuffix "-" "ends-with-"`
- trimPrefix: strings.TrimPrefix, but with the argument order reversed `trimPrefix "$" "$5"`
- upper: strings.ToUpper
- lower: strings.ToLower
- nospace: Remove all space characters from a string. `nospace "h e l l o"` becomes "hello"
- title: strings.Title
- untitle: Remove title casing
- repeat: strings.Repeat, but with the arguments switched: `repeat count str`. (This simplifies common pipelines)
- substr: Given string, start, and length, return a substr.
- initials: Given a multi-word string, return the initials. `initials "Matt Butcher"` returns "MB"
- randAlphaNum: Given a length, generate a random alphanumeric sequence
- randAlpha: Given a length, generate an alphabetic string
- randAscii: Given a length, generate a random ASCII string (symbols included)
- randNumeric: Given a length, generate a string of digits.
- swapcase: SwapCase swaps the case of a string using a word based algorithm. see https://godoc.org/github.com/Masterminds/goutils#SwapCase
- shuffle: Shuffle randomizes runes in a string and returns the result. It uses default random source in `math/rand`
- snakecase: convert all upper case characters in a string to underscore format.
- camelcase: convert all lower case characters behind underscores to upper case character
- wrap: Force a line wrap at the given width. `wrap 80 "imagine a longer string"`
- wrapWith: Wrap a line at the given length, but using 'sep' instead of a newline. `wrapWith 50, "<br>", $html`
- contains: strings.Contains, but with the arguments switched: `contains substr str`. (This simplifies common pipelines)
- hasPrefix: strings.hasPrefix, but with the arguments switched
- hasSuffix: strings.hasSuffix, but with the arguments switched
- quote: Wrap string(s) in double quotation marks, escape the contents by adding '\' before '"'.
- squote: Wrap string(s) in double quotation marks, does not escape content.
- cat: Concatenate strings, separating them by spaces. `cat $a $b $c`.
- indent: Indent a string using space characters. `indent 4 "foo\nbar"` produces " foo\n bar"
- nindent: Indent a string using space characters and prepend a new line. `indent 4 "foo\nbar"` produces "\n foo\n bar"
- replace: Replace an old with a new in a string: `$name | replace " " "-"`
- plural: Choose singular or plural based on length: `len $fish | plural "one anchovy" "many anchovies"`
- sha256sum: Generate a hex encoded sha256 hash of the input
- toString: Convert something to a string
String Slice Functions:
- join: strings.Join, but as `join SEP SLICE`
- split: strings.Split, but as `split SEP STRING`. The results are returned
as a map with the indexes set to _N, where N is an integer starting from 0.
Use it like this: `{{$v := "foo/bar/baz" | split "/"}}{{$v._0}}` (Prints `foo`)
- splitList: strings.Split, but as `split SEP STRING`. The results are returned
as an array.
- toStrings: convert a list to a list of strings. 'list 1 2 3 | toStrings' produces '["1" "2" "3"]'
- sortAlpha: sort a list lexicographically.
Integer Slice Functions:
- until: Given an integer, returns a slice of counting integers from 0 to one
less than the given integer: `range $i, $e := until 5`
- untilStep: Given start, stop, and step, return an integer slice starting at
'start', stopping at `stop`, and incrementing by 'step. This is the same
as Python's long-form of 'range'.
Conversions:
- atoi: Convert a string to an integer. 0 if the integer could not be parsed.
- int64: Convert a string or another numeric type to an int64.
- int: Convert a string or another numeric type to an int.
- float64: Convert a string or another numeric type to a float64.
Defaults:
- default: Give a default value. Used like this: trim " "| default "empty".
Since trim produces an empty string, the default value is returned. For
things with a length (strings, slices, maps), len(0) will trigger the default.
For numbers, the value 0 will trigger the default. For booleans, false will
trigger the default. For structs, the default is never returned (there is
no clear empty condition). For everything else, nil value triggers a default.
- empty: Return true if the given value is the zero value for its type.
Caveats: structs are always non-empty. This should match the behavior of
{{if pipeline}}, but can be used inside of a pipeline.
- coalesce: Given a list of items, return the first non-empty one.
This follows the same rules as 'empty'. '{{ coalesce .someVal 0 "hello" }}`
will return `.someVal` if set, or else return "hello". The 0 is skipped
because it is an empty value.
- compact: Return a copy of a list with all of the empty values removed.
'list 0 1 2 "" | compact' will return '[1 2]'
- ternary: Given a value,'true | ternary "b" "c"' will return "b".
'false | ternary "b" "c"' will return '"c"'. Similar to the JavaScript ternary
operator.
OS:
- env: Resolve an environment variable
- expandenv: Expand a string through the environment
File Paths:
- base: Return the last element of a path. https://golang.org/pkg/path#Base
- dir: Remove the last element of a path. https://golang.org/pkg/path#Dir
- clean: Clean a path to the shortest equivalent name. (e.g. remove "foo/.."
from "foo/../bar.html") https://golang.org/pkg/path#Clean
- ext: https://golang.org/pkg/path#Ext
- isAbs: https://golang.org/pkg/path#IsAbs
Encoding:
- b64enc: Base 64 encode a string.
- b64dec: Base 64 decode a string.
Reflection:
- typeOf: Takes an interface and returns a string representation of the type.
For pointers, this will return a type prefixed with an asterisk(`*`). So
a pointer to type `Foo` will be `*Foo`.
- typeIs: Compares an interface with a string name, and returns true if they match.
Note that a pointer will not match a reference. For example `*Foo` will not
match `Foo`.
- typeIsLike: Compares an interface with a string name and returns true if
the interface is that `name` or that `*name`. In other words, if the given
value matches the given type or is a pointer to the given type, this returns
true.
- kindOf: Takes an interface and returns a string representation of its kind.
- kindIs: Returns true if the given string matches the kind of the given interface.
Note: None of these can test whether or not something implements a given
interface, since doing so would require compiling the interface in ahead of
time.
Data Structures:
- tuple: Takes an arbitrary list of items and returns a slice of items. Its
tuple-ish properties are mainly gained through the template idiom, and not
through an API provided here. WARNING: The implementation of tuple will
change in the future.
- list: An arbitrary ordered list of items. (This is prefered over tuple.)
- dict: Takes a list of name/values and returns a map[string]interface{}.
The first parameter is converted to a string and stored as a key, the
second parameter is treated as the value. And so on, with odds as keys and
evens as values. If the function call ends with an odd, the last key will
be assigned the empty string. Non-string keys are converted to strings as
follows: []byte are converted, fmt.Stringers will have String() called.
errors will have Error() called. All others will be passed through
fmt.Sprintf("%v").
Lists Functions:
These are used to manipulate lists: '{{ list 1 2 3 | reverse | first }}'
- first: Get the first item in a 'list'. 'list 1 2 3 | first' prints '1'
- last: Get the last item in a 'list': 'list 1 2 3 | last ' prints '3'
- rest: Get all but the first item in a list: 'list 1 2 3 | rest' returns '[2 3]'
- initial: Get all but the last item in a list: 'list 1 2 3 | initial' returns '[1 2]'
- append: Add an item to the end of a list: 'append $list 4' adds '4' to the end of '$list'
- prepend: Add an item to the beginning of a list: 'prepend $list 4' puts 4 at the beginning of the list.
- reverse: Reverse the items in a list.
- uniq: Remove duplicates from a list.
- without: Return a list with the given values removed: 'without (list 1 2 3) 1' would return '[2 3]'
- has: Return 'true' if the item is found in the list: 'has "foo" $list' will return 'true' if the list contains "foo"
Dict Functions:
These are used to manipulate dicts.
- set: Takes a dict, a key, and a value, and sets that key/value pair in
the dict. `set $dict $key $value`. For convenience, it returns the dict,
even though the dict was modified in place.
- unset: Takes a dict and a key, and deletes that key/value pair from the
dict. `unset $dict $key`. This returns the dict for convenience.
- hasKey: Takes a dict and a key, and returns boolean true if the key is in
the dict.
- pluck: Given a key and one or more maps, get all of the values for that key.
- keys: Get an array of all of the keys in one or more dicts.
- pick: Select just the given keys out of the dict, and return a new dict.
- omit: Return a dict without the given keys.
Math Functions:
Integer functions will convert integers of any width to `int64`. If a
string is passed in, functions will attempt to convert with
`strconv.ParseInt(s, 1064)`. If this fails, the value will be treated as 0.
- add1: Increment an integer by 1
- add: Sum an arbitrary number of integers
- sub: Subtract the second integer from the first
- div: Divide the first integer by the second
- mod: Module of first integer divided by second
- mul: Multiply integers
- max: Return the biggest of a series of one or more integers
- min: Return the smallest of a series of one or more integers
- biggest: DEPRECATED. Return the biggest of a series of one or more integers
Crypto Functions:
- genPrivateKey: Generate a private key for the given cryptosystem. If no
argument is supplied, by default it will generate a private key using
the RSA algorithm. Accepted values are `rsa`, `dsa`, and `ecdsa`.
- derivePassword: Derive a password from the given parameters according to the ["Master Password" algorithm](http://masterpasswordapp.com/algorithm.html)
Given parameters (in order) are:
`counter` (starting with 1), `password_type` (maximum, long, medium, short, basic, or pin), `password`,
`user`, and `site`
SemVer Functions:
These functions provide version parsing and comparisons for SemVer 2 version
strings.
- semver: Parse a semantic version and return a Version object.
- semverCompare: Compare a SemVer range to a particular version.
*/
package sprig
+281
View File
@@ -0,0 +1,281 @@
package sprig
import (
"errors"
"html/template"
"os"
"path"
"strconv"
"strings"
ttemplate "text/template"
"time"
util "github.com/aokoli/goutils"
"github.com/huandu/xstrings"
)
// Produce the function map.
//
// Use this to pass the functions into the template engine:
//
// tpl := template.New("foo").Funcs(sprig.FuncMap()))
//
func FuncMap() template.FuncMap {
return HtmlFuncMap()
}
// HermeticTextFuncMap returns a 'text/template'.FuncMap with only repeatable functions.
func HermeticTxtFuncMap() ttemplate.FuncMap {
r := TxtFuncMap()
for _, name := range nonhermeticFunctions {
delete(r, name)
}
return r
}
// HermeticHtmlFuncMap returns an 'html/template'.Funcmap with only repeatable functions.
func HermeticHtmlFuncMap() template.FuncMap {
r := HtmlFuncMap()
for _, name := range nonhermeticFunctions {
delete(r, name)
}
return r
}
// TextFuncMap returns a 'text/template'.FuncMap
func TxtFuncMap() ttemplate.FuncMap {
return ttemplate.FuncMap(GenericFuncMap())
}
// HtmlFuncMap returns an 'html/template'.Funcmap
func HtmlFuncMap() template.FuncMap {
return template.FuncMap(GenericFuncMap())
}
// GenericFuncMap returns a copy of the basic function map as a map[string]interface{}.
func GenericFuncMap() map[string]interface{} {
gfm := make(map[string]interface{}, len(genericMap))
for k, v := range genericMap {
gfm[k] = v
}
return gfm
}
// These functions are not guaranteed to evaluate to the same result for given input, because they
// refer to the environemnt or global state.
var nonhermeticFunctions = []string{
// Date functions
"date",
"date_in_zone",
"date_modify",
"now",
"htmlDate",
"htmlDateInZone",
"dateInZone",
"dateModify",
// Strings
"randAlphaNum",
"randAlpha",
"randAscii",
"randNumeric",
"uuidv4",
// OS
"env",
"expandenv",
}
var genericMap = map[string]interface{}{
"hello": func() string { return "Hello!" },
// Date functions
"date": date,
"date_in_zone": dateInZone,
"date_modify": dateModify,
"now": func() time.Time { return time.Now() },
"htmlDate": htmlDate,
"htmlDateInZone": htmlDateInZone,
"dateInZone": dateInZone,
"dateModify": dateModify,
"ago": dateAgo,
"toDate": toDate,
// Strings
"abbrev": abbrev,
"abbrevboth": abbrevboth,
"trunc": trunc,
"trim": strings.TrimSpace,
"upper": strings.ToUpper,
"lower": strings.ToLower,
"title": strings.Title,
"untitle": untitle,
"substr": substring,
// Switch order so that "foo" | repeat 5
"repeat": func(count int, str string) string { return strings.Repeat(str, count) },
// Deprecated: Use trimAll.
"trimall": func(a, b string) string { return strings.Trim(b, a) },
// Switch order so that "$foo" | trimall "$"
"trimAll": func(a, b string) string { return strings.Trim(b, a) },
"trimSuffix": func(a, b string) string { return strings.TrimSuffix(b, a) },
"trimPrefix": func(a, b string) string { return strings.TrimPrefix(b, a) },
"nospace": util.DeleteWhiteSpace,
"initials": initials,
"randAlphaNum": randAlphaNumeric,
"randAlpha": randAlpha,
"randAscii": randAscii,
"randNumeric": randNumeric,
"swapcase": util.SwapCase,
"shuffle": xstrings.Shuffle,
"snakecase": xstrings.ToSnakeCase,
"camelcase": xstrings.ToCamelCase,
"wrap": func(l int, s string) string { return util.Wrap(s, l) },
"wrapWith": func(l int, sep, str string) string { return util.WrapCustom(str, l, sep, true) },
// Switch order so that "foobar" | contains "foo"
"contains": func(substr string, str string) bool { return strings.Contains(str, substr) },
"hasPrefix": func(substr string, str string) bool { return strings.HasPrefix(str, substr) },
"hasSuffix": func(substr string, str string) bool { return strings.HasSuffix(str, substr) },
"quote": quote,
"squote": squote,
"cat": cat,
"indent": indent,
"nindent": nindent,
"replace": replace,
"plural": plural,
"sha1sum": sha1sum,
"sha256sum": sha256sum,
"toString": strval,
// Wrap Atoi to stop errors.
"atoi": func(a string) int { i, _ := strconv.Atoi(a); return i },
"int64": toInt64,
"int": toInt,
"float64": toFloat64,
//"gt": func(a, b int) bool {return a > b},
//"gte": func(a, b int) bool {return a >= b},
//"lt": func(a, b int) bool {return a < b},
//"lte": func(a, b int) bool {return a <= b},
// split "/" foo/bar returns map[int]string{0: foo, 1: bar}
"split": split,
"splitList": func(sep, orig string) []string { return strings.Split(orig, sep) },
"toStrings": strslice,
"until": until,
"untilStep": untilStep,
// VERY basic arithmetic.
"add1": func(i interface{}) int64 { return toInt64(i) + 1 },
"add": func(i ...interface{}) int64 {
var a int64 = 0
for _, b := range i {
a += toInt64(b)
}
return a
},
"sub": func(a, b interface{}) int64 { return toInt64(a) - toInt64(b) },
"div": func(a, b interface{}) int64 { return toInt64(a) / toInt64(b) },
"mod": func(a, b interface{}) int64 { return toInt64(a) % toInt64(b) },
"mul": func(a interface{}, v ...interface{}) int64 {
val := toInt64(a)
for _, b := range v {
val = val * toInt64(b)
}
return val
},
"biggest": max,
"max": max,
"min": min,
"ceil": ceil,
"floor": floor,
"round": round,
// string slices. Note that we reverse the order b/c that's better
// for template processing.
"join": join,
"sortAlpha": sortAlpha,
// Defaults
"default": dfault,
"empty": empty,
"coalesce": coalesce,
"compact": compact,
"toJson": toJson,
"toPrettyJson": toPrettyJson,
"ternary": ternary,
// Reflection
"typeOf": typeOf,
"typeIs": typeIs,
"typeIsLike": typeIsLike,
"kindOf": kindOf,
"kindIs": kindIs,
// OS:
"env": func(s string) string { return os.Getenv(s) },
"expandenv": func(s string) string { return os.ExpandEnv(s) },
// File Paths:
"base": path.Base,
"dir": path.Dir,
"clean": path.Clean,
"ext": path.Ext,
"isAbs": path.IsAbs,
// Encoding:
"b64enc": base64encode,
"b64dec": base64decode,
"b32enc": base32encode,
"b32dec": base32decode,
// Data Structures:
"tuple": list, // FIXME: with the addition of append/prepend these are no longer immutable.
"list": list,
"dict": dict,
"set": set,
"unset": unset,
"hasKey": hasKey,
"pluck": pluck,
"keys": keys,
"pick": pick,
"omit": omit,
"merge": merge,
"append": push, "push": push,
"prepend": prepend,
"first": first,
"rest": rest,
"last": last,
"initial": initial,
"reverse": reverse,
"uniq": uniq,
"without": without,
"has": has,
// Crypto:
"genPrivateKey": generatePrivateKey,
"derivePassword": derivePassword,
"buildCustomCert": buildCustomCertificate,
"genCA": generateCertificateAuthority,
"genSelfSignedCert": generateSelfSignedCertificate,
"genSignedCert": generateSignedCertificate,
// UUIDs:
"uuidv4": uuidv4,
// SemVer:
"semver": semver,
"semverCompare": semverCompare,
// Flow Control:
"fail": func(msg string) (string, error) { return "", errors.New(msg) },
// Regex
"regexMatch": regexMatch,
"regexFindAll": regexFindAll,
"regexFind": regexFind,
"regexReplaceAll": regexReplaceAll,
"regexReplaceAllLiteral": regexReplaceAllLiteral,
"regexSplit": regexSplit,
}
+33
View File
@@ -0,0 +1,33 @@
hash: 770b6a1132b743dadf6a0bb5fb8bf7083b1a5209f6d6c07826234ab2a97aade9
updated: 2018-04-02T23:08:56.947456531+02:00
imports:
- name: github.com/aokoli/goutils
version: 9c37978a95bd5c709a15883b6242714ea6709e64
- name: github.com/google/uuid
version: 064e2069ce9c359c118179501254f67d7d37ba24
- name: github.com/huandu/xstrings
version: 3959339b333561bf62a38b424fd41517c2c90f40
- name: github.com/imdario/mergo
version: 7fe0c75c13abdee74b09fcacef5ea1c6bba6a874
- name: github.com/Masterminds/goutils
version: 3391d3790d23d03408670993e957e8f408993c34
- name: github.com/Masterminds/semver
version: 59c29afe1a994eacb71c833025ca7acf874bb1da
- name: github.com/stretchr/testify
version: e3a8ff8ce36581f87a15341206f205b1da467059
subpackages:
- assert
- name: golang.org/x/crypto
version: d172538b2cfce0c13cee31e647d0367aa8cd2486
subpackages:
- pbkdf2
- scrypt
testImports:
- name: github.com/davecgh/go-spew
version: 5215b55f46b2b919f50a1df0eaa5886afe4e3b3d
subpackages:
- spew
- name: github.com/pmezard/go-difflib
version: d8ed2627bdf02c080bf22230dbb337003b7aba2d
subpackages:
- difflib
+15
View File
@@ -0,0 +1,15 @@
package: github.com/Masterminds/sprig
import:
- package: github.com/Masterminds/goutils
version: ^1.0.0
- package: github.com/google/uuid
version: ^0.2
- package: golang.org/x/crypto
subpackages:
- scrypt
- package: github.com/Masterminds/semver
version: v1.2.2
- package: github.com/stretchr/testify
- package: github.com/imdario/mergo
version: ~0.2.2
- package: github.com/huandu/xstrings
+259
View File
@@ -0,0 +1,259 @@
package sprig
import (
"fmt"
"reflect"
"sort"
)
// Reflection is used in these functions so that slices and arrays of strings,
// ints, and other types not implementing []interface{} can be worked with.
// For example, this is useful if you need to work on the output of regexs.
func list(v ...interface{}) []interface{} {
return v
}
func push(list interface{}, v interface{}) []interface{} {
tp := reflect.TypeOf(list).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(list)
l := l2.Len()
nl := make([]interface{}, l)
for i := 0; i < l; i++ {
nl[i] = l2.Index(i).Interface()
}
return append(nl, v)
default:
panic(fmt.Sprintf("Cannot push on type %s", tp))
}
}
func prepend(list interface{}, v interface{}) []interface{} {
//return append([]interface{}{v}, list...)
tp := reflect.TypeOf(list).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(list)
l := l2.Len()
nl := make([]interface{}, l)
for i := 0; i < l; i++ {
nl[i] = l2.Index(i).Interface()
}
return append([]interface{}{v}, nl...)
default:
panic(fmt.Sprintf("Cannot prepend on type %s", tp))
}
}
func last(list interface{}) interface{} {
tp := reflect.TypeOf(list).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(list)
l := l2.Len()
if l == 0 {
return nil
}
return l2.Index(l - 1).Interface()
default:
panic(fmt.Sprintf("Cannot find last on type %s", tp))
}
}
func first(list interface{}) interface{} {
tp := reflect.TypeOf(list).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(list)
l := l2.Len()
if l == 0 {
return nil
}
return l2.Index(0).Interface()
default:
panic(fmt.Sprintf("Cannot find first on type %s", tp))
}
}
func rest(list interface{}) []interface{} {
tp := reflect.TypeOf(list).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(list)
l := l2.Len()
if l == 0 {
return nil
}
nl := make([]interface{}, l-1)
for i := 1; i < l; i++ {
nl[i-1] = l2.Index(i).Interface()
}
return nl
default:
panic(fmt.Sprintf("Cannot find rest on type %s", tp))
}
}
func initial(list interface{}) []interface{} {
tp := reflect.TypeOf(list).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(list)
l := l2.Len()
if l == 0 {
return nil
}
nl := make([]interface{}, l-1)
for i := 0; i < l-1; i++ {
nl[i] = l2.Index(i).Interface()
}
return nl
default:
panic(fmt.Sprintf("Cannot find initial on type %s", tp))
}
}
func sortAlpha(list interface{}) []string {
k := reflect.Indirect(reflect.ValueOf(list)).Kind()
switch k {
case reflect.Slice, reflect.Array:
a := strslice(list)
s := sort.StringSlice(a)
s.Sort()
return s
}
return []string{strval(list)}
}
func reverse(v interface{}) []interface{} {
tp := reflect.TypeOf(v).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(v)
l := l2.Len()
// We do not sort in place because the incoming array should not be altered.
nl := make([]interface{}, l)
for i := 0; i < l; i++ {
nl[l-i-1] = l2.Index(i).Interface()
}
return nl
default:
panic(fmt.Sprintf("Cannot find reverse on type %s", tp))
}
}
func compact(list interface{}) []interface{} {
tp := reflect.TypeOf(list).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(list)
l := l2.Len()
nl := []interface{}{}
var item interface{}
for i := 0; i < l; i++ {
item = l2.Index(i).Interface()
if !empty(item) {
nl = append(nl, item)
}
}
return nl
default:
panic(fmt.Sprintf("Cannot compact on type %s", tp))
}
}
func uniq(list interface{}) []interface{} {
tp := reflect.TypeOf(list).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(list)
l := l2.Len()
dest := []interface{}{}
var item interface{}
for i := 0; i < l; i++ {
item = l2.Index(i).Interface()
if !inList(dest, item) {
dest = append(dest, item)
}
}
return dest
default:
panic(fmt.Sprintf("Cannot find uniq on type %s", tp))
}
}
func inList(haystack []interface{}, needle interface{}) bool {
for _, h := range haystack {
if reflect.DeepEqual(needle, h) {
return true
}
}
return false
}
func without(list interface{}, omit ...interface{}) []interface{} {
tp := reflect.TypeOf(list).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(list)
l := l2.Len()
res := []interface{}{}
var item interface{}
for i := 0; i < l; i++ {
item = l2.Index(i).Interface()
if !inList(omit, item) {
res = append(res, item)
}
}
return res
default:
panic(fmt.Sprintf("Cannot find without on type %s", tp))
}
}
func has(needle interface{}, haystack interface{}) bool {
tp := reflect.TypeOf(haystack).Kind()
switch tp {
case reflect.Slice, reflect.Array:
l2 := reflect.ValueOf(haystack)
var item interface{}
l := l2.Len()
for i := 0; i < l; i++ {
item = l2.Index(i).Interface()
if reflect.DeepEqual(needle, item) {
return true
}
}
return false
default:
panic(fmt.Sprintf("Cannot find has on type %s", tp))
}
}
+159
View File
@@ -0,0 +1,159 @@
package sprig
import (
"math"
"reflect"
"strconv"
)
// toFloat64 converts 64-bit floats
func toFloat64(v interface{}) float64 {
if str, ok := v.(string); ok {
iv, err := strconv.ParseFloat(str, 64)
if err != nil {
return 0
}
return iv
}
val := reflect.Indirect(reflect.ValueOf(v))
switch val.Kind() {
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
return float64(val.Int())
case reflect.Uint8, reflect.Uint16, reflect.Uint32:
return float64(val.Uint())
case reflect.Uint, reflect.Uint64:
return float64(val.Uint())
case reflect.Float32, reflect.Float64:
return val.Float()
case reflect.Bool:
if val.Bool() == true {
return 1
}
return 0
default:
return 0
}
}
func toInt(v interface{}) int {
//It's not optimal. Bud I don't want duplicate toInt64 code.
return int(toInt64(v))
}
// toInt64 converts integer types to 64-bit integers
func toInt64(v interface{}) int64 {
if str, ok := v.(string); ok {
iv, err := strconv.ParseInt(str, 10, 64)
if err != nil {
return 0
}
return iv
}
val := reflect.Indirect(reflect.ValueOf(v))
switch val.Kind() {
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
return val.Int()
case reflect.Uint8, reflect.Uint16, reflect.Uint32:
return int64(val.Uint())
case reflect.Uint, reflect.Uint64:
tv := val.Uint()
if tv <= math.MaxInt64 {
return int64(tv)
}
// TODO: What is the sensible thing to do here?
return math.MaxInt64
case reflect.Float32, reflect.Float64:
return int64(val.Float())
case reflect.Bool:
if val.Bool() == true {
return 1
}
return 0
default:
return 0
}
}
func max(a interface{}, i ...interface{}) int64 {
aa := toInt64(a)
for _, b := range i {
bb := toInt64(b)
if bb > aa {
aa = bb
}
}
return aa
}
func min(a interface{}, i ...interface{}) int64 {
aa := toInt64(a)
for _, b := range i {
bb := toInt64(b)
if bb < aa {
aa = bb
}
}
return aa
}
func until(count int) []int {
step := 1
if count < 0 {
step = -1
}
return untilStep(0, count, step)
}
func untilStep(start, stop, step int) []int {
v := []int{}
if stop < start {
if step >= 0 {
return v
}
for i := start; i > stop; i += step {
v = append(v, i)
}
return v
}
if step <= 0 {
return v
}
for i := start; i < stop; i += step {
v = append(v, i)
}
return v
}
func floor(a interface{}) float64 {
aa := toFloat64(a)
return math.Floor(aa)
}
func ceil(a interface{}) float64 {
aa := toFloat64(a)
return math.Ceil(aa)
}
func round(a interface{}, p int, r_opt ...float64) float64 {
roundOn := .5
if len(r_opt) > 0 {
roundOn = r_opt[0]
}
val := toFloat64(a)
places := toFloat64(p)
var round float64
pow := math.Pow(10, places)
digit := pow * val
_, div := math.Modf(digit)
if div >= roundOn {
round = math.Ceil(digit)
} else {
round = math.Floor(digit)
}
return round / pow
}
+28
View File
@@ -0,0 +1,28 @@
package sprig
import (
"fmt"
"reflect"
)
// typeIs returns true if the src is the type named in target.
func typeIs(target string, src interface{}) bool {
return target == typeOf(src)
}
func typeIsLike(target string, src interface{}) bool {
t := typeOf(src)
return target == t || "*"+target == t
}
func typeOf(src interface{}) string {
return fmt.Sprintf("%T", src)
}
func kindIs(target string, src interface{}) bool {
return target == kindOf(src)
}
func kindOf(src interface{}) string {
return reflect.ValueOf(src).Kind().String()
}
+35
View File
@@ -0,0 +1,35 @@
package sprig
import (
"regexp"
)
func regexMatch(regex string, s string) bool {
match, _ := regexp.MatchString(regex, s)
return match
}
func regexFindAll(regex string, s string, n int) []string {
r := regexp.MustCompile(regex)
return r.FindAllString(s, n)
}
func regexFind(regex string, s string) string {
r := regexp.MustCompile(regex)
return r.FindString(s)
}
func regexReplaceAll(regex string, s string, repl string) string {
r := regexp.MustCompile(regex)
return r.ReplaceAllString(s, repl)
}
func regexReplaceAllLiteral(regex string, s string, repl string) string {
r := regexp.MustCompile(regex)
return r.ReplaceAllLiteralString(s, repl)
}
func regexSplit(regex string, s string, n int) []string {
r := regexp.MustCompile(regex)
return r.Split(s, n)
}
+23
View File
@@ -0,0 +1,23 @@
package sprig
import (
sv2 "github.com/Masterminds/semver"
)
func semverCompare(constraint, version string) (bool, error) {
c, err := sv2.NewConstraint(constraint)
if err != nil {
return false, err
}
v, err := sv2.NewVersion(version)
if err != nil {
return false, err
}
return c.Check(v), nil
}
func semver(version string) (*sv2.Version, error) {
return sv2.NewVersion(version)
}
+201
View File
@@ -0,0 +1,201 @@
package sprig
import (
"encoding/base32"
"encoding/base64"
"fmt"
"reflect"
"strconv"
"strings"
util "github.com/aokoli/goutils"
)
func base64encode(v string) string {
return base64.StdEncoding.EncodeToString([]byte(v))
}
func base64decode(v string) string {
data, err := base64.StdEncoding.DecodeString(v)
if err != nil {
return err.Error()
}
return string(data)
}
func base32encode(v string) string {
return base32.StdEncoding.EncodeToString([]byte(v))
}
func base32decode(v string) string {
data, err := base32.StdEncoding.DecodeString(v)
if err != nil {
return err.Error()
}
return string(data)
}
func abbrev(width int, s string) string {
if width < 4 {
return s
}
r, _ := util.Abbreviate(s, width)
return r
}
func abbrevboth(left, right int, s string) string {
if right < 4 || left > 0 && right < 7 {
return s
}
r, _ := util.AbbreviateFull(s, left, right)
return r
}
func initials(s string) string {
// Wrap this just to eliminate the var args, which templates don't do well.
return util.Initials(s)
}
func randAlphaNumeric(count int) string {
// It is not possible, it appears, to actually generate an error here.
r, _ := util.RandomAlphaNumeric(count)
return r
}
func randAlpha(count int) string {
r, _ := util.RandomAlphabetic(count)
return r
}
func randAscii(count int) string {
r, _ := util.RandomAscii(count)
return r
}
func randNumeric(count int) string {
r, _ := util.RandomNumeric(count)
return r
}
func untitle(str string) string {
return util.Uncapitalize(str)
}
func quote(str ...interface{}) string {
out := make([]string, len(str))
for i, s := range str {
out[i] = fmt.Sprintf("%q", strval(s))
}
return strings.Join(out, " ")
}
func squote(str ...interface{}) string {
out := make([]string, len(str))
for i, s := range str {
out[i] = fmt.Sprintf("'%v'", s)
}
return strings.Join(out, " ")
}
func cat(v ...interface{}) string {
r := strings.TrimSpace(strings.Repeat("%v ", len(v)))
return fmt.Sprintf(r, v...)
}
func indent(spaces int, v string) string {
pad := strings.Repeat(" ", spaces)
return pad + strings.Replace(v, "\n", "\n"+pad, -1)
}
func nindent(spaces int, v string) string {
return "\n" + indent(spaces, v)
}
func replace(old, new, src string) string {
return strings.Replace(src, old, new, -1)
}
func plural(one, many string, count int) string {
if count == 1 {
return one
}
return many
}
func strslice(v interface{}) []string {
switch v := v.(type) {
case []string:
return v
case []interface{}:
l := len(v)
b := make([]string, l)
for i := 0; i < l; i++ {
b[i] = strval(v[i])
}
return b
default:
val := reflect.ValueOf(v)
switch val.Kind() {
case reflect.Array, reflect.Slice:
l := val.Len()
b := make([]string, l)
for i := 0; i < l; i++ {
b[i] = strval(val.Index(i).Interface())
}
return b
default:
return []string{strval(v)}
}
}
}
func strval(v interface{}) string {
switch v := v.(type) {
case string:
return v
case []byte:
return string(v)
case error:
return v.Error()
case fmt.Stringer:
return v.String()
default:
return fmt.Sprintf("%v", v)
}
}
func trunc(c int, s string) string {
if len(s) <= c {
return s
}
return s[0:c]
}
func join(sep string, v interface{}) string {
return strings.Join(strslice(v), sep)
}
func split(sep, orig string) map[string]string {
parts := strings.Split(orig, sep)
res := make(map[string]string, len(parts))
for i, v := range parts {
res["_"+strconv.Itoa(i)] = v
}
return res
}
// substring creates a substring of the given string.
//
// If start is < 0, this calls string[:length].
//
// If start is >= 0 and length < 0, this calls string[start:]
//
// Otherwise, this calls string[start, length].
func substring(start, length int, s string) string {
if start < 0 {
return s[:length]
}
if length < 0 {
return s[start:]
}
return s[start:length]
}
+18
View File
@@ -0,0 +1,18 @@
language: go
go:
- 1.6
- 1.7
- 1.8
- tip
script:
- go test -v
notifications:
webhooks:
urls:
- https://webhooks.gitter.im/e/06e3328629952dabe3e0
on_success: change # options: [always|never|change] default: always
on_failure: always # options: [always|never|change] default: always
on_start: never # options: [always|never|change] default: always
+8
View File
@@ -0,0 +1,8 @@
# 1.0.1 (2017-05-31)
## Fixed
- #21: Fix generation of alphanumeric strings (thanks @dbarranco)
# 1.0.0 (2014-04-30)
- Initial release.
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+70
View File
@@ -0,0 +1,70 @@
GoUtils
===========
[![Stability: Maintenance](https://masterminds.github.io/stability/maintenance.svg)](https://masterminds.github.io/stability/maintenance.html)
[![GoDoc](https://godoc.org/github.com/Masterminds/goutils?status.png)](https://godoc.org/github.com/Masterminds/goutils) [![Build Status](https://travis-ci.org/Masterminds/goutils.svg?branch=master)](https://travis-ci.org/Masterminds/goutils) [![Build status](https://ci.appveyor.com/api/projects/status/sc2b1ew0m7f0aiju?svg=true)](https://ci.appveyor.com/project/mattfarina/goutils)
GoUtils provides users with utility functions to manipulate strings in various ways. It is a Go implementation of some
string manipulation libraries of Java Apache Commons. GoUtils includes the following Java Apache Commons classes:
* WordUtils
* RandomStringUtils
* StringUtils (partial implementation)
## Installation
If you have Go set up on your system, from the GOPATH directory within the command line/terminal, enter this:
go get github.com/Masterminds/goutils
If you do not have Go set up on your system, please follow the [Go installation directions from the documenation](http://golang.org/doc/install), and then follow the instructions above to install GoUtils.
## Documentation
GoUtils doc is available here: [![GoDoc](https://godoc.org/github.com/Masterminds/goutils?status.png)](https://godoc.org/github.com/Masterminds/goutils)
## Usage
The code snippets below show examples of how to use GoUtils. Some functions return errors while others do not. The first instance below, which does not return an error, is the `Initials` function (located within the `wordutils.go` file).
package main
import (
"fmt"
"github.com/Masterminds/goutils"
)
func main() {
// EXAMPLE 1: A goutils function which returns no errors
fmt.Println (goutils.Initials("John Doe Foo")) // Prints out "JDF"
}
Some functions return errors mainly due to illegal arguements used as parameters. The code example below illustrates how to deal with function that returns an error. In this instance, the function is the `Random` function (located within the `randomstringutils.go` file).
package main
import (
"fmt"
"github.com/Masterminds/goutils"
)
func main() {
// EXAMPLE 2: A goutils function which returns an error
rand1, err1 := goutils.Random (-1, 0, 0, true, true)
if err1 != nil {
fmt.Println(err1) // Prints out error message because -1 was entered as the first parameter in goutils.Random(...)
} else {
fmt.Println(rand1)
}
}
## License
GoUtils is licensed under the Apache License, Version 2.0. Please check the LICENSE.txt file or visit http://www.apache.org/licenses/LICENSE-2.0 for a copy of the license.
## Issue Reporting
Make suggestions or report issues using the Git issue tracker: https://github.com/Masterminds/goutils/issues
## Website
* [GoUtils webpage](http://Masterminds.github.io/goutils/)
+21
View File
@@ -0,0 +1,21 @@
version: build-{build}.{branch}
clone_folder: C:\gopath\src\github.com\Masterminds\goutils
shallow_clone: true
environment:
GOPATH: C:\gopath
platform:
- x64
build: off
install:
- go version
- go env
test_script:
- go test -v
deploy: off
+268
View File
@@ -0,0 +1,268 @@
/*
Copyright 2014 Alexander Okoli
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package goutils
import (
"fmt"
"math"
"math/rand"
"regexp"
"time"
"unicode"
)
// RANDOM provides the time-based seed used to generate random numbers
var RANDOM = rand.New(rand.NewSource(time.Now().UnixNano()))
/*
RandomNonAlphaNumeric creates a random string whose length is the number of characters specified.
Characters will be chosen from the set of all characters (ASCII/Unicode values between 0 to 2,147,483,647 (math.MaxInt32)).
Parameter:
count - the length of random string to create
Returns:
string - the random string
error - an error stemming from an invalid parameter within underlying function, RandomSeed(...)
*/
func RandomNonAlphaNumeric(count int) (string, error) {
return RandomAlphaNumericCustom(count, false, false)
}
/*
RandomAscii creates a random string whose length is the number of characters specified.
Characters will be chosen from the set of characters whose ASCII value is between 32 and 126 (inclusive).
Parameter:
count - the length of random string to create
Returns:
string - the random string
error - an error stemming from an invalid parameter within underlying function, RandomSeed(...)
*/
func RandomAscii(count int) (string, error) {
return Random(count, 32, 127, false, false)
}
/*
RandomNumeric creates a random string whose length is the number of characters specified.
Characters will be chosen from the set of numeric characters.
Parameter:
count - the length of random string to create
Returns:
string - the random string
error - an error stemming from an invalid parameter within underlying function, RandomSeed(...)
*/
func RandomNumeric(count int) (string, error) {
return Random(count, 0, 0, false, true)
}
/*
RandomAlphabetic creates a random string whose length is the number of characters specified.
Characters will be chosen from the set of alpha-numeric characters as indicated by the arguments.
Parameters:
count - the length of random string to create
letters - if true, generated string may include alphabetic characters
numbers - if true, generated string may include numeric characters
Returns:
string - the random string
error - an error stemming from an invalid parameter within underlying function, RandomSeed(...)
*/
func RandomAlphabetic(count int) (string, error) {
return Random(count, 0, 0, true, false)
}
/*
RandomAlphaNumeric creates a random string whose length is the number of characters specified.
Characters will be chosen from the set of alpha-numeric characters.
Parameter:
count - the length of random string to create
Returns:
string - the random string
error - an error stemming from an invalid parameter within underlying function, RandomSeed(...)
*/
func RandomAlphaNumeric(count int) (string, error) {
RandomString, err := Random(count, 0, 0, true, true)
if err != nil {
return "", fmt.Errorf("Error: %s", err)
}
match, err := regexp.MatchString("([0-9]+)", RandomString)
if err != nil {
panic(err)
}
if !match {
//Get the position between 0 and the length of the string-1 to insert a random number
position := rand.Intn(count)
//Insert a random number between [0-9] in the position
RandomString = RandomString[:position] + string('0'+rand.Intn(10)) + RandomString[position+1:]
return RandomString, err
}
return RandomString, err
}
/*
RandomAlphaNumericCustom creates a random string whose length is the number of characters specified.
Characters will be chosen from the set of alpha-numeric characters as indicated by the arguments.
Parameters:
count - the length of random string to create
letters - if true, generated string may include alphabetic characters
numbers - if true, generated string may include numeric characters
Returns:
string - the random string
error - an error stemming from an invalid parameter within underlying function, RandomSeed(...)
*/
func RandomAlphaNumericCustom(count int, letters bool, numbers bool) (string, error) {
return Random(count, 0, 0, letters, numbers)
}
/*
Random creates a random string based on a variety of options, using default source of randomness.
This method has exactly the same semantics as RandomSeed(int, int, int, bool, bool, []char, *rand.Rand), but
instead of using an externally supplied source of randomness, it uses the internal *rand.Rand instance.
Parameters:
count - the length of random string to create
start - the position in set of chars (ASCII/Unicode int) to start at
end - the position in set of chars (ASCII/Unicode int) to end before
letters - if true, generated string may include alphabetic characters
numbers - if true, generated string may include numeric characters
chars - the set of chars to choose randoms from. If nil, then it will use the set of all chars.
Returns:
string - the random string
error - an error stemming from an invalid parameter within underlying function, RandomSeed(...)
*/
func Random(count int, start int, end int, letters bool, numbers bool, chars ...rune) (string, error) {
return RandomSeed(count, start, end, letters, numbers, chars, RANDOM)
}
/*
RandomSeed creates a random string based on a variety of options, using supplied source of randomness.
If the parameters start and end are both 0, start and end are set to ' ' and 'z', the ASCII printable characters, will be used,
unless letters and numbers are both false, in which case, start and end are set to 0 and math.MaxInt32, respectively.
If chars is not nil, characters stored in chars that are between start and end are chosen.
This method accepts a user-supplied *rand.Rand instance to use as a source of randomness. By seeding a single *rand.Rand instance
with a fixed seed and using it for each call, the same random sequence of strings can be generated repeatedly and predictably.
Parameters:
count - the length of random string to create
start - the position in set of chars (ASCII/Unicode decimals) to start at
end - the position in set of chars (ASCII/Unicode decimals) to end before
letters - if true, generated string may include alphabetic characters
numbers - if true, generated string may include numeric characters
chars - the set of chars to choose randoms from. If nil, then it will use the set of all chars.
random - a source of randomness.
Returns:
string - the random string
error - an error stemming from invalid parameters: if count < 0; or the provided chars array is empty; or end <= start; or end > len(chars)
*/
func RandomSeed(count int, start int, end int, letters bool, numbers bool, chars []rune, random *rand.Rand) (string, error) {
if count == 0 {
return "", nil
} else if count < 0 {
err := fmt.Errorf("randomstringutils illegal argument: Requested random string length %v is less than 0.", count) // equiv to err := errors.New("...")
return "", err
}
if chars != nil && len(chars) == 0 {
err := fmt.Errorf("randomstringutils illegal argument: The chars array must not be empty")
return "", err
}
if start == 0 && end == 0 {
if chars != nil {
end = len(chars)
} else {
if !letters && !numbers {
end = math.MaxInt32
} else {
end = 'z' + 1
start = ' '
}
}
} else {
if end <= start {
err := fmt.Errorf("randomstringutils illegal argument: Parameter end (%v) must be greater than start (%v)", end, start)
return "", err
}
if chars != nil && end > len(chars) {
err := fmt.Errorf("randomstringutils illegal argument: Parameter end (%v) cannot be greater than len(chars) (%v)", end, len(chars))
return "", err
}
}
buffer := make([]rune, count)
gap := end - start
// high-surrogates range, (\uD800-\uDBFF) = 55296 - 56319
// low-surrogates range, (\uDC00-\uDFFF) = 56320 - 57343
for count != 0 {
count--
var ch rune
if chars == nil {
ch = rune(random.Intn(gap) + start)
} else {
ch = chars[random.Intn(gap)+start]
}
if letters && unicode.IsLetter(ch) || numbers && unicode.IsDigit(ch) || !letters && !numbers {
if ch >= 56320 && ch <= 57343 { // low surrogate range
if count == 0 {
count++
} else {
// Insert low surrogate
buffer[count] = ch
count--
// Insert high surrogate
buffer[count] = rune(55296 + random.Intn(128))
}
} else if ch >= 55296 && ch <= 56191 { // High surrogates range (Partial)
if count == 0 {
count++
} else {
// Insert low surrogate
buffer[count] = rune(56320 + random.Intn(128))
count--
// Insert high surrogate
buffer[count] = ch
}
} else if ch >= 56192 && ch <= 56319 {
// private high surrogate, skip it
count++
} else {
// not one of the surrogates*
buffer[count] = ch
}
} else {
count++
}
}
return string(buffer), nil
}
+224
View File
@@ -0,0 +1,224 @@
/*
Copyright 2014 Alexander Okoli
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package goutils
import (
"bytes"
"fmt"
"strings"
"unicode"
)
// Typically returned by functions where a searched item cannot be found
const INDEX_NOT_FOUND = -1
/*
Abbreviate abbreviates a string using ellipses. This will turn the string "Now is the time for all good men" into "Now is the time for..."
Specifically, the algorithm is as follows:
- If str is less than maxWidth characters long, return it.
- Else abbreviate it to (str[0:maxWidth - 3] + "...").
- If maxWidth is less than 4, return an illegal argument error.
- In no case will it return a string of length greater than maxWidth.
Parameters:
str - the string to check
maxWidth - maximum length of result string, must be at least 4
Returns:
string - abbreviated string
error - if the width is too small
*/
func Abbreviate(str string, maxWidth int) (string, error) {
return AbbreviateFull(str, 0, maxWidth)
}
/*
AbbreviateFull abbreviates a string using ellipses. This will turn the string "Now is the time for all good men" into "...is the time for..."
This function works like Abbreviate(string, int), but allows you to specify a "left edge" offset. Note that this left edge is not
necessarily going to be the leftmost character in the result, or the first character following the ellipses, but it will appear
somewhere in the result.
In no case will it return a string of length greater than maxWidth.
Parameters:
str - the string to check
offset - left edge of source string
maxWidth - maximum length of result string, must be at least 4
Returns:
string - abbreviated string
error - if the width is too small
*/
func AbbreviateFull(str string, offset int, maxWidth int) (string, error) {
if str == "" {
return "", nil
}
if maxWidth < 4 {
err := fmt.Errorf("stringutils illegal argument: Minimum abbreviation width is 4")
return "", err
}
if len(str) <= maxWidth {
return str, nil
}
if offset > len(str) {
offset = len(str)
}
if len(str)-offset < (maxWidth - 3) { // 15 - 5 < 10 - 3 = 10 < 7
offset = len(str) - (maxWidth - 3)
}
abrevMarker := "..."
if offset <= 4 {
return str[0:maxWidth-3] + abrevMarker, nil // str.substring(0, maxWidth - 3) + abrevMarker;
}
if maxWidth < 7 {
err := fmt.Errorf("stringutils illegal argument: Minimum abbreviation width with offset is 7")
return "", err
}
if (offset + maxWidth - 3) < len(str) { // 5 + (10-3) < 15 = 12 < 15
abrevStr, _ := Abbreviate(str[offset:len(str)], (maxWidth - 3))
return abrevMarker + abrevStr, nil // abrevMarker + abbreviate(str.substring(offset), maxWidth - 3);
}
return abrevMarker + str[(len(str)-(maxWidth-3)):len(str)], nil // abrevMarker + str.substring(str.length() - (maxWidth - 3));
}
/*
DeleteWhiteSpace deletes all whitespaces from a string as defined by unicode.IsSpace(rune).
It returns the string without whitespaces.
Parameter:
str - the string to delete whitespace from, may be nil
Returns:
the string without whitespaces
*/
func DeleteWhiteSpace(str string) string {
if str == "" {
return str
}
sz := len(str)
var chs bytes.Buffer
count := 0
for i := 0; i < sz; i++ {
ch := rune(str[i])
if !unicode.IsSpace(ch) {
chs.WriteRune(ch)
count++
}
}
if count == sz {
return str
}
return chs.String()
}
/*
IndexOfDifference compares two strings, and returns the index at which the strings begin to differ.
Parameters:
str1 - the first string
str2 - the second string
Returns:
the index where str1 and str2 begin to differ; -1 if they are equal
*/
func IndexOfDifference(str1 string, str2 string) int {
if str1 == str2 {
return INDEX_NOT_FOUND
}
if IsEmpty(str1) || IsEmpty(str2) {
return 0
}
var i int
for i = 0; i < len(str1) && i < len(str2); i++ {
if rune(str1[i]) != rune(str2[i]) {
break
}
}
if i < len(str2) || i < len(str1) {
return i
}
return INDEX_NOT_FOUND
}
/*
IsBlank checks if a string is whitespace or empty (""). Observe the following behavior:
goutils.IsBlank("") = true
goutils.IsBlank(" ") = true
goutils.IsBlank("bob") = false
goutils.IsBlank(" bob ") = false
Parameter:
str - the string to check
Returns:
true - if the string is whitespace or empty ("")
*/
func IsBlank(str string) bool {
strLen := len(str)
if str == "" || strLen == 0 {
return true
}
for i := 0; i < strLen; i++ {
if unicode.IsSpace(rune(str[i])) == false {
return false
}
}
return true
}
/*
IndexOf returns the index of the first instance of sub in str, with the search beginning from the
index start point specified. -1 is returned if sub is not present in str.
An empty string ("") will return -1 (INDEX_NOT_FOUND). A negative start position is treated as zero.
A start position greater than the string length returns -1.
Parameters:
str - the string to check
sub - the substring to find
start - the start position; negative treated as zero
Returns:
the first index where the sub string was found (always >= start)
*/
func IndexOf(str string, sub string, start int) int {
if start < 0 {
start = 0
}
if len(str) < start {
return INDEX_NOT_FOUND
}
if IsEmpty(str) || IsEmpty(sub) {
return INDEX_NOT_FOUND
}
partialIndex := strings.Index(str[start:len(str)], sub)
if partialIndex == -1 {
return INDEX_NOT_FOUND
}
return partialIndex + start
}
// IsEmpty checks if a string is empty (""). Returns true if empty, and false otherwise.
func IsEmpty(str string) bool {
return len(str) == 0
}
+356
View File
@@ -0,0 +1,356 @@
/*
Copyright 2014 Alexander Okoli
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
/*
Package goutils provides utility functions to manipulate strings in various ways.
The code snippets below show examples of how to use goutils. Some functions return
errors while others do not, so usage would vary as a result.
Example:
package main
import (
"fmt"
"github.com/aokoli/goutils"
)
func main() {
// EXAMPLE 1: A goutils function which returns no errors
fmt.Println (goutils.Initials("John Doe Foo")) // Prints out "JDF"
// EXAMPLE 2: A goutils function which returns an error
rand1, err1 := goutils.Random (-1, 0, 0, true, true)
if err1 != nil {
fmt.Println(err1) // Prints out error message because -1 was entered as the first parameter in goutils.Random(...)
} else {
fmt.Println(rand1)
}
}
*/
package goutils
import (
"bytes"
"strings"
"unicode"
)
// VERSION indicates the current version of goutils
const VERSION = "1.0.0"
/*
Wrap wraps a single line of text, identifying words by ' '.
New lines will be separated by '\n'. Very long words, such as URLs will not be wrapped.
Leading spaces on a new line are stripped. Trailing spaces are not stripped.
Parameters:
str - the string to be word wrapped
wrapLength - the column (a column can fit only one character) to wrap the words at, less than 1 is treated as 1
Returns:
a line with newlines inserted
*/
func Wrap(str string, wrapLength int) string {
return WrapCustom(str, wrapLength, "", false)
}
/*
WrapCustom wraps a single line of text, identifying words by ' '.
Leading spaces on a new line are stripped. Trailing spaces are not stripped.
Parameters:
str - the string to be word wrapped
wrapLength - the column number (a column can fit only one character) to wrap the words at, less than 1 is treated as 1
newLineStr - the string to insert for a new line, "" uses '\n'
wrapLongWords - true if long words (such as URLs) should be wrapped
Returns:
a line with newlines inserted
*/
func WrapCustom(str string, wrapLength int, newLineStr string, wrapLongWords bool) string {
if str == "" {
return ""
}
if newLineStr == "" {
newLineStr = "\n" // TODO Assumes "\n" is seperator. Explore SystemUtils.LINE_SEPARATOR from Apache Commons
}
if wrapLength < 1 {
wrapLength = 1
}
inputLineLength := len(str)
offset := 0
var wrappedLine bytes.Buffer
for inputLineLength-offset > wrapLength {
if rune(str[offset]) == ' ' {
offset++
continue
}
end := wrapLength + offset + 1
spaceToWrapAt := strings.LastIndex(str[offset:end], " ") + offset
if spaceToWrapAt >= offset {
// normal word (not longer than wrapLength)
wrappedLine.WriteString(str[offset:spaceToWrapAt])
wrappedLine.WriteString(newLineStr)
offset = spaceToWrapAt + 1
} else {
// long word or URL
if wrapLongWords {
end := wrapLength + offset
// long words are wrapped one line at a time
wrappedLine.WriteString(str[offset:end])
wrappedLine.WriteString(newLineStr)
offset += wrapLength
} else {
// long words aren't wrapped, just extended beyond limit
end := wrapLength + offset
spaceToWrapAt = strings.IndexRune(str[end:len(str)], ' ') + end
if spaceToWrapAt >= 0 {
wrappedLine.WriteString(str[offset:spaceToWrapAt])
wrappedLine.WriteString(newLineStr)
offset = spaceToWrapAt + 1
} else {
wrappedLine.WriteString(str[offset:len(str)])
offset = inputLineLength
}
}
}
}
wrappedLine.WriteString(str[offset:len(str)])
return wrappedLine.String()
}
/*
Capitalize capitalizes all the delimiter separated words in a string. Only the first letter of each word is changed.
To convert the rest of each word to lowercase at the same time, use CapitalizeFully(str string, delimiters ...rune).
The delimiters represent a set of characters understood to separate words. The first string character
and the first non-delimiter character after a delimiter will be capitalized. A "" input string returns "".
Capitalization uses the Unicode title case, normally equivalent to upper case.
Parameters:
str - the string to capitalize
delimiters - set of characters to determine capitalization, exclusion of this parameter means whitespace would be delimeter
Returns:
capitalized string
*/
func Capitalize(str string, delimiters ...rune) string {
var delimLen int
if delimiters == nil {
delimLen = -1
} else {
delimLen = len(delimiters)
}
if str == "" || delimLen == 0 {
return str
}
buffer := []rune(str)
capitalizeNext := true
for i := 0; i < len(buffer); i++ {
ch := buffer[i]
if isDelimiter(ch, delimiters...) {
capitalizeNext = true
} else if capitalizeNext {
buffer[i] = unicode.ToTitle(ch)
capitalizeNext = false
}
}
return string(buffer)
}
/*
CapitalizeFully converts all the delimiter separated words in a string into capitalized words, that is each word is made up of a
titlecase character and then a series of lowercase characters. The delimiters represent a set of characters understood
to separate words. The first string character and the first non-delimiter character after a delimiter will be capitalized.
Capitalization uses the Unicode title case, normally equivalent to upper case.
Parameters:
str - the string to capitalize fully
delimiters - set of characters to determine capitalization, exclusion of this parameter means whitespace would be delimeter
Returns:
capitalized string
*/
func CapitalizeFully(str string, delimiters ...rune) string {
var delimLen int
if delimiters == nil {
delimLen = -1
} else {
delimLen = len(delimiters)
}
if str == "" || delimLen == 0 {
return str
}
str = strings.ToLower(str)
return Capitalize(str, delimiters...)
}
/*
Uncapitalize uncapitalizes all the whitespace separated words in a string. Only the first letter of each word is changed.
The delimiters represent a set of characters understood to separate words. The first string character and the first non-delimiter
character after a delimiter will be uncapitalized. Whitespace is defined by unicode.IsSpace(char).
Parameters:
str - the string to uncapitalize fully
delimiters - set of characters to determine capitalization, exclusion of this parameter means whitespace would be delimeter
Returns:
uncapitalized string
*/
func Uncapitalize(str string, delimiters ...rune) string {
var delimLen int
if delimiters == nil {
delimLen = -1
} else {
delimLen = len(delimiters)
}
if str == "" || delimLen == 0 {
return str
}
buffer := []rune(str)
uncapitalizeNext := true // TODO Always makes capitalize/un apply to first char.
for i := 0; i < len(buffer); i++ {
ch := buffer[i]
if isDelimiter(ch, delimiters...) {
uncapitalizeNext = true
} else if uncapitalizeNext {
buffer[i] = unicode.ToLower(ch)
uncapitalizeNext = false
}
}
return string(buffer)
}
/*
SwapCase swaps the case of a string using a word based algorithm.
Conversion algorithm:
Upper case character converts to Lower case
Title case character converts to Lower case
Lower case character after Whitespace or at start converts to Title case
Other Lower case character converts to Upper case
Whitespace is defined by unicode.IsSpace(char).
Parameters:
str - the string to swap case
Returns:
the changed string
*/
func SwapCase(str string) string {
if str == "" {
return str
}
buffer := []rune(str)
whitespace := true
for i := 0; i < len(buffer); i++ {
ch := buffer[i]
if unicode.IsUpper(ch) {
buffer[i] = unicode.ToLower(ch)
whitespace = false
} else if unicode.IsTitle(ch) {
buffer[i] = unicode.ToLower(ch)
whitespace = false
} else if unicode.IsLower(ch) {
if whitespace {
buffer[i] = unicode.ToTitle(ch)
whitespace = false
} else {
buffer[i] = unicode.ToUpper(ch)
}
} else {
whitespace = unicode.IsSpace(ch)
}
}
return string(buffer)
}
/*
Initials extracts the initial letters from each word in the string. The first letter of the string and all first
letters after the defined delimiters are returned as a new string. Their case is not changed. If the delimiters
parameter is excluded, then Whitespace is used. Whitespace is defined by unicode.IsSpacea(char). An empty delimiter array returns an empty string.
Parameters:
str - the string to get initials from
delimiters - set of characters to determine words, exclusion of this parameter means whitespace would be delimeter
Returns:
string of initial letters
*/
func Initials(str string, delimiters ...rune) string {
if str == "" {
return str
}
if delimiters != nil && len(delimiters) == 0 {
return ""
}
strLen := len(str)
var buf bytes.Buffer
lastWasGap := true
for i := 0; i < strLen; i++ {
ch := rune(str[i])
if isDelimiter(ch, delimiters...) {
lastWasGap = true
} else if lastWasGap {
buf.WriteRune(ch)
lastWasGap = false
}
}
return buf.String()
}
// private function (lower case func name)
func isDelimiter(ch rune, delimiters ...rune) bool {
if delimiters == nil {
return unicode.IsSpace(ch)
}
for _, delimiter := range delimiters {
if ch == delimiter {
return true
}
}
return false
}
+9
View File
@@ -0,0 +1,9 @@
language: go
go:
- 1.4.3
- 1.5.3
- tip
script:
- go test -v ./...
+10
View File
@@ -0,0 +1,10 @@
# How to contribute
We definitely welcome patches and contribution to this project!
### Legal requirements
In order to protect both you and ourselves, you will need to sign the
[Contributor License Agreement](https://cla.developers.google.com/clas).
You may have already signed it for other Google projects.
+9
View File
@@ -0,0 +1,9 @@
Paul Borman <borman@google.com>
bmatsuo
shawnps
theory
jboverfelt
dsymonds
cd1
wallclockbuilder
dansouza
+27
View File
@@ -0,0 +1,27 @@
Copyright (c) 2009,2014 Google Inc. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+23
View File
@@ -0,0 +1,23 @@
**This package is currently in development and the API may not be stable.**
The API will become stable with v1.
# uuid ![build status](https://travis-ci.org/google/uuid.svg?branch=master)
The uuid package generates and inspects UUIDs based on
[RFC 4122](http://tools.ietf.org/html/rfc4122)
and DCE 1.1: Authentication and Security Services.
This package is based on the github.com/pborman/uuid package (previously named
code.google.com/p/go-uuid). It differs from these earlier packages in that
a UUID is a 16 byte array rather than a byte slice. One loss due to this
change is the ability to represent an invalid UUID (vs a NIL UUID).
###### Install
`go get github.com/google/uuid`
###### Documentation
[![GoDoc](https://godoc.org/github.com/google/uuid?status.svg)](http://godoc.org/github.com/google/uuid)
Full `go doc` style documentation for the package can be viewed online without
installing this package by using the GoDoc site here:
http://godoc.org/github.com/google/uuid
+80
View File
@@ -0,0 +1,80 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"encoding/binary"
"fmt"
"os"
)
// A Domain represents a Version 2 domain
type Domain byte
// Domain constants for DCE Security (Version 2) UUIDs.
const (
Person = Domain(0)
Group = Domain(1)
Org = Domain(2)
)
// NewDCESecurity returns a DCE Security (Version 2) UUID.
//
// The domain should be one of Person, Group or Org.
// On a POSIX system the id should be the users UID for the Person
// domain and the users GID for the Group. The meaning of id for
// the domain Org or on non-POSIX systems is site defined.
//
// For a given domain/id pair the same token may be returned for up to
// 7 minutes and 10 seconds.
func NewDCESecurity(domain Domain, id uint32) (UUID, error) {
uuid, err := NewUUID()
if err == nil {
uuid[6] = (uuid[6] & 0x0f) | 0x20 // Version 2
uuid[9] = byte(domain)
binary.BigEndian.PutUint32(uuid[0:], id)
}
return uuid, err
}
// NewDCEPerson returns a DCE Security (Version 2) UUID in the person
// domain with the id returned by os.Getuid.
//
// NewDCEPerson(Person, uint32(os.Getuid()))
func NewDCEPerson() (UUID, error) {
return NewDCESecurity(Person, uint32(os.Getuid()))
}
// NewDCEGroup returns a DCE Security (Version 2) UUID in the group
// domain with the id returned by os.Getgid.
//
// NewDCEGroup(Group, uint32(os.Getgid()))
func NewDCEGroup() (UUID, error) {
return NewDCESecurity(Group, uint32(os.Getgid()))
}
// Domain returns the domain for a Version 2 UUID. Domains are only defined
// for Version 2 UUIDs.
func (uuid UUID) Domain() Domain {
return Domain(uuid[9])
}
// ID returns the id for a Version 2 UUID. IDs are only defined for Version 2
// UUIDs.
func (uuid UUID) ID() uint32 {
return binary.BigEndian.Uint32(uuid[0:4])
}
func (d Domain) String() string {
switch d {
case Person:
return "Person"
case Group:
return "Group"
case Org:
return "Org"
}
return fmt.Sprintf("Domain%d", int(d))
}
+12
View File
@@ -0,0 +1,12 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package uuid generates and inspects UUIDs.
//
// UUIDs are based on RFC 4122 and DCE 1.1: Authentication and Security
// Services.
//
// A UUID is a 16 byte (128 bit) array. UUIDs may be used as keys to
// maps or compared directly.
package uuid
+53
View File
@@ -0,0 +1,53 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"crypto/md5"
"crypto/sha1"
"hash"
)
// Well known namespace IDs and UUIDs
var (
NameSpaceDNS = Must(Parse("6ba7b810-9dad-11d1-80b4-00c04fd430c8"))
NameSpaceURL = Must(Parse("6ba7b811-9dad-11d1-80b4-00c04fd430c8"))
NameSpaceOID = Must(Parse("6ba7b812-9dad-11d1-80b4-00c04fd430c8"))
NameSpaceX500 = Must(Parse("6ba7b814-9dad-11d1-80b4-00c04fd430c8"))
Nil UUID // empty UUID, all zeros
)
// NewHash returns a new UUID derived from the hash of space concatenated with
// data generated by h. The hash should be at least 16 byte in length. The
// first 16 bytes of the hash are used to form the UUID. The version of the
// UUID will be the lower 4 bits of version. NewHash is used to implement
// NewMD5 and NewSHA1.
func NewHash(h hash.Hash, space UUID, data []byte, version int) UUID {
h.Reset()
h.Write(space[:])
h.Write([]byte(data))
s := h.Sum(nil)
var uuid UUID
copy(uuid[:], s)
uuid[6] = (uuid[6] & 0x0f) | uint8((version&0xf)<<4)
uuid[8] = (uuid[8] & 0x3f) | 0x80 // RFC 4122 variant
return uuid
}
// NewMD5 returns a new MD5 (Version 3) UUID based on the
// supplied name space and data. It is the same as calling:
//
// NewHash(md5.New(), space, data, 3)
func NewMD5(space UUID, data []byte) UUID {
return NewHash(md5.New(), space, data, 3)
}
// NewSHA1 returns a new SHA1 (Version 5) UUID based on the
// supplied name space and data. It is the same as calling:
//
// NewHash(sha1.New(), space, data, 5)
func NewSHA1(space UUID, data []byte) UUID {
return NewHash(sha1.New(), space, data, 5)
}
+39
View File
@@ -0,0 +1,39 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import "fmt"
// MarshalText implements encoding.TextMarshaler.
func (uuid UUID) MarshalText() ([]byte, error) {
var js [36]byte
encodeHex(js[:], uuid)
return js[:], nil
}
// UnmarshalText implements encoding.TextUnmarshaler.
func (uuid *UUID) UnmarshalText(data []byte) error {
// See comment in ParseBytes why we do this.
// id, err := ParseBytes(data)
id, err := ParseBytes(data)
if err == nil {
*uuid = id
}
return err
}
// MarshalBinary implements encoding.BinaryMarshaler.
func (uuid UUID) MarshalBinary() ([]byte, error) {
return uuid[:], nil
}
// UnmarshalBinary implements encoding.BinaryUnmarshaler.
func (uuid *UUID) UnmarshalBinary(data []byte) error {
if len(data) != 16 {
return fmt.Errorf("invalid UUID (got %d bytes)", len(data))
}
copy(uuid[:], data)
return nil
}
+103
View File
@@ -0,0 +1,103 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"net"
"sync"
)
var (
nodeMu sync.Mutex
interfaces []net.Interface // cached list of interfaces
ifname string // name of interface being used
nodeID [6]byte // hardware for version 1 UUIDs
zeroID [6]byte // nodeID with only 0's
)
// NodeInterface returns the name of the interface from which the NodeID was
// derived. The interface "user" is returned if the NodeID was set by
// SetNodeID.
func NodeInterface() string {
defer nodeMu.Unlock()
nodeMu.Lock()
return ifname
}
// SetNodeInterface selects the hardware address to be used for Version 1 UUIDs.
// If name is "" then the first usable interface found will be used or a random
// Node ID will be generated. If a named interface cannot be found then false
// is returned.
//
// SetNodeInterface never fails when name is "".
func SetNodeInterface(name string) bool {
defer nodeMu.Unlock()
nodeMu.Lock()
return setNodeInterface(name)
}
func setNodeInterface(name string) bool {
if interfaces == nil {
var err error
interfaces, err = net.Interfaces()
if err != nil && name != "" {
return false
}
}
for _, ifs := range interfaces {
if len(ifs.HardwareAddr) >= 6 && (name == "" || name == ifs.Name) {
copy(nodeID[:], ifs.HardwareAddr)
ifname = ifs.Name
return true
}
}
// We found no interfaces with a valid hardware address. If name
// does not specify a specific interface generate a random Node ID
// (section 4.1.6)
if name == "" {
randomBits(nodeID[:])
return true
}
return false
}
// NodeID returns a slice of a copy of the current Node ID, setting the Node ID
// if not already set.
func NodeID() []byte {
defer nodeMu.Unlock()
nodeMu.Lock()
if nodeID == zeroID {
setNodeInterface("")
}
nid := nodeID
return nid[:]
}
// SetNodeID sets the Node ID to be used for Version 1 UUIDs. The first 6 bytes
// of id are used. If id is less than 6 bytes then false is returned and the
// Node ID is not set.
func SetNodeID(id []byte) bool {
if len(id) < 6 {
return false
}
defer nodeMu.Unlock()
nodeMu.Lock()
copy(nodeID[:], id)
ifname = "user"
return true
}
// NodeID returns the 6 byte node id encoded in uuid. It returns nil if uuid is
// not valid. The NodeID is only well defined for version 1 and 2 UUIDs.
func (uuid UUID) NodeID() []byte {
if len(uuid) != 16 {
return nil
}
var node [6]byte
copy(node[:], uuid[10:])
return node[:]
}
+58
View File
@@ -0,0 +1,58 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"database/sql/driver"
"fmt"
)
// Scan implements sql.Scanner so UUIDs can be read from databases transparently
// Currently, database types that map to string and []byte are supported. Please
// consult database-specific driver documentation for matching types.
func (uuid *UUID) Scan(src interface{}) error {
switch src.(type) {
case string:
// if an empty UUID comes from a table, we return a null UUID
if src.(string) == "" {
return nil
}
// see Parse for required string format
u, err := Parse(src.(string))
if err != nil {
return fmt.Errorf("Scan: %v", err)
}
*uuid = u
case []byte:
b := src.([]byte)
// if an empty UUID comes from a table, we return a null UUID
if len(b) == 0 {
return nil
}
// assumes a simple slice of bytes if 16 bytes
// otherwise attempts to parse
if len(b) != 16 {
return uuid.Scan(string(b))
}
copy((*uuid)[:], b)
default:
return fmt.Errorf("Scan: unable to scan type %T into UUID", src)
}
return nil
}
// Value implements sql.Valuer so that UUIDs can be written to databases
// transparently. Currently, UUIDs map to strings. Please consult
// database-specific driver documentation for matching types.
func (uuid UUID) Value() (driver.Value, error) {
return uuid.String(), nil
}
+123
View File
@@ -0,0 +1,123 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"encoding/binary"
"sync"
"time"
)
// A Time represents a time as the number of 100's of nanoseconds since 15 Oct
// 1582.
type Time int64
const (
lillian = 2299160 // Julian day of 15 Oct 1582
unix = 2440587 // Julian day of 1 Jan 1970
epoch = unix - lillian // Days between epochs
g1582 = epoch * 86400 // seconds between epochs
g1582ns100 = g1582 * 10000000 // 100s of a nanoseconds between epochs
)
var (
timeMu sync.Mutex
lasttime uint64 // last time we returned
clockSeq uint16 // clock sequence for this run
timeNow = time.Now // for testing
)
// UnixTime converts t the number of seconds and nanoseconds using the Unix
// epoch of 1 Jan 1970.
func (t Time) UnixTime() (sec, nsec int64) {
sec = int64(t - g1582ns100)
nsec = (sec % 10000000) * 100
sec /= 10000000
return sec, nsec
}
// GetTime returns the current Time (100s of nanoseconds since 15 Oct 1582) and
// clock sequence as well as adjusting the clock sequence as needed. An error
// is returned if the current time cannot be determined.
func GetTime() (Time, uint16, error) {
defer timeMu.Unlock()
timeMu.Lock()
return getTime()
}
func getTime() (Time, uint16, error) {
t := timeNow()
// If we don't have a clock sequence already, set one.
if clockSeq == 0 {
setClockSequence(-1)
}
now := uint64(t.UnixNano()/100) + g1582ns100
// If time has gone backwards with this clock sequence then we
// increment the clock sequence
if now <= lasttime {
clockSeq = ((clockSeq + 1) & 0x3fff) | 0x8000
}
lasttime = now
return Time(now), clockSeq, nil
}
// ClockSequence returns the current clock sequence, generating one if not
// already set. The clock sequence is only used for Version 1 UUIDs.
//
// The uuid package does not use global static storage for the clock sequence or
// the last time a UUID was generated. Unless SetClockSequence is used, a new
// random clock sequence is generated the first time a clock sequence is
// requested by ClockSequence, GetTime, or NewUUID. (section 4.2.1.1)
func ClockSequence() int {
defer timeMu.Unlock()
timeMu.Lock()
return clockSequence()
}
func clockSequence() int {
if clockSeq == 0 {
setClockSequence(-1)
}
return int(clockSeq & 0x3fff)
}
// SetClockSeq sets the clock sequence to the lower 14 bits of seq. Setting to
// -1 causes a new sequence to be generated.
func SetClockSequence(seq int) {
defer timeMu.Unlock()
timeMu.Lock()
setClockSequence(seq)
}
func setClockSequence(seq int) {
if seq == -1 {
var b [2]byte
randomBits(b[:]) // clock sequence
seq = int(b[0])<<8 | int(b[1])
}
old_seq := clockSeq
clockSeq = uint16(seq&0x3fff) | 0x8000 // Set our variant
if old_seq != clockSeq {
lasttime = 0
}
}
// Time returns the time in 100s of nanoseconds since 15 Oct 1582 encoded in
// uuid. The time is only defined for version 1 and 2 UUIDs.
func (uuid UUID) Time() Time {
time := int64(binary.BigEndian.Uint32(uuid[0:4]))
time |= int64(binary.BigEndian.Uint16(uuid[4:6])) << 32
time |= int64(binary.BigEndian.Uint16(uuid[6:8])&0xfff) << 48
return Time(time)
}
// ClockSequence returns the clock sequence encoded in uuid.
// The clock sequence is only well defined for version 1 and 2 UUIDs.
func (uuid UUID) ClockSequence() int {
return int(binary.BigEndian.Uint16(uuid[8:10])) & 0x3fff
}
+43
View File
@@ -0,0 +1,43 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"io"
)
// randomBits completely fills slice b with random data.
func randomBits(b []byte) {
if _, err := io.ReadFull(rander, b); err != nil {
panic(err.Error()) // rand should never fail
}
}
// xvalues returns the value of a byte as a hexadecimal digit or 255.
var xvalues = [256]byte{
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 255, 255, 255, 255, 255, 255,
255, 10, 11, 12, 13, 14, 15, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 10, 11, 12, 13, 14, 15, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
}
// xtob converts hex characters x1 and x2 into a byte.
func xtob(x1, x2 byte) (byte, bool) {
b1 := xvalues[x1]
b2 := xvalues[x2]
return (b1 << 4) | b2, b1 != 255 && b2 != 255
}
+191
View File
@@ -0,0 +1,191 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"bytes"
"crypto/rand"
"encoding/hex"
"errors"
"fmt"
"io"
"strings"
)
// A UUID is a 128 bit (16 byte) Universal Unique IDentifier as defined in RFC
// 4122.
type UUID [16]byte
// A Version represents a UUID's version.
type Version byte
// A Variant represents a UUID's variant.
type Variant byte
// Constants returned by Variant.
const (
Invalid = Variant(iota) // Invalid UUID
RFC4122 // The variant specified in RFC4122
Reserved // Reserved, NCS backward compatibility.
Microsoft // Reserved, Microsoft Corporation backward compatibility.
Future // Reserved for future definition.
)
var rander = rand.Reader // random function
// Parse decodes s into a UUID or returns an error. Both the UUID form of
// xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx and
// urn:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx are decoded.
func Parse(s string) (UUID, error) {
var uuid UUID
if len(s) != 36 {
if len(s) != 36+9 {
return uuid, fmt.Errorf("invalid UUID length: %d", len(s))
}
if strings.ToLower(s[:9]) != "urn:uuid:" {
return uuid, fmt.Errorf("invalid urn prefix: %q", s[:9])
}
s = s[9:]
}
if s[8] != '-' || s[13] != '-' || s[18] != '-' || s[23] != '-' {
return uuid, errors.New("invalid UUID format")
}
for i, x := range [16]int{
0, 2, 4, 6,
9, 11,
14, 16,
19, 21,
24, 26, 28, 30, 32, 34} {
if v, ok := xtob(s[x], s[x+1]); !ok {
return uuid, errors.New("invalid UUID format")
} else {
uuid[i] = v
}
}
return uuid, nil
}
// ParseBytes is like Parse, except it parses a byte slice instead of a string.
func ParseBytes(b []byte) (UUID, error) {
var uuid UUID
if len(b) != 36 {
if len(b) != 36+9 {
return uuid, fmt.Errorf("invalid UUID length: %d", len(b))
}
if !bytes.Equal(bytes.ToLower(b[:9]), []byte("urn:uuid:")) {
return uuid, fmt.Errorf("invalid urn prefix: %q", b[:9])
}
b = b[9:]
}
if b[8] != '-' || b[13] != '-' || b[18] != '-' || b[23] != '-' {
return uuid, errors.New("invalid UUID format")
}
for i, x := range [16]int{
0, 2, 4, 6,
9, 11,
14, 16,
19, 21,
24, 26, 28, 30, 32, 34} {
if v, ok := xtob(b[x], b[x+1]); !ok {
return uuid, errors.New("invalid UUID format")
} else {
uuid[i] = v
}
}
return uuid, nil
}
// Must returns uuid if err is nil and panics otherwise.
func Must(uuid UUID, err error) UUID {
if err != nil {
panic(err)
}
return uuid
}
// String returns the string form of uuid, xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
// , or "" if uuid is invalid.
func (uuid UUID) String() string {
var buf [36]byte
encodeHex(buf[:], uuid)
return string(buf[:])
}
// URN returns the RFC 2141 URN form of uuid,
// urn:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, or "" if uuid is invalid.
func (uuid UUID) URN() string {
var buf [36 + 9]byte
copy(buf[:], "urn:uuid:")
encodeHex(buf[9:], uuid)
return string(buf[:])
}
func encodeHex(dst []byte, uuid UUID) {
hex.Encode(dst[:], uuid[:4])
dst[8] = '-'
hex.Encode(dst[9:13], uuid[4:6])
dst[13] = '-'
hex.Encode(dst[14:18], uuid[6:8])
dst[18] = '-'
hex.Encode(dst[19:23], uuid[8:10])
dst[23] = '-'
hex.Encode(dst[24:], uuid[10:])
}
// Variant returns the variant encoded in uuid.
func (uuid UUID) Variant() Variant {
switch {
case (uuid[8] & 0xc0) == 0x80:
return RFC4122
case (uuid[8] & 0xe0) == 0xc0:
return Microsoft
case (uuid[8] & 0xe0) == 0xe0:
return Future
default:
return Reserved
}
}
// Version returns the version of uuid.
func (uuid UUID) Version() Version {
return Version(uuid[6] >> 4)
}
func (v Version) String() string {
if v > 15 {
return fmt.Sprintf("BAD_VERSION_%d", v)
}
return fmt.Sprintf("VERSION_%d", v)
}
func (v Variant) String() string {
switch v {
case RFC4122:
return "RFC4122"
case Reserved:
return "Reserved"
case Microsoft:
return "Microsoft"
case Future:
return "Future"
case Invalid:
return "Invalid"
}
return fmt.Sprintf("BadVariant%d", int(v))
}
// SetRand sets the random number generator to r, which implents io.Reader.
// If r.Read returns an error when the package requests random data then
// a panic will be issued.
//
// Calling SetRand with nil sets the random number generator to the default
// generator.
func SetRand(r io.Reader) {
if r == nil {
rander = rand.Reader
return
}
rander = r
}
+44
View File
@@ -0,0 +1,44 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import (
"encoding/binary"
)
// NewUUID returns a Version 1 UUID based on the current NodeID and clock
// sequence, and the current time. If the NodeID has not been set by SetNodeID
// or SetNodeInterface then it will be set automatically. If the NodeID cannot
// be set NewUUID returns nil. If clock sequence has not been set by
// SetClockSequence then it will be set automatically. If GetTime fails to
// return the current NewUUID returns Nil and an error.
//
// In most cases, New should be used.
func NewUUID() (UUID, error) {
nodeMu.Lock()
if nodeID == zeroID {
setNodeInterface("")
}
nodeMu.Unlock()
var uuid UUID
now, seq, err := GetTime()
if err != nil {
return uuid, err
}
timeLow := uint32(now & 0xffffffff)
timeMid := uint16((now >> 32) & 0xffff)
timeHi := uint16((now >> 48) & 0x0fff)
timeHi |= 0x1000 // Version 1
binary.BigEndian.PutUint32(uuid[0:], timeLow)
binary.BigEndian.PutUint16(uuid[4:], timeMid)
binary.BigEndian.PutUint16(uuid[6:], timeHi)
binary.BigEndian.PutUint16(uuid[8:], seq)
copy(uuid[10:], nodeID[:])
return uuid, nil
}
+38
View File
@@ -0,0 +1,38 @@
// Copyright 2016 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package uuid
import "io"
// New is creates a new random UUID or panics. New is equivalent to
// the expression
//
// uuid.Must(uuid.NewRandom())
func New() UUID {
return Must(NewRandom())
}
// NewRandom returns a Random (Version 4) UUID or panics.
//
// The strength of the UUIDs is based on the strength of the crypto/rand
// package.
//
// A note about uniqueness derived from from the UUID Wikipedia entry:
//
// Randomly generated UUIDs have 122 random bits. One's annual risk of being
// hit by a meteorite is estimated to be one chance in 17 billion, that
// means the probability is about 0.00000000006 (6 × 10−11),
// equivalent to the odds of creating a few tens of trillions of UUIDs in a
// year and having one duplicate.
func NewRandom() (UUID, error) {
var uuid UUID
_, err := io.ReadFull(rander, uuid[:])
if err != nil {
return Nil, err
}
uuid[6] = (uuid[6] & 0x0f) | 0x40 // Version 4
uuid[8] = (uuid[8] & 0x3f) | 0x80 // Variant is 10
return uuid, nil
}
+24
View File
@@ -0,0 +1,24 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
*.test
*.prof
+1
View File
@@ -0,0 +1 @@
language: go

Some files were not shown because too many files have changed in this diff Show More