Compare commits

...
172 Commits
Author SHA1 Message Date
KUOKA Yusuke f84ed93754 fix: helmfle should not break when run without -f (#453)
Fixes #451
2019-01-25 12:13:56 +09:00
KUOKA Yusuke bf42d2519d fix: Relative path in helmfile not relative to file in multi file setup (#448)
Fixes #431
2019-01-24 10:48:01 +09:00
Maxim Mironenko 99ce8570c7 helmfiles examples link URL fix (#447)
* helmfiles examples link URL fix

* Update README.md

Co-Authored-By: maximmi <mmironenko@chanzuckerberg.com>
2019-01-24 10:46:59 +09:00
KUOKA Yusuke 571f351a8f feat: configurable concurrency for helmfile test (#442)
`helmfile test --concurency N` to set a concurrency number.
It is automatically ceiled at the number of releases just to reduce wasting computing resources.

Also, I've refactored the scatter-gather logic scattered across the code-base.

Resolves #433
2019-01-23 15:46:24 +09:00
KUOKA Yusuke b9a097ed14 fix: first-pass rendering of helmfile.yaml (#441)
There was a regression introduced by #439 that broke the first-pass rendering completely. This fixes that.
2019-01-22 15:55:03 +09:00
Yusuke KUOKA 344c20d790 release: fix make tasks for auto-tagging 2019-01-22 11:25:55 +09:00
KUOKA Yusuke 5f52f96709 fix missingFileHandler (#440)
- Fix panics when `missingFileHandler` is not defined
- Fix `missingFileHandler: Error` had been writing errors at the Warn level
- Add `Info` and `Debug` as valid values

Ref https://github.com/roboll/helmfile/pull/439#issuecomment-456155521
2019-01-22 11:22:53 +09:00
Yusuke KUOKA 171eacfb3c release: fix hack/semtag when there are one or more git remotes before origin 2019-01-22 01:29:28 +09:00
Yusuke KUOKA 7d976e5271 Semi-automate minor/patch version tagging 2019-01-22 01:23:22 +09:00
KUOKA Yusuke f813ac2642 feat: Release Template (#439)
This feature is supposed to help advanced use-cases like Conventional Directory Structure explained in several issues like #428.

Newly added configuration keys `templates`, `missingFileHandler`, and the ability to defer executing template expressions in `values`, `secrets`, `namespace`, and `chart` of releases allows you to abstract away repetitions into a reusable template:

```yaml
templates:
  default: &default
    missingFileHandler: Warn
    namespace: "{{`{{ .Release.Name }}`}}"
    chart: stable/{{`{{ .Release.Name }}`}}
    values:
    - config/{{`{{ .Release.Name }}`}}/values.yaml
    - config/{{`{{ .Release.Name }}`}}/{{`{{ .Environment.Name }}`}}.yaml
    secrets:
    - config/{{`{{ .Release.Name }}`}}/secrets.yaml
    - config/{{`{{ .Release.Name }}`}}/{{`{{ .Environment.Name }}`}}-secrets.yaml

releases:
- name: envoy
  <<: *default
```

See the updated documentation for more details.

Resolves #428
2019-01-22 01:19:07 +09:00
KUOKA Yusuke 23178b398c docs: Remove go get from the recommended installation methods (#437)
Add a contribution guide instead and note about the `go get` way there according to the discussion made in the relevant issue.

Resolves #393
2019-01-17 12:03:58 +09:00
Vincent Behar fea62032e9 fix: cleanup hook executed too early for apply command (#426)
fixes https://github.com/roboll/helmfile/issues/412

the `apply` command first runs the `diff` command, which triggers the execution of the `cleanup` hooks at the end of the `diff`.
2019-01-01 01:33:16 +09:00
Vincent Behar 1f7a8ddbe3 fix: prepare hook is not executed for diff command (#425)
fixes https://github.com/roboll/helmfile/issues/424

note that the "cleanup" hook is already executed as it should
2019-01-01 01:30:25 +09:00
Raphael Sampaio 413ab4f55b adds Homebrew as an installation method (#418) 2018-12-21 12:06:33 +09:00
Hung Tran Duc 39bc28c23f feat: configurable common path prefix for values files (#408)
Removes duplicated declaration path for cleaner helmfile, for example, this configuration.

```
- chart: projectABC
  name: {{ .Environment.Name }}-projectABC
  namespace: abc
  version: {{ .Environment.Values.projectABC.chart.version }}
  valuesPathPrefix: ../values/{{ .Environment.Name }}/
  values:
  - common.yaml
  - project-abc.yaml
  - platform.yaml
  - ...
  secrets:
  - secrets.yaml
```
2018-12-13 18:07:50 +09:00
Matteo Gazzetta 4b22d4fc78 Update Dockerfile and CircleCI (#417) 2018-12-11 07:01:23 +09:00
Eric Chin 2a35b6864c Remove extra appending of chartName for lint and template (#406)
Closes #407
2018-12-11 06:54:57 +09:00
Anatoly Rugalev 4b23213ce4 Added new installation method via AUR (#405) 2018-11-27 14:13:45 -08:00
Anatoly Rugalev 971cbcf1db Added path normalization in - set directive. (#404)
Fixes #399
2018-11-27 14:13:04 -08:00
Anatoly Rugalev d898dfbd0d Updated documentation related to deprecation of context directive (#403) 2018-11-27 14:12:40 -08:00
Anatoly Rugalev b41b44c313 Fixes #383. Deprecated context directive in favor of helmDefaults.kubeContext (#401) 2018-11-22 12:39:03 +09:00
Sam Weston b3e27db8b3 Improve documentation of Environment Secrets (#395) 2018-11-09 16:38:23 +09:00
harmjanblok 9cab7b00f6 Bump helm to v2.11.0 (#394) 2018-11-09 16:36:25 +09:00
Sam Weston bfc86de92d Correct minor spelling and grammar issues in README.md (#385) 2018-10-23 15:21:00 +09:00
Martin Mauch 5ad916003f Add instructions for installing via scoop on Windows (#372) 2018-10-16 16:22:08 +09:00
KUOKA Yusuke b0f9062e08 fix: incomplete environment values merging (#378)
Fixes #377

Ref https://github.com/imdario/mergo/pull/81
2018-10-03 22:39:37 +09:00
KUOKA Yusuke 770c3daa5f feat: get and getOrNil template funcs to allow defaulting in templates (#370)
* feat: `get` and `getOrNil` template funcs to allow defaulting in templates

Ref #357

* Add docs about missing keys and default values in templates
2018-09-28 11:44:49 +09:00
KUOKA Yusuke 6f0dc6e069 bump go to 1.11 (#376)
In go 1.11, the code formatter seems to have changed its formatting rule a bit, that resulted in `make fmt` producing changes introduced in this commit/pr.
2018-09-28 10:48:07 +09:00
KUOKA Yusuke f23a43c80c fix(ci,doc): follow up for default non-interactivity of helmfile (#375)
The integration test was failing due to that it was still using `--auto-approve` flag which was removed. The documentation was still refering to `--auto-approve` and had no explanation about the new `--interactive` flag. This fixes all these issues.

This a follow up for #368
See https://github.com/roboll/helmfile/pull/374#issuecomment-425291468 for more context.
2018-09-28 10:40:57 +09:00
David Genest 942f9a20b0 fix first pass rendering crash on syntax error (#374) 2018-09-28 10:27:11 +09:00
KUOKA Yusuke 429c42d429 doc: mark release as installed/uninstalled (#371)
Resolves #362
2018-09-27 11:17:00 +09:00
KUOKA Yusuke 7d1787d0b6 One Ctrl-C should exit helmfile gracefully (#369)
Perhaps in latest commits I unexpectedly changed it to require double Ctrl-C to actually interrupt. This fixes it so that one Ctrl-C exists helmfile.

On the way to the fix, I changed the exit-code to 128 + SIGNAL(2 for SIGINT, 15 for SIGTERM) according to common *nix commands.
2018-09-27 04:36:34 +09:00
KUOKA Yusuke 98be623701 helmfile should be non-interactive by default (#368)
Request for user confirmation only when a global `--interactive` flag is provided.

Resolves #354
2018-09-27 04:36:14 +09:00
KUOKA Yusuke 76122738c8 fix: cleanup hook commands output should be printed when debug logging enabled (#367)
There was a race condition due to that helmfile had not been waiting for goroutines to exit

Ref #363
2018-09-27 03:21:58 +09:00
KUOKA Yusuke 973d47bfc0 fix: sync repos always failing (#366) 2018-09-27 02:32:56 +09:00
KUOKA Yusuke 2eb13c25c5 fix: helmfile should pass release names to helm template (#365)
Fixes #360
2018-09-27 02:22:18 +09:00
KUOKA Yusuke b94265122f feat: least frequent repository update (#356)
Prevents helmfile from consuming unnecessarily much time in running `helm repo update` over and over.

helmfile now marks which repository was updated, and skip second and further `helm repo update` when all the `repositories` found in a helmfile.yaml was marked as already updated.

Let's say you had two helmfiles, the first one with repositories `foo` and `bar`, an the second one with only `bar`. `helmfile repos` will run `helm update repo` for the first helmfile, marking `foo` and `bar` as already updated. The second helmfile.yaml contains only `bar`, which is marked as already updated. So helmfile won't run `helm repo update` for the second.

This applies to all the helmfile command that results in `helm repo update`, like `repos`, `sync`, `diff` and so on.

Resolves #335
2018-09-27 02:10:11 +09:00
KUOKA Yusuke f2b610afdf feat: helmfile repos updates all repos regardless of selector and releases (#353)
Resolves #338
2018-09-27 01:44:05 +09:00
KUOKA Yusuke 6cba77d4f2 fix(hook): event hooks should run on remote charts, too (#364)
And fixed the bug that "diff" was not triggering "prepare" hooks.

Ref #363
2018-09-27 01:01:25 +09:00
KUOKA Yusuke b9de22b256 feat: prepare and cleanup release event hooks (#349)
Resolves #295
Resolves #330
Resolves #329 (Supports templating of only `releases[].hooks[].command` and `args` right now
Resolves #324
2018-09-21 10:35:12 +09:00
KUOKA Yusuke 4da13b44ee fix: --helm-binary does not work with sync when updating repo (#352)
Fixes #350
2018-09-20 09:31:16 +09:00
KUOKA Yusuke 5d1a92d3a5 fix: Forbid extraneous arguments (#348)
Resolves #296
2018-09-20 09:12:12 +09:00
KUOKA Yusuke 9808849986 feat: Merge multiple docs inside a single helmfile.yaml (#270)
Resolves #254
2018-09-19 15:42:01 +09:00
Dan Helfman 1ade353c1a Use --namespace value within Helmfile template (#343)
* Use --namespace value within Helmfile template (#326).

* Tabs and spaces.

* Tab/spaces fix.

* Pacifying "go fmt".

* Hard-coding namespace in test convenience function.

* MR feedback: Add comment to exported identifier.

* MR feedback: Put comment in correct location.

* fix(ci): fix never-ending build issue by auto-approving `helmfile delete`
2018-09-17 21:28:45 +09:00
Terry Corley 6453768f2e Fix small typo in error string in state.go (#345) 2018-09-17 20:27:39 +09:00
KUOKA Yusuke cf31d02f25 feat: Option to mark release as not installed (#342)
Resolves #197
2018-09-16 22:44:47 +09:00
KUOKA Yusuke c5bc932ba2 feat: Prompt before delete (#341)
Resolves #182
2018-09-16 22:43:29 +09:00
KUOKA Yusuke 81d796c167 fix: readFile and exec should run in the directory the template file resides in (#340)
Fixes #299
2018-09-16 22:43:17 +09:00
KUOKA Yusuke 37ea442515 feat: delete in reverse order (#339)
Resolves #312
2018-09-16 22:42:13 +09:00
David Genest 43a54486a1 docs: add precision on what value files will be rendered (#337) 2018-09-14 21:44:38 +09:00
David Genest b3da9a9a07 fix: evaluate helm-binary BEFORE any helm invocation (#333)
closes #332
2018-09-13 09:59:35 +09:00
David Genest 30155d8742 fix: use channel to harvest download results (#328)
Fixes #327
2018-09-12 09:12:23 +09:00
David Genest 7bfb58c0e4 feat: double render the helmfile (#308)
This allows using the environment values defined in the environments: section of helmfile.yaml to be used from other sections of the file.

This works by having two template renderers, the first-pass and the second-pass renderer.

The first-pass render renders a helmfile.yaml template with replacing template functions has side-effects with noop. So, use only funcs that don't have side-effects to compose your environment values.

Then the second-pass renderer renders the same helmfile.yaml template, but with the environment values loaded by the first-pass renderer.

The implementation uses a buffer instead of re-reading the file twice.

Resolves #297
2018-09-12 08:55:42 +09:00
KUOKA Yusuke 751e549253 fix: prevent panicing on invalid selector format (#323)
Fixes #322
2018-09-09 17:20:44 +09:00
KUOKA Yusuke 4f073167b2 fix: prevent panicing when an undefined environment name was specified via --environment (#321)
Fixes #320
2018-09-09 08:35:15 +09:00
KUOKA Yusuke 7c65b2ed2c fix: unexpected no releases found for any helmfiles on helmfiles: config (#318)
Fixes #315
Fixes #316
2018-09-08 13:50:51 +09:00
Shane Starcher 595c70f85b feat: add devel flag support (#313)
Resolves #161

Changelog:

* add support for devel flag

* allow for explicit disable of the devel default
2018-09-08 08:32:14 +09:00
KUOKA Yusuke 8f96f0aff5 fix: do not panic on invalid yaml (#314)
Fixes #311
2018-09-08 02:15:39 +09:00
Takuma Hashimoto c3934495fe fix: {{ .Environment.Name }} being always blank in helmfile.yaml
Correctly sets the environment name passed via the flag `--environment NAME`, so that it can be referenced by `{{ .Environment.Name }}`
2018-09-06 18:12:32 +09:00
KUOKA Yusuke a205f0d586 fix: concurrency issue in diff and sync (#309)
fixes #301
2018-09-06 10:14:10 +09:00
David Genest 2e44da1b54 ensure helmfiles are searched relative to base helmfile (#307) 2018-09-06 10:08:45 +09:00
Karl Stoney d0a4008965 Add missing repos on lint command. (#305)
https://github.com/roboll/helmfile/issues/303
2018-09-06 10:07:43 +09:00
Karl Stoney af121b85b5 feat: Concurrent chart download in template and lint commands
This enables `helmfile lint` and `helmfile template` commands to fetch and untar all the required charts concurrently. The concurrency is configurable via the `--concurrency` flag, that defaults to `0`.

Ref #292
2018-09-05 08:19:57 +09:00
KUOKA Yusuke 1c3bfcca10 feat(environment): Allow missing environment in helmfile.yaml, partly. (#294)
helmfile as of today ensures that all the targeted helmfile.yaml to have the specified environment defined in it.
That is, `helmfile --environment prod -f helmfile.d/ sync` fails if any helmfile under `helmfile.d/` is missing the `production` environment.

This changes the validation logic, so that helmfile fails only when all the helmfiles miss the environment.

Resolves #279
2018-09-04 22:04:37 +09:00
Hans Kristian Flaatten 7d7ca74a05 docs: update link to documentation for latest released version (#293) 2018-09-04 21:49:21 +09:00
KUOKA Yusuke f1ac50bf44 feat: apply run sync against only affected releases (#291)
Enhance the `diff` functionality to be able to return affected releases that has any changes,
so that the succeeding `sync` can be run against only the affected releases.

This provides us extra idempotency.

Resolves #277
2018-09-04 21:48:43 +09:00
KUOKA Yusuke 60843cc224 fix: Setting concurrency>1 with single release breaks sync (#290)
Fixes #287
2018-09-04 20:59:24 +09:00
Karl Stoney 53dea091e3 Fix set in helmfile template and helmfile lint (#289)
This removes --concurrenty flag from `helmfile template` and `helmfile lint`. YAGNI.
Also separates out the downloadCharts into a function so it can be used by lint too.

Fixes #288
2018-09-04 20:55:32 +09:00
KUOKA Yusuke 5510de4765 feat(apply): update repos and and deps before running diff (#286)
Resolves #271
2018-09-04 16:12:43 +09:00
Karl Stoney 93c5d4c219 feat: helmfile template (#284)
`helmfile template` runs `helm template` over releases within the helmfiles, and provide you a stream of generated yaml documents of Kubernetes resources via stdout.

Resolves #283
2018-09-04 11:31:43 +09:00
KUOKA Yusuke 8a90e5320c feat: Ability to call arbitrary command from a template (#282)
Resolves #244
2018-09-03 16:48:03 +09:00
rmartinez3 6be53b1bcc WIP: adding to be albe to set multiple flags for --set, -f, --value in args (#278)
* adding to be albe to set multiple flags for --set, -f, --value in args

* add test for passing a set flag with two or more key-values
2018-09-03 16:45:02 +09:00
KUOKA Yusuke b0cc7ba96c fix: sync partially run helm upgrade even one of releases failed while loading values.yaml (#281)
Fixes #280
2018-09-03 16:43:57 +09:00
KUOKA Yusuke 54f1567294 feat: --suppress-secrets of diff and apply commands (#272)
Adds `--suppress-secrets` to `helmfile apply` and `helmfile diff`, so that the diff command omits the contents of secrets from its output. This is a security feature that should always be turned on for CI/CD use-cases.

With `--suppress-secrets`, the output when there is any change looks like:

```
Comparing bar stable/grafana
default, baz-grafana, Secret (v1) has changed:
+ Changes suppressed on sensitive content of type Secret
```

Resolves #269
2018-09-02 21:21:29 +09:00
KUOKA Yusuke 046281c70f feat: Environment Secrets (#274)
Resolves #255
2018-09-02 21:07:35 +09:00
KUOKA Yusuke 98617f78e3 fix: --args "--set foo=bar" always result in "unknown flag: --set foo" error (#268)
Ref #248
2018-09-02 21:06:38 +09:00
KUOKA Yusuke ed0854a5c0 feat: Environment and Environment Values (#267)
Resolves #253
2018-08-31 22:59:27 +09:00
KUOKA Yusuke 7c793fdb88 feat: helmfiles: <ordered glob patterns of helmfiles> configuration (#266)
Resolves #247
2018-08-31 12:03:18 +09:00
Dan Wendorf b70956b5d8 Bump Helm 2.9.1 -> 2.10.0 (#265) 2018-08-31 10:17:51 +09:00
KUOKA Yusuke 3840605e04 feat: helmfile apply [--auto-approve] (#263)
This command syncs releases only if there is any difference between the desired and the current state. It asks for an confirmation by default. Provide `--auto-approve` flag after the `apply` command to skip it.

Resolves #205
2018-08-31 10:15:02 +09:00
KUOKA Yusuke bb3b44e511 fix: follow up for the relative paths improvement (#262)
Fixes for the bugs that are introduced by #261, that is values.yaml files specified in `values:` have redundant base path in their prefixes, and remaining .dec files after secrets decryption(https://github.com/roboll/helmfile/issues/251#issuecomment-417285854)
2018-08-30 21:59:59 +09:00
KUOKA Yusuke 421299c883 feat: All the paths are relative to helmfile.yaml (#261)
`helmfile lint` works with relative chart reference (#252)
The tempalte function `readFile` accepts the path relative to helmfile.yaml

Resolves #246
Fixes #252
2018-08-30 19:47:52 +09:00
KUOKA Yusuke 79f0e70ce8 fix: avoid "cross-device link" errors while decrypting secrets (#260)
ref https://github.com/roboll/helmfile/issues/251#issuecomment-417166296
2018-08-30 17:20:38 +09:00
KUOKA Yusuke 0c8a89cbaf fix: coordinate secret decryption (#259)
Fixes #258
2018-08-30 17:03:45 +09:00
KUOKA Yusuke b3ebd4cdd0 fix: Keep backward-compatibility broken after introduction of values file template (#257)
Fixes #249
2018-08-30 16:57:37 +09:00
Shane Starcher 9b71c64ef2 fix: use temporary file for secret file to prevent deletion collision (#250)
Fixes #167
2018-08-30 09:28:26 +09:00
KUOKA Yusuke 822cc13e72 feat: Template functions to replace sets (#242)
Resolves #227
2018-08-28 22:00:51 +09:00
KUOKA Yusuke 0ac8401d1e feat: feat: Configurable default values for --verify, --force, --recreate-pods, and --timeout (#241)
Resolves #230
2018-08-28 15:11:05 +09:00
KUOKA Yusuke 815ee1f85b feat: --force, --recreate-pods, --timeout as first-class citizens in helmfile.yaml (#239)
Resolves #229
2018-08-27 23:06:16 +09:00
KUOKA Yusuke 2c36640ad2 To reduce integration test flakiness 2018-08-27 23:04:16 +09:00
KUOKA Yusuke faaf172615 fix: panic on flag parsing error (#238)
This is a follow-up for #235.

We setup the default logger only after the flags are parsed successfully.
That's because we init the logger according to the --log-level flag value.
2018-08-27 22:58:25 +09:00
KUOKA Yusuke fcbd6aff29 fix: Relative path in KUBECONFIG envvar (#237)
Resolves #173
2018-08-27 22:53:17 +09:00
Shane Starcher 543a653864 improve error handling for value rendering (#235)
Fixes #233

Output on values render error:

```
err: failed to render [/Users/sstarcher/xxx/xxx/values.yaml], because of template: stringTemplate:10:18: executing "stringTemplate" at <requiredEnv "HELM_AC...>: error calling requiredEnv: required env var `HELM_ACCOUNT` is not set
```

Also removes panic and sets the output as `apps.Run()` can and will return errors.  Panic makes no sense.

Changelog:

* improve error handling for value rendering

* only output if error exists

* add exit status
2018-08-27 12:53:25 +09:00
rmartinez3 fa69ac2cfa fix: Args being reordered (#224)
Fixes #212
2018-08-27 10:41:00 +09:00
KUOKA Yusuke 2121fe57b2 Merge pull request #232 from mumoshu/set-array
feat: set array
2018-08-24 17:12:17 +09:00
Yusuke KUOKA cfcafa2e46 feat: set array
Fixes #148
2018-08-24 17:01:34 +09:00
KUOKA Yusuke b6d2ee9d43 Merge pull request #231 from mumoshu/partly-allow-duplicate-release-name
fix: helmfile should fail on duplicate release name after filtered by labels
2018-08-24 16:43:38 +09:00
Yusuke KUOKA 35542b5516 fix: helmfile should fail on duplicate release name after filtered by labels
This is a follow-up for #218, fixes the unintentional degradation that broken the use-case described in https://github.com/roboll/helmfile/issues/193#issuecomment-415434408
2018-08-24 16:18:08 +09:00
KUOKA Yusuke 45d0f1c8e4 Merge pull request #228 from mumoshu/set-file
feat: Support for helm's --set-file
2018-08-24 13:44:07 +09:00
KUOKA Yusuke 96d0781b85 Merge pull request #226 from mumoshu/usage-without-internet
doc: Using helmfile without an Internet connection
2018-08-24 13:43:53 +09:00
Yusuke KUOKA 48e65791c0 feat: Support for helm's --set-file
Resolves #207
2018-08-24 13:06:31 +09:00
Yusuke KUOKA 2398ad2f35 doc: Using helmfile without an Internet connection
Resolves #155
2018-08-24 12:10:18 +09:00
KUOKA Yusuke 013276f576 Merge pull request #225 from mumoshu/unmatched-selector-exit-code
feat: exit code 2 on unmatched selectors
2018-08-24 11:47:10 +09:00
Yusuke KUOKA 3a16b7ed91 feat: exit code 2 on unmatched selectors
Along with the fix for the bug introduced via #217

Resolves #219
2018-08-24 11:40:30 +09:00
KUOKA Yusuke a5d42e741b Merge pull request #223 from mumoshu/show-which-release-failed
feat: emit error message containing which release in which helmfile an upgrade failed
2018-08-23 12:00:15 +09:00
Yusuke KUOKA 313b5de378 feat: emit error message containing which release in which helmfile an upgrade failed
Resolves #66
2018-08-23 11:45:43 +09:00
KUOKA Yusuke 0541731350 Merge pull request #222 from mumoshu/wait-for-release
feat: `wait: true` in each release
2018-08-23 10:43:01 +09:00
Yusuke KUOKA b4623f0515 feat: wait: true in each release
To instruct `helm upgrade` to wait for resources via `--wait`

Resolves #65
2018-08-23 10:37:15 +09:00
KUOKA Yusuke ffa40beff9 Merge pull request #221 from mumoshu/env-files
doc: Instruction to use env files
2018-08-23 09:47:01 +09:00
Yusuke KUOKA 4fae044f0e doc: Instruction to use env files
Closes #203
2018-08-23 09:44:42 +09:00
KUOKA Yusuke d8aa9f305b Merge pull request #218 from mumoshu/should-fail-on-duplicate-release-name
fix: helmfile should fail on duplicate release name
2018-08-23 09:32:03 +09:00
KUOKA Yusuke c402759363 Merge pull request #217 from mumoshu/less-verbose-unmatched-selector-logging
fix: Make logging on unmatched selectors less verbose
2018-08-23 09:27:32 +09:00
Yusuke KUOKA e84ff05a63 fix: helmfile should fail on duplicate release name
Fixes #193
2018-08-23 09:26:39 +09:00
Yusuke KUOKA fcb75fa851 fix: Make logging on unmatched selectors less verbose
Fixes #200
2018-08-23 09:23:55 +09:00
KUOKA Yusuke d609dab848 feat: Template values files (#216)
Resolves #97
2018-08-22 18:53:32 +09:00
KUOKA Yusuke 6c89fac8e5 Merge pull request #215 from mumoshu/diff-target-namespace
feat(diff): Pass namespace to helm-diff
2018-08-22 18:51:10 +09:00
KUOKA Yusuke 3550e5f335 Merge pull request #213 from chadasapp/master
Typofix: required_env -> requiredEnv
2018-08-22 15:05:29 +09:00
Yusuke KUOKA 127dd857df feat(diff): Pass namespace to helm-diff
Tested manually by running `helmfile diff` with debug logging:

```yaml
helmDefaults:
  tillerNamespace: foo

charts:
  - name: grafana
    namespace: grafana
    chart: stable/grafana
```

```console
$ ./helmfile --log-level debug diff
Comparing grafana stable/grafana
exec: helm diff upgrade --allow-unreleased grafana stable/grafana --namespace grafana --tiller-namespace=foo
```

Resolves #179
2018-08-22 11:20:55 +09:00
Chad Horohoe 2cf2dbb868 Typofix: required_env -> requiredEnv
The name is `requiredEnv` and I cannot find any references to `required_env` in the repository anywhere.
2018-08-20 11:10:01 -07:00
KUOKA Yusuke 545a4a3efa Merge pull request #208 from cloudposse/add-examples
Add link to more examples
2018-08-20 21:32:26 +09:00
KUOKA Yusuke 0ed1355573 Merge pull request #206 from InTheCloudDan/patch-1
remove outdated warning.
2018-08-07 15:28:05 +09:00
Erik Osterman 9fef3f7d0f Add link to more examples 2018-08-06 23:20:49 -07:00
KUOKA Yusuke 0d457c4df0 Merge pull request #201 from maver1ck/helmfile-111
Add --args to delete
2018-08-03 09:36:48 +09:00
KUOKA Yusuke 6be52bfbbb Merge pull request #199 from mumoshu/optional-detailed-exitcode-for-diff
feat: Optional detailed exitcodes for `helmfile diff`
2018-08-03 09:36:08 +09:00
Dan O'Brien 089c158084 remove outdated warning. 2018-08-02 20:19:29 -04:00
Maciej Bryński d67573299c Add --args to delete 2018-07-31 10:18:31 +02:00
Yusuke KUOKA 3b4ce90a5a feat: Optional detailed exitcodes for helmfile diff
Adds the `--detailed-exitcode` to the `helmfile diff` command to return `1` on failure, and `2` when no error but diff is seen.

This feature requires the latest `helm-diff` containing https://github.com/databus23/helm-diff/pull/78, and `helm` containing https://github.com/helm/helm/pull/4367.

This is verified to work by manually running commands like the followings:

```bash
./helmfile --helm-binary helm211dev -f ./examples/helmfile.d diff --detailed-exitcode; echo $?
./helmfile --helm-binary helm211dev -f ./examples/helmfile.d diff; echo $?
```

Note that, in above example commands, `helm211dev` is a custom `helm` binary that is built from helm's master branch containing [the necessary enhancement to allow propagate non-zero plugin exit code](https://github.com/helm/helm/pull/4367).
2018-07-26 00:16:47 +09:00
KUOKA Yusuke 92d09a3e92 Merge pull request #198 from mumoshu/fix-ci-flakiness
fix(ci): Reduce flakiness
2018-07-25 23:24:44 +09:00
Yusuke KUOKA 1a5558a306 fix(ci): Reduce flakiness
Fix #103
2018-07-25 11:21:01 +09:00
KUOKA Yusuke ee3632cebc Merge pull request #191 from mumoshu/fix-cmd-descrs
Fix command descriptions
2018-07-19 12:02:07 +09:00
KUOKA Yusuke 2297093f84 Merge pull request #190 from mumoshu/fix-helmfile.d-defaulting
fix: `helmfile sync` doesn't recognize the default `helmfile.d/` directory
2018-07-19 12:01:49 +09:00
KUOKA Yusuke 506de12fcd fix: helmfile -f dir/ -l k=v sync fails on first helmfile w/ no match (#188)
Fixes #187

Changelog:

* fix: `helmfile -f dir/ -l k=v sync` fails on first helmfile w/ no match

* fix(ux): present the filename if the selector did not match any releases

```console
$ helmfile -f helmfile.d/ -l foo=bar sync
2018/07/12 23:37:29 specified selector did not match any releases in helmfile.d/00-backend.yaml
2018/07/12 23:37:29 specified selector did not match any releases in helmfile.d/01-frontend.yaml
2018/07/12 23:37:29 err: specified selector did not match any releases in any helmfile
```
2018-07-19 12:01:14 +09:00
rmartinez3 01fe5b4213 Adding merging for helmDefaults args and --args, adding tillerNamspace and kubeContext to helmDefaults (#186)
Add tillerNamespace and kubeContext to helmDefaults. Also add merging of args capability. 

People can start adding additional fields under helmDefaults on top of this. The additional fields are merged to args as long as they are implemented using the framework introduced by this commit.

Changelog:

* adding merging for helm defaults args and --args, adding tillernamspace, kubecontext to helmdefaults, adding getargs to lint command

* adding docs for helmdefaults

* checking if empty default values
2018-07-19 11:53:09 +09:00
Simon Li 06a1d245ae Multi-level logging (#185)
Closes #93

Summary of changes:

- Adds the [uber-go-/zap](https://github.com/uber-go/zap) library for leveled logging (69040996e0bcbd2f211d0d8d576147027e928e3b)
- A new CLI arg `--log-level` which can take the values `debug` `info` `warn` `error` `fatal` `panic`. The default is `info`, and the existing `--quiet` flag is treated as `--log-level warn`
- All `helm.exec` calls are preceded by a one-line summary. The current information on the full helm command line arguments `exec: helm exec ... ` is only output at `debug` level. This means sensitive command line arguments which may include passwords and `--set` arguments should be hidden by default.
2018-07-19 11:33:57 +09:00
Yusuke KUOKA 438d19852a Fix command descriptions
Resolves #170
2018-07-12 16:09:46 +09:00
Yusuke KUOKA c74db5d281 fix: helmfile sync doesn't recognize the default helmfile.d/ directory
Fixes #189
2018-07-12 14:13:56 +09:00
KUOKA Yusuke 35732f3a93 Bugfix for helm-binary flag. to global (#184)
Two follow-up commits for #181 

* Make the --helm-binary flag global
* Add helm-binary support to the delete command
2018-07-05 11:56:09 +09:00
hori-ryota 7f37c6b39f Add helm-binary opt to delete command 2018-07-04 22:10:17 +09:00
hori-ryota fa0ff32b00 Bugfix for helm-binary flag. to global 2018-07-04 19:01:46 +09:00
Hori Ryota 9f57f8be7d Add the option to specify an alternative helm binary (#181)
`helm` may not be version compatible. Add an option to specify an alternative `helm` command on runtime, possibly according to the version of tiller running on your cluster.
2018-07-03 22:19:02 +09:00
KUOKA Yusuke 88e84e5ae8 Merge pull request #177 from amnk/amnk/issue-176
Add helm-secrets plugin to Dockerfile
2018-06-26 22:37:44 +09:00
KUOKA Yusuke c52504cf8e Merge pull request #178 from Scukerman/master
Fix typo in README.md
2018-06-26 22:33:37 +09:00
Scukerman a8212587b8 Fix typo in README.md 2018-06-26 16:25:42 +03:00
amnk e1040c2aa1 Add helm-secrets plugin to Dockerfile
Closes: #176
2018-06-26 10:42:47 +02:00
rmartinez3 58bc2a2997 adding args to be added in helmfile along with context to be set in helmspec (#171)
Ref #114 

Changelog:

* adding args to be added in helmfile along with context to be set in helmspec

* making command args take precedence
2018-06-26 10:12:38 +09:00
KUOKA Yusuke af1914c575 Merge pull request #172 from mumoshu/diff-against-new-release
feat: `helmfile diff` also works against non-existent releases
2018-06-25 16:15:09 +09:00
Yusuke KUOKA da337e8724 feat: helmfile diff also works against non-existent releases
Resolves #127
2018-06-14 22:45:26 +09:00
Johan Lyheden 6856c6e979 Add helmfile lint support (#162)
The use case is to have a list of helmfile releases version controlled together with all settings and have a CI pipeline that will lint all releases with settings before running sync. The new functionality was mostly copy pasted from the Diff implementation with some extra handling for fetching remote charts.

Notes:

* Added release name to chart path to avoid potential race condition when fetching the chart
2018-06-14 22:35:09 +09:00
KUOKA Yusuke 2fba241122 Ordered helmfile(s) from a directory (#160)
Resolves #151 #137

Notes:

- You can have either helmfile.d or helmfile.yaml/charts.yaml, not both
2018-06-14 22:20:15 +09:00
KUOKA Yusuke df264e2736 Merge pull request #165 from cmeury/no-match-error
fail when no releases match selector
2018-06-12 23:58:33 +09:00
Cedric Meury 575198424a fail when no releases match selector 2018-06-12 14:19:06 +02:00
KUOKA Yusuke afa45930c1 Merge pull request #159 from wendorf/dockerfile-helm-diff
Dockerfile installs helm-diff
2018-06-01 12:06:06 +09:00
KUOKA Yusuke 3044255f7a Merge pull request #156 from wendorf/dockerfile-helm-2.9.1
Docker image uses Helm v2.9.1 (from v2.8.2)
2018-06-01 12:04:35 +09:00
Dan Wendorf 98d3897262 Dockerfile installs helm-diff
Enables the `helmfile diff` command

- git and bash are required to install helm-diff
2018-05-31 15:52:33 -07:00
Dan Wendorf a9a8fb8388 Docker image uses Helm v2.9.1 (from v2.8.2) 2018-05-31 13:43:50 -07:00
Dan Wendorf fb6ac7bba2 Dockerfile uses ARG instead of ENV
Allows customization at image build time
2018-05-31 13:41:36 -07:00
Dan O'Brien 1a4f342f25 Add basic repository authentication (#154)
* basic repository authentication via new `username` and `password` keys

* add warning to readme
2018-05-31 10:42:38 +09:00
KUOKA Yusuke 1768e5dea7 Add tests for helmfile test (#152)
Follow up for #150
2018-05-22 17:19:10 +09:00
Mike Eves 37f6ae8557 Add helmfile test sub-command (#150)
**Feature**
An additional sub-command to the helmfile binary; helmfile test

**Why**
Helm provides helm test (https://github.com/kubernetes/helm/blob/master/docs/chart_tests.md) as a method to run automated tests post chart install to ensure things are working as they should be.

It would be nice to be able to run something like helmfile test against a particular helmfile in order to run helm tests against all charts/releases defined in the file. Either as part of the sync (i.e. helmfile sync --test) to be ran immediately after the corresponding chart is installed, or as a separate command ran after a sync (i.e. helmfile test).

A chart without tests will exit with a 0 status, so it can be safely ran against any charts.

**Notes**

`--cleanup` (bool) & `--timeout` (default: 300) are available as first class arguments. Additional arguments can be passed to the helm binary as with other sub commands using `--args=`

Resolves #144
2018-05-22 17:12:48 +09:00
KUOKA Yusuke 5735efa8c0 Merge pull request #147 from mumoshu/fix-static-linking-for-linux-binary-on-circle
Fix static linking for linux binaries built on CircleCI
2018-05-17 16:22:49 +09:00
Yusuke KUOKA f08b981814 Fix static linking for linux binaries built on CircleCI
Fixes #146
2018-05-16 22:46:35 +09:00
KUOKA Yusuke a7e35b7316 Merge pull request #142 from mumoshu/make-purge-optional
Make purge optional when running "helmfile delete"
2018-05-16 08:59:35 +09:00
KUOKA Yusuke f288032d60 Merge pull request #145 from ConnectedHomes/absolute-value-path
Don't prefix base path to absolute path
2018-05-15 22:36:40 +09:00
David Bliss 4fe8d988d5 Don't prefix base path to absolute path
If an entry in the `values` section of a helmfile is an absolute
path, then it would be erroneously prefixed with the base path.

This would cause `helmfile sync` to fail because the value.yaml file
would not exist at the incorrectly prefixed path.
2018-05-15 11:22:38 +01:00
KUOKA Yusuke 09f19047bd Merge pull request #143 from mumoshu/fix-helm-diff-deprecation-warning
Fix the helm-diff deprecation warning
2018-05-11 21:56:15 +09:00
Yusuke KUOKA 52507de39b Mention about the minimum ver. of helm-diff 2018-05-11 21:52:08 +09:00
Yusuke KUOKA ac70e45bff Fix the helm-diff deprecation warning
Resolves #130
2018-05-11 16:51:44 +09:00
Yusuke KUOKA 3979c18ad3 Make purge optional when running "helmfile delete"
`helmfile delete` has been implying `--purge` but it is not the case since this change.

The new behavior is `helmfile delete --purge` to actually purge releases.
Run just `helmfile delete` to delete releases but not purge them.

Resolves #71
2018-05-11 16:41:02 +09:00
KUOKA Yusuke fb781be560 Merge pull request #139 from int128/inline-values
Inline values support
2018-05-11 14:34:25 +09:00
Hidetake Iwata 5db3ba2112 Add inline values support
This allows `values` accepts the following types:
- string type for values file path
- map type for inline values
2018-05-11 11:55:01 +09:00
KUOKA Yusuke 568d750f75 Merge pull request #141 from mumoshu/always-reset-values
Force `--reset-values` on sync
2018-05-11 03:05:45 +09:00
Yusuke KUOKA c41dbc4c9f Force --reset-values on sync
Resolves #63
2018-05-11 03:04:36 +09:00
119 changed files with 14449 additions and 774 deletions
+10 -8
View File
@@ -4,7 +4,7 @@ jobs:
build:
docker:
- image: circleci/golang:1.10.1
- image: circleci/golang:1.11.2
working_directory: /go/src/github.com/roboll/helmfile
steps:
- checkout
@@ -16,7 +16,7 @@ jobs:
test:
docker:
- image: circleci/golang:1.10.1
- image: circleci/golang:1.11.2
working_directory: /go/src/github.com/roboll/helmfile
steps:
- checkout
@@ -26,9 +26,12 @@ jobs:
# thanks to https://raw.githubusercontent.com/weaveworks/launcher/master/.circleci/config.yml
integration_tests:
machine: true
machine:
image: circleci/classic:201808-01
environment:
CHANGE_MINIKUBE_NONE_USER: true
MINIKUBE_WANTUPDATENOTIFICATION: false
MINIKUBE_WANTREPORTERRORPROMPT: false
steps:
- checkout
- run: mkdir ~/build
@@ -38,7 +41,7 @@ jobs:
- run:
name: Install helm
environment:
HELM_VERSION: v2.9.0
HELM_VERSION: v2.12.0
command: |
HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
curl -Lo ${HELM_FILENAME} "https://kubernetes-helm.storage.googleapis.com/${HELM_FILENAME}"
@@ -49,14 +52,13 @@ jobs:
name: Deploy minikube
environment:
CHANGE_MINIKUBE_NONE_USER: true
K8S_VERSION: v1.9.0
MINIKUBE_VERSION: v0.25.2
K8S_VERSION: v1.12.3
MINIKUBE_VERSION: v0.30.0
command: |
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/linux/amd64/kubectl
chmod +x kubectl && sudo mv kubectl /usr/local/bin/
curl -Lo minikube https://storage.googleapis.com/minikube/releases/${MINIKUBE_VERSION}/minikube-linux-amd64
chmod +x minikube && sudo mv minikube /usr/local/bin/
sudo minikube config set WantReportErrorPrompt false
sudo -E minikube start --vm-driver=none --kubernetes-version=${K8S_VERSION}
sudo -E minikube update-context
- run:
@@ -72,7 +74,7 @@ jobs:
release:
docker:
- image: circleci/golang:1.10.1
- image: circleci/golang:1.11.2
working_directory: /go/src/github.com/roboll/helmfile
steps:
- checkout
+39
View File
@@ -0,0 +1,39 @@
# DCO + License
By contributing to `helmfile`, you accept and agree to the following DCO and license terms and
conditions for your present and future Contributions submitted to the `helmfile` project.
[DCO](https://developercertificate.org/)
[License](https://github.com/roboll/helmfile/blob/master/LICENSE)
# Developing helmfile
Locate your `GOPATH`, usually `~/go`, and run:
```console
$ go get github.com/roboll/helmfile
$ cd $GOPATH/src/github.com/roboll/helmfile
$ git checkout -b your-shiny-new-feature origin/master
...
$ git commit -m 'feat: do whatever for whatever purpose
This adds ... by:
- Adding ...
- Changing ...
- Removing...
Resolves #ISSUE_NUMBER
'
$ hub fork
$ git push YOUR_GITHUB_USER your-shiny-new-feature
$ hub pull-request
```
Note that the above tutorial uses [hub](https://github.com/github/hub) just for ease of explanation.
Please use whatever tool or way to author your pull request!
+11 -7
View File
@@ -1,4 +1,4 @@
FROM golang:1.10.1-alpine3.7 as builder
FROM golang:1.11.2-alpine3.8 as builder
RUN apk add --no-cache make git
WORKDIR /go/src/github.com/roboll/helmfile/
@@ -7,19 +7,23 @@ RUN make static-linux
# -----------------------------------------------------------------------------
FROM alpine:3.7
FROM alpine:3.8
RUN apk add --no-cache ca-certificates
RUN apk add --no-cache ca-certificates git bash curl
ENV HELM_VERSION=v2.8.2
ENV HELM_LOCATION="https://kubernetes-helm.storage.googleapis.com"
ENV HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
ENV HELM_SHA256="614b5ac79de4336b37c9b26d528c6f2b94ee6ccacb94b0f4b8d9583a8dd122d3"
ARG HELM_VERSION=v2.12.0
ARG HELM_LOCATION="https://kubernetes-helm.storage.googleapis.com"
ARG HELM_FILENAME="helm-${HELM_VERSION}-linux-amd64.tar.gz"
ARG HELM_SHA256="9f96a6e4fc52b5df906da381532cc2eb2f3f57cc203ccaec2b11cf5dc26a7dfc"
RUN wget ${HELM_LOCATION}/${HELM_FILENAME} && \
sha256sum ${HELM_FILENAME} | grep -q "${HELM_SHA256}" && \
tar zxf ${HELM_FILENAME} && mv /linux-amd64/helm /usr/local/bin/ && \
rm ${HELM_FILENAME} && rm -r /linux-amd64
RUN mkdir -p "$(helm home)/plugins"
RUN helm plugin install https://github.com/databus23/helm-diff && \
helm plugin install https://github.com/futuresimple/helm-secrets
COPY --from=builder /go/src/github.com/roboll/helmfile/dist/helmfile_linux_amd64 /usr/local/bin/helmfile
CMD ["/usr/local/bin/helmfile"]
Generated
+61 -5
View File
@@ -2,64 +2,120 @@
[[projects]]
digest = "1:d9141c149f722217d35834972878499a4781fe5ae46a152e8ac843ff15efc80f"
name = "github.com/Masterminds/semver"
packages = ["."]
pruneopts = "UT"
revision = "8d82589cda2d7b5b9167396841975c4535c9bec6"
version = "v1.4.1"
[[projects]]
digest = "1:46a054a232ea2b7f0a35398d682b433d26ba9975fce9197b1784824402059f5b"
name = "github.com/Masterminds/sprig"
packages = ["."]
pruneopts = "UT"
revision = "6b2a58267f6a8b1dc8e2eb5519b984008fa85e8c"
version = "v2.15.0"
[[projects]]
digest = "1:8f5416c7f59da8600725ae1ff00a99af1da8b04c211ae6f3c8f8bcab0164f650"
name = "github.com/aokoli/goutils"
packages = ["."]
pruneopts = "UT"
revision = "3391d3790d23d03408670993e957e8f408993c34"
version = "v1.0.1"
[[projects]]
digest = "1:8f8811f9be822914c3a25c6a071e93beb4c805d7b026cbf298bc577bc1cc945b"
name = "github.com/google/uuid"
packages = ["."]
pruneopts = "UT"
revision = "064e2069ce9c359c118179501254f67d7d37ba24"
version = "0.2"
[[projects]]
digest = "1:6b1e7618931d0ed9a17fec1c1d84271907aa61172c50b8272327f6cd4fafb650"
name = "github.com/huandu/xstrings"
packages = ["."]
pruneopts = "UT"
revision = "2bf18b218c51864a87384c06996e40ff9dcff8e1"
version = "v1.0.0"
[[projects]]
digest = "1:8eb1de8112c9924d59bf1d3e5c26f5eaa2bfc2a5fcbb92dc1c2e4546d695f277"
name = "github.com/imdario/mergo"
packages = ["."]
revision = "9d5f1277e9a8ed20c3684bda8fde67c05628518c"
version = "v0.3.4"
pruneopts = "UT"
revision = "9f23e2d6bd2a77f959b2bf6acdbefd708a83a4a4"
version = "v0.3.6"
[[projects]]
digest = "1:55d0f7127962b0657acfd69650634057b3a08958226feee1b64a379666bb915e"
name = "github.com/urfave/cli"
packages = ["."]
pruneopts = "UT"
revision = "6011f165dc288c72abd8acd7722f837c5c64198d"
[[projects]]
digest = "1:3c1a69cdae3501bf75e76d0d86dc6f2b0a7421bc205c0cb7b96b19eed464a34d"
name = "go.uber.org/atomic"
packages = ["."]
pruneopts = "UT"
revision = "1ea20fb1cbb1cc08cbd0d913a96dead89aa18289"
version = "v1.3.2"
[[projects]]
digest = "1:60bf2a5e347af463c42ed31a493d817f8a72f102543060ed992754e689805d1a"
name = "go.uber.org/multierr"
packages = ["."]
pruneopts = "UT"
revision = "3c4937480c32f4c13a875a1829af76c98ca3d40a"
version = "v1.1.0"
[[projects]]
digest = "1:9580b1b079114140ade8cec957685344d14f00119e0241f6b369633cb346eeb3"
name = "go.uber.org/zap"
packages = [
".",
"buffer",
"internal/bufferpool",
"internal/color",
"internal/exit",
"zapcore",
]
pruneopts = "UT"
revision = "eeedf312bc6c57391d84767a4cd413f02a917974"
version = "v1.8.0"
[[projects]]
branch = "master"
digest = "1:7b1c378a71f617ec07229a1fcbb90b6a5344295823e1d732398f50fd9ae4bbec"
name = "golang.org/x/crypto"
packages = [
"pbkdf2",
"scrypt"
"scrypt",
]
pruneopts = "UT"
revision = "b2aa35443fbc700ab74c586ae79b81c171851023"
[[projects]]
branch = "v2"
digest = "1:342378ac4dcb378a5448dd723f0784ae519383532f5e70ade24132c4c8693202"
name = "gopkg.in/yaml.v2"
packages = ["."]
pruneopts = "UT"
revision = "5420a8b6744d3b0345ab293f6fcba19c978f1183"
version = "v2.2.1"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "2f414b2f5156dc8144a90cdbbb9685d5b8af323024f986bb52e66601f02f3c0f"
input-imports = [
"github.com/Masterminds/sprig",
"github.com/imdario/mergo",
"github.com/urfave/cli",
"go.uber.org/zap",
"go.uber.org/zap/zapcore",
"gopkg.in/yaml.v2",
]
solver-name = "gps-cdcl"
solver-version = 1
+4
View File
@@ -9,3 +9,7 @@
[prune]
go-tests = true
unused-packages = true
[[constraint]]
name = "github.com/imdario/mergo"
version = "0.3.6"
+15 -1
View File
@@ -26,7 +26,7 @@ integration:
.PHONY: integration
cross:
gox -os '!freebsd !netbsd' -arch '!arm' -output "dist/{{.Dir}}_{{.OS}}_{{.Arch}}" -ldflags '-X main.Version=${TAG}' ${TARGETS}
env CGO_ENABLED=0 gox -os '!freebsd !netbsd' -arch '!arm' -output "dist/{{.Dir}}_{{.OS}}_{{.Arch}}" -ldflags '-X main.Version=${TAG}' ${TARGETS}
.PHONY: cross
static-linux:
@@ -61,3 +61,17 @@ tools:
TAG = $(shell git describe --tags --abbrev=0 HEAD)
LAST = $(shell git describe --tags --abbrev=0 HEAD^)
BODY = "`git log ${LAST}..HEAD --oneline --decorate` `printf '\n\#\#\# [Build Info](${BUILD_URL})'`"
release/minor:
git checkout master
git pull --rebase origin master
bash -c 'if git branch | grep autorelease; then git branch -D autorelease; else echo no branch to be cleaned; fi'
git checkout -b autorelease origin/master
bash -c 'SEMTAG_REMOTE=origin hack/semtag final -s minor'
release/patch:
git checkout master
git pull --rebase origin master
bash -c 'if git branch | grep autorelease; then git branch -D autorelease; else echo no branch to be cleaned; fi'
git checkout -b autorelease origin/master
bash -c 'SEMTAG_REMOTE=origin hack/semtag final -s patch'
+589 -40
View File
@@ -24,7 +24,7 @@ To avoid upgrades for each iteration of `helm`, the `helmfile` executable delega
## configuration syntax
**CAUTION**: This documentation is for the development version of helmfile. If you are looking for the documentation for any of releases, please switch to the corresponding branch like [v0.12.0](https://github.com/roboll/helmfile/tree/v0.12.0).
**CAUTION**: This documentation is for the development version of Helmfile. If you are looking for the documentation for any of releases, please switch to the corresponding release tag like [v0.31.0](https://github.com/roboll/helmfile/tree/v0.31.0).
The default helmfile is `helmfile.yaml`:
@@ -34,8 +34,24 @@ repositories:
url: http://roboll.io/charts
certFile: optional_client_cert
keyFile: optional_client_key
username: optional_username
password: optional_password
context: kube-context # kube-context (--kube-context)
# context: kube-context # this directive is deprecated, please consider using helmDefaults.kubeContext
#default values to set for args along with dedicated keys that can be set by contributers, cli args take precedence over these
helmDefaults:
tillerNamespace: tiller-namespace #dedicated default key for tiller-namespace
kubeContext: kube-context #dedicated default key for kube-context (--kube-context)
# additional and global args passed to helm
args:
- "--set k=v"
# defaults for verify, wait, force, timeout and recreatePods under releases[]
verify: true
wait: true
timeout: 600
recreatePods: true
force: true
releases:
# Published chart example
@@ -45,18 +61,44 @@ releases:
foo: bar
chart: roboll/vault-secret-manager # the chart being installed to create this release, referenced by `repository/chart` syntax
version: ~1.24.1 # the semver of the chart. range constraint is supported
values: [ vault.yaml ] # value files (--values)
values:
# value files passed via --values
- vault.yaml
# inline values, passed via a temporary values file and --values
- address: https://vault.example.com
db:
username: {{ requiredEnv "DB_USERNAME" }}
# value taken from environment variable. Quotes are necessary. Will throw an error if the environment variable is not set. $DB_PASSWORD needs to be set in the calling environment ex: export DB_PASSWORD='password1'
password: {{ requiredEnv "DB_PASSWORD" }}
proxy:
# Interpolate environment variable with a fixed string
domain: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
scheme: {{ env "SCHEME" | default "https" }}
set:
# single value loaded from a local file, translates to --set-file foo.config=path/to/file
- name: foo.config
file: path/to/file
# set a single array value in an array, translates to --set bar[0]={1,2}
- name: bar[0]
values:
- 1
- 2
# set a templated value
- name: namespace
value: {{ .Namespace }}
# will attempt to decrypt it using helm-secrets plugin
secrets:
- vault_secret.yaml # will attempt to decrypt it using helm-secrets plugin
set: # values (--set)
- name: address
value: https://vault.example.com
- name: db.password
value: {{ requiredEnv "DB_PASSWORD" }} # value taken from environment variable. Quotes are necessary. Will throw an error if the environment variable is not set. $DB_PASSWORD needs to be set in the calling environment ex: export DB_PASSWORD='password1'
- name: proxy.domain
value: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com # Interpolate environment variable with a fixed string
- name: proxy.scheme
value: {{ env "SCHEME" | default "https" }}
- vault_secret.yaml
# wait for k8s resources via --wait. Defaults to `false`
wait: true
# time in seconds to wait for any individual Kubernetes operation (like Jobs for hooks, and waits on pod/pvc/svc/deployment readiness) (default 300)
timeout: 60
# performs pods restart for the resource if applicable
recreatePods: true
# forces resource update through delete/recreate if needed
force: true
# set `false` to uninstall on sync
installed: true
# Local chart example
- name: grafana # name of this release
@@ -65,6 +107,7 @@ releases:
values:
- "../../my-values/grafana/values.yaml" # Values file (relative path to manifest)
- ./values/{{ requiredEnv "PLATFORM_ENV" }}/config.yaml # Values file taken from path with environment variable. $PLATFORM_ENV must be set in the calling environment.
wait: true
```
@@ -72,13 +115,13 @@ releases:
Helmfile uses [Go templates](https://godoc.org/text/template) for templating your helmfile.yaml. While go ships several built-in functions, we have added all of the functions in the [Sprig library](https://godoc.org/github.com/Masterminds/sprig).
We also added one special template function: `requiredEnv`.
The `required_env` function allows you to declare a particular environment variable as required for template rendering.
We also added one special template function: `requiredEnv`.
The `requiredEnv` function allows you to declare a particular environment variable as required for template rendering.
If the environment variable is unset or empty, the template rendering will fail with an error message.
## Using environment variables
Environment variables can be used in most places for templating the helmfile. Currently this is supported for `name`, `namespace`, `value` (in set) and `url` (in repositories).
Environment variables can be used in most places for templating the helmfile. Currently this is supported for `name`, `namespace`, `value` (in set), `values` and `url` (in repositories).
Examples:
@@ -93,6 +136,10 @@ releases:
- name: {{ requiredEnv "NAME" }}-vault
namespace: {{ requiredEnv "NAME" }}
chart: roboll/vault-secret-manager
values:
- db:
username: {{ requiredEnv "DB_USERNAME" }}
password: {{ requiredEnv "DB_PASSWORD" }}
set:
- name: proxy.domain
value: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
@@ -102,9 +149,11 @@ releases:
## installation
- `go get github.com/roboll/helmfile` or
- download one of [releases](https://github.com/roboll/helmfile/releases) or
- run as a [container](https://quay.io/roboll/helmfile)
- run as a [container](https://quay.io/roboll/helmfile) or
- install from [AUR](https://aur.archlinux.org/packages/kubernetes-helmfile-bin/) for Archlinux or
- Windows (using [scoop](https://scoop.sh/)): `scoop install helmfile`
- macOS (using [homebrew](https://brew.sh/)): `brew install helmfile`
## getting started
@@ -125,12 +174,16 @@ releases:
Sync your Kubernetes cluster state to the desired one by running:
```console
helmfile sync
helmfile apply
```
Congratulations! You now have your first Prometheus deployment running inside your cluster.
Iterate on the `helmfile.yaml` by referencing the [configuration syntax](#configuration-syntax) and the [cli reference](#cli-reference).
Iterate on the `helmfile.yaml` by referencing:
- [Configuration syntax](#configuration-syntax)
- [CLI reference](#cli-reference).
- [Helmfile Best Practices Guide](https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md)
## cli reference
@@ -142,24 +195,32 @@ USAGE:
helmfile [global options] command [command options] [arguments...]
COMMANDS:
repos sync repositories from state file (helm repo add && helm repo update)
charts sync charts from state file (helm upgrade --install)
diff diff charts from state file against env (helm diff)
sync sync all resources from state file (repos, charts and local chart deps)
status retrieve status of releases in state file
delete delete charts from state file (helm delete)
repos sync repositories from state file (helm repo add && helm repo update)
charts sync releases from state file (helm upgrade --install)
diff diff releases from state file against env (helm diff)
template template releases from state file against env (helm template)
lint lint charts from state file (helm lint)
sync sync all resources from state file (repos, releases and chart deps)
apply apply all resources from state file only when there are changes
status retrieve status of releases in state file
delete delete releases from state file (helm delete)
test test releases from state file (helm test)
GLOBAL OPTIONS:
--file FILE, -f FILE load config from FILE (default: "helmfile.yaml")
--quiet, -q silence output
--namespace value, -n value Set namespace. Uses the namespace set in the context by default
--selector,l value Only run using the releases that match labels. Labels can take the form of foo=bar or foo!=bar.
A release must match all labels in a group in order to be used. Multiple groups can be specified at once.
--selector tier=frontend,tier!=proxy --selector tier=backend. Will match all frontend, non-proxy releases AND all backend releases.
The name of a release can be used as a label. --selector name=myrelease
--kube-context value Set kubectl context. Uses current context by default
--help, -h show help
--version, -v print the version
--helm-binary value, -b value path to helm binary
--file helmfile.yaml, -f helmfile.yaml load config from file or directory. defaults to helmfile.yaml or `helmfile.d`(means `helmfile.d/*.yaml`) in this preference
--environment default, -e default specify the environment name. defaults to default
--quiet, -q Silence output. Equivalent to log-level warn
--kube-context value Set kubectl context. Uses current context by default
--log-level value Set log level, default info
--namespace value, -n value Set namespace. Uses the namespace set in the context by default, and is available in templates as {{ .Namespace }}
--selector value, -l value Only run using the releases that match labels. Labels can take the form of foo=bar or foo!=bar.
A release must match all labels in a group in order to be used. Multiple groups can be specified at once.
--selector tier=frontend,tier!=proxy --selector tier=backend. Will match all frontend, non-proxy releases AND all backend releases.
The name of a release can be used as a label. --selector name=myrelease
--interactive, -i Request confirmation before attempting to modify clusters
--help, -h show help
--version, -v print the version
```
### sync
@@ -169,15 +230,31 @@ The `helmfile sync` sub-command sync your cluster state as described in your `he
Under the covers, Helmfile executes `helm upgrade --install` for each `release` declared in the manifest, by optionally decrypting [secrets](#secrets) to be consumed as helm chart values. It also updates specified chart repositories and updates the
dependencies of any referenced local charts.
For Helm 2.9+ you can use a username and password to authenticate to a remote repository.
### diff
The `helmfile diff` sub-command executes the [helm-diff](https://github.com/databus23/helm-diff) plugin across all of
the charts/releases defined in the manifest.
To supply the diff functionality Helmfile needs the `helm diff` plugin installed. For Helm 2.3+
To supply the diff functionality Helmfile needs the [helm-diff](https://github.com/databus23/helm-diff) plugin v2.9.0+1 or greater installed. For Helm 2.3+
you should be able to simply execute `helm plugin install https://github.com/databus23/helm-diff`. For more details
please look at their [documentation](https://github.com/databus23/helm-diff#helm-diff-plugin).
### apply
The `helmfile apply` sub-command begins by executing `diff`. If `diff` finds that there is any changes, `sync` is executed. Adding `--interactive` instructs Helmfile to request your confirmation before `sync`.
An expected use-case of `apply` is to schedule it to run periodically, so that you can auto-fix skews between the desired and the current state of your apps running on Kubernetes clusters.
### delete
The `helmfile delete` sub-command deletes all the releases defined in the manifests.
`helmfile --interactive delete` instructs Helmfile to request your confirmation before actually deleting releases.
Note that `delete` doesn't purge releases. So `helmfile delete && helmfile sync` results in sync failed due to that releases names are not deleted but preserved for future references. If you really want to remove releases for reuse, add `--purge` flag to run it like `helmfile delete --purge`.
### secrets
The `secrets` parameter in a `helmfile.yaml` causes the [helm-secrets](https://github.com/futuresimple/helm-secrets) plugin to be executed to decrypt the file.
@@ -186,19 +263,29 @@ To supply the secret functionality Helmfile needs the `helm secrets` plugin inst
you should be able to simply execute `helm plugin install https://github.com/futuresimple/helm-secrets
`.
### test
The `helmfile test` sub-command runs a `helm test` against specified releases in the manifest, default to all
Use `--cleanup` to delete pods upon completion.
### lint
The `helmfile lint` sub-command runs a `helm lint` across all of the charts/releases defined in the manifest. Non local charts will be fetched into a temporary folder which will be deleted once the task is completed.
## Paths Overview
Using manifest files in conjunction with command line argument can be a bit confusing.
A few rules to clear up this ambiguity:
- Absolute paths are always resolved as absolute paths
- Relative paths referenced *in* the helmfile manifest itself are relative to that manifest
- Relative paths referenced *in* the Helmfile manifest itself are relative to that manifest
- Relative paths referenced on the command line are relative to the current working directory the user is in
For additional context, take a look at [paths examples](PATHS.md)
## Labels Overview
A selector can be used to only target a subset of releases when running helmfile. This is useful for large helmfiles with releases that are logically grouped together.
A selector can be used to only target a subset of releases when running Helmfile. This is useful for large helmfiles with releases that are logically grouped together.
Labels are simple key value pairs that are an optional field of the release spec. When selecting by label, the search can be inverted. `tier!=backend` would match all releases that do NOT have the `tier: backend` label. `tier=fronted` would only match releases with the `tier: frontend` label.
@@ -208,6 +295,468 @@ The `selector` parameter can be specified multiple times. Each parameter is reso
`--selector tier=frontend --selector tier=backend` will select all the charts
## Templates
You can use go's text/template expressions in `helmfile.yaml` and `values.yaml.gotmpl` (templated helm values files). `values.yaml` references will be used verbatim. In other words:
- for value files ending with `.gotmpl`, template expressions will be rendered
- for plain value files (ending in `.yaml`), content will be used as-is
In addition to built-in ones, the following custom template functions are available:
- `readFile` reads the specified local file and generate a golang string
- `fromYaml` reads a golang string and generates a map
- `setValueAtPath PATH NEW_VALUE` traverses a golang map, replaces the value at the PATH with NEW_VALUE
- `toYaml` marshals a map into a string
### Values Files Templates
You can reference a template of values file in your `helmfile.yaml` like below:
```yaml
releases
- name: myapp
chart: mychart
values:
- values.yaml.gotmpl
```
Every values file whose file extension is `.gotmpl` is considered as a template file.
Suppose `values.yaml.gotmpl` was something like:
```yaml
{{ readFile "values.yaml" | fromYaml | setValueAtPath "foo.bar" "FOO_BAR" | toYaml }}
```
And `values.yaml` was:
```yaml
foo:
bar: ""
```
The resulting, temporary values.yaml that is generated from `values.yaml.tpl` would become:
```yaml
foo:
# Notice `setValueAtPath "foo.bar" "FOO_BAR"` in the template above
bar: FOO_BAR
```
## Refactoring `helmfile.yaml` with values files templates
One of expected use-cases of values files templates is to keep `helmfile.yaml` small and concise.
See the example `helmfile.yaml` below:
```yaml
releases:
- name: {{ requiredEnv "NAME" }}-vault
namespace: {{ requiredEnv "NAME" }}
chart: roboll/vault-secret-manager
values:
- db:
username: {{ requiredEnv "DB_USERNAME" }}
password: {{ requiredEnv "DB_PASSWORD" }}
set:
- name: proxy.domain
value: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
- name: proxy.scheme
value: {{ env "SCHEME" | default "https" }}
```
The `values` and `set` sections of the config file can be separated out into a template:
`helmfile.yaml`:
```yaml
releases:
- name: {{ requiredEnv "NAME" }}-vault
namespace: {{ requiredEnv "NAME" }}
chart: roboll/vault-secret-manager
values:
- values.yaml.tmpl
```
`values.yaml.tmpl`:
```yaml
db:
username: {{ requiredEnv "DB_USERNAME" }}
password: {{ requiredEnv "DB_PASSWORD" }}
proxy:
domain: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
scheme: {{ env "SCHEME" | default "https" }}
```
## Environment
When you want to customize the contents of `helmfile.yaml` or `values.yaml` files per environment, use this feature.
You can define as many environments as you want under `environments` in `helmfile.yaml`.
The environment name defaults to `default`, that is, `helmfile sync` implies the `default` environment.
The selected environment name can be referenced from `helmfile.yaml` and `values.yaml.gotmpl` by `{{ .Environment.Name }}`.
If you want to specify a non-default environment, provide a `--environment NAME` flag to `helmfile` like `helmfile --environment production sync`.
The below example shows how to define a production-only release:
```yaml
environments:
default:
production:
releases:
{{ if eq .Environment.Name "production" }}
- name: newrelic-agent
# snip
{{ end }}
- name: myapp
# snip
```
## Environment Values
Environment Values allows you to inject a set of values specific to the selected environment, into values.yaml templates.
Use it to inject common values from the environment to multiple values files, to make your configuration DRY.
Suppose you have three files `helmfile.yaml`, `production.yaml` and `values.yaml.gotmpl`:
`helmfile.yaml`
```yaml
environments:
production:
values:
- production.yaml
releases:
- name: myapp
values:
- values.yaml.gotmpl
```
`production.yaml`
```yaml
domain: prod.example.com
releaseName: prod
```
`values.yaml.gotmpl`
```yaml
domain: {{ .Environment.Values | getOrNil "my.domain" | default "dev.example.com" }}
```
`helmfile sync` installs `myapp` with the value `domain=dev.example.com`,
whereas `helmfile --environment production sync` installs the app with the value `domain=production.example.com`.
For even more flexibility, you can now use values declared in the `environments:` section in other parts of your helmfiles:
consider:
`default.yaml`
```yaml
domain: dev.example.com
releaseName: dev
```
```yaml
environments:
default:
values:
- default.yaml
production:
values:
- production.yaml # bare .yaml file, content will be used verbatim
- other.yaml.gotmpl # template directives with potential side-effects like `exec` and `readFile` will be honoured
releases:
- name: myapp-{{ .Environment.Values.releaseName }} # release name will be one of `dev` or `prod` depending on selected environment
values:
- values.yaml.gotmpl
{{ if eq (.Environment.Values.releaseName "prod" ) }}
# this release would be installed only if selected environment is `production`
- name: production-specific-release
...
{{ end }}
```
## Environment Secrets
Environment Secrets (not to be confused with Kubernetes Secrets) are encrypted versions of `Environment Values`.
You can list any number of `secrets.yaml` files created using `helm secrets` or `sops`, so that
Helmfile could automatically decrypt and merge the secrets into the environment values.
First you must have the [helm-secrets](https://github.com/futuresimple/helm-secrets) plugin installed along with a
`.sops.yaml` file to configure the method of encryption (this can be in the same directory as your helmfile or
in the sub-directory containing your secrets files).
Then suppose you have a a foo.bar secret defined in `environments/production/secrets.yaml`:
```yaml
foo.bar: "mysupersecretstring"
```
You can then encrypt it with `helm secrets enc environments/production/secrets.yaml`
Then reference that encrypted file in `helmfile.yaml`:
```yaml
environments:
production:
secrets:
- environments/production/secrets.yaml
releases:
- name: myapp
chart: mychart
values:
- values.yaml.gotmpl
```
Then the environment secret `foo.bar` can be referenced by the below template expression in your `values.yaml.gotmpl`:
```yaml
{{ .Environment.Values.foo.bar }}
```
## Separating helmfile.yaml into multiple independent files
Once your `helmfile.yaml` got to contain too many releases,
split it into multiple yaml files.
Recommended granularity of helmfile.yaml files is "per microservice" or "per team".
And there are two ways to organize your files.
- Single directory
- Glob patterns
### Single directory
`helmfile -f path/to/directory` loads and runs all the yaml files under the specified directory, each file as an independent helmfile.yaml.
The default helmfile directory is `helmfile.d`, that is,
in case helmfile is unable to locate `helmfile.yaml`, it tries to locate `helmfile.d/*.yaml`.
All the yaml files under the specified directory are processed in the alphabetical order. For example, you can use a `<two digit number>-<microservice>.yaml` naming convention to control the sync order.
- `helmfile.d`/
- `00-database.yaml`
- `00-backend.yaml`
- `01-frontend.yaml`
### Glob patterns
In case you want more control over how multiple `helmfile.yaml` files are organized, use `helmfiles:` configuration key in the `helmfile.yaml`:
Suppose you have multiple microservices organized in a Git repository that looks like:
- `myteam/` (sometimes it is equivalent to a k8s ns, that is `kube-system` for `clusterops` team)
- `apps/`
- `filebeat/`
- `helmfile.yaml` (no `charts/` exists, because it depends on the stable/filebeat chart hosted on the official helm charts repository)
- `README.md` (each app managed by my team has a dedicated README maintained by the owners of the app)
- `metricbeat/`
- `helmfile.yaml`
- `README.md`
- `elastalert-operator/`
- `helmfile.yaml`
- `README.md`
- `charts/`
- `elastalert-operator/`
- `<the content of the local helm chart>`
The benefits of this structure is that you can run `git diff` to locate in which directory=microservice a git commit has changes.
It allows your CI system to run a workflow for the changed microservice only.
A downside of this is that you don't have an obvious way to sync all microservices at once. That is, you have to run:
```bash
for d in apps/*; do helmfile -f $d diff; if [ $? -eq 2 ]; then helmfile -f $d sync; fi; done
```
At this point, you'll start writing a `Makefile` under `myteam/` so that `make sync-all` will do the job.
It does work, but you can rely on the Helmfile feature instead.
Put `myteam/helmfile.yaml` that looks like:
```yaml
helmfiles:
- apps/*/helmfile.yaml
```
So that you can get rid of the `Makefile` and the bash snippet.
Just run `helmfile sync` inside `myteam/`, and you are done.
All the files are sorted alphabetically per group = array item inside `helmfiles:`, so that you have granular control over ordering, too.
## Importing values from any source
The `exec` template function that is available in `values.yaml.gotmpl` is useful for importing values from any source
that is accessible by running a command:
A usual usage of `exec` would look like this:
```
mysetting: |
{{ exec "./mycmd" (list "arg1" "arg2" "--flag1") | indent 2 }}
```
Or even with a pipeline:
```
mysetting: |
{{ yourinput | exec "./mycmd-consume-stdin" (list "arg1" "arg2") | indent 2 }}
```
The possibility is endless. Try importing values from your golang app, bash script, jsonnet, or anything!
## Hooks
A Helmfile hook is a per-release extension point that is composed of:
- `events`
- `command`
- `args`
Helmfile triggers various `events` while it is running.
Once `events` are triggered, associated `hooks` are executed, by running the `command` with `args`.
Currently supported `events` are:
- `prepare`
- `cleanup`
Hooks associated to `prepare` events are triggered after each release in your helmfile is loaded from YAML, before execution.
Hooks associated to `cleanup` events are triggered after each release is processed.
The following is an example hook that just prints the contextual information provided to hook:
```
releases:
- name: myapp
chart: mychart
# *snip*
hooks:
- events: ["prepare", "cleanup"]
command: "echo"
args: ["{{`{{.Environment.Name}}`}}", "{{`{{.Release.Name}}`}}", "{{`{{.HelmfileCommand}}`}}\
"]
```
Let's say you ran `helmfile --environment prod sync`, the above hook results in executing:
```
echo {{Environment.Name}} {{.Release.Name}} {{.HelmfileCommand}}
```
Whereas the template expressions are executed thus the command becomes:
```
echo prod myapp sync
```
Now, replace `echo` with any command you like, and rewrite `args` that actually conforms to the command, so that you can integrate any command that does:
- templating
- linting
- testing
For templating, imagine that you created a hook that generates a helm chart on-the-fly by running an external tool like ksonnet, kustomize, or your own template engine.
It will allow you to write your helm releases with any language you like, while still leveraging goodies provided by helm.
### Helmfile + Kustomize
Do you prefer `kustomize` to write and organize your Kubernetes apps, but still want to leverage helm's useful features
like rollback, history, and so on? This section is for you!
The combination of `hooks` and [helmify-kustomize](https://gist.github.com/mumoshu/f9d0bd98e0eb77f636f79fc2fb130690)
enables you to integrate [kustomize](https://github.com/kubernetes-sigs/kustomize) into Helmfile.
That is, you can use `kustommize` to build a local helm chart from a kustomize overlay.
Let's assume you have a kustomize project named `foo-kustomize` like this:
```
foo-kustomize/
├── base
│   ├── configMap.yaml
│   ├── deployment.yaml
│   ├── kustomization.yaml
│   └── service.yaml
└── overlays
├── default
│   ├── kustomization.yaml
│   └── map.yaml
├── production
│   ├── deployment.yaml
│   └── kustomization.yaml
└── staging
├── kustomization.yaml
└── map.yaml
5 directories, 10 files
```
Write `helmfile.yaml`:
```yaml
- name: kustomize
chart: ./foo
hooks:
- events: ["prepare", "cleanup"]
command: "../helmify"
args: ["{{`{{if eq .Event.Name \"prepare\"}}build{{else}}clean{{end}}`}}", "{{`{{.Release.Ch\
art}}`}}", "{{`{{.Environment.Name}}`}}"]
```
Run `helmfile --environment staging sync` and see it results in helmfile running `kustomize build foo-kustomize/overlays/staging > foo/templates/all.yaml`.
Voilà! You can mix helm releases that are backed by remote charts, local charts, and even kustomize overlays.
## Guides
Use the [Helmfile Best Practices Guide](/docs/writing-helmfile.md) to write advanced helmfiles that feature:
- Default values
- Layering
## Using env files
Helmfile itself doesn't have an ability to load env files. But you can write some bash script to achieve the goal:
```console
set -a; . .env; set +a; helmfile sync
```
Please see #203 for more context.
## Running helmfile interactively
`helmfile --interactive [apply|delete]` requests confirmation from you before actually modifying your cluster.
Use it when you're running `helmfile` manually on your local machine or a kind of secure administrative hosts.
For your local use-case, aliasing it like `alias hi='helmfile --interactive'` would be convenient.
## Running Helmfile without an Internet connection
Once you download all required charts into your machine, you can run `helmfile charts` to deploy your apps.
It basically run only `helm upgrade --install` with your already-downloaded charts, hence no Internet connection is required.
See #155 for more information on this topic.
## Examples
For more examples, see [examples/REAMDME.md](https://github.com/roboll/helmfile/blob/master/examples/README.md).
For more examples, see the [examples/README.md](https://github.com/roboll/helmfile/blob/master/examples/README.md) or the [`helmfile`](https://github.com/cloudposse/helmfiles/tree/master/releases) distribution by [Cloud Posse](https://github.com/cloudposse/).
# Attribution
We use:
- [semtag](https://github.com/pnikosis/semtag) for automated semver tagging. I greatly appreciate the author(pnikosis)'s effort on creating it and their kindness to share it!
+403
View File
@@ -0,0 +1,403 @@
package main
import (
"fmt"
"github.com/roboll/helmfile/helmexec"
"github.com/roboll/helmfile/state"
"io/ioutil"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
)
type testFs struct {
wd string
dirs map[string]bool
files map[string]string
}
func appWithFs(app *app, files map[string]string) *app {
fs := newTestFs(files)
return injectFs(app, fs)
}
func injectFs(app *app, fs *testFs) *app {
app.readFile = fs.readFile
app.glob = fs.glob
app.abs = fs.abs
app.getwd = fs.getwd
app.chdir = fs.chdir
app.fileExistsAt = fs.fileExistsAt
app.directoryExistsAt = fs.directoryExistsAt
return app
}
func newTestFs(files map[string]string) *testFs {
dirs := map[string]bool{}
for abs, _ := range files {
d := filepath.Dir(abs)
dirs[d] = true
}
return &testFs{
wd: "/path/to",
dirs: dirs,
files: files,
}
}
func (f *testFs) fileExistsAt(path string) bool {
var ok bool
if strings.Contains(path, "/") {
_, ok = f.files[path]
} else {
_, ok = f.files[filepath.Join(f.wd, path)]
}
return ok
}
func (f *testFs) directoryExistsAt(path string) bool {
var ok bool
if strings.Contains(path, "/") {
_, ok = f.dirs[path]
} else {
_, ok = f.dirs[filepath.Join(f.wd, path)]
}
return ok
}
func (f *testFs) readFile(filename string) ([]byte, error) {
var str string
var ok bool
if strings.Contains(filename, "/") {
str, ok = f.files[filename]
} else {
str, ok = f.files[filepath.Join(f.wd, filename)]
}
if !ok {
return []byte(nil), fmt.Errorf("no file found: %s", filename)
}
return []byte(str), nil
}
func (f *testFs) glob(pattern string) ([]string, error) {
matches := []string{}
for name, _ := range f.files {
matched, err := filepath.Match(pattern, name)
if err != nil {
return nil, err
}
if matched {
matches = append(matches, name)
}
}
if len(matches) == 0 {
return []string(nil), fmt.Errorf("no file matched: %s", pattern)
}
return matches, nil
}
func (f *testFs) abs(path string) (string, error) {
var p string
if path[0] == '/' {
p = path
} else {
p = filepath.Join(f.wd, path)
}
return filepath.Clean(p), nil
}
func (f *testFs) getwd() (string, error) {
return f.wd, nil
}
func (f *testFs) chdir(dir string) error {
if dir == "/path/to" || dir == "/path/to/helmfile.d" {
f.wd = dir
return nil
}
return fmt.Errorf("unexpected chdir \"%s\"", dir)
}
// See https://github.com/roboll/helmfile/issues/193
func TestVisitDesiredStatesWithReleasesFiltered(t *testing.T) {
files := map[string]string{
"/path/to/helmfile.yaml": `
helmfiles:
- helmfile.d/a*.yaml
- helmfile.d/b*.yaml
`,
"/path/to/helmfile.d/a1.yaml": `
releases:
- name: zipkin
chart: stable/zipkin
`,
"/path/to/helmfile.d/a2.yaml": `
releases:
- name: prometheus
chart: stable/prometheus
`,
"/path/to/helmfile.d/b.yaml": `
releases:
- name: grafana
chart: stable/grafana
`,
}
noop := func(st *state.HelmState, helm helmexec.Interface) []error {
return []error{}
}
testcases := []struct {
name string
expectErr bool
}{
{name: "prometheus", expectErr: false},
{name: "zipkin", expectErr: false},
{name: "grafana", expectErr: false},
{name: "elasticsearch", expectErr: true},
}
for _, testcase := range testcases {
app := appWithFs(&app{
kubeContext: "default",
logger: helmexec.NewLogger(os.Stderr, "debug"),
selectors: []string{fmt.Sprintf("name=%s", testcase.name)},
namespace: "",
env: "default",
}, files)
err := app.VisitDesiredStatesWithReleasesFiltered(
"helmfile.yaml", noop,
)
if testcase.expectErr && err == nil {
t.Errorf("error expected but not happened for name=%s", testcase.name)
} else if !testcase.expectErr && err != nil {
t.Errorf("unexpected error for name=%s: %v", testcase.name, err)
}
}
}
// See https://github.com/roboll/helmfile/issues/320
func TestVisitDesiredStatesWithReleasesFiltered_UndefinedEnv(t *testing.T) {
files := map[string]string{
"/path/to/helmfile.yaml": `
environments:
prod:
helmfiles:
- helmfile.d/a*.yaml
`,
"/path/to/helmfile.d/a1.yaml": `
environments:
prod:
releases:
- name: zipkin
chart: stable/zipkin
`,
}
noop := func(st *state.HelmState, helm helmexec.Interface) []error {
return []error{}
}
testcases := []struct {
name string
expectErr bool
}{
{name: "undefined_env", expectErr: true},
{name: "default", expectErr: false},
{name: "prod", expectErr: false},
}
for _, testcase := range testcases {
app := appWithFs(&app{
kubeContext: "default",
logger: helmexec.NewLogger(os.Stderr, "debug"),
namespace: "",
selectors: []string{},
env: testcase.name,
}, files)
err := app.VisitDesiredStatesWithReleasesFiltered(
"helmfile.yaml", noop,
)
if testcase.expectErr && err == nil {
t.Errorf("error expected but not happened for environment=%s", testcase.name)
} else if !testcase.expectErr && err != nil {
t.Errorf("unexpected error for environment=%s: %v", testcase.name, err)
}
}
}
// See https://github.com/roboll/helmfile/issues/322
func TestVisitDesiredStatesWithReleasesFiltered_Selectors(t *testing.T) {
files := map[string]string{
"/path/to/helmfile.yaml": `
helmfiles:
- helmfile.d/a*.yaml
- helmfile.d/b*.yaml
`,
"/path/to/helmfile.d/a1.yaml": `
releases:
- name: zipkin
chart: stable/zipkin
`,
"/path/to/helmfile.d/a2.yaml": `
releases:
- name: prometheus
chart: stable/prometheus
`,
"/path/to/helmfile.d/b.yaml": `
releases:
- name: grafana
chart: stable/grafana
- name: foo
chart: charts/foo
labels:
duplicated: yes
- name: foo
chart: charts/foo
labels:
duplicated: yes
`,
}
testcases := []struct {
label string
expectedCount int
expectErr bool
errMsg string
}{
{label: "name=prometheus", expectedCount: 1, expectErr: false},
{label: "name=", expectedCount: 0, expectErr: true, errMsg: "failed processing /path/to/helmfile.d/a1.yaml: Malformed label: name=. Expected label in form k=v or k!=v"},
{label: "name!=", expectedCount: 0, expectErr: true, errMsg: "failed processing /path/to/helmfile.d/a1.yaml: Malformed label: name!=. Expected label in form k=v or k!=v"},
{label: "name", expectedCount: 0, expectErr: true, errMsg: "failed processing /path/to/helmfile.d/a1.yaml: Malformed label: name. Expected label in form k=v or k!=v"},
// See https://github.com/roboll/helmfile/issues/193
{label: "duplicated=yes", expectedCount: 0, expectErr: true, errMsg: "failed processing /path/to/helmfile.d/b.yaml: duplicate release \"foo\" found: there were 2 releases named \"foo\" matching specified selector"},
}
for _, testcase := range testcases {
actual := []string{}
collectReleases := func(st *state.HelmState, helm helmexec.Interface) []error {
for _, r := range st.Releases {
actual = append(actual, r.Name)
}
return []error{}
}
app := appWithFs(&app{
kubeContext: "default",
logger: helmexec.NewLogger(os.Stderr, "debug"),
namespace: "",
selectors: []string{testcase.label},
env: "default",
}, files)
err := app.VisitDesiredStatesWithReleasesFiltered(
"helmfile.yaml", collectReleases,
)
if testcase.expectErr {
if err == nil {
t.Errorf("error expected but not happened for selector %s", testcase.label)
} else if err.Error() != testcase.errMsg {
t.Errorf("unexpected error message: expected=\"%s\", actual=\"%s\"", testcase.errMsg, err.Error())
}
} else if !testcase.expectErr && err != nil {
t.Errorf("unexpected error for selector %s: %v", testcase.label, err)
}
if len(actual) != testcase.expectedCount {
t.Errorf("unexpected release count for selector %s: expected=%d, actual=%d", testcase.label, testcase.expectedCount, len(actual))
}
}
}
// See https://github.com/roboll/helmfile/issues/312
func TestVisitDesiredStatesWithReleasesFiltered_ReverseOrder(t *testing.T) {
files := map[string]string{
"/path/to/helmfile.yaml": `
helmfiles:
- helmfile.d/a*.yaml
- helmfile.d/b*.yaml
`,
"/path/to/helmfile.d/a1.yaml": `
releases:
- name: zipkin
chart: stable/zipkin
`,
"/path/to/helmfile.d/a2.yaml": `
releases:
- name: prometheus
chart: stable/prometheus
- name: elasticsearch
chart: stable/elasticsearch
`,
"/path/to/helmfile.d/b.yaml": `
releases:
- name: grafana
chart: stable/grafana
`,
}
expected := []string{"grafana", "elasticsearch", "prometheus", "zipkin"}
testcases := []struct {
reverse bool
expected []string
}{
{reverse: false, expected: []string{"zipkin", "prometheus", "elasticsearch", "grafana"}},
{reverse: true, expected: []string{"grafana", "elasticsearch", "prometheus", "zipkin"}},
}
for _, testcase := range testcases {
actual := []string{}
collectReleases := func(st *state.HelmState, helm helmexec.Interface) []error {
for _, r := range st.Releases {
actual = append(actual, r.Name)
}
return []error{}
}
app := appWithFs(&app{
kubeContext: "default",
logger: helmexec.NewLogger(os.Stderr, "debug"),
reverse: testcase.reverse,
namespace: "",
selectors: []string{},
env: "default",
}, files)
err := app.VisitDesiredStatesWithReleasesFiltered(
"helmfile.yaml", collectReleases,
)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(testcase.expected, actual) {
t.Errorf("releases did not match: expected=%v actual=%v", expected, actual)
}
}
}
func TestLoadDesiredStateFromYaml_DuplicateReleaseName(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease1
chart: mychart1
labels:
stage: pre
foo: bar
- name: myrelease1
chart: mychart2
labels:
stage: post
`)
app := &app{
readFile: ioutil.ReadFile,
glob: filepath.Glob,
abs: filepath.Abs,
kubeContext: "default",
logger: logger,
}
_, err := app.loadDesiredStateFromYaml(yamlContent, yamlFile, "default", "default")
if err != nil {
t.Error("unexpected error")
}
}
+115
View File
@@ -0,0 +1,115 @@
package args
import (
"fmt"
"strings"
"github.com/roboll/helmfile/state"
)
type keyVal struct {
key string
val string
spaceFlag bool
}
type argMap struct {
//m map[string]string
m map[string][]*keyVal
flags []string
}
func (a *argMap) SetArg(flag, arg string, isSpace bool) {
if _, exists := a.m[flag]; !exists {
keyarg := &keyVal{key: flag, val: arg, spaceFlag: isSpace}
a.m[flag] = append(a.m[flag], keyarg)
a.flags = append(a.flags, flag)
} else if flag == "--set" || flag == "-f" || flag == "--values" {
keyarg := &keyVal{key: flag, val: arg, spaceFlag: isSpace}
a.m[flag] = append(a.m[flag], keyarg)
}
}
func newArgMap() *argMap {
return &argMap{m: map[string][]*keyVal{}}
}
func GetArgs(args string, state *state.HelmState) []string {
argsMap := newArgMap()
if len(args) > 0 {
argsVals := strings.Split(args, " ")
for index, arg := range argsVals {
if strings.HasPrefix(arg, "--") {
argVal := strings.SplitN(arg, "=", 2)
if len(argVal) > 1 {
arg := argVal[0]
value := argVal[1]
argsMap.SetArg(arg, value, false)
} else {
//check if next value is arg to flag
if index+1 < len(argsVals) {
nextVal := argsVals[index+1]
if strings.HasPrefix(nextVal, "--") {
argsMap.SetArg(arg, "", false)
} else {
argsMap.SetArg(arg, nextVal, true)
}
} else {
argsMap.SetArg(arg, "", false)
}
}
}
}
}
if len(state.HelmDefaults.Args) > 0 {
for _, arg := range state.HelmDefaults.Args {
var flag string
var val string
argsNum, _ := fmt.Sscanf(arg, "--%s %s", &flag, &val)
if argsNum == 2 {
argsMap.SetArg(flag, val, true)
} else {
argVal := strings.SplitN(arg, "=", 2)
argFirst := argVal[0]
if len(argVal) > 1 {
val = argVal[1]
argsMap.SetArg(argFirst, val, false)
} else {
argsMap.SetArg(argFirst, "", false)
}
}
}
}
if state.HelmDefaults.TillerNamespace != "" {
argsMap.SetArg("--tiller-namespace", state.HelmDefaults.TillerNamespace, false)
}
if state.HelmDefaults.KubeContext != "" {
argsMap.SetArg("--kube-context", state.HelmDefaults.KubeContext, false)
}
var argArr []string
for _, flag := range argsMap.flags {
val := argsMap.m[flag]
for _, obj := range val {
if obj.val != "" {
if obj.spaceFlag {
argArr = append(argArr, obj.key, obj.val)
} else {
argArr = append(argArr, fmt.Sprintf("%s=%s", obj.key, obj.val))
}
} else {
argArr = append(argArr, fmt.Sprintf("%s", obj.key))
}
}
}
state.HelmDefaults.Args = argArr
return state.HelmDefaults.Args
}
+46
View File
@@ -0,0 +1,46 @@
package args
import (
"strings"
"testing"
"github.com/roboll/helmfile/state"
)
func TestGetArgs(t *testing.T) {
args := "--timeout=3600 --set app1.bootstrap=true --set app2.bootstrap=false --tiller-namespace ns"
defaultArgs := []string{"--recreate-pods", "--force"}
Helmdefaults := state.HelmSpec{KubeContext: "test", TillerNamespace: "test-namespace", Args: defaultArgs}
testState := &state.HelmState{HelmDefaults: Helmdefaults}
receivedArgs := GetArgs(args, testState)
expectedOutput := "--timeout=3600 --set app1.bootstrap=true --set app2.bootstrap=false --tiller-namespace ns --recreate-pods --force --kube-context=test"
if compareArgs(expectedOutput, receivedArgs) == false {
t.Errorf("expected %s, got %s", expectedOutput, strings.Join(receivedArgs, " "))
}
}
func Test2(t *testing.T) {
args := "--timeout=3600 --set app1.bootstrap=true --set app2.bootstrap=false,app3.bootstrap=true --tiller-namespace ns"
defaultArgs := []string{"--recreate-pods", "--force"}
Helmdefaults := state.HelmSpec{KubeContext: "test", TillerNamespace: "test-namespace", Args: defaultArgs}
testState := &state.HelmState{HelmDefaults: Helmdefaults}
receivedArgs := GetArgs(args, testState)
expectedOutput := "--timeout=3600 --set app1.bootstrap=true --set app2.bootstrap=false,app3.bootstrap=true --tiller-namespace ns --recreate-pods --force --kube-context=test"
if compareArgs(expectedOutput, receivedArgs) == false {
t.Errorf("expected %s, got %s", expectedOutput, strings.Join(receivedArgs, " "))
}
}
func compareArgs(expectedArgs string, args []string) bool {
if strings.Compare(strings.Join(args, " "), expectedArgs) != 0 {
return false
}
return true
}
+33
View File
@@ -0,0 +1,33 @@
package main
import (
"bufio"
"fmt"
"log"
"os"
"strings"
)
// Copyright (c) 2017 Roland Singer [roland.singer@desertbit.com]
//
// Shamelessly borrowed from @r0l1's awesome work that is available at https://gist.github.com/r0l1/3dcbb0c8f6cfe9c66ab8008f55f8f28b
func askForConfirmation(s string) bool {
reader := bufio.NewReader(os.Stdin)
for {
fmt.Printf("%s [y/n]: ", s)
response, err := reader.ReadString('\n')
if err != nil {
log.Fatal(err)
}
response = strings.ToLower(strings.TrimSpace(response))
if response == "y" || response == "yes" {
return true
} else if response == "n" || response == "no" {
return false
}
}
}
+161
View File
@@ -0,0 +1,161 @@
# The Helmfile Best Practices Guide
This guide covers the Helmfile’s considered patterns for writing advanced helmfiles. It focuses on how helmfile should be structured and executed.
## Missing keys and Default values
helmfile tries its best to inform users for noticing potential mistakes.
One example of how helmfile achieves it is that, `helmfile` fails when you tried to access missing keys in environment values.
That is, the following example let `helmfile` fail when you have no `eventApi.replicas` defined in environment values.
```
{{ .Environment.Values.eventApi.replicas | default 1 }}
```
In case it isn't a mistake and you do want to allow missing keys, use the `getOrNil` template function:
```
{{ .Environment.Values | getOrNil "eventApi.replicas" }}
```
This result in printing `<no value` in your template, that may or may not result in a failure.
If you want a kind of default values that is used when a missing key was referenced, use `default` like:
```
{{ .Environment.Values | getOrNil "eventApi.replicas" | default 1 }}
```
Now, you get `1` when there is no `eventApi.replicas` defined in environment values.
## Release Template / Conventional Directory Structure
Introducing helmfile into a large-scale project that involes dozens of releases often results in a lot of repetitions in `helmfile.yaml` files.
The example below shows repetitions in `namespace`, `chart`, `values`, and `secrets`:
```yaml
releases:
# *snip*
- name: heapster
namespace: kube-system
chart: stable/heapster
version: 0.3.2
values:
- "./config/heapster/values.yaml"
- "./config/heapster/{{ .Environment.Name }}.yaml"
secrets:
- "./config/heapster/secrets.yaml"
- "./config/heapster/{{ .Environment.Name }}-secrets.yaml"
- name: kubernetes-dashboard
namespace: kube-system
chart: stable/kubernetes-dashboard
version: 0.10.0
values:
- "./config/kubernetes-dashboard/values.yaml"
- "./config/kubernetes-dashboard/{{ .Environment.Name }}.yaml"
values:
- "./config/kubernetes-dashboard/secrets.yaml"
- "./config/kubernetes-dashboard/{{ .Environment.Name }}-secrets.yaml"
```
This is where Helmfile's advanced feature called Release Template comes handy.
It allows you to abstract away the repetitions in releases into a template, which is then included and executed by using YAML anchor/alias:
```yaml
templates:
default: &default
chart: stable/{{`{{ .Release.Name }}`}}
namespace: kube-system
# This prevents helmfile exiting when it encounters a missing file
# Valid values are "Error", "Warn", "Info", "Debug". The default is "Error"
# Use "Debug" to make missing files errors invisible at the default log level(--log-level=INFO)
missingFileHandler: Warn
values:
- config/{{`{{ .Release.Name }}`}}/values.yaml
- config/{{`{{ .Release.Name }}`}}/{{`{{ .Environment.Name }}`}}.yaml
secrets:
- config/{{`{{ .Release.Name }}`}}/secrets.yaml
- config/{{`{{ .Release.Name }}`}}/{{`{{ .Environment.Name }}`}}-secrets.yaml
releases:
- name: heapster
<<: *default
- name: kubernetes-dashboard
<<: *default
```
See the [issue 428](https://github.com/roboll/helmfile/issues/428) for more context on how this is supposed to work.
## Layering
You may occasionally end up with many helmfiles that shares common parts like which repositories to use, and whichi release to be bundled by default.
Use Layering to extract te common parts into a dedicated *library helmfile*s, so that each helmfile becomes DRY.
Let's assume that your `helmfile.yaml` looks like:
```
{ readFile "commons.yaml" }}
---
{{ readFile "environments.yaml" }}
---
releases:
- name: myapp
chart: mychart
```
Whereas `commons.yaml` contained a monitoring agent:
```yaml
releases:
- name: metricbaet
chart: stable/metricbeat
```
And `environments.yaml` contained well-known environments:
```yaml
environments:
development:
production:
```
At run time, template expressions in your `helmfile.yaml` are executed:
```yaml
releases:
- name: metricbaet
chart: stable/metricbeat
---
environments:
development:
production:
---
releases:
- name: myapp
chart: mychart
```
Resulting YAML documents are merged in the order of occurrence,
so that your `helmfile.yaml` becomes:
```yaml
environments:
development:
production:
releases:
- name: metricbaet
chart: stable/metricbeat
- name: myapp
chart: mychart
```
Great!
Now, repeat the above steps for each your `helmfile.yaml`, so that all your helmfiles becomes DRY.
+8
View File
@@ -0,0 +1,8 @@
package environment
type Environment struct {
Name string
Values map[string]interface{}
}
var EmptyEnvironment Environment
+102
View File
@@ -0,0 +1,102 @@
package event
import (
"fmt"
"github.com/roboll/helmfile/environment"
"github.com/roboll/helmfile/helmexec"
"github.com/roboll/helmfile/tmpl"
"go.uber.org/zap"
"os"
)
type Hook struct {
Name string `yaml:"name"`
Events []string `yaml:"events"`
Command string `yaml:"command"`
Args []string `yaml:"args"`
}
type event struct {
Name string
}
type Bus struct {
Runner helmexec.Runner
Hooks []Hook
BasePath string
StateFilePath string
Namespace string
Env environment.Environment
ReadFile func(string) ([]byte, error)
Logger *zap.SugaredLogger
}
func (bus *Bus) Trigger(evt string, context map[string]interface{}) (bool, error) {
if bus.Runner == nil {
bus.Runner = helmexec.ShellRunner{
Dir: bus.BasePath,
}
}
executed := false
for _, hook := range bus.Hooks {
contained := false
for _, e := range hook.Events {
contained = contained || e == evt
}
if !contained {
continue
}
var err error
name := hook.Name
if name == "" {
name = hook.Command
}
fmt.Fprintf(os.Stderr, "%s: basePath=%s\n", bus.StateFilePath, bus.BasePath)
data := map[string]interface{}{
"Environment": bus.Env,
"Namespace": bus.Namespace,
"Event": event{
Name: evt,
},
}
for k, v := range context {
data[k] = v
}
render := tmpl.NewTextRenderer(bus.ReadFile, bus.BasePath, data)
bus.Logger.Debugf("hook[%s]: triggered by event \"%s\"\n", name, evt)
command, err := render.RenderTemplateText(hook.Command)
if err != nil {
return false, fmt.Errorf("hook[%s]: %v", name, err)
}
args := make([]string, len(hook.Args))
for i, raw := range hook.Args {
args[i], err = render.RenderTemplateText(raw)
if err != nil {
return false, fmt.Errorf("hook[%s]: %v", name, err)
}
}
bytes, err := bus.Runner.Execute(command, args)
bus.Logger.Debugf("hook[%s]: %s\n", name, string(bytes))
if err != nil {
return false, fmt.Errorf("hook[%s]: command `%s` failed: %v", name, command, err)
}
executed = true
}
return executed, nil
}
+113
View File
@@ -0,0 +1,113 @@
package event
import (
"fmt"
"github.com/roboll/helmfile/environment"
"github.com/roboll/helmfile/helmexec"
"os"
"testing"
)
var logger = helmexec.NewLogger(os.Stdout, "warn")
type runner struct {
}
func (r *runner) Execute(cmd string, args []string) ([]byte, error) {
if cmd == "ng" {
return nil, fmt.Errorf("cmd failed due to invalid cmd: %s", cmd)
}
for _, a := range args {
if a == "ng" {
return nil, fmt.Errorf("cmd failed due to invalid arg: %s", a)
}
}
return []byte(""), nil
}
func TestTrigger(t *testing.T) {
cases := []struct {
name string
hook *Hook
triggeredEvt string
expectedResult bool
expectedErr string
}{
{
"okhook1",
&Hook{"okhook1", []string{"foo"}, "ok", []string{}},
"foo",
true,
"",
},
{
"missinghook1",
&Hook{"okhook1", []string{"foo"}, "ok", []string{}},
"bar",
false,
"",
},
{
"nohook1",
nil,
"bar",
false,
"",
},
{
"nghook1",
&Hook{"nghook1", []string{"foo"}, "ng", []string{}},
"foo",
false,
"hook[nghook1]: command `ng` failed: cmd failed due to invalid cmd: ng",
},
{
"nghook2",
&Hook{"nghook2", []string{"foo"}, "ok", []string{"ng"}},
"foo",
false,
"hook[nghook2]: command `ok` failed: cmd failed due to invalid arg: ng",
},
}
readFile := func(filename string) ([]byte, error) {
return nil, fmt.Errorf("unexpected call to readFile: %s", filename)
}
for _, c := range cases {
hooks := []Hook{}
if c.hook != nil {
hooks = append(hooks, *c.hook)
}
bus := &Bus{
Hooks: hooks,
StateFilePath: "path/to/helmfile.yaml",
BasePath: "path/to",
Namespace: "myns",
Env: environment.Environment{Name: "prod"},
Logger: logger,
ReadFile: readFile,
}
bus.Runner = &runner{}
data := map[string]interface{}{
"Release": "myrel",
"HelmfileCommand": "mycmd",
}
ok, err := bus.Trigger(c.triggeredEvt, data)
if ok != c.expectedResult {
t.Errorf("unexpected result for case \"%s\": expected=%v, actual=%v", c.name, c.expectedResult, ok)
}
if c.expectedErr != "" {
if err == nil {
t.Error("error expected, but not occurred")
} else if err.Error() != c.expectedErr {
t.Errorf("unexpected error for case \"%s\": expected=%s, actual=%v", c.name, c.expectedErr, err)
}
} else {
if err != nil {
t.Errorf("unexpected error for case \"%s\": %v", c.name, err)
}
}
}
}
Executable
+623
View File
@@ -0,0 +1,623 @@
#!/usr/bin/env bash
PROG=semtag
PROG_VERSION="v0.1.0"
SEMVER_REGEX="^v?(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(\-[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$"
IDENTIFIER_REGEX="^\-([0-9A-Za-z-]+)\.([0-9A-Za-z-]+)*$"
# Global variables
FIRST_VERSION="v0.0.0"
finalversion=$FIRST_VERSION
lastversion=$FIRST_VERSION
hasversiontag="false"
scope="patch"
displayonly="false"
forcetag="false"
forcedversion=
versionname=
identifier=
HELP="\
Usage:
$PROG
$PROG getlast
$PROG getfinal
$PROG (final|alpha|beta|candidate) [-s <scope> (major|minor|patch|auto) | -o]
$PROG --help
$PROG --version
Options:
-s The scope that must be increased, can be major, minor or patch.
The resulting version will match X.Y.Z(-PRERELEASE)(+BUILD)
where X, Y and Z are positive integers, PRERELEASE is an optionnal
string composed of alphanumeric characters describing if the build is
a release candidate, alpha or beta version, with a number.
BUILD is also an optional string composed of alphanumeric
characters and hyphens.
Setting the scope as 'auto', the script will chose the scope between
'minor' and 'patch', depending on the amount of lines added (<10% will
choose patch).
-v Specifies manually the version to be tagged, must be a valid semantic version
in the format X.Y.Z where X, Y and Z are positive integers.
-o Output the version only, shows the bumped version, but doesn't tag.
-f Forces to tag, even if there are unstaged or uncommited changes.
Commands:
--help Print this help message.
--version Prints the program's version.
get Returns both current final version and last tagged version.
getlast Returns the latest tagged version.
getfinal Returns the latest tagged final version.
getcurrent Returns the current version, based on the latest one, if there are uncommited or
unstaged changes, they will be reflected in the version, adding the number of
pending commits, current branch and commit hash.
final Tags the current build as a final version, this only can be done on the master branch.
candidate Tags the current build as a release candidate, the tag will contain all
the commits from the last final version.
alpha Tags the current build as an alpha version, the tag will contain all
the commits from the last final version.
beta Tags the current build as a beta version, the tag will contain all
the commits from the last final version."
# Commands and options
ACTION="getlast"
ACTION="$1"
shift
# We get the parameters
while getopts "v:s:of" opt; do
case $opt in
v)
forcedversion="$OPTARG"
;;
s)
scope="$OPTARG"
;;
o)
displayonly="true"
;;
f)
forcetag="true"
;;
\?)
echo "Invalid option: -$OPTARG" >&2
exit 1
;;
:)
echo "Option -$OPTARG requires an argument." >&2
exit 1
;;
esac
done
# Gets a string with the version and returns an array of maximum size of 5 with all the parts of the sematinc version
# $1 The string containing the version in semantic format
# $2 The variable to store the result array:
# position 0: major number
# position 1: minor number
# position 2: patch number
# position 3: identifier (or prerelease identifier)
# position 4: build info
function explode_version {
local __version=$1
local __result=$2
if [[ $__version =~ $SEMVER_REGEX ]] ; then
local __major=${BASH_REMATCH[1]}
local __minor=${BASH_REMATCH[2]}
local __patch=${BASH_REMATCH[3]}
local __prere=${BASH_REMATCH[4]}
local __build=${BASH_REMATCH[5]}
eval "$__result=(\"$__major\" \"$__minor\" \"$__patch\" \"$__prere\" \"$__build\")"
else
eval "$__result="
fi
}
# Compare two versions and returns -1, 0 or 1
# $1 The first version to compare
# $2 The second version to compare
# $3 The variable where to store the result
function compare_versions {
local __first
local __second
explode_version $1 __first
explode_version $2 __second
local lv=$3
# Compares MAJOR, MINOR and PATCH
for i in 0 1 2; do
local __numberfirst=${__first[$i]}
local __numbersecond=${__second[$i]}
case $(($__numberfirst - $__numbersecond)) in
0)
;;
-[0-9]*)
eval "$lv=-1"
return 0
;;
[0-9]*)
eval "$lv=1"
return 0
;;
esac
done
# Identifiers should compare with the ASCII order.
local __identifierfirst=${__first[3]}
local __identifiersecond=${__second[3]}
if [[ -n "$__identifierfirst" ]] && [[ -n "$__identifiersecond" ]]; then
if [[ "$__identifierfirst" > "$__identifiersecond" ]]; then
eval "$lv=1"
return 0
elif [[ "$__identifierfirst" < "$__identifiersecond" ]]; then
eval "$lv=-1"
return 0
fi
elif [[ -z "$__identifierfirst" ]] && [[ -n "$__identifiersecond" ]]; then
eval "$lv=1"
return 0
elif [[ -n "$__identifierfirst" ]] && [[ -z "$__identifiersecond" ]]; then
eval "$lv=-1"
return 0
fi
eval "$lv=0"
}
# Returns the last version of two
# $1 The first version to compare
# $2 The second version to compare
# $3 The variable where to store the last one
function get_latest_of_two {
local __first=$1
local __second=$2
local __result
local __latest=$3
compare_versions $__first $__second __result
case $__result in
0)
eval "$__latest=$__second"
;;
-1)
eval "$__latest=$__second"
;;
1)
eval "$__latest=$__first"
;;
esac
}
# Assigns a 2 size array with the identifier, having the identifier at pos 0, and the number in pos 1
# $1 The identifier in the format -id.#
# $2 The vferiable where to store the 2 size array
function explode_identifier {
local __identifier=$1
local __result=$2
if [[ $__identifier =~ $IDENTIFIER_REGEX ]] ; then
local __id=${BASH_REMATCH[1]}
local __number=${BASH_REMATCH[2]}
if [[ -z "$__number" ]]; then
__number=1
fi
eval "$__result=(\"$__id\" \"$__number\")"
else
eval "$__result="
fi
}
# Gets a list of tags and assigns the base and latest versions
# Receives an array with the tags containing the versions
# Assigns to the global variables finalversion and lastversion the final version and the latest version
function get_latest {
local __taglist=("$@")
local __tagsnumber=${#__taglist[@]}
local __current
case $__tagsnumber in
0)
finalversion=$FIRST_VERSION
lastversion=$FIRST_VERSION
;;
1)
__current=${__taglist[0]}
explode_version $__current ver
if [ -n "$ver" ]; then
if [ -n "${ver[3]}" ]; then
finalversion=$FIRST_VERSION
else
finalversion=$__current
fi
lastversion=$__current
else
finalversion=$FIRST_VERSION
lastversion=$FIRST_VERSION
fi
;;
*)
local __lastpos=$(($__tagsnumber-1))
for i in $(seq 0 $__lastpos)
do
__current=${__taglist[i]}
explode_version ${__taglist[i]} ver
if [ -n "$ver" ]; then
if [ -z "${ver[3]}" ]; then
get_latest_of_two $finalversion $__current finalversion
get_latest_of_two $lastversion $finalversion lastversion
else
get_latest_of_two $lastversion $__current lastversion
fi
fi
done
;;
esac
if git rev-parse -q --verify "refs/tags/$lastversion" >/dev/null; then
hasversiontag="true"
else
hasversiontag="false"
fi
}
# Gets the next version given the provided scope
# $1 The version that is going to be bumped
# $2 The scope to bump
# $3 The variable where to stoer the result
function get_next_version {
local __exploded
local __fromversion=$1
local __scope=$2
local __result=$3
explode_version $__fromversion __exploded
case $__scope in
major)
__exploded[0]=$((${__exploded[0]}+1))
__exploded[1]=0
__exploded[2]=0
;;
minor)
__exploded[1]=$((${__exploded[1]}+1))
__exploded[2]=0
;;
patch)
__exploded[2]=$((${__exploded[2]}+1))
;;
esac
eval "$__result=v${__exploded[0]}.${__exploded[1]}.${__exploded[2]}"
}
function bump_version {
## First we try to get the next version based on the existing last one
if [ "$scope" == "auto" ]; then
get_scope_auto scope
fi
local __candidatefromlast=$FIRST_VERSION
local __explodedlast
explode_version $lastversion __explodedlast
if [[ -n "${__explodedlast[3]}" ]]; then
# Last version is not final
local __idlast
explode_identifier ${__explodedlast[3]} __idlast
# We get the last, given the desired id based on the scope
__candidatefromlast="v${__explodedlast[0]}.${__explodedlast[1]}.${__explodedlast[2]}"
if [[ -n "$identifier" ]]; then
local __nextid="$identifier.1"
if [ "$identifier" == "${__idlast[0]}" ]; then
# We target the same identifier as the last so we increase one
__nextid="$identifier.$(( ${__idlast[1]}+1 ))"
__candidatefromlast="$__candidatefromlast-$__nextid"
else
# Different identifiers, we make sure we are assigning a higher identifier, if not, we increase the version
__candidatefromlast="$__candidatefromlast-$__nextid"
local __comparedwithlast
compare_versions $__candidatefromlast $lastversion __comparedwithlast
if [ "$__comparedwithlast" == -1 ]; then
get_next_version $__candidatefromlast $scope __candidatefromlast
__candidatefromlast="$__candidatefromlast-$__nextid"
fi
fi
fi
fi
# Then we try to get the version based on the latest final one
local __candidatefromfinal=$FIRST_VERSION
get_next_version $finalversion $scope __candidatefromfinal
if [[ -n "$identifier" ]]; then
__candidatefromfinal="$__candidatefromfinal-$identifier.1"
fi
# Finally we compare both candidates
local __resultversion
local __result
compare_versions $__candidatefromlast $__candidatefromfinal __result
case $__result in
0)
__resultversion=$__candidatefromlast
;;
-1)
__resultversion="$__candidatefromfinal"
;;
1)
__resultversion=$__candidatefromlast
;;
esac
eval "$1=$__resultversion"
}
function increase_version {
local __version=
if [ -z $forcedversion ]; then
bump_version __version
else
if [[ $forcedversion =~ $SEMVER_REGEX ]] ; then
compare_versions $forcedversion $lastversion __result
if [ $__result -le 0 ]; then
echo "Version can't be lower than last version: $lastversion"
exit 1
fi
else
echo "Non valid version to bump"
exit 1
fi
__version=$forcedversion
fi
if [ "$displayonly" == "true" ]; then
echo "$__version"
else
if [ "$forcetag" == "false" ]; then
check_git_dirty_status
fi
local __commitlist
if [ "$finalversion" == "$FIRST_VERSION" ] || [ "$hasversiontag" != "true" ]; then
__commitlist="$(git log --pretty=oneline | cat)"
else
__commitlist="$(git log --pretty=oneline $finalversion... | cat)"
fi
# If we are forcing a bump, we add bump to the commit list
if [[ -z $__commitlist && "$forcetag" == "true" ]]; then
__commitlist="bump"
fi
if [[ -z $__commitlist ]]; then
echo "No commits since the last final version, not bumping version"
else
if [[ -z $versionname ]]; then
versionname=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
fi
local __message="$versionname
$__commitlist"
# We check we have info on the user
local __username=$(git config user.name)
if [ -z "$__username" ]; then
__username=$(id -u -n)
git config user.name $__username
fi
local __useremail=$(git config user.email)
if [ -z "$__useremail" ]; then
__useremail=$(hostname)
git config user.email "$__username@$__useremail"
fi
git tag -a $__version -m "$__message"
# If we have a remote, we push there
local __remote=${SEMTAG_REMOTE:-$(git remote)}
if [[ -n $__remote ]]; then
git push $__remote $__version > /dev/null
if [ $? -eq 0 ]; then
echo "$__version"
else
echo "Error pushing the tag $__version to $__remote"
exit 1
fi
else
echo "$__version"
fi
fi
fi
}
function check_git_dirty_status {
local __repostatus=
get_work_tree_status __repostatus
if [ "$__repostatus" == "uncommitted" ]; then
echo "ERROR: You have uncommitted changes"
git status --porcelain
exit 1
fi
if [ "$__repostatus" == "unstaged" ]; then
echo "ERROR: You have unstaged changes"
git status --porcelain
exit 1
fi
}
# Get the total amount of lines of code in the repo
function get_total_lines {
local __empty_id="$(git hash-object -t tree /dev/null)"
local __changes="$(git diff --numstat $__empty_id | cat)"
local __added_deleted=$1
get_changed_lines "$__changes" $__added_deleted
}
# Get the total amount of lines of code since the provided tag
function get_sincetag_lines {
local __sincetag=$1
local __changes="$(git diff --numstat $__sincetag | cat)"
local __added_deleted=$2
get_changed_lines "$__changes" $__added_deleted
}
function get_changed_lines {
local __changes_numstat=$1
local __result=$2
IFS=$'\n' read -rd '' -a __changes_array <<<"$__changes_numstat"
local __diff_regex="^([0-9]+)[[:space:]]+([0-9]+)[[:space:]]+.+$"
local __total_added=0
local __total_deleted=0
for i in "${__changes_array[@]}"
do
if [[ $i =~ $__diff_regex ]] ; then
local __added=${BASH_REMATCH[1]}
local __deleted=${BASH_REMATCH[2]}
__total_added=$(( $__total_added+$__added ))
__total_deleted=$(( $__total_deleted+$__deleted ))
fi
done
eval "$2=( $__total_added $__total_deleted )"
}
function get_scope_auto {
local __verbose=$2
local __total=0
local __since=0
local __scope=
get_total_lines __total
get_sincetag_lines $finalversion __since
local __percentage=0
if [ "$__total" != "0" ]; then
local __percentage=$(( 100*$__since/$__total ))
if [ $__percentage -gt "10" ]; then
__scope="minor"
else
__scope="patch"
fi
fi
eval "$1=$__scope"
if [[ -n "$__verbose" ]]; then
echo "[Auto Scope] Percentage of lines changed: $__percentage"
echo "[Auto Scope] : $__scope"
fi
}
function get_work_tree_status {
# Update the index
git update-index -q --ignore-submodules --refresh > /dev/null
eval "$1="
if ! git diff-files --quiet --ignore-submodules -- > /dev/null
then
eval "$1=unstaged"
fi
if ! git diff-index --cached --quiet HEAD --ignore-submodules -- > /dev/null
then
eval "$1=uncommitted"
fi
}
function get_current {
if [ "$hasversiontag" == "true" ]; then
local __commitcount="$(git rev-list $lastversion.. --count)"
else
local __commitcount="$(git rev-list --count HEAD)"
fi
local __status=
get_work_tree_status __status
if [ "$__commitcount" == "0" ] && [ -z "$__status" ]; then
eval "$1=$lastversion"
else
local __buildinfo="$(git rev-parse --short HEAD)"
local __currentbranch="$(git rev-parse --abbrev-ref HEAD)"
if [ "$__currentbranch" != "master" ]; then
__buildinfo="$__currentbranch.$__buildinfo"
fi
local __suffix=
if [ "$__commitcount" != "0" ]; then
if [ -n "$__suffix" ]; then
__suffix="$__suffix."
fi
__suffix="$__suffix$__commitcount"
fi
if [ -n "$__status" ]; then
if [ -n "$__suffix" ]; then
__suffix="$__suffix."
fi
__suffix="$__suffix$__status"
fi
__suffix="$__suffix+$__buildinfo"
if [ "$lastversion" == "$finalversion" ]; then
scope="patch"
identifier=
local __bumped=
bump_version __bumped
eval "$1=$__bumped-dev.$__suffix"
else
eval "$1=$lastversion.$__suffix"
fi
fi
}
function init {
git fetch > /dev/null
TAGS="$(git tag)"
IFS=$'\n' read -rd '' -a TAG_ARRAY <<<"$TAGS"
get_latest ${TAG_ARRAY[@]}
currentbranch="$(git rev-parse --abbrev-ref HEAD)"
}
case $ACTION in
--help)
echo -e "$HELP"
;;
--version)
echo -e "${PROG}: $PROG_VERSION"
;;
final)
init
diff=$(git diff master | cat)
if [ "$forcetag" == "false" ]; then
if [ -n "$diff" ]; then
echo "ERROR: Branch must be updated with master for final versions"
exit 1
fi
fi
increase_version
;;
alpha|beta)
init
identifier="$ACTION"
increase_version
;;
candidate)
init
identifier="rc"
increase_version
;;
getlast)
init
echo "$lastversion"
;;
getfinal)
init
echo "$finalversion"
;;
getcurrent)
init
get_current current
echo "$current"
;;
get)
init
echo "Current final version: $finalversion"
echo "Last tagged version: $lastversion"
;;
*)
echo "'$ACTION' is not a valid command, see --help for available commands."
;;
esac
+121 -21
View File
@@ -1,9 +1,15 @@
package helmexec
import (
"fmt"
"io"
"io/ioutil"
"os"
"strings"
"sync"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
)
const (
@@ -11,16 +17,36 @@ const (
)
type execer struct {
runner Runner
writer io.Writer
kubeContext string
extra []string
helmBinary string
runner Runner
logger *zap.SugaredLogger
kubeContext string
extra []string
decryptionMutex sync.Mutex
}
func NewLogger(writer io.Writer, logLevel string) *zap.SugaredLogger {
var cfg zapcore.EncoderConfig
cfg.MessageKey = "message"
out := zapcore.AddSync(writer)
var level zapcore.Level
err := level.Set(logLevel)
if err != nil {
panic(err)
}
core := zapcore.NewCore(
zapcore.NewConsoleEncoder(cfg),
out,
level,
)
return zap.New(core).Sugar()
}
// New for running helm commands
func New(writer io.Writer, kubeContext string) *execer {
func New(logger *zap.SugaredLogger, kubeContext string) *execer {
return &execer{
writer: writer,
helmBinary: command,
logger: logger,
kubeContext: kubeContext,
runner: &ShellRunner{},
}
@@ -30,57 +56,131 @@ func (helm *execer) SetExtraArgs(args ...string) {
helm.extra = args
}
func (helm *execer) AddRepo(name, repository, certfile, keyfile string) error {
func (helm *execer) SetHelmBinary(bin string) {
helm.helmBinary = bin
}
func (helm *execer) AddRepo(name, repository, certfile, keyfile, username, password string) error {
var args []string
args = append(args, "repo", "add", name, repository)
if certfile != "" && keyfile != "" {
args = append(args, "--cert-file", certfile, "--key-file", keyfile)
}
if username != "" && password != "" {
args = append(args, "--username", username, "--password", password)
}
helm.logger.Infof("Adding repo %v %v", name, repository)
out, err := helm.exec(args...)
helm.write(out)
return err
}
func (helm *execer) UpdateRepo() error {
helm.logger.Info("Updating repo")
out, err := helm.exec("repo", "update")
helm.write(out)
return err
}
func (helm *execer) UpdateDeps(chart string) error {
helm.logger.Infof("Updating dependency %v", chart)
out, err := helm.exec("dependency", "update", chart)
helm.write(out)
return err
}
func (helm *execer) SyncRelease(name, chart string, flags ...string) error {
out, err := helm.exec(append([]string{"upgrade", "--install", name, chart}, flags...)...)
helm.logger.Infof("Upgrading %v", chart)
out, err := helm.exec(append([]string{"upgrade", "--install", "--reset-values", name, chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) ReleaseStatus(name string) error {
helm.logger.Infof("Getting status %v", name)
out, err := helm.exec(append([]string{"status", name})...)
if helm.writer != nil {
helm.writer.Write(out)
}
helm.write(out)
return err
}
func (helm *execer) DecryptSecret(name string) (string, error) {
// Prevents https://github.com/roboll/helmfile/issues/258
helm.decryptionMutex.Lock()
defer helm.decryptionMutex.Unlock()
helm.logger.Infof("Decrypting secret %v", name)
out, err := helm.exec(append([]string{"secrets", "dec", name})...)
helm.write(out)
return name + ".dec", err
tmpFile, err := ioutil.TempFile("", "secret")
if err != nil {
return "", err
}
defer tmpFile.Close()
// os.Rename seems to results in "cross-device link` errors in some cases
// Instead of moving, copy it to the destination temp file as a work-around
// See https://github.com/roboll/helmfile/issues/251#issuecomment-417166296f
decFilename := name + ".dec"
decFile, err := os.Open(decFilename)
if err != nil {
return "", err
}
defer decFile.Close()
_, err = io.Copy(tmpFile, decFile)
if err != nil {
return "", err
}
if err := decFile.Close(); err != nil {
return "", err
}
if err := os.Remove(decFilename); err != nil {
return "", err
}
return tmpFile.Name(), err
}
func (helm *execer) DiffRelease(name, chart string, flags ...string) error {
out, err := helm.exec(append([]string{"diff", name, chart}, flags...)...)
func (helm *execer) TemplateRelease(chart string, flags ...string) error {
out, err := helm.exec(append([]string{"template", chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) DeleteRelease(name string) error {
out, err := helm.exec("delete", "--purge", name)
func (helm *execer) DiffRelease(name, chart string, flags ...string) error {
helm.logger.Infof("Comparing %v %v", name, chart)
out, err := helm.exec(append([]string{"diff", "upgrade", "--allow-unreleased", name, chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) Lint(chart string, flags ...string) error {
helm.logger.Infof("Linting %v", chart)
out, err := helm.exec(append([]string{"lint", chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) Fetch(chart string, flags ...string) error {
helm.logger.Infof("Fetching %v", chart)
out, err := helm.exec(append([]string{"fetch", chart}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) DeleteRelease(name string, flags ...string) error {
helm.logger.Infof("Deleting %v", name)
out, err := helm.exec(append([]string{"delete", name}, flags...)...)
helm.write(out)
return err
}
func (helm *execer) TestRelease(name string, flags ...string) error {
helm.logger.Infof("Testing %v", name)
out, err := helm.exec(append([]string{"test", name}, flags...)...)
helm.write(out)
return err
}
@@ -93,12 +193,12 @@ func (helm *execer) exec(args ...string) ([]byte, error) {
if helm.kubeContext != "" {
cmdargs = append(cmdargs, "--kube-context", helm.kubeContext)
}
helm.write([]byte(fmt.Sprintf("exec: helm %s\n", strings.Join(cmdargs, " "))))
return helm.runner.Execute(command, cmdargs)
helm.logger.Debugf("exec: %s %s", helm.helmBinary, strings.Join(cmdargs, " "))
return helm.runner.Execute(helm.helmBinary, cmdargs)
}
func (helm *execer) write(out []byte) {
if helm.writer != nil {
helm.writer.Write(out)
if len(out) > 0 {
helm.logger.Infof("%s", out)
}
}
+182 -32
View File
@@ -2,9 +2,11 @@ package helmexec
import (
"bytes"
"io"
"os"
"reflect"
"testing"
"go.uber.org/zap"
)
// Mocking the command-line runner
@@ -18,8 +20,8 @@ func (mock *mockRunner) Execute(cmd string, args []string) ([]byte, error) {
return []byte{}, nil
}
func MockExecer(writer io.Writer, kubeContext string) *execer {
execer := New(writer, kubeContext)
func MockExecer(logger *zap.SugaredLogger, kubeContext string) *execer {
execer := New(logger, kubeContext)
execer.runner = &mockRunner{}
return execer
}
@@ -28,7 +30,8 @@ func MockExecer(writer io.Writer, kubeContext string) *execer {
func TestNewHelmExec(t *testing.T) {
buffer := bytes.NewBufferString("something")
helm := New(buffer, "dev")
logger := NewLogger(buffer, "debug")
helm := New(logger, "dev")
if helm.kubeContext != "dev" {
t.Error("helmexec.New() - kubeContext")
}
@@ -41,7 +44,7 @@ func TestNewHelmExec(t *testing.T) {
}
func Test_SetExtraArgs(t *testing.T) {
helm := New(new(bytes.Buffer), "dev")
helm := New(NewLogger(os.Stdout, "info"), "dev")
helm.SetExtraArgs()
if len(helm.extra) != 0 {
t.Error("helmexec.SetExtraArgs() - passing no arguments should not change extra field")
@@ -56,18 +59,43 @@ func Test_SetExtraArgs(t *testing.T) {
}
}
func Test_SetHelmBinary(t *testing.T) {
helm := New(NewLogger(os.Stdout, "info"), "dev")
if helm.helmBinary != "helm" {
t.Error("helmexec.command - default command is not helm")
}
helm.SetHelmBinary("foo")
if helm.helmBinary != "foo" {
t.Errorf("helmexec.SetHelmBinary() - actual = %s expect = foo", helm.helmBinary)
}
}
func Test_AddRepo(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
helm.AddRepo("myRepo", "https://repo.example.com/", "cert.pem", "key.pem")
expected := "exec: helm repo add myRepo https://repo.example.com/ --cert-file cert.pem --key-file key.pem --kube-context dev\n"
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.AddRepo("myRepo", "https://repo.example.com/", "cert.pem", "key.pem", "", "")
expected := `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --cert-file cert.pem --key-file key.pem --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.AddRepo("myRepo", "https://repo.example.com/", "", "")
expected = "exec: helm repo add myRepo https://repo.example.com/ --kube-context dev\n"
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "", "")
expected = `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "example_user", "example_password")
expected = `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
@@ -75,9 +103,12 @@ func Test_AddRepo(t *testing.T) {
func Test_UpdateRepo(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.UpdateRepo()
expected := "exec: helm repo update --kube-context dev\n"
expected := `Updating repo
exec: helm repo update --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.UpdateRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
@@ -85,34 +116,44 @@ func Test_UpdateRepo(t *testing.T) {
func Test_SyncRelease(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.SyncRelease("release", "chart", "--timeout 10", "--wait")
expected := "exec: helm upgrade --install release chart --timeout 10 --wait --kube-context dev\n"
expected := `Upgrading chart
exec: helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SyncRelease("release", "chart")
expected = "exec: helm upgrade --install release chart --kube-context dev\n"
expected = `Upgrading chart
exec: helm upgrade --install --reset-values release chart --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_UpdateDeps(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.UpdateDeps("./chart/foo")
expected := "exec: helm dependency update ./chart/foo --kube-context dev\n"
expected := `Updating dependency ./chart/foo
exec: helm dependency update ./chart/foo --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
t.Errorf("helmexec.UpdateDeps()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.SetExtraArgs("--verify")
helm.UpdateDeps("./chart/foo")
expected = "exec: helm dependency update ./chart/foo --verify --kube-context dev\n"
expected = `Updating dependency ./chart/foo
exec: helm dependency update ./chart/foo --verify --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
@@ -120,9 +161,12 @@ func Test_UpdateDeps(t *testing.T) {
func Test_DecryptSecret(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DecryptSecret("secretName")
expected := "exec: helm secrets dec secretName --kube-context dev\n"
expected := `Decrypting secret secretName
exec: helm secrets dec secretName --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.DecryptSecret()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
@@ -130,16 +174,21 @@ func Test_DecryptSecret(t *testing.T) {
func Test_DiffRelease(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DiffRelease("release", "chart", "--timeout 10", "--wait")
expected := "exec: helm diff release chart --timeout 10 --wait --kube-context dev\n"
expected := `Comparing release chart
exec: helm diff upgrade --allow-unreleased release chart --timeout 10 --wait --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm.DiffRelease("release", "chart")
expected = "exec: helm diff release chart --kube-context dev\n"
expected = `Comparing release chart
exec: helm diff upgrade --allow-unreleased release chart --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
@@ -147,19 +196,62 @@ func Test_DiffRelease(t *testing.T) {
func Test_DeleteRelease(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DeleteRelease("release")
expected := "exec: helm delete --purge release --kube-context dev\n"
expected := `Deleting release
exec: helm delete release --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.DeleteRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_DeleteRelease_Flags(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.DeleteRelease("release", "--purge")
expected := `Deleting release
exec: helm delete release --purge --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.DeleteRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_TestRelease(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.TestRelease("release")
expected := `Testing release
exec: helm test release --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.TestRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_TestRelease_Flags(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.TestRelease("release", "--cleanup", "--timeout", "60")
expected := `Testing release
exec: helm test release --cleanup --timeout 60 --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.TestRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_ReleaseStatus(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "dev")
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.ReleaseStatus("myRelease")
expected := "exec: helm status myRelease --kube-context dev\n"
expected := `Getting status myRelease
exec: helm status myRelease --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.ReleaseStatus()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
@@ -167,21 +259,22 @@ func Test_ReleaseStatus(t *testing.T) {
func Test_exec(t *testing.T) {
var buffer bytes.Buffer
helm := MockExecer(&buffer, "")
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "")
helm.exec("version")
expected := "exec: helm version\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
helm = MockExecer(nil, "dev")
helm = MockExecer(logger, "dev")
ret, _ := helm.exec("diff")
if len(ret) != 0 {
t.Error("helmexec.exec() - expected empty return value")
}
buffer.Reset()
helm = MockExecer(&buffer, "dev")
helm = MockExecer(logger, "dev")
helm.exec("diff", "release", "chart", "--timeout 10", "--wait")
expected = "exec: helm diff release chart --timeout 10 --wait --kube-context dev\n"
if buffer.String() != expected {
@@ -202,4 +295,61 @@ func Test_exec(t *testing.T) {
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
buffer.Reset()
helm = MockExecer(logger, "")
helm.SetHelmBinary("overwritten")
helm.exec("version")
expected = "exec: overwritten version\n"
if buffer.String() != expected {
t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_Lint(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.Lint("path/to/chart", "--values", "file.yml")
expected := `Linting path/to/chart
exec: helm lint path/to/chart --values file.yml --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.Lint()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
func Test_Fetch(t *testing.T) {
var buffer bytes.Buffer
logger := NewLogger(&buffer, "debug")
helm := MockExecer(logger, "dev")
helm.Fetch("chart", "--version", "1.2.3", "--untar", "--untardir", "/tmp/dir")
expected := `Fetching chart
exec: helm fetch chart --version 1.2.3 --untar --untardir /tmp/dir --kube-context dev
`
if buffer.String() != expected {
t.Errorf("helmexec.Lint()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
var logLevelTests = map[string]string{
"debug": `Adding repo myRepo https://repo.example.com/
exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password
`,
"info": `Adding repo myRepo https://repo.example.com/
`,
"warn": ``,
}
func Test_LogLevels(t *testing.T) {
var buffer bytes.Buffer
for logLevel, expected := range logLevelTests {
buffer.Reset()
logger := NewLogger(&buffer, logLevel)
helm := MockExecer(logger, "")
helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "example_user", "example_password")
if buffer.String() != expected {
t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
}
}
}
+7 -2
View File
@@ -3,14 +3,19 @@ package helmexec
// Interface for executing helm commands
type Interface interface {
SetExtraArgs(args ...string)
SetHelmBinary(bin string)
AddRepo(name, repository, certfile, keyfile string) error
AddRepo(name, repository, certfile, keyfile, username, password string) error
UpdateRepo() error
UpdateDeps(chart string) error
SyncRelease(name, chart string, flags ...string) error
DiffRelease(name, chart string, flags ...string) error
TemplateRelease(chart string, flags ...string) error
Fetch(chart string, flags ...string) error
Lint(chart string, flags ...string) error
ReleaseStatus(name string) error
DeleteRelease(name string) error
DeleteRelease(name string, flags ...string) error
TestRelease(name string, flags ...string) error
DecryptSecret(name string) (string, error)
}
+4 -10
View File
@@ -1,8 +1,6 @@
package helmexec
import (
"io/ioutil"
"os"
"os/exec"
)
@@ -17,17 +15,13 @@ type Runner interface {
}
// ShellRunner implemention for shell commands
type ShellRunner struct{}
type ShellRunner struct {
Dir string
}
// Execute a shell command
func (shell ShellRunner) Execute(cmd string, args []string) ([]byte, error) {
dir, err := ioutil.TempDir("", tmpPrefix+cmd+tmpSuffix)
if err != nil {
return nil, err
}
defer os.RemoveAll(dir)
preparedCmd := exec.Command(cmd, args...)
preparedCmd.Dir = dir
preparedCmd.Dir = shell.Dir
return preparedCmd.CombinedOutput()
}
+911 -136
View File
File diff suppressed because it is too large Load Diff
+256
View File
@@ -0,0 +1,256 @@
package main
import (
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/roboll/helmfile/helmexec"
"github.com/roboll/helmfile/state"
"gopkg.in/yaml.v2"
)
func makeRenderer(readFile func(string) ([]byte, error), env string) *twoPassRenderer {
return &twoPassRenderer{
reader: readFile,
env: env,
namespace: "namespace",
filename: "",
logger: helmexec.NewLogger(os.Stdout, "debug"),
abs: filepath.Abs,
}
}
func TestReadFromYaml_MakeEnvironmentHasNoSideEffects(t *testing.T) {
yamlContent := []byte(`
environments:
staging:
values:
- default/values.yaml
production:
releases:
- name: {{ readFile "other/default/values.yaml" }}
chart: mychart1
`)
fileReaderCalls := 0
// make a reader that returns a simulated context
fileReader := func(filename string) ([]byte, error) {
expectedFilename := filepath.Clean("default/values.yaml")
if !strings.HasSuffix(filename, expectedFilename) {
return nil, fmt.Errorf("unexpected filename: expected=%s, actual=%s", expectedFilename, filename)
}
fileReaderCalls++
if fileReaderCalls == 2 {
return []byte("SecondPass"), nil
}
return []byte(""), nil
}
r := makeRenderer(fileReader, "staging")
yamlBuf, err := r.renderTemplate(yamlContent)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
var state state.HelmState
err = yaml.Unmarshal(yamlBuf.Bytes(), &state)
if fileReaderCalls > 2 {
t.Error("reader should be called only twice")
}
if state.Releases[0].Name != "SecondPass" {
t.Errorf("release name should have ben set as SecondPass")
}
}
func TestReadFromYaml_RenderTemplate(t *testing.T) {
defaultValuesYaml := []byte(`
releaseName: "hello"
conditionalReleaseTag: "yes"
`)
yamlContent := []byte(`
environments:
staging:
values:
- default/values.yaml
production:
releases:
- name: {{ .Environment.Values.releaseName }}
chart: mychart1
{{ if (eq .Environment.Values.conditionalReleaseTag "yes") }}
- name: conditionalRelease
{{ end }}
`)
// make a reader that returns a simulated context
fileReader := func(filename string) ([]byte, error) {
expectedFilename := filepath.Clean("default/values.yaml")
if !strings.HasSuffix(filename, expectedFilename) {
return nil, fmt.Errorf("unexpected filename: expected=%s, actual=%s", expectedFilename, filename)
}
return defaultValuesYaml, nil
}
r := makeRenderer(fileReader, "staging")
// test the double rendering
yamlBuf, err := r.renderTemplate(yamlContent)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
var state state.HelmState
err = yaml.Unmarshal(yamlBuf.Bytes(), &state)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if len(state.Releases) != 2 {
t.Fatal("there should be 2 releases")
}
if state.Releases[0].Name != "hello" {
t.Errorf("release name should be hello")
}
if state.Releases[1].Name != "conditionalRelease" {
t.Error("conditional release should have been present")
}
}
func TestReadFromYaml_RenderTemplateWithValuesReferenceError(t *testing.T) {
defaultValuesYaml := []byte("")
yamlContent := []byte(`
environments:
staging:
values:
- default/values.yaml
production:
{{ if (eq .Environment.Values.releaseName "a") }} # line 8
releases:
- name: a
chart: mychart1
{{ end }}
`)
// make a reader that returns a simulated context
fileReader := func(filename string) ([]byte, error) {
return defaultValuesYaml, nil
}
r := makeRenderer(fileReader, "staging")
// test the double rendering
_, err := r.renderTemplate(yamlContent)
if !strings.Contains(err.Error(), "stringTemplate:8") {
t.Fatalf("error should contain a stringTemplate error (reference to unknow key) %v", err)
}
}
// This test shows that a gotmpl reference will get rendered correctly
// even if the pre-render disables the readFile and exec functions.
// This does not apply to .gotmpl files, which is a nice side-effect.
func TestReadFromYaml_RenderTemplateWithGotmpl(t *testing.T) {
defaultValuesYamlGotmpl := []byte(`
releaseName: {{ readFile "nonIgnoredFile" }}
`)
yamlContent := []byte(`
environments:
staging:
values:
- values.yaml.gotmpl
production:
{{ if (eq .Environment.Values.releaseName "release-a") }} # line 8
releases:
- name: a
chart: mychart1
{{ end }}
`)
fileReader := func(filename string) ([]byte, error) {
if strings.HasSuffix(filename, "nonIgnoredFile") {
return []byte("release-a"), nil
}
return defaultValuesYamlGotmpl, nil
}
r := makeRenderer(fileReader, "staging")
rendered, _ := r.renderTemplate(yamlContent)
var state state.HelmState
yaml.Unmarshal(rendered.Bytes(), &state)
if len(state.Releases) != 1 {
t.Fatal("there should be 1 release")
}
if state.Releases[0].Name != "a" {
t.Fatal("release should have been declared")
}
}
func TestReadFromYaml_RenderTemplateWithNamespace(t *testing.T) {
defaultValuesYaml := []byte(``)
yamlContent := []byte(`releases:
- name: {{ .Namespace }}-myrelease
chart: mychart
`)
// make a reader that returns a simulated context
fileReader := func(filename string) ([]byte, error) {
return defaultValuesYaml, nil
}
r := makeRenderer(fileReader, "staging")
yamlBuf, err := r.renderTemplate(yamlContent)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
var state state.HelmState
err = yaml.Unmarshal(yamlBuf.Bytes(), &state)
if state.Releases[0].Name != "namespace-myrelease" {
t.Errorf("release name should be namespace-myrelease")
}
}
func TestReadFromYaml_HelfileShouldBeResilentToTemplateErrors(t *testing.T) {
yamlContent := []byte(`environments:
staging:
production:
releases:
{{ if (eq .Environment.Name "production" }} # notice syntax error: unclosed left paren
- name: prod-myrelease
{{ else }}
- name: myapp
{{ end }}
chart: mychart
`)
fileReader := func(filename string) ([]byte, error) {
return yamlContent, nil
}
r := makeRenderer(fileReader, "staging")
_, err := r.renderTemplate(yamlContent)
if err == nil {
t.Fatalf("wanted error, none returned")
}
}
+172
View File
@@ -0,0 +1,172 @@
package state
import (
"bytes"
"fmt"
"github.com/imdario/mergo"
"github.com/roboll/helmfile/environment"
"github.com/roboll/helmfile/helmexec"
"github.com/roboll/helmfile/tmpl"
"go.uber.org/zap"
"gopkg.in/yaml.v2"
"io"
"io/ioutil"
"os"
"path/filepath"
)
type StateLoadError struct {
msg string
Cause error
}
func (e *StateLoadError) Error() string {
return fmt.Sprintf("%s: %v", e.msg, e.Cause)
}
type UndefinedEnvError struct {
msg string
}
func (e *UndefinedEnvError) Error() string {
return e.msg
}
func createFromYaml(content []byte, file string, env string, logger *zap.SugaredLogger) (*HelmState, error) {
c := &creator{
logger,
ioutil.ReadFile,
filepath.Abs,
true,
}
return c.CreateFromYaml(content, file, env)
}
type creator struct {
logger *zap.SugaredLogger
readFile func(string) ([]byte, error)
abs func(string) (string, error)
Strict bool
}
func NewCreator(logger *zap.SugaredLogger, readFile func(string) ([]byte, error), abs func(string) (string, error)) *creator {
return &creator{
logger: logger,
readFile: readFile,
abs: abs,
Strict: true,
}
}
func (c *creator) CreateFromYaml(content []byte, file string, env string) (*HelmState, error) {
var state HelmState
basePath, err := c.abs(filepath.Dir(file))
if err != nil {
return nil, &StateLoadError{fmt.Sprintf("failed to read %s", file), err}
}
state.FilePath = file
state.basePath = basePath
decoder := yaml.NewDecoder(bytes.NewReader(content))
if !c.Strict {
decoder.SetStrict(false)
} else {
decoder.SetStrict(true)
}
i := 0
for {
i++
var intermediate HelmState
err := decoder.Decode(&intermediate)
if err == io.EOF {
break
} else if err != nil {
return nil, &StateLoadError{fmt.Sprintf("failed to read %s: reading document at index %d", file, i), err}
}
if err := mergo.Merge(&state, &intermediate, mergo.WithAppendSlice); err != nil {
return nil, &StateLoadError{fmt.Sprintf("failed to read %s: merging document at index %d", file, i), err}
}
}
if len(state.DeprecatedReleases) > 0 {
if len(state.Releases) > 0 {
return nil, fmt.Errorf("failed to parse %s: you can't specify both `charts` and `releases` sections", file)
}
state.Releases = state.DeprecatedReleases
state.DeprecatedReleases = []ReleaseSpec{}
}
if state.DeprecatedContext != "" && state.HelmDefaults.KubeContext == "" {
state.HelmDefaults.KubeContext = state.DeprecatedContext
}
state.logger = c.logger
e, err := state.loadEnv(env, c.readFile)
if err != nil {
return nil, &StateLoadError{fmt.Sprintf("failed to read %s", file), err}
}
state.Env = *e
state.readFile = c.readFile
return &state, nil
}
func (st *HelmState) loadEnv(name string, readFile func(string) ([]byte, error)) (*environment.Environment, error) {
envVals := map[string]interface{}{}
envSpec, ok := st.Environments[name]
if ok {
for _, envvalFile := range envSpec.Values {
envvalFullPath := filepath.Join(st.basePath, envvalFile)
tmplData := EnvironmentTemplateData{Environment: environment.EmptyEnvironment, Namespace: ""}
r := tmpl.NewFileRenderer(readFile, filepath.Dir(envvalFullPath), tmplData)
bytes, err := r.RenderToBytes(envvalFullPath)
if err != nil {
return nil, fmt.Errorf("failed to load environment values file \"%s\": %v", envvalFile, err)
}
m := map[string]interface{}{}
if err := yaml.Unmarshal(bytes, &m); err != nil {
return nil, fmt.Errorf("failed to load environment values file \"%s\": %v", envvalFile, err)
}
if err := mergo.Merge(&envVals, &m, mergo.WithOverride); err != nil {
return nil, fmt.Errorf("failed to load \"%s\": %v", envvalFile, err)
}
}
if len(envSpec.Secrets) > 0 {
helm := helmexec.New(st.logger, "")
for _, secFile := range envSpec.Secrets {
path := filepath.Join(st.basePath, secFile)
if _, err := os.Stat(path); os.IsNotExist(err) {
return nil, err
}
decFile, err := helm.DecryptSecret(path)
if err != nil {
return nil, err
}
bytes, err := readFile(decFile)
if err != nil {
return nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", secFile, err)
}
m := map[string]interface{}{}
if err := yaml.Unmarshal(bytes, &m); err != nil {
return nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", secFile, err)
}
if err := mergo.Merge(&envVals, &m, mergo.WithOverride); err != nil {
return nil, fmt.Errorf("failed to load \"%s\": %v", secFile, err)
}
}
}
} else if name != DefaultEnv {
return nil, &UndefinedEnvError{msg: fmt.Sprintf("environment \"%s\" is not defined", name)}
}
return &environment.Environment{Name: name, Values: envVals}, nil
}
+249
View File
@@ -0,0 +1,249 @@
package state
import (
"fmt"
"path/filepath"
"reflect"
"testing"
)
func TestReadFromYaml(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease
namespace: mynamespace
chart: mychart
`)
state, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err != nil {
t.Errorf("unxpected error: %v", err)
}
if state.Releases[0].Name != "myrelease" {
t.Errorf("unexpected release name: expected=myrelease actual=%s", state.Releases[0].Name)
}
if state.Releases[0].Namespace != "mynamespace" {
t.Errorf("unexpected chart namespace: expected=mynamespace actual=%s", state.Releases[0].Chart)
}
if state.Releases[0].Chart != "mychart" {
t.Errorf("unexpected chart name: expected=mychart actual=%s", state.Releases[0].Chart)
}
}
func TestReadFromYaml_InexistentEnv(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease
namespace: mynamespace
chart: mychart
`)
_, err := createFromYaml(yamlContent, yamlFile, "production", logger)
if err == nil {
t.Error("expected error")
}
}
func TestReadFromYaml_NonDefaultEnv(t *testing.T) {
yamlFile := "/example/path/to/helmfile.yaml"
yamlContent := []byte(`environments:
production:
values:
- foo.yaml
- bar.yaml.gotmpl
releases:
- name: myrelease
namespace: mynamespace
chart: mychart
values:
- values.yaml.gotmpl
`)
fooYamlFile := "/example/path/to/foo.yaml"
fooYamlContent := []byte(`foo: foo
# As this file doesn't have an file extension ".gotmpl", this template expression should not be evaluated
baz: "{{ readFile \"baz.txt\" }}"`)
barYamlFile := "/example/path/to/bar.yaml.gotmpl"
barYamlContent := []byte(`foo: FOO
bar: {{ readFile "bar.txt" }}
`)
barTextFile := "/example/path/to/bar.txt"
barTextContent := []byte("BAR")
expected := map[string]interface{}{
"foo": "FOO",
"bar": "BAR",
// As the file doesn't have an file extension ".gotmpl", this template expression should not be evaluated
"baz": "{{ readFile \"baz.txt\" }}",
}
valuesFile := "/example/path/to/values.yaml.gotmpl"
valuesContent := []byte(`env: {{ .Environment.Name }}`)
expectedValues := `env: production`
readFile := func(filename string) ([]byte, error) {
switch filename {
case fooYamlFile:
return fooYamlContent, nil
case barYamlFile:
return barYamlContent, nil
case barTextFile:
return barTextContent, nil
case valuesFile:
return valuesContent, nil
}
return nil, fmt.Errorf("unexpected filename: %s", filename)
}
state, err := NewCreator(logger, readFile, filepath.Abs).CreateFromYaml(yamlContent, yamlFile, "production")
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := state.Env.Values
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected environment values: expected=%v, actual=%v", expected, actual)
}
actualValuesData, err := state.RenderValuesFileToBytes(valuesFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actualValues := string(actualValuesData)
if !reflect.DeepEqual(expectedValues, actualValues) {
t.Errorf("unexpected values: expected=%v, actual=%v", expectedValues, actualValues)
}
}
func TestReadFromYaml_StrictUnmarshalling(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease
namespace: mynamespace
releases: mychart
`)
_, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err == nil {
t.Error("expected an error for wrong key 'releases' which is not in struct")
}
}
func TestReadFromYaml_DeprecatedReleaseReferences(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`charts:
- name: myrelease
chart: mychart
`)
state, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err != nil {
t.Errorf("unxpected error: %v", err)
}
if state.Releases[0].Name != "myrelease" {
t.Errorf("unexpected release name: expected=myrelease actual=%s", state.Releases[0].Name)
}
if state.Releases[0].Chart != "mychart" {
t.Errorf("unexpected chart name: expected=mychart actual=%s", state.Releases[0].Chart)
}
}
func TestReadFromYaml_ConflictingReleasesConfig(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`charts:
- name: myrelease1
chart: mychart1
releases:
- name: myrelease2
chart: mychart2
`)
_, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err == nil {
t.Error("expected error")
}
}
func TestReadFromYaml_FilterReleasesOnLabels(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease1
chart: mychart1
labels:
tier: frontend
foo: bar
- name: myrelease2
chart: mychart2
labels:
tier: frontend
- name: myrelease3
chart: mychart3
labels:
tier: backend
`)
cases := []struct {
filter LabelFilter
results []bool
}{
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}}},
[]bool{true, true, false}},
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}, []string{"foo", "bar"}}},
[]bool{true, false, false}},
{LabelFilter{negativeLabels: [][]string{[]string{"tier", "frontend"}}},
[]bool{false, false, true}},
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}}, negativeLabels: [][]string{[]string{"foo", "bar"}}},
[]bool{false, true, false}},
}
state, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
for idx, c := range cases {
for idx2, expected := range c.results {
if f := c.filter.Match(state.Releases[idx2]); f != expected {
t.Errorf("[case: %d][outcome: %d] Unexpected outcome wanted %t, got %t", idx, idx2, expected, f)
}
}
}
}
func TestReadFromYaml_FilterNegatives(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease1
chart: mychart1
labels:
stage: pre
foo: bar
- name: myrelease2
chart: mychart2
labels:
stage: post
- name: myrelease3
chart: mychart3
`)
cases := []struct {
filter LabelFilter
results []bool
}{
{LabelFilter{positiveLabels: [][]string{[]string{"stage", "pre"}}},
[]bool{true, false, false}},
{LabelFilter{positiveLabels: [][]string{[]string{"stage", "post"}}},
[]bool{false, true, false}},
{LabelFilter{negativeLabels: [][]string{[]string{"stage", "pre"}, []string{"stage", "post"}}},
[]bool{false, false, true}},
}
state, err := createFromYaml(yamlContent, yamlFile, DefaultEnv, logger)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
for idx, c := range cases {
for idx2, expected := range c.results {
if f := c.filter.Match(state.Releases[idx2]); f != expected {
t.Errorf("[case: %d][outcome: %d] Unexpected outcome wanted %t, got %t", idx, idx2, expected, f)
}
}
}
}
+6
View File
@@ -0,0 +1,6 @@
package state
type EnvironmentSpec struct {
Values []string `yaml:"values"`
Secrets []string `yaml:"secrets"`
}
+69
View File
@@ -0,0 +1,69 @@
package state
import (
"fmt"
"github.com/roboll/helmfile/tmpl"
"gopkg.in/yaml.v2"
)
func (r ReleaseSpec) ExecuteTemplateExpressions(renderer *tmpl.FileRenderer) (*ReleaseSpec, error) {
var result *ReleaseSpec
var err error
result, err = r.Clone()
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\": %v", r.Name, err)
}
{
ts := result.Chart
result.Chart, err = renderer.RenderTemplateContentToString([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".chart = \"%s\": %v", r.Name, ts, err)
}
}
{
ts := result.Namespace
result.Namespace, err = renderer.RenderTemplateContentToString([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".namespace = \"%s\": %v", r.Name, ts, err)
}
}
for i, t := range result.Values {
switch ts := t.(type) {
case string:
s, err := renderer.RenderTemplateContentToBuffer([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".values[%d] = \"%s\": %v", r.Name, i, ts, err)
}
result.Values[i] = s.String()
}
}
for i, ts := range result.Secrets {
s, err := renderer.RenderTemplateContentToBuffer([]byte(ts))
if err != nil {
return nil, fmt.Errorf("failed executing template expressions in release \"%s\".secrets[%d] = \"%s\": %v", r.Name, i, ts, err)
}
result.Secrets[i] = s.String()
}
return result, nil
}
func (r ReleaseSpec) Clone() (*ReleaseSpec, error) {
serialized, err := yaml.Marshal(r)
if err != nil {
return nil, fmt.Errorf("failed cloning release \"%s\": %v", r.Name, err)
}
var deserialized ReleaseSpec
if err := yaml.Unmarshal(serialized, &deserialized); err != nil {
return nil, fmt.Errorf("failed cloning release \"%s\": %v", r.Name, err)
}
return &deserialized, nil
}
+1 -1
View File
@@ -62,7 +62,7 @@ func ParseLabels(l string) (LabelFilter, error) {
kv := strings.Split(label, "=")
lf.positiveLabels = append(lf.positiveLabels, kv)
} else { // malformed case
err = fmt.Errorf("Malformed label: %s. Expected label in form k=v or k!=v", label)
return lf, fmt.Errorf("Malformed label: %s. Expected label in form k=v or k!=v", label)
}
}
return lf, err
+824 -258
View File
File diff suppressed because it is too large Load Diff
+32
View File
@@ -0,0 +1,32 @@
package state
import (
"fmt"
"github.com/roboll/helmfile/tmpl"
)
func (st *HelmState) envTemplateData() EnvironmentTemplateData {
return EnvironmentTemplateData{
st.Env,
st.Namespace,
}
}
func (st *HelmState) ExecuteTemplates() (*HelmState, error) {
r := *st
for i, rt := range st.Releases {
tmplData := ReleaseTemplateData{
Environment: st.Env,
Release: rt,
}
renderer := tmpl.NewFileRenderer(st.readFile, st.basePath, tmplData)
r, err := rt.ExecuteTemplateExpressions(renderer)
if err != nil {
return nil, fmt.Errorf("failed executing templates in release \"%s\".\"%s\": %v", st.FilePath, rt.Name, err)
}
st.Releases[i] = *r
}
return &r, nil
}
+76
View File
@@ -0,0 +1,76 @@
package state
import (
"reflect"
"testing"
"github.com/roboll/helmfile/environment"
)
func TestHelmState_executeTemplates(t *testing.T) {
tests := []struct {
name string
input ReleaseSpec
want ReleaseSpec
}{
{
name: "Has template expressions in chart, values, and secrets",
input: ReleaseSpec{
Chart: "test-charts/{{ .Release.Name }}",
Version: "0.1",
Verify: nil,
Name: "test-app",
Namespace: "test-namespace-{{ .Release.Name }}",
Values: []interface{}{"config/{{ .Environment.Name }}/{{ .Release.Name }}/values.yaml"},
Secrets: []string{"config/{{ .Environment.Name }}/{{ .Release.Name }}/secrets.yaml"},
},
want: ReleaseSpec{
Chart: "test-charts/test-app",
Version: "0.1",
Verify: nil,
Name: "test-app",
Namespace: "test-namespace-test-app",
Values: []interface{}{"config/test_env/test-app/values.yaml"},
Secrets: []string{"config/test_env/test-app/secrets.yaml"},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
state := &HelmState{
basePath: ".",
HelmDefaults: HelmSpec{
KubeContext: "test_context",
},
Env: environment.Environment{Name: "test_env"},
Namespace: "test-namespace_",
Repositories: nil,
Releases: []ReleaseSpec{
tt.input,
},
}
r, err := state.ExecuteTemplates()
if err != nil {
t.Errorf("Unexpected error: %v", err)
t.FailNow()
}
actual := r.Releases[0]
if !reflect.DeepEqual(actual.Chart, tt.want.Chart) {
t.Errorf("expected %+v, got %+v", tt.want.Chart, actual.Chart)
}
if !reflect.DeepEqual(actual.Namespace, tt.want.Namespace) {
t.Errorf("expected %+v, got %+v", tt.want.Namespace, actual.Namespace)
}
if !reflect.DeepEqual(actual.Values, tt.want.Values) {
t.Errorf("expected %+v, got %+v", tt.want.Values, actual.Values)
}
if !reflect.DeepEqual(actual.Secrets, tt.want.Secrets) {
t.Errorf("expected %+v, got %+v", tt.want.Secrets, actual.Secrets)
}
})
}
}
+88
View File
@@ -0,0 +1,88 @@
package state
import (
"fmt"
"github.com/roboll/helmfile/helmexec"
"sync"
)
type result struct {
release ReleaseSpec
err error
}
func (st *HelmState) scatterGather(concurrency int, items int, produceInputs func(), receiveInputsAndProduceIntermediates func(int), aggregateIntermediates func()) {
numReleases := len(st.Releases)
if concurrency < 1 {
concurrency = numReleases
} else if concurrency > numReleases {
concurrency = numReleases
}
// WaitGroup is required to wait until goroutine per job in job queue cleanly stops.
var waitGroup sync.WaitGroup
waitGroup.Add(concurrency)
go produceInputs()
for w := 1; w <= concurrency; w++ {
go func(id int) {
st.logger.Debugf("worker %d/%d started", id, concurrency)
receiveInputsAndProduceIntermediates(id)
st.logger.Debugf("worker %d/%d finished", id, concurrency)
waitGroup.Done()
}(w)
}
aggregateIntermediates()
// Wait until all the goroutines to gracefully finish
waitGroup.Wait()
}
func (st *HelmState) scatterGatherReleases(helm helmexec.Interface, concurrency int, do func(ReleaseSpec) error) []error {
var errs []error
inputs := st.Releases
inputsSize := len(inputs)
releases := make(chan ReleaseSpec)
results := make(chan result)
st.scatterGather(
concurrency,
inputsSize,
func() {
for _, release := range inputs {
releases <- release
}
close(releases)
},
func(id int) {
for release := range releases {
err := do(release)
st.logger.Debugf("sending result for release: %s\n", release.Name)
results <- result{release: release, err: err}
st.logger.Debugf("sent result for release: %s\n", release.Name)
}
},
func() {
for i := range inputs {
st.logger.Debugf("receiving result %d", i)
r := <-results
if r.err != nil {
errs = append(errs, fmt.Errorf("release \"%s\" failed: %v", r.release.Name, r.err))
} else {
st.logger.Debugf("received result for release \"%s\"", r.release.Name)
}
st.logger.Debugf("received result for %d", i)
}
},
)
if len(errs) != 0 {
return errs
}
return nil
}
+440 -243
View File
@@ -7,159 +7,11 @@ import (
"errors"
"strings"
"github.com/roboll/helmfile/helmexec"
)
func TestReadFromYaml(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease
namespace: mynamespace
chart: mychart
`)
state, err := readFromYaml(yamlContent, yamlFile)
if err != nil {
t.Errorf("unxpected error: %v", err)
}
if state.Releases[0].Name != "myrelease" {
t.Errorf("unexpected release name: expected=myrelease actual=%s", state.Releases[0].Name)
}
if state.Releases[0].Namespace != "mynamespace" {
t.Errorf("unexpected chart namespace: expected=mynamespace actual=%s", state.Releases[0].Chart)
}
if state.Releases[0].Chart != "mychart" {
t.Errorf("unexpected chart name: expected=mychart actual=%s", state.Releases[0].Chart)
}
}
func TestReadFromYaml_StrictUnmarshalling(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease
namespace: mynamespace
releases: mychart
`)
_, err := readFromYaml(yamlContent, yamlFile)
if err == nil {
t.Error("expected an error for wrong key 'releases' which is not in struct")
}
}
func TestReadFromYaml_DeprecatedReleaseReferences(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`charts:
- name: myrelease
chart: mychart
`)
state, err := readFromYaml(yamlContent, yamlFile)
if err != nil {
t.Errorf("unxpected error: %v", err)
}
if state.Releases[0].Name != "myrelease" {
t.Errorf("unexpected release name: expected=myrelease actual=%s", state.Releases[0].Name)
}
if state.Releases[0].Chart != "mychart" {
t.Errorf("unexpected chart name: expected=mychart actual=%s", state.Releases[0].Chart)
}
}
func TestReadFromYaml_ConflictingReleasesConfig(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`charts:
- name: myrelease1
chart: mychart1
releases:
- name: myrelease2
chart: mychart2
`)
_, err := readFromYaml(yamlContent, yamlFile)
if err == nil {
t.Error("expected error")
}
}
func TestReadFromYaml_FilterReleasesOnLabels(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease1
chart: mychart1
labels:
tier: frontend
foo: bar
- name: myrelease2
chart: mychart2
labels:
tier: frontend
- name: myrelease3
chart: mychart3
labels:
tier: backend
`)
cases := []struct {
filter LabelFilter
results []bool
}{
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}}},
[]bool{true, true, false}},
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}, []string{"foo", "bar"}}},
[]bool{true, false, false}},
{LabelFilter{negativeLabels: [][]string{[]string{"tier", "frontend"}}},
[]bool{false, false, true}},
{LabelFilter{positiveLabels: [][]string{[]string{"tier", "frontend"}}, negativeLabels: [][]string{[]string{"foo", "bar"}}},
[]bool{false, true, false}},
}
state, err := readFromYaml(yamlContent, yamlFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
for idx, c := range cases {
for idx2, expected := range c.results {
if f := c.filter.Match(state.Releases[idx2]); f != expected {
t.Errorf("[case: %d][outcome: %d] Unexpected outcome wanted %t, got %t", idx, idx2, expected, f)
}
}
}
}
func TestReadFromYaml_FilterNegatives(t *testing.T) {
yamlFile := "example/path/to/yaml/file"
yamlContent := []byte(`releases:
- name: myrelease1
chart: mychart1
labels:
stage: pre
foo: bar
- name: myrelease2
chart: mychart2
labels:
stage: post
- name: myrelease3
chart: mychart3
`)
cases := []struct {
filter LabelFilter
results []bool
}{
{LabelFilter{positiveLabels: [][]string{[]string{"stage", "pre"}}},
[]bool{true, false, false}},
{LabelFilter{positiveLabels: [][]string{[]string{"stage", "post"}}},
[]bool{false, true, false}},
{LabelFilter{negativeLabels: [][]string{[]string{"stage", "pre"}, []string{"stage", "post"}}},
[]bool{false, false, true}},
}
state, err := readFromYaml(yamlContent, yamlFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
for idx, c := range cases {
for idx2, expected := range c.results {
if f := c.filter.Match(state.Releases[idx2]); f != expected {
t.Errorf("[case: %d][outcome: %d] Unexpected outcome wanted %t, got %t", idx, idx2, expected, f)
}
}
}
}
var logger = helmexec.NewLogger(os.Stdout, "warn")
func TestLabelParsing(t *testing.T) {
cases := []struct {
@@ -185,6 +37,7 @@ func TestLabelParsing(t *testing.T) {
}
}
}
func TestHelmState_applyDefaultsTo(t *testing.T) {
type fields struct {
BaseChartPath string
@@ -197,10 +50,11 @@ func TestHelmState_applyDefaultsTo(t *testing.T) {
type args struct {
spec ReleaseSpec
}
verify := false
specWithNamespace := ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Verify: false,
Verify: &verify,
Name: "test-charts",
Namespace: "test-namespace",
Values: nil,
@@ -261,8 +115,8 @@ func TestHelmState_applyDefaultsTo(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
state := &HelmState{
BaseChartPath: tt.fields.BaseChartPath,
Context: tt.fields.Context,
basePath: tt.fields.BaseChartPath,
DeprecatedContext: tt.fields.Context,
DeprecatedReleases: tt.fields.DeprecatedReleases,
Namespace: tt.fields.Namespace,
Repositories: tt.fields.Repositories,
@@ -275,117 +129,269 @@ func TestHelmState_applyDefaultsTo(t *testing.T) {
}
}
func Test_renderTemplateString(t *testing.T) {
type args struct {
s string
envs map[string]string
func TestHelmState_flagsForUpgrade(t *testing.T) {
enable := true
disable := false
some := func(v int) *int {
return &v
}
tests := []struct {
name string
args args
want string
wantErr bool
name string
defaults HelmSpec
release *ReleaseSpec
want []string
}{
{
name: "simple replacement",
args: args{
s: "{{ env \"HF_TEST_VAR\" }}",
envs: map[string]string{
"HF_TEST_VAR": "content",
},
name: "no-options",
defaults: HelmSpec{
Verify: false,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Verify: &disable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--namespace", "test-namespace",
},
want: "content",
wantErr: false,
},
{
name: "two replacements",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" }}{{ env \"HF_TEST_BETA\" }}",
envs: map[string]string{
"HF_TEST_ALPHA": "first",
"HF_TEST_BETA": "second",
},
name: "verify",
defaults: HelmSpec{
Verify: false,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Verify: &enable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--verify",
"--namespace", "test-namespace",
},
want: "firstsecond",
wantErr: false,
},
{
name: "replacement and comment",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" }}{{/* comment */}}",
envs: map[string]string{
"HF_TEST_ALPHA": "first",
},
name: "verify-from-default",
defaults: HelmSpec{
Verify: true,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Verify: &disable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--verify",
"--namespace", "test-namespace",
},
want: "first",
wantErr: false,
},
{
name: "global template function",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" | len }}",
envs: map[string]string{
"HF_TEST_ALPHA": "abcdefg",
},
name: "force",
defaults: HelmSpec{
Force: false,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Force: &enable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--force",
"--namespace", "test-namespace",
},
want: "7",
wantErr: false,
},
{
name: "env var not set",
args: args{
s: "{{ env \"HF_TEST_NONE\" }}",
envs: map[string]string{
"HF_TEST_THIS": "first",
},
name: "force-from-default",
defaults: HelmSpec{
Force: true,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Force: &disable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--force",
"--namespace", "test-namespace",
},
want: "",
},
{
name: "undefined function",
args: args{
s: "{{ env foo }}",
envs: map[string]string{
"foo": "bar",
},
name: "recreate-pods",
defaults: HelmSpec{
RecreatePods: false,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
RecreatePods: &enable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--recreate-pods",
"--namespace", "test-namespace",
},
wantErr: true,
},
{
name: "required env var",
args: args{
s: "{{ requiredEnv \"HF_TEST\" }}",
envs: map[string]string{
"HF_TEST": "value",
},
name: "recreate-pods-from-default",
defaults: HelmSpec{
RecreatePods: true,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
RecreatePods: &disable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--recreate-pods",
"--namespace", "test-namespace",
},
want: "value",
wantErr: false,
},
{
name: "required env var not set",
args: args{
s: "{{ requiredEnv \"HF_TEST_NONE\" }}",
envs: map[string]string{},
name: "wait",
defaults: HelmSpec{
Wait: false,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Wait: &enable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--wait",
"--namespace", "test-namespace",
},
},
{
name: "devel",
defaults: HelmSpec{
Devel: true,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Wait: &enable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--devel",
"--wait",
"--namespace", "test-namespace",
},
},
{
name: "devel-release",
defaults: HelmSpec{
Devel: true,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Devel: &disable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--namespace", "test-namespace",
},
},
{
name: "wait-from-default",
defaults: HelmSpec{
Wait: true,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Wait: &disable,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--wait",
"--namespace", "test-namespace",
},
},
{
name: "timeout",
defaults: HelmSpec{
Timeout: 0,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Timeout: some(123),
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--timeout", "123",
"--namespace", "test-namespace",
},
},
{
name: "timeout-from-default",
defaults: HelmSpec{
Timeout: 123,
},
release: &ReleaseSpec{
Chart: "test/chart",
Version: "0.1",
Timeout: nil,
Name: "test-charts",
Namespace: "test-namespace",
},
want: []string{
"--version", "0.1",
"--timeout", "123",
"--namespace", "test-namespace",
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
for k, v := range tt.args.envs {
err := os.Setenv(k, v)
if err != nil {
t.Error("renderTemplateString() could not set env var for testing")
}
state := &HelmState{
basePath: "./",
DeprecatedContext: "default",
Releases: []ReleaseSpec{*tt.release},
HelmDefaults: tt.defaults,
}
got, err := renderTemplateString(tt.args.s)
if (err != nil) != tt.wantErr {
t.Errorf("renderTemplateString() for %s error = %v, wantErr %v", tt.name, err, tt.wantErr)
return
helm := helmexec.New(logger, "default")
args, err := state.flagsForUpgrade(helm, tt.release)
if err != nil {
t.Errorf("unexpected error flagsForUpgade: %v", err)
}
if got != tt.want {
t.Errorf("renderTemplateString() for %s = %v, want %v", tt.name, got, tt.want)
if !reflect.DeepEqual(args, tt.want) {
t.Errorf("flagsForUpgrade returned = %v, want %v", args, tt.want)
}
})
}
@@ -439,7 +445,7 @@ func Test_isLocalChart(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isLocalChart(tt.args.chart); got != tt.want {
t.Errorf("isLocalChart() = %v, want %v", got, tt.want)
t.Errorf("pathExists() = %v, want %v", got, tt.want)
}
})
}
@@ -521,8 +527,11 @@ func (helm *mockHelmExec) UpdateDeps(chart string) error {
func (helm *mockHelmExec) SetExtraArgs(args ...string) {
return
}
func (helm *mockHelmExec) AddRepo(name, repository, certfile, keyfile string) error {
helm.repo = []string{name, repository, certfile, keyfile}
func (helm *mockHelmExec) SetHelmBinary(bin string) {
return
}
func (helm *mockHelmExec) AddRepo(name, repository, certfile, keyfile, username, password string) error {
helm.repo = []string{name, repository, certfile, keyfile, username, password}
return nil
}
func (helm *mockHelmExec) UpdateRepo() error {
@@ -546,13 +555,28 @@ func (helm *mockHelmExec) ReleaseStatus(release string) error {
helm.releases = append(helm.releases, mockRelease{name: release, flags: []string{}})
return nil
}
func (helm *mockHelmExec) DeleteRelease(name string) error {
func (helm *mockHelmExec) DeleteRelease(name string, flags ...string) error {
return nil
}
func (helm *mockHelmExec) DecryptSecret(name string) (string, error) {
return "", nil
}
func (helm *mockHelmExec) TestRelease(name string, flags ...string) error {
if strings.Contains(name, "error") {
return errors.New("error")
}
helm.releases = append(helm.releases, mockRelease{name: name, flags: flags})
return nil
}
func (helm *mockHelmExec) Fetch(chart string, flags ...string) error {
return nil
}
func (helm *mockHelmExec) Lint(chart string, flags ...string) error {
return nil
}
func (helm *mockHelmExec) TemplateRelease(chart string, flags ...string) error {
return nil
}
func TestHelmState_SyncRepos(t *testing.T) {
tests := []struct {
name string
@@ -569,10 +593,12 @@ func TestHelmState_SyncRepos(t *testing.T) {
URL: "http://example.com/",
CertFile: "",
KeyFile: "",
Username: "",
Password: "",
},
},
helm: &mockHelmExec{},
want: []string{"name", "http://example.com/", "", ""},
want: []string{"name", "http://example.com/", "", "", "", ""},
},
{
name: "repository with cert and key",
@@ -582,10 +608,27 @@ func TestHelmState_SyncRepos(t *testing.T) {
URL: "http://example.com/",
CertFile: "certfile",
KeyFile: "keyfile",
Username: "",
Password: "",
},
},
helm: &mockHelmExec{},
want: []string{"name", "http://example.com/", "certfile", "keyfile"},
want: []string{"name", "http://example.com/", "certfile", "keyfile", "", ""},
},
{
name: "repository with username and password",
repos: []RepositorySpec{
{
Name: "name",
URL: "http://example.com/",
CertFile: "",
KeyFile: "",
Username: "example_user",
Password: "example_password",
},
},
helm: &mockHelmExec{},
want: []string{"name", "http://example.com/", "", "", "example_user", "example_password"},
},
}
for _, tt := range tests {
@@ -643,13 +686,59 @@ func TestHelmState_SyncReleases(t *testing.T) {
},
},
helm: &mockHelmExec{},
wantReleases: []mockRelease{{"releaseName", []string{"--set", "someList=a\\,b\\,c,json=\\{\"name\": \"john\"\\}"}}},
wantReleases: []mockRelease{{"releaseName", []string{"--set", "someList=a\\,b\\,c", "--set", "json=\\{\"name\": \"john\"\\}"}}},
},
{
name: "set single value from file",
releases: []ReleaseSpec{
{
Name: "releaseName",
Chart: "foo",
SetValues: []SetValue{
{
Name: "foo",
Value: "FOO",
},
{
Name: "bar",
File: "path/to/bar",
},
{
Name: "baz",
Value: "BAZ",
},
},
},
},
helm: &mockHelmExec{},
wantReleases: []mockRelease{{"releaseName", []string{"--set", "foo=FOO", "--set-file", "bar=path/to/bar", "--set", "baz=BAZ"}}},
},
{
name: "set single array value in an array",
releases: []ReleaseSpec{
{
Name: "releaseName",
Chart: "foo",
SetValues: []SetValue{
{
Name: "foo.bar[0]",
Values: []string{
"A",
"B",
},
},
},
},
},
helm: &mockHelmExec{},
wantReleases: []mockRelease{{"releaseName", []string{"--set", "foo.bar[0]={A,B}"}}},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
state := &HelmState{
Releases: tt.releases,
logger: logger,
}
if _ = state.SyncReleases(tt.helm, []string{}, 1); !reflect.DeepEqual(tt.helm.releases, tt.wantReleases) {
t.Errorf("HelmState.SyncReleases() for [%s] = %v, want %v", tt.name, tt.helm.releases, tt.wantReleases)
@@ -660,7 +749,7 @@ func TestHelmState_SyncReleases(t *testing.T) {
func TestHelmState_UpdateDeps(t *testing.T) {
state := &HelmState{
BaseChartPath: "/src",
basePath: "/src",
Releases: []ReleaseSpec{
{
Chart: "./..",
@@ -678,7 +767,7 @@ func TestHelmState_UpdateDeps(t *testing.T) {
Chart: "published/deeper",
},
{
Chart: "./error",
Chart: ".error",
},
},
}
@@ -727,6 +816,7 @@ func TestHelmState_ReleaseStatuses(t *testing.T) {
i := func(t *testing.T) {
state := &HelmState{
Releases: tt.releases,
logger: logger,
}
errs := state.ReleaseStatuses(tt.helm, 1)
if (errs != nil) != tt.wantErr {
@@ -740,3 +830,110 @@ func TestHelmState_ReleaseStatuses(t *testing.T) {
t.Run(tt.name, i)
}
}
func TestHelmState_TestReleasesNoCleanUp(t *testing.T) {
tests := []struct {
name string
cleanup bool
releases []ReleaseSpec
helm *mockHelmExec
want []mockRelease
wantErr bool
}{
{
name: "happy path",
releases: []ReleaseSpec{
{
Name: "releaseA",
},
},
helm: &mockHelmExec{},
want: []mockRelease{{"releaseA", []string{"--timeout", "1"}}},
},
{
name: "do cleanup",
cleanup: true,
releases: []ReleaseSpec{
{
Name: "releaseB",
},
},
helm: &mockHelmExec{},
want: []mockRelease{{"releaseB", []string{"--cleanup", "--timeout", "1"}}},
},
{
name: "happy path",
releases: []ReleaseSpec{
{
Name: "error",
},
},
helm: &mockHelmExec{},
wantErr: true,
},
}
for _, tt := range tests {
i := func(t *testing.T) {
state := &HelmState{
Releases: tt.releases,
logger: logger,
}
errs := state.TestReleases(tt.helm, tt.cleanup, 1, 1)
if (errs != nil) != tt.wantErr {
t.Errorf("TestReleases() for %s error = %v, wantErr %v", tt.name, errs, tt.wantErr)
return
}
if !reflect.DeepEqual(tt.helm.releases, tt.want) {
t.Errorf("HelmState.TestReleases() for [%s] = %v, want %v", tt.name, tt.helm.releases, tt.want)
}
}
t.Run(tt.name, i)
}
}
func TestHelmState_NoReleaseMatched(t *testing.T) {
releases := []ReleaseSpec{
{
Name: "releaseA",
Labels: map[string]string{
"foo": "bar",
},
},
}
tests := []struct {
name string
labels string
wantErr bool
}{
{
name: "happy path",
labels: "foo=bar",
wantErr: false,
},
{
name: "name does not exist",
labels: "name=releaseB",
wantErr: false,
},
{
name: "label does not match anything",
labels: "foo=notbar",
wantErr: false,
},
}
for _, tt := range tests {
i := func(t *testing.T) {
state := &HelmState{
Releases: releases,
logger: logger,
}
errs := state.FilterReleases([]string{tt.labels})
if (errs != nil) != tt.wantErr {
t.Errorf("ReleaseStatuses() for %s error = %v, wantErr %v", tt.name, errs, tt.wantErr)
return
}
}
t.Run(tt.name, i)
}
}
+24
View File
@@ -0,0 +1,24 @@
package state
import "github.com/roboll/helmfile/environment"
// TemplateSpec defines the structure of a reusable and composable template for helm releases.
type TemplateSpec struct {
ReleaseSpec `yaml:",inline"`
}
// EnvironmentTemplateData provides variables accessible while executing golang text/template expressions in helmfile and values YAML files
type EnvironmentTemplateData struct {
// Environment is accessible as `.Environment` from any template executed by the renderer
Environment environment.Environment
// Namespace is accessible as `.Namespace` from any non-values template executed by the renderer
Namespace string
}
// ReleaseTemplateData provides variables accessible while executing golang text/template expressions in releases of a helmfile YAML file
type ReleaseTemplateData struct {
// Environment is accessible as `.Environment` from any template expression executed by the renderer
Environment environment.Environment
// Release is accessible as `.Release` from any template expression executed by the renderer
Release ReleaseSpec
}
+2 -1
View File
@@ -2,7 +2,8 @@ repositories:
- name: stable
url: https://kubernetes-charts.storage.googleapis.com/
context: minikube
helmDefaults:
kubeContext: minikube
releases:
+4 -3
View File
@@ -36,7 +36,8 @@ function retry() {
${command} && break # substitute your command here
retry_result=$?
n=$[$n+1]
sleep 1
# approximated binary exponential backoff to reduce flakiness
sleep $((n ** 2))
done
}
@@ -57,7 +58,7 @@ test_start "happypath - simple rollout of httpbin chart"
info "Syncing ${dir}/happypath.yaml"
${helmfile} -f ${dir}/happypath.yaml sync
wait_deploy_ready httpbin-httpbin
retry 3 "curl --fail $(minikube service --url --namespace=${test_ns} httpbin-httpbin)/status/200"
retry 5 "curl --fail $(minikube service --url --namespace=${test_ns} httpbin-httpbin)/status/200"
[ ${retry_result} -eq 0 ] || fail "httpbin failed to return 200 OK"
info "Deleting release"
${helmfile} -f ${dir}/happypath.yaml delete
@@ -67,4 +68,4 @@ test_pass "happypath"
# ALL DONE -----------------------------------------------------------------------------------------------------------
all_tests_passed
all_tests_passed
+7
View File
@@ -0,0 +1,7 @@
package tmpl
type Context struct {
preRender bool
basePath string
readFile func(string) ([]byte, error)
}
+194
View File
@@ -0,0 +1,194 @@
package tmpl
import (
"fmt"
"gopkg.in/yaml.v2"
"io"
"os"
"os/exec"
"path/filepath"
"reflect"
"strings"
"text/template"
)
type Values = map[string]interface{}
func (c *Context) createFuncMap() template.FuncMap {
funcMap := template.FuncMap{
"exec": c.Exec,
"readFile": c.ReadFile,
"toYaml": ToYaml,
"fromYaml": FromYaml,
"setValueAtPath": SetValueAtPath,
"requiredEnv": RequiredEnv,
"get": get,
"getOrNil": getOrNil,
}
if c.preRender {
// disable potential side-effect template calls
funcMap["exec"] = func(string, []interface{}, ...string) (string, error) {
return "", nil
}
funcMap["readFile"] = func(string) (string, error) {
return "", nil
}
}
return funcMap
}
func (c *Context) Exec(command string, args []interface{}, inputs ...string) (string, error) {
var input string
if len(inputs) > 0 {
input = inputs[0]
}
strArgs := make([]string, len(args))
for i, a := range args {
switch a.(type) {
case string:
strArgs[i] = a.(string)
default:
return "", fmt.Errorf("unexpected type of arg \"%s\" in args %v at index %d", reflect.TypeOf(a), args, i)
}
}
cmd := exec.Command(command, strArgs...)
cmd.Dir = c.basePath
writeErrs := make(chan error)
cmdErrs := make(chan error)
cmdOuts := make(chan []byte)
if len(input) > 0 {
stdin, err := cmd.StdinPipe()
if err != nil {
return "", err
}
go func(input string, stdin io.WriteCloser) {
defer stdin.Close()
defer close(writeErrs)
size := len(input)
var n int
var err error
i := 0
for {
n, err = io.WriteString(stdin, input[i:])
if err != nil {
writeErrs <- fmt.Errorf("failed while writing %d bytes to stdin of \"%s\": %v", len(input), command, err)
break
}
i += n
if n == size {
break
}
}
}(input, stdin)
}
go func() {
defer close(cmdOuts)
defer close(cmdErrs)
bytes, err := cmd.Output()
if err != nil {
cmdErrs <- fmt.Errorf("exec cmd=%s args=[%s] failed: %v", command, strings.Join(strArgs, ", "), err)
} else {
cmdOuts <- bytes
}
}()
for {
select {
case bytes := <-cmdOuts:
return string(bytes), nil
case err := <-cmdErrs:
return "", err
case err := <-writeErrs:
return "", err
}
}
}
func (c *Context) ReadFile(filename string) (string, error) {
path := filepath.Join(c.basePath, filename)
bytes, err := c.readFile(path)
if err != nil {
return "", err
}
return string(bytes), nil
}
func ToYaml(v interface{}) (string, error) {
data, err := yaml.Marshal(v)
if err != nil {
return "", err
}
return string(data), nil
}
func FromYaml(str string) (Values, error) {
m := Values{}
if err := yaml.Unmarshal([]byte(str), &m); err != nil {
return nil, err
}
return m, nil
}
func SetValueAtPath(path string, value interface{}, values Values) (Values, error) {
var current interface{}
current = values
components := strings.Split(path, ".")
pathToMap := components[:len(components)-1]
key := components[len(components)-1]
for _, k := range pathToMap {
var elem interface{}
switch typedCurrent := current.(type) {
case map[string]interface{}:
v, exists := typedCurrent[k]
if !exists {
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" does not exist", path, k)
}
elem = v
case map[interface{}]interface{}:
v, exists := typedCurrent[k]
if !exists {
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" does not exist", path, k)
}
elem = v
default:
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" was not a map", path, k)
}
switch typedElem := elem.(type) {
case map[string]interface{}, map[interface{}]interface{}:
current = typedElem
default:
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" was not a map", path, k)
}
}
switch typedCurrent := current.(type) {
case map[string]interface{}:
typedCurrent[key] = value
case map[interface{}]interface{}:
typedCurrent[key] = value
default:
return nil, fmt.Errorf("failed to set value at path \"%s\": value for key \"%s\" was not a map", path, key)
}
return values, nil
}
func RequiredEnv(name string) (string, error) {
if val, exists := os.LookupEnv(name); exists && len(val) > 0 {
return val, nil
}
return "", fmt.Errorf("required env var `%s` is not set", name)
}
+99
View File
@@ -0,0 +1,99 @@
package tmpl
import (
"fmt"
"reflect"
"testing"
)
func TestReadFile(t *testing.T) {
expected := `foo:
bar: BAR
`
expectedFilename := "values.yaml"
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(expected), nil
}}
actual, err := ctx.ReadFile(expectedFilename)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestToYaml(t *testing.T) {
expected := `foo:
bar: BAR
`
vals := Values(map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "BAR",
},
})
actual, err := ToYaml(vals)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestFromYaml(t *testing.T) {
raw := `foo:
bar: BAR
`
expected := Values(map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "BAR",
},
})
actual, err := FromYaml(raw)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestSetValueAtPath_OneComponent(t *testing.T) {
input := map[string]interface{}{
"foo": "",
}
expected := map[string]interface{}{
"foo": "FOO",
}
actual, err := SetValueAtPath("foo", "FOO", input)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestSetValueAtPath_TwoComponents(t *testing.T) {
input := map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "",
},
}
expected := map[string]interface{}{
"foo": map[interface{}]interface{}{
"bar": "FOO_BAR",
},
}
actual, err := SetValueAtPath("foo.bar", "FOO_BAR", input)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
+41
View File
@@ -0,0 +1,41 @@
package tmpl
import (
"bytes"
"github.com/Masterminds/sprig"
"text/template"
)
func (c *Context) stringTemplate() *template.Template {
funcMap := sprig.TxtFuncMap()
for name, f := range c.createFuncMap() {
funcMap[name] = f
}
tmpl := template.New("stringTemplate").Funcs(funcMap)
if c.preRender {
tmpl.Option("missingkey=zero")
} else {
tmpl.Option("missingkey=error")
}
return tmpl
}
func (c *Context) RenderTemplateToBuffer(s string, data ...interface{}) (*bytes.Buffer, error) {
var t, parseErr = c.stringTemplate().Parse(s)
if parseErr != nil {
return nil, parseErr
}
var tplString bytes.Buffer
var d interface{}
if len(data) > 0 {
d = data[0]
}
var execErr = t.Execute(&tplString, d)
if execErr != nil {
return &tplString, execErr
}
return &tplString, nil
}
+238
View File
@@ -0,0 +1,238 @@
package tmpl
import (
"fmt"
"os"
"reflect"
"testing"
)
func TestRenderTemplate_Values(t *testing.T) {
valuesYamlContent := `foo:
bar: BAR
`
expected := `foo:
bar: FOO_BAR
`
expectedFilename := "values.yaml"
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(valuesYamlContent), nil
}}
buf, err := ctx.RenderTemplateToBuffer(`{{ readFile "values.yaml" | fromYaml | setValueAtPath "foo.bar" "FOO_BAR" | toYaml }}`)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := buf.String()
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestRenderTemplate_WithData(t *testing.T) {
valuesYamlContent := `foo:
bar: {{ .foo.bar }}
`
expected := `foo:
bar: FOO_BAR
`
expectedFilename := "values.yaml"
data := map[string]interface{}{
"foo": map[string]interface{}{
"bar": "FOO_BAR",
},
}
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(valuesYamlContent), nil
}}
buf, err := ctx.RenderTemplateToBuffer(valuesYamlContent, data)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := buf.String()
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestRenderTemplate_AccessingMissingKeyWithGetOrNil(t *testing.T) {
valuesYamlContent := `foo:
bar: {{ . | getOrNil "foo.bar" }}
`
expected := `foo:
bar: <no value>
`
expectedFilename := "values.yaml"
data := map[string]interface{}{}
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(valuesYamlContent), nil
}}
buf, err := ctx.RenderTemplateToBuffer(valuesYamlContent, data)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := buf.String()
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestRenderTemplate_Defaulting(t *testing.T) {
valuesYamlContent := `foo:
bar: {{ . | getOrNil "foo.bar" | default "DEFAULT" }}
`
expected := `foo:
bar: DEFAULT
`
expectedFilename := "values.yaml"
data := map[string]interface{}{}
ctx := &Context{readFile: func(filename string) ([]byte, error) {
if filename != expectedFilename {
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", expectedFilename, filename)
}
return []byte(valuesYamlContent), nil
}}
buf, err := ctx.RenderTemplateToBuffer(valuesYamlContent, data)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := buf.String()
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func renderTemplateToString(s string) (string, error) {
ctx := &Context{readFile: func(filename string) ([]byte, error) {
return nil, fmt.Errorf("unexpected call to readFile: filename=%s", filename)
}}
tplString, err := ctx.RenderTemplateToBuffer(s)
if err != nil {
return "", err
}
return tplString.String(), nil
}
func Test_renderTemplateToString(t *testing.T) {
type args struct {
s string
envs map[string]string
}
tests := []struct {
name string
args args
want string
wantErr bool
}{
{
name: "simple replacement",
args: args{
s: "{{ env \"HF_TEST_VAR\" }}",
envs: map[string]string{
"HF_TEST_VAR": "content",
},
},
want: "content",
wantErr: false,
},
{
name: "two replacements",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" }}{{ env \"HF_TEST_BETA\" }}",
envs: map[string]string{
"HF_TEST_ALPHA": "first",
"HF_TEST_BETA": "second",
},
},
want: "firstsecond",
wantErr: false,
},
{
name: "replacement and comment",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" }}{{/* comment */}}",
envs: map[string]string{
"HF_TEST_ALPHA": "first",
},
},
want: "first",
wantErr: false,
},
{
name: "global template function",
args: args{
s: "{{ env \"HF_TEST_ALPHA\" | len }}",
envs: map[string]string{
"HF_TEST_ALPHA": "abcdefg",
},
},
want: "7",
wantErr: false,
},
{
name: "env var not set",
args: args{
s: "{{ env \"HF_TEST_NONE\" }}",
envs: map[string]string{
"HF_TEST_THIS": "first",
},
},
want: "",
},
{
name: "undefined function",
args: args{
s: "{{ env foo }}",
envs: map[string]string{
"foo": "bar",
},
},
wantErr: true,
},
{
name: "required env var",
args: args{
s: "{{ requiredEnv \"HF_TEST\" }}",
envs: map[string]string{
"HF_TEST": "value",
},
},
want: "value",
wantErr: false,
},
{
name: "required env var not set",
args: args{
s: "{{ requiredEnv \"HF_TEST_NONE\" }}",
envs: map[string]string{},
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
for k, v := range tt.args.envs {
err := os.Setenv(k, v)
if err != nil {
t.Error("renderTemplateToString() could not set env var for testing")
}
}
got, err := renderTemplateToString(tt.args.s)
if (err != nil) != tt.wantErr {
t.Errorf("renderTemplateToString() for %s error = %v, wantErr %v", tt.name, err, tt.wantErr)
return
}
if got != tt.want {
t.Errorf("renderTemplateToString() for %s = %v, want %v", tt.name, got, tt.want)
}
})
}
}
+78
View File
@@ -0,0 +1,78 @@
package tmpl
import (
"bytes"
"io/ioutil"
"fmt"
"strings"
)
type FileRenderer struct {
ReadFile func(string) ([]byte, error)
Context *Context
Data interface{}
}
func NewFileRenderer(readFile func(filename string) ([]byte, error), basePath string, data interface{}) *FileRenderer {
return &FileRenderer{
ReadFile: readFile,
Context: &Context{
basePath: basePath,
readFile: readFile,
},
Data: data,
}
}
func NewFirstPassRenderer(basePath string, data interface{}) *FileRenderer {
return &FileRenderer{
ReadFile: ioutil.ReadFile,
Context: &Context{
preRender: true,
basePath: basePath,
readFile: ioutil.ReadFile,
},
Data: data,
}
}
func (r *FileRenderer) RenderTemplateFileToBuffer(file string) (*bytes.Buffer, error) {
content, err := r.ReadFile(file)
if err != nil {
return nil, err
}
return r.RenderTemplateContentToBuffer(content)
}
func (r *FileRenderer) RenderToBytes(path string) ([]byte, error) {
var yamlBytes []byte
splits := strings.Split(path, ".")
if len(splits) > 0 && splits[len(splits)-1] == "gotmpl" {
yamlBuf, err := r.RenderTemplateFileToBuffer(path)
if err != nil {
return nil, fmt.Errorf("failed to render [%s], because of %v", path, err)
}
yamlBytes = yamlBuf.Bytes()
} else {
var err error
yamlBytes, err = r.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("failed to load [%s]: %v", path, err)
}
}
return yamlBytes, nil
}
func (r *FileRenderer) RenderTemplateContentToBuffer(content []byte) (*bytes.Buffer, error) {
return r.Context.RenderTemplateToBuffer(string(content), r.Data)
}
func (r *FileRenderer) RenderTemplateContentToString(content []byte) (string, error) {
buf, err := r.Context.RenderTemplateToBuffer(string(content), r.Data)
if err != nil {
return "", err
}
return buf.String(), nil
}
+67
View File
@@ -0,0 +1,67 @@
package tmpl
import (
"fmt"
"github.com/roboll/helmfile/environment"
"reflect"
"testing"
)
var emptyEnvTmplData = map[string]interface{}{
"Environment": environment.EmptyEnvironment,
"Namespace": "",
}
func TestRenderToBytes_Gotmpl(t *testing.T) {
valuesYamlTmplContent := `foo:
bar: '{{ readFile "data.txt" }}'
`
dataFileContent := "FOO_BAR"
expected := `foo:
bar: 'FOO_BAR'
`
dataFile := "data.txt"
valuesTmplFile := "values.yaml.gotmpl"
r := NewFileRenderer(func(filename string) ([]byte, error) {
switch filename {
case valuesTmplFile:
return []byte(valuesYamlTmplContent), nil
case dataFile:
return []byte(dataFileContent), nil
}
return nil, fmt.Errorf("unexpected filename: expected=%v or %v, actual=%s", dataFile, valuesTmplFile, filename)
}, "", emptyEnvTmplData)
buf, err := r.RenderToBytes(valuesTmplFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := string(buf)
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
func TestRenderToBytes_Yaml(t *testing.T) {
valuesYamlContent := `foo:
bar: '{{ readFile "data.txt" }}'
`
expected := `foo:
bar: '{{ readFile "data.txt" }}'
`
valuesFile := "values.yaml"
r := NewFileRenderer(func(filename string) ([]byte, error) {
switch filename {
case valuesFile:
return []byte(valuesYamlContent), nil
}
return nil, fmt.Errorf("unexpected filename: expected=%v, actual=%s", valuesFile, filename)
}, "", emptyEnvTmplData)
buf, err := r.RenderToBytes(valuesFile)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
actual := string(buf)
if !reflect.DeepEqual(actual, expected) {
t.Errorf("unexpected result: expected=%v, actual=%v", expected, actual)
}
}
+60
View File
@@ -0,0 +1,60 @@
package tmpl
import (
"fmt"
"reflect"
"strings"
)
type noValueError struct {
msg string
}
func (e *noValueError) Error() string {
return e.msg
}
func get(path string, obj interface{}) (interface{}, error) {
if path == "" {
return obj, nil
}
keys := strings.Split(path, ".")
switch typedObj := obj.(type) {
case map[string]interface{}:
v, ok := typedObj[keys[0]]
if !ok {
return nil, &noValueError{fmt.Sprintf("no value exist for key \"%s\" in %v", keys[0], typedObj)}
}
return get(strings.Join(keys[1:], "."), v)
case map[interface{}]interface{}:
v, ok := typedObj[keys[0]]
if !ok {
return nil, &noValueError{fmt.Sprintf("no value exist for key \"%s\" in %v", keys[0], typedObj)}
}
return get(strings.Join(keys[1:], "."), v)
default:
maybeStruct := reflect.ValueOf(typedObj)
if maybeStruct.NumField() < 1 {
return nil, &noValueError{fmt.Sprintf("unexpected type(%v) of value for key \"%s\": it must be either map[string]interface{} or any struct", reflect.TypeOf(obj), keys[0])}
}
f := maybeStruct.FieldByName(keys[0])
if !f.IsValid() {
return nil, &noValueError{fmt.Sprintf("no field named \"%s\" exist in %v", keys[0], typedObj)}
}
v := f.Interface()
return get(strings.Join(keys[1:], "."), v)
}
}
func getOrNil(path string, o interface{}) (interface{}, error) {
v, err := get(path, o)
if err != nil {
switch err.(type) {
case *noValueError:
return nil, nil
default:
return nil, err
}
}
return v, nil
}
+91
View File
@@ -0,0 +1,91 @@
package tmpl
import (
"testing"
)
func TestGetStruct(t *testing.T) {
type Foo struct{ Bar string }
obj := struct{ Foo }{Foo{Bar: "Bar"}}
v1, err := get("Foo.Bar", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v1 != "Bar" {
t.Errorf("unexpected value for path Foo.Bar in %v: expected=Bar, actual=%v", obj, v1)
}
_, err = get("Foo.baz", obj)
if err == nil {
t.Errorf("expected error but was not occurred")
}
}
func TestGetMap(t *testing.T) {
obj := map[string]interface{}{"Foo": map[string]interface{}{"Bar": "Bar"}}
v1, err := get("Foo.Bar", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v1 != "Bar" {
t.Errorf("unexpected value for path Foo.Bar in %v: expected=Bar, actual=%v", obj, v1)
}
_, err = get("Foo.baz", obj)
if err == nil {
t.Errorf("expected error but was not occurred")
}
}
func TestGetOrNilStruct(t *testing.T) {
type Foo struct{ Bar string }
obj := struct{ Foo }{Foo{Bar: "Bar"}}
v1, err := getOrNil("Foo.Bar", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v1 != "Bar" {
t.Errorf("unexpected value for path Foo.Bar in %v: expected=Bar, actual=%v", obj, v1)
}
v2, err := getOrNil("Foo.baz", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v2 != nil {
t.Errorf("unexpected value for path Foo.baz in %v: expected=nil, actual=%v", obj, v2)
}
}
func TestGetOrNilMap(t *testing.T) {
obj := map[string]interface{}{"Foo": map[string]interface{}{"Bar": "Bar"}}
v1, err := getOrNil("Foo.Bar", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v1 != "Bar" {
t.Errorf("unexpected value for path Foo.Bar in %v: expected=Bar, actual=%v", obj, v1)
}
v2, err := getOrNil("Foo.baz", obj)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if v2 != nil {
t.Errorf("unexpected value for path Foo.baz in %v: expected=nil, actual=%v", obj, v2)
}
}
+30
View File
@@ -0,0 +1,30 @@
package tmpl
type templateTextRenderer struct {
ReadText func(string) ([]byte, error)
Context *Context
Data interface{}
}
type TextRenderer interface {
RenderTemplateText(text string) (string, error)
}
func NewTextRenderer(readFile func(filename string) ([]byte, error), basePath string, data interface{}) *templateTextRenderer {
return &templateTextRenderer{
ReadText: readFile,
Context: &Context{
basePath: basePath,
readFile: readFile,
},
Data: data,
}
}
func (r *templateTextRenderer) RenderTemplateText(text string) (string, error) {
buf, err := r.Context.RenderTemplateToBuffer(text, r.Data)
if err != nil {
return "", err
}
return buf.String(), nil
}
+1 -1
View File
@@ -27,7 +27,7 @@ It is ready for production use. [It is used in several projects by Docker, Googl
### Latest release
[Release v0.3.4](https://github.com/imdario/mergo/releases/tag/v0.3.4).
[Release v0.3.6](https://github.com/imdario/mergo/releases/tag/v0.3.6).
### Important note
+23 -4
View File
@@ -9,6 +9,7 @@
package mergo
import (
"fmt"
"reflect"
)
@@ -103,7 +104,15 @@ func deepMerge(dst, src reflect.Value, visited map[uintptr]*visit, depth int, co
case reflect.Ptr:
fallthrough
case reflect.Map:
if err = deepMerge(dstElement, srcElement, visited, depth+1, config); err != nil {
srcMapElm := srcElement
dstMapElm := dstElement
if srcMapElm.CanInterface() {
srcMapElm = reflect.ValueOf(srcMapElm.Interface())
if dstMapElm.IsValid() {
dstMapElm = reflect.ValueOf(dstMapElm.Interface())
}
}
if err = deepMerge(dstMapElm, srcMapElm, visited, depth+1, config); err != nil {
return
}
case reflect.Slice:
@@ -116,7 +125,14 @@ func deepMerge(dst, src reflect.Value, visited map[uintptr]*visit, depth int, co
dstSlice = reflect.ValueOf(dstElement.Interface())
}
dstSlice = reflect.AppendSlice(dstSlice, srcSlice)
if !isEmptyValue(src) && (overwrite || isEmptyValue(dst)) && !config.AppendSlice {
dstSlice = srcSlice
} else if config.AppendSlice {
if srcSlice.Type() != dstSlice.Type() {
return fmt.Errorf("cannot append two slice with different type (%s, %s)", srcSlice.Type(), dstSlice.Type())
}
dstSlice = reflect.AppendSlice(dstSlice, srcSlice)
}
dst.SetMapIndex(key, dstSlice)
}
}
@@ -124,7 +140,7 @@ func deepMerge(dst, src reflect.Value, visited map[uintptr]*visit, depth int, co
continue
}
if srcElement.IsValid() && (overwrite || (!dstElement.IsValid() || isEmptyValue(dst))) {
if srcElement.IsValid() && (overwrite || (!dstElement.IsValid() || isEmptyValue(dstElement))) {
if dst.IsNil() {
dst.Set(reflect.MakeMap(dst.Type()))
}
@@ -137,7 +153,10 @@ func deepMerge(dst, src reflect.Value, visited map[uintptr]*visit, depth int, co
}
if !isEmptyValue(src) && (overwrite || isEmptyValue(dst)) && !config.AppendSlice {
dst.Set(src)
} else {
} else if config.AppendSlice {
if src.Type() != dst.Type() {
return fmt.Errorf("cannot append two slice with different type (%s, %s)", src.Type(), dst.Type())
}
dst.Set(reflect.AppendSlice(dst, src))
}
case reflect.Ptr:
+6 -1
View File
@@ -45,7 +45,12 @@ func isEmptyValue(v reflect.Value) bool {
return v.Uint() == 0
case reflect.Float32, reflect.Float64:
return v.Float() == 0
case reflect.Interface, reflect.Ptr, reflect.Func:
case reflect.Interface, reflect.Ptr:
if v.IsNil() {
return true
}
return isEmptyValue(v.Elem())
case reflect.Func:
return v.IsNil()
case reflect.Invalid:
return true
+15
View File
@@ -0,0 +1,15 @@
coverage:
range: 80..100
round: down
precision: 2
status:
project: # measuring the overall project coverage
default: # context, you can create multiple ones with custom titles
enabled: yes # must be yes|true to enable this status
target: 100 # specify the target coverage for each commit status
# option: "auto" (must increase from parent commit or pull request base)
# option: "X%" a static target percentage to hit
if_not_found: success # if parent is not found report status as success, error, or failure
if_ci_failed: error # if ci fails report status as success, error, or failure
+11
View File
@@ -0,0 +1,11 @@
.DS_Store
/vendor
/cover
cover.out
lint.log
# Binaries
*.test
# Profiling output
*.prof
+23
View File
@@ -0,0 +1,23 @@
sudo: false
language: go
go_import_path: go.uber.org/atomic
go:
- 1.7
- 1.8
- 1.9
cache:
directories:
- vendor
install:
- make install_ci
script:
- make test_ci
- scripts/test-ubergo.sh
- make lint
after_success:
- bash <(curl -s https://codecov.io/bash)
+19
View File
@@ -0,0 +1,19 @@
Copyright (c) 2016 Uber Technologies, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
+64
View File
@@ -0,0 +1,64 @@
PACKAGES := $(shell glide nv)
# Many Go tools take file globs or directories as arguments instead of packages.
PACKAGE_FILES ?= *.go
# The linting tools evolve with each Go version, so run them only on the latest
# stable release.
GO_VERSION := $(shell go version | cut -d " " -f 3)
GO_MINOR_VERSION := $(word 2,$(subst ., ,$(GO_VERSION)))
LINTABLE_MINOR_VERSIONS := 7 8
ifneq ($(filter $(LINTABLE_MINOR_VERSIONS),$(GO_MINOR_VERSION)),)
SHOULD_LINT := true
endif
export GO15VENDOREXPERIMENT=1
.PHONY: build
build:
go build -i $(PACKAGES)
.PHONY: install
install:
glide --version || go get github.com/Masterminds/glide
glide install
.PHONY: test
test:
go test -cover -race $(PACKAGES)
.PHONY: install_ci
install_ci: install
go get github.com/wadey/gocovmerge
go get github.com/mattn/goveralls
go get golang.org/x/tools/cmd/cover
ifdef SHOULD_LINT
go get github.com/golang/lint/golint
endif
.PHONY: lint
lint:
ifdef SHOULD_LINT
@rm -rf lint.log
@echo "Checking formatting..."
@gofmt -d -s $(PACKAGE_FILES) 2>&1 | tee lint.log
@echo "Checking vet..."
@$(foreach dir,$(PACKAGE_FILES),go tool vet $(dir) 2>&1 | tee -a lint.log;)
@echo "Checking lint..."
@$(foreach dir,$(PKGS),golint $(dir) 2>&1 | tee -a lint.log;)
@echo "Checking for unresolved FIXMEs..."
@git grep -i fixme | grep -v -e vendor -e Makefile | tee -a lint.log
@[ ! -s lint.log ]
else
@echo "Skipping linters on" $(GO_VERSION)
endif
.PHONY: test_ci
test_ci: install_ci build
./scripts/cover.sh $(shell go list $(PACKAGES))
+36
View File
@@ -0,0 +1,36 @@
# atomic [![GoDoc][doc-img]][doc] [![Build Status][ci-img]][ci] [![Coverage Status][cov-img]][cov] [![Go Report Card][reportcard-img]][reportcard]
Simple wrappers for primitive types to enforce atomic access.
## Installation
`go get -u go.uber.org/atomic`
## Usage
The standard library's `sync/atomic` is powerful, but it's easy to forget which
variables must be accessed atomically. `go.uber.org/atomic` preserves all the
functionality of the standard library, but wraps the primitive types to
provide a safer, more convenient API.
```go
var atom atomic.Uint32
atom.Store(42)
atom.Sub(2)
atom.CAS(40, 11)
```
See the [documentation][doc] for a complete API specification.
## Development Status
Stable.
<hr>
Released under the [MIT License](LICENSE.txt).
[doc-img]: https://godoc.org/github.com/uber-go/atomic?status.svg
[doc]: https://godoc.org/go.uber.org/atomic
[ci-img]: https://travis-ci.org/uber-go/atomic.svg?branch=master
[ci]: https://travis-ci.org/uber-go/atomic
[cov-img]: https://codecov.io/gh/uber-go/atomic/branch/master/graph/badge.svg
[cov]: https://codecov.io/gh/uber-go/atomic
[reportcard-img]: https://goreportcard.com/badge/go.uber.org/atomic
[reportcard]: https://goreportcard.com/report/go.uber.org/atomic
+351
View File
@@ -0,0 +1,351 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
// Package atomic provides simple wrappers around numerics to enforce atomic
// access.
package atomic
import (
"math"
"sync/atomic"
"time"
)
// Int32 is an atomic wrapper around an int32.
type Int32 struct{ v int32 }
// NewInt32 creates an Int32.
func NewInt32(i int32) *Int32 {
return &Int32{i}
}
// Load atomically loads the wrapped value.
func (i *Int32) Load() int32 {
return atomic.LoadInt32(&i.v)
}
// Add atomically adds to the wrapped int32 and returns the new value.
func (i *Int32) Add(n int32) int32 {
return atomic.AddInt32(&i.v, n)
}
// Sub atomically subtracts from the wrapped int32 and returns the new value.
func (i *Int32) Sub(n int32) int32 {
return atomic.AddInt32(&i.v, -n)
}
// Inc atomically increments the wrapped int32 and returns the new value.
func (i *Int32) Inc() int32 {
return i.Add(1)
}
// Dec atomically decrements the wrapped int32 and returns the new value.
func (i *Int32) Dec() int32 {
return i.Sub(1)
}
// CAS is an atomic compare-and-swap.
func (i *Int32) CAS(old, new int32) bool {
return atomic.CompareAndSwapInt32(&i.v, old, new)
}
// Store atomically stores the passed value.
func (i *Int32) Store(n int32) {
atomic.StoreInt32(&i.v, n)
}
// Swap atomically swaps the wrapped int32 and returns the old value.
func (i *Int32) Swap(n int32) int32 {
return atomic.SwapInt32(&i.v, n)
}
// Int64 is an atomic wrapper around an int64.
type Int64 struct{ v int64 }
// NewInt64 creates an Int64.
func NewInt64(i int64) *Int64 {
return &Int64{i}
}
// Load atomically loads the wrapped value.
func (i *Int64) Load() int64 {
return atomic.LoadInt64(&i.v)
}
// Add atomically adds to the wrapped int64 and returns the new value.
func (i *Int64) Add(n int64) int64 {
return atomic.AddInt64(&i.v, n)
}
// Sub atomically subtracts from the wrapped int64 and returns the new value.
func (i *Int64) Sub(n int64) int64 {
return atomic.AddInt64(&i.v, -n)
}
// Inc atomically increments the wrapped int64 and returns the new value.
func (i *Int64) Inc() int64 {
return i.Add(1)
}
// Dec atomically decrements the wrapped int64 and returns the new value.
func (i *Int64) Dec() int64 {
return i.Sub(1)
}
// CAS is an atomic compare-and-swap.
func (i *Int64) CAS(old, new int64) bool {
return atomic.CompareAndSwapInt64(&i.v, old, new)
}
// Store atomically stores the passed value.
func (i *Int64) Store(n int64) {
atomic.StoreInt64(&i.v, n)
}
// Swap atomically swaps the wrapped int64 and returns the old value.
func (i *Int64) Swap(n int64) int64 {
return atomic.SwapInt64(&i.v, n)
}
// Uint32 is an atomic wrapper around an uint32.
type Uint32 struct{ v uint32 }
// NewUint32 creates a Uint32.
func NewUint32(i uint32) *Uint32 {
return &Uint32{i}
}
// Load atomically loads the wrapped value.
func (i *Uint32) Load() uint32 {
return atomic.LoadUint32(&i.v)
}
// Add atomically adds to the wrapped uint32 and returns the new value.
func (i *Uint32) Add(n uint32) uint32 {
return atomic.AddUint32(&i.v, n)
}
// Sub atomically subtracts from the wrapped uint32 and returns the new value.
func (i *Uint32) Sub(n uint32) uint32 {
return atomic.AddUint32(&i.v, ^(n - 1))
}
// Inc atomically increments the wrapped uint32 and returns the new value.
func (i *Uint32) Inc() uint32 {
return i.Add(1)
}
// Dec atomically decrements the wrapped int32 and returns the new value.
func (i *Uint32) Dec() uint32 {
return i.Sub(1)
}
// CAS is an atomic compare-and-swap.
func (i *Uint32) CAS(old, new uint32) bool {
return atomic.CompareAndSwapUint32(&i.v, old, new)
}
// Store atomically stores the passed value.
func (i *Uint32) Store(n uint32) {
atomic.StoreUint32(&i.v, n)
}
// Swap atomically swaps the wrapped uint32 and returns the old value.
func (i *Uint32) Swap(n uint32) uint32 {
return atomic.SwapUint32(&i.v, n)
}
// Uint64 is an atomic wrapper around a uint64.
type Uint64 struct{ v uint64 }
// NewUint64 creates a Uint64.
func NewUint64(i uint64) *Uint64 {
return &Uint64{i}
}
// Load atomically loads the wrapped value.
func (i *Uint64) Load() uint64 {
return atomic.LoadUint64(&i.v)
}
// Add atomically adds to the wrapped uint64 and returns the new value.
func (i *Uint64) Add(n uint64) uint64 {
return atomic.AddUint64(&i.v, n)
}
// Sub atomically subtracts from the wrapped uint64 and returns the new value.
func (i *Uint64) Sub(n uint64) uint64 {
return atomic.AddUint64(&i.v, ^(n - 1))
}
// Inc atomically increments the wrapped uint64 and returns the new value.
func (i *Uint64) Inc() uint64 {
return i.Add(1)
}
// Dec atomically decrements the wrapped uint64 and returns the new value.
func (i *Uint64) Dec() uint64 {
return i.Sub(1)
}
// CAS is an atomic compare-and-swap.
func (i *Uint64) CAS(old, new uint64) bool {
return atomic.CompareAndSwapUint64(&i.v, old, new)
}
// Store atomically stores the passed value.
func (i *Uint64) Store(n uint64) {
atomic.StoreUint64(&i.v, n)
}
// Swap atomically swaps the wrapped uint64 and returns the old value.
func (i *Uint64) Swap(n uint64) uint64 {
return atomic.SwapUint64(&i.v, n)
}
// Bool is an atomic Boolean.
type Bool struct{ v uint32 }
// NewBool creates a Bool.
func NewBool(initial bool) *Bool {
return &Bool{boolToInt(initial)}
}
// Load atomically loads the Boolean.
func (b *Bool) Load() bool {
return truthy(atomic.LoadUint32(&b.v))
}
// CAS is an atomic compare-and-swap.
func (b *Bool) CAS(old, new bool) bool {
return atomic.CompareAndSwapUint32(&b.v, boolToInt(old), boolToInt(new))
}
// Store atomically stores the passed value.
func (b *Bool) Store(new bool) {
atomic.StoreUint32(&b.v, boolToInt(new))
}
// Swap sets the given value and returns the previous value.
func (b *Bool) Swap(new bool) bool {
return truthy(atomic.SwapUint32(&b.v, boolToInt(new)))
}
// Toggle atomically negates the Boolean and returns the previous value.
func (b *Bool) Toggle() bool {
return truthy(atomic.AddUint32(&b.v, 1) - 1)
}
func truthy(n uint32) bool {
return n&1 == 1
}
func boolToInt(b bool) uint32 {
if b {
return 1
}
return 0
}
// Float64 is an atomic wrapper around float64.
type Float64 struct {
v uint64
}
// NewFloat64 creates a Float64.
func NewFloat64(f float64) *Float64 {
return &Float64{math.Float64bits(f)}
}
// Load atomically loads the wrapped value.
func (f *Float64) Load() float64 {
return math.Float64frombits(atomic.LoadUint64(&f.v))
}
// Store atomically stores the passed value.
func (f *Float64) Store(s float64) {
atomic.StoreUint64(&f.v, math.Float64bits(s))
}
// Add atomically adds to the wrapped float64 and returns the new value.
func (f *Float64) Add(s float64) float64 {
for {
old := f.Load()
new := old + s
if f.CAS(old, new) {
return new
}
}
}
// Sub atomically subtracts from the wrapped float64 and returns the new value.
func (f *Float64) Sub(s float64) float64 {
return f.Add(-s)
}
// CAS is an atomic compare-and-swap.
func (f *Float64) CAS(old, new float64) bool {
return atomic.CompareAndSwapUint64(&f.v, math.Float64bits(old), math.Float64bits(new))
}
// Duration is an atomic wrapper around time.Duration
// https://godoc.org/time#Duration
type Duration struct {
v Int64
}
// NewDuration creates a Duration.
func NewDuration(d time.Duration) *Duration {
return &Duration{v: *NewInt64(int64(d))}
}
// Load atomically loads the wrapped value.
func (d *Duration) Load() time.Duration {
return time.Duration(d.v.Load())
}
// Store atomically stores the passed value.
func (d *Duration) Store(n time.Duration) {
d.v.Store(int64(n))
}
// Add atomically adds to the wrapped time.Duration and returns the new value.
func (d *Duration) Add(n time.Duration) time.Duration {
return time.Duration(d.v.Add(int64(n)))
}
// Sub atomically subtracts from the wrapped time.Duration and returns the new value.
func (d *Duration) Sub(n time.Duration) time.Duration {
return time.Duration(d.v.Sub(int64(n)))
}
// Swap atomically swaps the wrapped time.Duration and returns the old value.
func (d *Duration) Swap(n time.Duration) time.Duration {
return time.Duration(d.v.Swap(int64(n)))
}
// CAS is an atomic compare-and-swap.
func (d *Duration) CAS(old, new time.Duration) bool {
return d.v.CAS(int64(old), int64(new))
}
// Value shadows the type of the same name from sync/atomic
// https://godoc.org/sync/atomic#Value
type Value struct{ atomic.Value }
+17
View File
@@ -0,0 +1,17 @@
hash: f14d51408e3e0e4f73b34e4039484c78059cd7fc5f4996fdd73db20dc8d24f53
updated: 2016-10-27T00:10:51.16960137-07:00
imports: []
testImports:
- name: github.com/davecgh/go-spew
version: 5215b55f46b2b919f50a1df0eaa5886afe4e3b3d
subpackages:
- spew
- name: github.com/pmezard/go-difflib
version: d8ed2627bdf02c080bf22230dbb337003b7aba2d
subpackages:
- difflib
- name: github.com/stretchr/testify
version: d77da356e56a7428ad25149ca77381849a6a5232
subpackages:
- assert
- require
+6
View File
@@ -0,0 +1,6 @@
package: go.uber.org/atomic
testImport:
- package: github.com/stretchr/testify
subpackages:
- assert
- require
+49
View File
@@ -0,0 +1,49 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package atomic
// String is an atomic type-safe wrapper around Value for strings.
type String struct{ v Value }
// NewString creates a String.
func NewString(str string) *String {
s := &String{}
if str != "" {
s.Store(str)
}
return s
}
// Load atomically loads the wrapped string.
func (s *String) Load() string {
v := s.v.Load()
if v == nil {
return ""
}
return v.(string)
}
// Store atomically stores the passed string.
// Note: Converting the string to an interface{} to store in the Value
// requires an allocation.
func (s *String) Store(str string) {
s.v.Store(str)
}
+15
View File
@@ -0,0 +1,15 @@
coverage:
range: 80..100
round: down
precision: 2
status:
project: # measuring the overall project coverage
default: # context, you can create multiple ones with custom titles
enabled: yes # must be yes|true to enable this status
target: 100 # specify the target coverage for each commit status
# option: "auto" (must increase from parent commit or pull request base)
# option: "X%" a static target percentage to hit
if_not_found: success # if parent is not found report status as success, error, or failure
if_ci_failed: error # if ci fails report status as success, error, or failure
+1
View File
@@ -0,0 +1 @@
/vendor
+33
View File
@@ -0,0 +1,33 @@
sudo: false
language: go
go_import_path: go.uber.org/multierr
env:
global:
- GO15VENDOREXPERIMENT=1
go:
- 1.7
- 1.8
- tip
cache:
directories:
- vendor
before_install:
- go version
install:
- |
set -e
make install_ci
script:
- |
set -e
make lint
make test_ci
after_success:
- bash <(curl -s https://codecov.io/bash)
+28
View File
@@ -0,0 +1,28 @@
Releases
========
v1.1.0 (2017-06-30)
===================
- Added an `Errors(error) []error` function to extract the underlying list of
errors for a multierr error.
v1.0.0 (2017-05-31)
===================
No changes since v0.2.0. This release is committing to making no breaking
changes to the current API in the 1.X series.
v0.2.0 (2017-04-11)
===================
- Repeatedly appending to the same error is now faster due to fewer
allocations.
v0.1.0 (2017-31-03)
===================
- Initial release
+19
View File
@@ -0,0 +1,19 @@
Copyright (c) 2017 Uber Technologies, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
+74
View File
@@ -0,0 +1,74 @@
export GO15VENDOREXPERIMENT=1
PACKAGES := $(shell glide nv)
GO_FILES := $(shell \
find . '(' -path '*/.*' -o -path './vendor' ')' -prune \
-o -name '*.go' -print | cut -b3-)
.PHONY: install
install:
glide --version || go get github.com/Masterminds/glide
glide install
.PHONY: build
build:
go build -i $(PACKAGES)
.PHONY: test
test:
go test -cover -race $(PACKAGES)
.PHONY: gofmt
gofmt:
$(eval FMT_LOG := $(shell mktemp -t gofmt.XXXXX))
@gofmt -e -s -l $(GO_FILES) > $(FMT_LOG) || true
@[ ! -s "$(FMT_LOG)" ] || (echo "gofmt failed:" | cat - $(FMT_LOG) && false)
.PHONY: govet
govet:
$(eval VET_LOG := $(shell mktemp -t govet.XXXXX))
@go vet $(PACKAGES) 2>&1 \
| grep -v '^exit status' > $(VET_LOG) || true
@[ ! -s "$(VET_LOG)" ] || (echo "govet failed:" | cat - $(VET_LOG) && false)
.PHONY: golint
golint:
@go get github.com/golang/lint/golint
$(eval LINT_LOG := $(shell mktemp -t golint.XXXXX))
@cat /dev/null > $(LINT_LOG)
@$(foreach pkg, $(PACKAGES), golint $(pkg) >> $(LINT_LOG) || true;)
@[ ! -s "$(LINT_LOG)" ] || (echo "golint failed:" | cat - $(LINT_LOG) && false)
.PHONY: staticcheck
staticcheck:
@go get honnef.co/go/tools/cmd/staticcheck
$(eval STATICCHECK_LOG := $(shell mktemp -t staticcheck.XXXXX))
@staticcheck $(PACKAGES) 2>&1 > $(STATICCHECK_LOG) || true
@[ ! -s "$(STATICCHECK_LOG)" ] || (echo "staticcheck failed:" | cat - $(STATICCHECK_LOG) && false)
.PHONY: lint
lint: gofmt govet golint staticcheck
.PHONY: cover
cover:
./scripts/cover.sh $(shell go list $(PACKAGES))
go tool cover -html=cover.out -o cover.html
update-license:
@go get go.uber.org/tools/update-license
@update-license \
$(shell go list -json $(PACKAGES) | \
jq -r '.Dir + "/" + (.GoFiles | .[])')
##############################################################################
.PHONY: install_ci
install_ci: install
go get github.com/wadey/gocovmerge
go get github.com/mattn/goveralls
go get golang.org/x/tools/cmd/cover
.PHONY: test_ci
test_ci: install_ci
./scripts/cover.sh $(shell go list $(PACKAGES))
+23
View File
@@ -0,0 +1,23 @@
# multierr [![GoDoc][doc-img]][doc] [![Build Status][ci-img]][ci] [![Coverage Status][cov-img]][cov]
`multierr` allows combining one or more Go `error`s together.
## Installation
go get -u go.uber.org/multierr
## Status
Stable: No breaking changes will be made before 2.0.
-------------------------------------------------------------------------------
Released under the [MIT License].
[MIT License]: LICENSE.txt
[doc-img]: https://godoc.org/go.uber.org/multierr?status.svg
[doc]: https://godoc.org/go.uber.org/multierr
[ci-img]: https://travis-ci.org/uber-go/multierr.svg?branch=master
[cov-img]: https://codecov.io/gh/uber-go/multierr/branch/master/graph/badge.svg
[ci]: https://travis-ci.org/uber-go/multierr
[cov]: https://codecov.io/gh/uber-go/multierr
+401
View File
@@ -0,0 +1,401 @@
// Copyright (c) 2017 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
// Package multierr allows combining one or more errors together.
//
// Overview
//
// Errors can be combined with the use of the Combine function.
//
// multierr.Combine(
// reader.Close(),
// writer.Close(),
// conn.Close(),
// )
//
// If only two errors are being combined, the Append function may be used
// instead.
//
// err = multierr.Combine(reader.Close(), writer.Close())
//
// This makes it possible to record resource cleanup failures from deferred
// blocks with the help of named return values.
//
// func sendRequest(req Request) (err error) {
// conn, err := openConnection()
// if err != nil {
// return err
// }
// defer func() {
// err = multierr.Append(err, conn.Close())
// }()
// // ...
// }
//
// The underlying list of errors for a returned error object may be retrieved
// with the Errors function.
//
// errors := multierr.Errors(err)
// if len(errors) > 0 {
// fmt.Println("The following errors occurred:")
// }
//
// Advanced Usage
//
// Errors returned by Combine and Append MAY implement the following
// interface.
//
// type errorGroup interface {
// // Returns a slice containing the underlying list of errors.
// //
// // This slice MUST NOT be modified by the caller.
// Errors() []error
// }
//
// Note that if you need access to list of errors behind a multierr error, you
// should prefer using the Errors function. That said, if you need cheap
// read-only access to the underlying errors slice, you can attempt to cast
// the error to this interface. You MUST handle the failure case gracefully
// because errors returned by Combine and Append are not guaranteed to
// implement this interface.
//
// var errors []error
// group, ok := err.(errorGroup)
// if ok {
// errors = group.Errors()
// } else {
// errors = []error{err}
// }
package multierr // import "go.uber.org/multierr"
import (
"bytes"
"fmt"
"io"
"strings"
"sync"
"go.uber.org/atomic"
)
var (
// Separator for single-line error messages.
_singlelineSeparator = []byte("; ")
_newline = []byte("\n")
// Prefix for multi-line messages
_multilinePrefix = []byte("the following errors occurred:")
// Prefix for the first and following lines of an item in a list of
// multi-line error messages.
//
// For example, if a single item is:
//
// foo
// bar
//
// It will become,
//
// - foo
// bar
_multilineSeparator = []byte("\n - ")
_multilineIndent = []byte(" ")
)
// _bufferPool is a pool of bytes.Buffers.
var _bufferPool = sync.Pool{
New: func() interface{} {
return &bytes.Buffer{}
},
}
type errorGroup interface {
Errors() []error
}
// Errors returns a slice containing zero or more errors that the supplied
// error is composed of. If the error is nil, the returned slice is empty.
//
// err := multierr.Append(r.Close(), w.Close())
// errors := multierr.Errors(err)
//
// If the error is not composed of other errors, the returned slice contains
// just the error that was passed in.
//
// Callers of this function are free to modify the returned slice.
func Errors(err error) []error {
if err == nil {
return nil
}
// Note that we're casting to multiError, not errorGroup. Our contract is
// that returned errors MAY implement errorGroup. Errors, however, only
// has special behavior for multierr-specific error objects.
//
// This behavior can be expanded in the future but I think it's prudent to
// start with as little as possible in terms of contract and possibility
// of misuse.
eg, ok := err.(*multiError)
if !ok {
return []error{err}
}
errors := eg.Errors()
result := make([]error, len(errors))
copy(result, errors)
return result
}
// multiError is an error that holds one or more errors.
//
// An instance of this is guaranteed to be non-empty and flattened. That is,
// none of the errors inside multiError are other multiErrors.
//
// multiError formats to a semi-colon delimited list of error messages with
// %v and with a more readable multi-line format with %+v.
type multiError struct {
copyNeeded atomic.Bool
errors []error
}
var _ errorGroup = (*multiError)(nil)
// Errors returns the list of underlying errors.
//
// This slice MUST NOT be modified.
func (merr *multiError) Errors() []error {
if merr == nil {
return nil
}
return merr.errors
}
func (merr *multiError) Error() string {
if merr == nil {
return ""
}
buff := _bufferPool.Get().(*bytes.Buffer)
buff.Reset()
merr.writeSingleline(buff)
result := buff.String()
_bufferPool.Put(buff)
return result
}
func (merr *multiError) Format(f fmt.State, c rune) {
if c == 'v' && f.Flag('+') {
merr.writeMultiline(f)
} else {
merr.writeSingleline(f)
}
}
func (merr *multiError) writeSingleline(w io.Writer) {
first := true
for _, item := range merr.errors {
if first {
first = false
} else {
w.Write(_singlelineSeparator)
}
io.WriteString(w, item.Error())
}
}
func (merr *multiError) writeMultiline(w io.Writer) {
w.Write(_multilinePrefix)
for _, item := range merr.errors {
w.Write(_multilineSeparator)
writePrefixLine(w, _multilineIndent, fmt.Sprintf("%+v", item))
}
}
// Writes s to the writer with the given prefix added before each line after
// the first.
func writePrefixLine(w io.Writer, prefix []byte, s string) {
first := true
for len(s) > 0 {
if first {
first = false
} else {
w.Write(prefix)
}
idx := strings.IndexByte(s, '\n')
if idx < 0 {
idx = len(s) - 1
}
io.WriteString(w, s[:idx+1])
s = s[idx+1:]
}
}
type inspectResult struct {
// Number of top-level non-nil errors
Count int
// Total number of errors including multiErrors
Capacity int
// Index of the first non-nil error in the list. Value is meaningless if
// Count is zero.
FirstErrorIdx int
// Whether the list contains at least one multiError
ContainsMultiError bool
}
// Inspects the given slice of errors so that we can efficiently allocate
// space for it.
func inspect(errors []error) (res inspectResult) {
first := true
for i, err := range errors {
if err == nil {
continue
}
res.Count++
if first {
first = false
res.FirstErrorIdx = i
}
if merr, ok := err.(*multiError); ok {
res.Capacity += len(merr.errors)
res.ContainsMultiError = true
} else {
res.Capacity++
}
}
return
}
// fromSlice converts the given list of errors into a single error.
func fromSlice(errors []error) error {
res := inspect(errors)
switch res.Count {
case 0:
return nil
case 1:
// only one non-nil entry
return errors[res.FirstErrorIdx]
case len(errors):
if !res.ContainsMultiError {
// already flat
return &multiError{errors: errors}
}
}
nonNilErrs := make([]error, 0, res.Capacity)
for _, err := range errors[res.FirstErrorIdx:] {
if err == nil {
continue
}
if nested, ok := err.(*multiError); ok {
nonNilErrs = append(nonNilErrs, nested.errors...)
} else {
nonNilErrs = append(nonNilErrs, err)
}
}
return &multiError{errors: nonNilErrs}
}
// Combine combines the passed errors into a single error.
//
// If zero arguments were passed or if all items are nil, a nil error is
// returned.
//
// Combine(nil, nil) // == nil
//
// If only a single error was passed, it is returned as-is.
//
// Combine(err) // == err
//
// Combine skips over nil arguments so this function may be used to combine
// together errors from operations that fail independently of each other.
//
// multierr.Combine(
// reader.Close(),
// writer.Close(),
// pipe.Close(),
// )
//
// If any of the passed errors is a multierr error, it will be flattened along
// with the other errors.
//
// multierr.Combine(multierr.Combine(err1, err2), err3)
// // is the same as
// multierr.Combine(err1, err2, err3)
//
// The returned error formats into a readable multi-line error message if
// formatted with %+v.
//
// fmt.Sprintf("%+v", multierr.Combine(err1, err2))
func Combine(errors ...error) error {
return fromSlice(errors)
}
// Append appends the given errors together. Either value may be nil.
//
// This function is a specialization of Combine for the common case where
// there are only two errors.
//
// err = multierr.Append(reader.Close(), writer.Close())
//
// The following pattern may also be used to record failure of deferred
// operations without losing information about the original error.
//
// func doSomething(..) (err error) {
// f := acquireResource()
// defer func() {
// err = multierr.Append(err, f.Close())
// }()
func Append(left error, right error) error {
switch {
case left == nil:
return right
case right == nil:
return left
}
if _, ok := right.(*multiError); !ok {
if l, ok := left.(*multiError); ok && !l.copyNeeded.Swap(true) {
// Common case where the error on the left is constantly being
// appended to.
errs := append(l.errors, right)
return &multiError{errors: errs}
} else if !ok {
// Both errors are single errors.
return &multiError{errors: []error{left, right}}
}
}
// Either right or both, left and right, are multiErrors. Rely on usual
// expensive logic.
errors := [2]error{left, right}
return fromSlice(errors[0:])
}
+19
View File
@@ -0,0 +1,19 @@
hash: b53b5e9a84b9cb3cc4b2d0499e23da2feca1eec318ce9bb717ecf35bf24bf221
updated: 2017-04-10T13:34:45.671678062-07:00
imports:
- name: go.uber.org/atomic
version: 3b8db5e93c4c02efbc313e17b2e796b0914a01fb
testImports:
- name: github.com/davecgh/go-spew
version: 6d212800a42e8ab5c146b8ace3490ee17e5225f9
subpackages:
- spew
- name: github.com/pmezard/go-difflib
version: d8ed2627bdf02c080bf22230dbb337003b7aba2d
subpackages:
- difflib
- name: github.com/stretchr/testify
version: 69483b4bd14f5845b5a1e55bca19e954e827f1d0
subpackages:
- assert
- require
+8
View File
@@ -0,0 +1,8 @@
package: go.uber.org/multierr
import:
- package: go.uber.org/atomic
version: ^1
testImport:
- package: github.com/stretchr/testify
subpackages:
- assert
+17
View File
@@ -0,0 +1,17 @@
coverage:
range: 80..100
round: down
precision: 2
status:
project: # measuring the overall project coverage
default: # context, you can create multiple ones with custom titles
enabled: yes # must be yes|true to enable this status
target: 95% # specify the target coverage for each commit status
# option: "auto" (must increase from parent commit or pull request base)
# option: "X%" a static target percentage to hit
if_not_found: success # if parent is not found report status as success, error, or failure
if_ci_failed: error # if ci fails report status as success, error, or failure
ignore:
- internal/readme/readme.go
+28
View File
@@ -0,0 +1,28 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
vendor
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
*.test
*.prof
*.pprof
*.out
*.log
+108
View File
@@ -0,0 +1,108 @@
# :zap: zap [![GoDoc][doc-img]][doc] [![Build Status][ci-img]][ci] [![Coverage Status][cov-img]][cov]
Blazing fast, structured, leveled logging in Go.
## Installation
`go get -u go.uber.org/zap`
Note that zap only supports the two most recent minor versions of Go.
## Quick Start
In contexts where performance is nice, but not critical, use the
`SugaredLogger`. It's 4-10x faster than than other structured logging
packages and includes both structured and `printf`-style APIs.
```go
logger, _ := zap.NewProduction()
defer logger.Sync() // flushes buffer, if any
sugar := logger.Sugar()
sugar.Infow("failed to fetch URL",
// Structured context as loosely typed key-value pairs.
"url", url,
"attempt", 3,
"backoff", time.Second,
)
sugar.Infof("Failed to fetch URL: %s", url)
```
When performance and type safety are critical, use the `Logger`. It's even
faster than the `SugaredLogger` and allocates far less, but it only supports
structured logging.
```go
logger, _ := zap.NewProduction()
defer logger.Sync()
logger.Info("failed to fetch URL",
// Structured context as strongly typed Field values.
zap.String("url", url),
zap.Int("attempt", 3),
zap.Duration("backoff", time.Second),
)
```
See the [documentation][doc] and [FAQ](FAQ.md) for more details.
## Performance
For applications that log in the hot path, reflection-based serialization and
string formatting are prohibitively expensive &mdash; they're CPU-intensive
and make many small allocations. Put differently, using `encoding/json` and
`fmt.Fprintf` to log tons of `interface{}`s makes your application slow.
Zap takes a different approach. It includes a reflection-free, zero-allocation
JSON encoder, and the base `Logger` strives to avoid serialization overhead
and allocations wherever possible. By building the high-level `SugaredLogger`
on that foundation, zap lets users *choose* when they need to count every
allocation and when they'd prefer a more familiar, loosely typed API.
As measured by its own [benchmarking suite][], not only is zap more performant
than comparable structured logging packages &mdash; it's also faster than the
standard library. Like all benchmarks, take these with a grain of salt.<sup
id="anchor-versions">[1](#footnote-versions)</sup>
Log a message and 10 fields:
{{.BenchmarkAddingFields}}
Log a message with a logger that already has 10 fields of context:
{{.BenchmarkAccumulatedContext}}
Log a static string, without any context or `printf`-style templating:
{{.BenchmarkWithoutFields}}
## Development Status: Stable
All APIs are finalized, and no breaking changes will be made in the 1.x series
of releases. Users of semver-aware dependency management systems should pin
zap to `^1`.
## Contributing
We encourage and support an active, healthy community of contributors &mdash;
including you! Details are in the [contribution guide](CONTRIBUTING.md) and
the [code of conduct](CODE_OF_CONDUCT.md). The zap maintainers keep an eye on
issues and pull requests, but you can also report any negative conduct to
oss-conduct@uber.com. That email list is a private, safe space; even the zap
maintainers don't have access, so don't hesitate to hold us to a high
standard.
<hr>
Released under the [MIT License](LICENSE.txt).
<sup id="footnote-versions">1</sup> In particular, keep in mind that we may be
benchmarking against slightly older versions of other packages. Versions are
pinned in zap's [glide.lock][] file. [↩](#anchor-versions)
[doc-img]: https://godoc.org/go.uber.org/zap?status.svg
[doc]: https://godoc.org/go.uber.org/zap
[ci-img]: https://travis-ci.org/uber-go/zap.svg?branch=master
[ci]: https://travis-ci.org/uber-go/zap
[cov-img]: https://codecov.io/gh/uber-go/zap/branch/master/graph/badge.svg
[cov]: https://codecov.io/gh/uber-go/zap
[benchmarking suite]: https://github.com/uber-go/zap/tree/master/benchmarks
[glide.lock]: https://github.com/uber-go/zap/blob/master/glide.lock
+21
View File
@@ -0,0 +1,21 @@
language: go
sudo: false
go:
- "1.9"
- "1.10"
go_import_path: go.uber.org/zap
env:
global:
- TEST_TIMEOUT_SCALE=10
cache:
directories:
- vendor
install:
- make dependencies
script:
- make lint
- make test
- make bench
after_success:
- make cover
- bash <(curl -s https://codecov.io/bash)
+286
View File
@@ -0,0 +1,286 @@
# Changelog
## v1.8.0 (13 Apr 2018)
Enhancements:
* [#508][]: Make log level configurable when redirecting the standard
library's logger.
* [#518][]: Add a logger that writes to a `*testing.TB`.
* [#577][]: Add a top-level alias for `zapcore.Field` to clean up GoDoc.
Bugfixes:
* [#574][]: Add a missing import comment to `go.uber.org/zap/buffer`.
Thanks to @DiSiqueira and @djui for their contributions to this release.
## v1.7.1 (25 Sep 2017)
Bugfixes:
* [#504][]: Store strings when using AddByteString with the map encoder.
## v1.7.0 (21 Sep 2017)
Enhancements:
* [#487][]: Add `NewStdLogAt`, which extends `NewStdLog` by allowing the user
to specify the level of the logged messages.
## v1.6.0 (30 Aug 2017)
Enhancements:
* [#491][]: Omit zap stack frames from stacktraces.
* [#490][]: Add a `ContextMap` method to observer logs for simpler
field validation in tests.
## v1.5.0 (22 Jul 2017)
Enhancements:
* [#460][] and [#470][]: Support errors produced by `go.uber.org/multierr`.
* [#465][]: Support user-supplied encoders for logger names.
Bugfixes:
* [#477][]: Fix a bug that incorrectly truncated deep stacktraces.
Thanks to @richard-tunein and @pavius for their contributions to this release.
## v1.4.1 (08 Jun 2017)
This release fixes two bugs.
Bugfixes:
* [#435][]: Support a variety of case conventions when unmarshaling levels.
* [#444][]: Fix a panic in the observer.
## v1.4.0 (12 May 2017)
This release adds a few small features and is fully backward-compatible.
Enhancements:
* [#424][]: Add a `LineEnding` field to `EncoderConfig`, allowing users to
override the Unix-style default.
* [#425][]: Preserve time zones when logging times.
* [#431][]: Make `zap.AtomicLevel` implement `fmt.Stringer`, which makes a
variety of operations a bit simpler.
## v1.3.0 (25 Apr 2017)
This release adds an enhancement to zap's testing helpers as well as the
ability to marshal an AtomicLevel. It is fully backward-compatible.
Enhancements:
* [#415][]: Add a substring-filtering helper to zap's observer. This is
particularly useful when testing the `SugaredLogger`.
* [#416][]: Make `AtomicLevel` implement `encoding.TextMarshaler`.
## v1.2.0 (13 Apr 2017)
This release adds a gRPC compatibility wrapper. It is fully backward-compatible.
Enhancements:
* [#402][]: Add a `zapgrpc` package that wraps zap's Logger and implements
`grpclog.Logger`.
## v1.1.0 (31 Mar 2017)
This release fixes two bugs and adds some enhancements to zap's testing helpers.
It is fully backward-compatible.
Bugfixes:
* [#385][]: Fix caller path trimming on Windows.
* [#396][]: Fix a panic when attempting to use non-existent directories with
zap's configuration struct.
Enhancements:
* [#386][]: Add filtering helpers to zaptest's observing logger.
Thanks to @moitias for contributing to this release.
## v1.0.0 (14 Mar 2017)
This is zap's first stable release. All exported APIs are now final, and no
further breaking changes will be made in the 1.x release series. Anyone using a
semver-aware dependency manager should now pin to `^1`.
Breaking changes:
* [#366][]: Add byte-oriented APIs to encoders to log UTF-8 encoded text without
casting from `[]byte` to `string`.
* [#364][]: To support buffering outputs, add `Sync` methods to `zapcore.Core`,
`zap.Logger`, and `zap.SugaredLogger`.
* [#371][]: Rename the `testutils` package to `zaptest`, which is less likely to
clash with other testing helpers.
Bugfixes:
* [#362][]: Make the ISO8601 time formatters fixed-width, which is friendlier
for tab-separated console output.
* [#369][]: Remove the automatic locks in `zapcore.NewCore`, which allows zap to
work with concurrency-safe `WriteSyncer` implementations.
* [#347][]: Stop reporting errors when trying to `fsync` standard out on Linux
systems.
* [#373][]: Report the correct caller from zap's standard library
interoperability wrappers.
Enhancements:
* [#348][]: Add a registry allowing third-party encodings to work with zap's
built-in `Config`.
* [#327][]: Make the representation of logger callers configurable (like times,
levels, and durations).
* [#376][]: Allow third-party encoders to use their own buffer pools, which
removes the last performance advantage that zap's encoders have over plugins.
* [#346][]: Add `CombineWriteSyncers`, a convenience function to tee multiple
`WriteSyncer`s and lock the result.
* [#365][]: Make zap's stacktraces compatible with mid-stack inlining (coming in
Go 1.9).
* [#372][]: Export zap's observing logger as `zaptest/observer`. This makes it
easier for particularly punctilious users to unit test their application's
logging.
Thanks to @suyash, @htrendev, @flisky, @Ulexus, and @skipor for their
contributions to this release.
## v1.0.0-rc.3 (7 Mar 2017)
This is the third release candidate for zap's stable release. There are no
breaking changes.
Bugfixes:
* [#339][]: Byte slices passed to `zap.Any` are now correctly treated as binary blobs
rather than `[]uint8`.
Enhancements:
* [#307][]: Users can opt into colored output for log levels.
* [#353][]: In addition to hijacking the output of the standard library's
package-global logging functions, users can now construct a zap-backed
`log.Logger` instance.
* [#311][]: Frames from common runtime functions and some of zap's internal
machinery are now omitted from stacktraces.
Thanks to @ansel1 and @suyash for their contributions to this release.
## v1.0.0-rc.2 (21 Feb 2017)
This is the second release candidate for zap's stable release. It includes two
breaking changes.
Breaking changes:
* [#316][]: Zap's global loggers are now fully concurrency-safe
(previously, users had to ensure that `ReplaceGlobals` was called before the
loggers were in use). However, they must now be accessed via the `L()` and
`S()` functions. Users can update their projects with
```
gofmt -r "zap.L -> zap.L()" -w .
gofmt -r "zap.S -> zap.S()" -w .
```
* [#309][] and [#317][]: RC1 was mistakenly shipped with invalid
JSON and YAML struct tags on all config structs. This release fixes the tags
and adds static analysis to prevent similar bugs in the future.
Bugfixes:
* [#321][]: Redirecting the standard library's `log` output now
correctly reports the logger's caller.
Enhancements:
* [#325][] and [#333][]: Zap now transparently supports non-standard, rich
errors like those produced by `github.com/pkg/errors`.
* [#326][]: Though `New(nil)` continues to return a no-op logger, `NewNop()` is
now preferred. Users can update their projects with `gofmt -r 'zap.New(nil) ->
zap.NewNop()' -w .`.
* [#300][]: Incorrectly importing zap as `github.com/uber-go/zap` now returns a
more informative error.
Thanks to @skipor and @chapsuk for their contributions to this release.
## v1.0.0-rc.1 (14 Feb 2017)
This is the first release candidate for zap's stable release. There are multiple
breaking changes and improvements from the pre-release version. Most notably:
* **Zap's import path is now "go.uber.org/zap"** &mdash; all users will
need to update their code.
* User-facing types and functions remain in the `zap` package. Code relevant
largely to extension authors is now in the `zapcore` package.
* The `zapcore.Core` type makes it easy for third-party packages to use zap's
internals but provide a different user-facing API.
* `Logger` is now a concrete type instead of an interface.
* A less verbose (though slower) logging API is included by default.
* Package-global loggers `L` and `S` are included.
* A human-friendly console encoder is included.
* A declarative config struct allows common logger configurations to be managed
as configuration instead of code.
* Sampling is more accurate, and doesn't depend on the standard library's shared
timer heap.
## v0.1.0-beta.1 (6 Feb 2017)
This is a minor version, tagged to allow users to pin to the pre-1.0 APIs and
upgrade at their leisure. Since this is the first tagged release, there are no
backward compatibility concerns and all functionality is new.
Early zap adopters should pin to the 0.1.x minor version until they're ready to
upgrade to the upcoming stable release.
[#316]: https://github.com/uber-go/zap/pull/316
[#309]: https://github.com/uber-go/zap/pull/309
[#317]: https://github.com/uber-go/zap/pull/317
[#321]: https://github.com/uber-go/zap/pull/321
[#325]: https://github.com/uber-go/zap/pull/325
[#333]: https://github.com/uber-go/zap/pull/333
[#326]: https://github.com/uber-go/zap/pull/326
[#300]: https://github.com/uber-go/zap/pull/300
[#339]: https://github.com/uber-go/zap/pull/339
[#307]: https://github.com/uber-go/zap/pull/307
[#353]: https://github.com/uber-go/zap/pull/353
[#311]: https://github.com/uber-go/zap/pull/311
[#366]: https://github.com/uber-go/zap/pull/366
[#364]: https://github.com/uber-go/zap/pull/364
[#371]: https://github.com/uber-go/zap/pull/371
[#362]: https://github.com/uber-go/zap/pull/362
[#369]: https://github.com/uber-go/zap/pull/369
[#347]: https://github.com/uber-go/zap/pull/347
[#373]: https://github.com/uber-go/zap/pull/373
[#348]: https://github.com/uber-go/zap/pull/348
[#327]: https://github.com/uber-go/zap/pull/327
[#376]: https://github.com/uber-go/zap/pull/376
[#346]: https://github.com/uber-go/zap/pull/346
[#365]: https://github.com/uber-go/zap/pull/365
[#372]: https://github.com/uber-go/zap/pull/372
[#385]: https://github.com/uber-go/zap/pull/385
[#396]: https://github.com/uber-go/zap/pull/396
[#386]: https://github.com/uber-go/zap/pull/386
[#402]: https://github.com/uber-go/zap/pull/402
[#415]: https://github.com/uber-go/zap/pull/415
[#416]: https://github.com/uber-go/zap/pull/416
[#424]: https://github.com/uber-go/zap/pull/424
[#425]: https://github.com/uber-go/zap/pull/425
[#431]: https://github.com/uber-go/zap/pull/431
[#435]: https://github.com/uber-go/zap/pull/435
[#444]: https://github.com/uber-go/zap/pull/444
[#477]: https://github.com/uber-go/zap/pull/477
[#465]: https://github.com/uber-go/zap/pull/465
[#460]: https://github.com/uber-go/zap/pull/460
[#470]: https://github.com/uber-go/zap/pull/470
[#487]: https://github.com/uber-go/zap/pull/487
[#490]: https://github.com/uber-go/zap/pull/490
[#491]: https://github.com/uber-go/zap/pull/491
[#504]: https://github.com/uber-go/zap/pull/504
[#508]: https://github.com/uber-go/zap/pull/508
[#518]: https://github.com/uber-go/zap/pull/518
[#577]: https://github.com/uber-go/zap/pull/577
[#574]: https://github.com/uber-go/zap/pull/574
+75
View File
@@ -0,0 +1,75 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age,
body size, disability, ethnicity, gender identity and expression, level of
experience, nationality, personal appearance, race, religion, or sexual
identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an
appointed representative at an online or offline event. Representation of a
project may be further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at oss-conduct@uber.com. The project
team will review and investigate all complaints, and will respond in a way
that it deems appropriate to the circumstances. The project team is obligated
to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 1.4, available at
[http://contributor-covenant.org/version/1/4][version].
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/
+81
View File
@@ -0,0 +1,81 @@
# Contributing
We'd love your help making zap the very best structured logging library in Go!
If you'd like to add new exported APIs, please [open an issue][open-issue]
describing your proposal &mdash; discussing API changes ahead of time makes
pull request review much smoother. In your issue, pull request, and any other
communications, please remember to treat your fellow contributors with
respect! We take our [code of conduct](CODE_OF_CONDUCT.md) seriously.
Note that you'll need to sign [Uber's Contributor License Agreement][cla]
before we can accept any of your contributions. If necessary, a bot will remind
you to accept the CLA when you open your pull request.
## Setup
[Fork][fork], then clone the repository:
```
mkdir -p $GOPATH/src/go.uber.org
cd $GOPATH/src/go.uber.org
git clone git@github.com:your_github_username/zap.git
cd zap
git remote add upstream https://github.com/uber-go/zap.git
git fetch upstream
```
Install zap's dependencies:
```
make dependencies
```
Make sure that the tests and the linters pass:
```
make test
make lint
```
If you're not using the minor version of Go specified in the Makefile's
`LINTABLE_MINOR_VERSIONS` variable, `make lint` doesn't do anything. This is
fine, but it means that you'll only discover lint failures after you open your
pull request.
## Making Changes
Start by creating a new branch for your changes:
```
cd $GOPATH/src/go.uber.org/zap
git checkout master
git fetch upstream
git rebase upstream/master
git checkout -b cool_new_feature
```
Make your changes, then ensure that `make lint` and `make test` still pass. If
you're satisfied with your changes, push them to your fork.
```
git push origin cool_new_feature
```
Then use the GitHub UI to open a pull request.
At this point, you're waiting on us to review your changes. We *try* to respond
to issues and pull requests within a few business days, and we may suggest some
improvements or alternatives. Once your changes are approved, one of the
project maintainers will merge them.
We're much more likely to approve your changes if you:
* Add tests for new functionality.
* Write a [good commit message][commit-message].
* Maintain backward compatibility.
[fork]: https://github.com/uber-go/zap/fork
[open-issue]: https://github.com/uber-go/zap/issues/new
[cla]: https://cla-assistant.io/uber-go/zap
[commit-message]: http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html
+154
View File
@@ -0,0 +1,154 @@
# Frequently Asked Questions
## Design
### Why spend so much effort on logger performance?
Of course, most applications won't notice the impact of a slow logger: they
already take tens or hundreds of milliseconds for each operation, so an extra
millisecond doesn't matter.
On the other hand, why *not* make structured logging fast? The `SugaredLogger`
isn't any harder to use than other logging packages, and the `Logger` makes
structured logging possible in performance-sensitive contexts. Across a fleet
of Go microservices, making each application even slightly more efficient adds
up quickly.
### Why aren't `Logger` and `SugaredLogger` interfaces?
Unlike the familiar `io.Writer` and `http.Handler`, `Logger` and
`SugaredLogger` interfaces would include *many* methods. As [Rob Pike points
out][go-proverbs], "The bigger the interface, the weaker the abstraction."
Interfaces are also rigid &mdash; *any* change requires releasing a new major
version, since it breaks all third-party implementations.
Making the `Logger` and `SugaredLogger` concrete types doesn't sacrifice much
abstraction, and it lets us add methods without introducing breaking changes.
Your applications should define and depend upon an interface that includes
just the methods you use.
### Why sample application logs?
Applications often experience runs of errors, either because of a bug or
because of a misbehaving user. Logging errors is usually a good idea, but it
can easily make this bad situation worse: not only is your application coping
with a flood of errors, it's also spending extra CPU cycles and I/O logging
those errors. Since writes are typically serialized, logging limits throughput
when you need it most.
Sampling fixes this problem by dropping repetitive log entries. Under normal
conditions, your application writes out every entry. When similar entries are
logged hundreds or thousands of times each second, though, zap begins dropping
duplicates to preserve throughput.
### Why do the structured logging APIs take a message in addition to fields?
Subjectively, we find it helpful to accompany structured context with a brief
description. This isn't critical during development, but it makes debugging
and operating unfamiliar systems much easier.
More concretely, zap's sampling algorithm uses the message to identify
duplicate entries. In our experience, this is a practical middle ground
between random sampling (which often drops the exact entry that you need while
debugging) and hashing the complete entry (which is prohibitively expensive).
### Why include package-global loggers?
Since so many other logging packages include a global logger, many
applications aren't designed to accept loggers as explicit parameters.
Changing function signatures is often a breaking change, so zap includes
global loggers to simplify migration.
Avoid them where possible.
### Why include dedicated Panic and Fatal log levels?
In general, application code should handle errors gracefully instead of using
`panic` or `os.Exit`. However, every rule has exceptions, and it's common to
crash when an error is truly unrecoverable. To avoid losing any information
&mdash; especially the reason for the crash &mdash; the logger must flush any
buffered entries before the process exits.
Zap makes this easy by offering `Panic` and `Fatal` logging methods that
automatically flush before exiting. Of course, this doesn't guarantee that
logs will never be lost, but it eliminates a common error.
See the discussion in uber-go/zap#207 for more details.
### What's `DPanic`?
`DPanic` stands for "panic in development." In development, it logs at
`PanicLevel`; otherwise, it logs at `ErrorLevel`. `DPanic` makes it easier to
catch errors that are theoretically possible, but shouldn't actually happen,
*without* crashing in production.
If you've ever written code like this, you need `DPanic`:
```go
if err != nil {
panic(fmt.Sprintf("shouldn't ever get here: %v", err))
}
```
## Installation
### What does the error `expects import "go.uber.org/zap"` mean?
Either zap was installed incorrectly or you're referencing the wrong package
name in your code.
Zap's source code happens to be hosted on GitHub, but the [import
path][import-path] is `go.uber.org/zap`. This gives us, the project
maintainers, the freedom to move the source code if necessary. However, it
means that you need to take a little care when installing and using the
package.
If you follow two simple rules, everything should work: install zap with `go
get -u go.uber.org/zap`, and always import it in your code with `import
"go.uber.org/zap"`. Your code shouldn't contain *any* references to
`github.com/uber-go/zap`.
## Usage
### Does zap support log rotation?
Zap doesn't natively support rotating log files, since we prefer to leave this
to an external program like `logrotate`.
However, it's easy to integrate a log rotation package like
[`gopkg.in/natefinch/lumberjack.v2`][lumberjack] as a `zapcore.WriteSyncer`.
```go
// lumberjack.Logger is already safe for concurrent use, so we don't need to
// lock it.
w := zapcore.AddSync(&lumberjack.Logger{
Filename: "/var/log/myapp/foo.log",
MaxSize: 500, // megabytes
MaxBackups: 3,
MaxAge: 28, // days
})
core := zapcore.NewCore(
zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
w,
zap.InfoLevel,
)
logger := zap.New(core)
```
## Extensions
We'd love to support every logging need within zap itself, but we're only
familiar with a handful of log ingestion systems, flag-parsing packages, and
the like. Rather than merging code that we can't effectively debug and
support, we'd rather grow an ecosystem of zap extensions.
We're aware of the following extensions, but haven't used them ourselves:
| Package | Integration |
| --- | --- |
| `github.com/tchap/zapext` | Sentry, syslog |
| `github.com/fgrosse/zaptest` | Ginkgo |
[go-proverbs]: https://go-proverbs.github.io/
[import-path]: https://golang.org/cmd/go/#hdr-Remote_import_paths
[lumberjack]: https://godoc.org/gopkg.in/natefinch/lumberjack.v2
+19
View File
@@ -0,0 +1,19 @@
Copyright (c) 2016-2017 Uber Technologies, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
+76
View File
@@ -0,0 +1,76 @@
export GO15VENDOREXPERIMENT=1
BENCH_FLAGS ?= -cpuprofile=cpu.pprof -memprofile=mem.pprof -benchmem
PKGS ?= $(shell glide novendor)
# Many Go tools take file globs or directories as arguments instead of packages.
PKG_FILES ?= *.go zapcore benchmarks buffer zapgrpc zaptest zaptest/observer internal/bufferpool internal/exit internal/color internal/ztest
# The linting tools evolve with each Go version, so run them only on the latest
# stable release.
GO_VERSION := $(shell go version | cut -d " " -f 3)
GO_MINOR_VERSION := $(word 2,$(subst ., ,$(GO_VERSION)))
LINTABLE_MINOR_VERSIONS := 10
ifneq ($(filter $(LINTABLE_MINOR_VERSIONS),$(GO_MINOR_VERSION)),)
SHOULD_LINT := true
endif
.PHONY: all
all: lint test
.PHONY: dependencies
dependencies:
@echo "Installing Glide and locked dependencies..."
glide --version || go get -u -f github.com/Masterminds/glide
glide install
@echo "Installing test dependencies..."
go install ./vendor/github.com/axw/gocov/gocov
go install ./vendor/github.com/mattn/goveralls
ifdef SHOULD_LINT
@echo "Installing golint..."
go install ./vendor/github.com/golang/lint/golint
else
@echo "Not installing golint, since we don't expect to lint on" $(GO_VERSION)
endif
# Disable printf-like invocation checking due to testify.assert.Error()
VET_RULES := -printf=false
.PHONY: lint
lint:
ifdef SHOULD_LINT
@rm -rf lint.log
@echo "Checking formatting..."
@gofmt -d -s $(PKG_FILES) 2>&1 | tee lint.log
@echo "Installing test dependencies for vet..."
@go test -i $(PKGS)
@echo "Checking vet..."
@$(foreach dir,$(PKG_FILES),go tool vet $(VET_RULES) $(dir) 2>&1 | tee -a lint.log;)
@echo "Checking lint..."
@$(foreach dir,$(PKGS),golint $(dir) 2>&1 | tee -a lint.log;)
@echo "Checking for unresolved FIXMEs..."
@git grep -i fixme | grep -v -e vendor -e Makefile | tee -a lint.log
@echo "Checking for license headers..."
@./check_license.sh | tee -a lint.log
@[ ! -s lint.log ]
else
@echo "Skipping linters on" $(GO_VERSION)
endif
.PHONY: test
test:
go test -race $(PKGS)
.PHONY: cover
cover:
./scripts/cover.sh $(PKGS)
.PHONY: bench
BENCH ?= .
bench:
@$(foreach pkg,$(PKGS),go test -bench=$(BENCH) -run="^$$" $(BENCH_FLAGS) $(pkg);)
.PHONY: updatereadme
updatereadme:
rm -f README.md
cat .readme.tmpl | go run internal/readme/readme.go > README.md
+136
View File
@@ -0,0 +1,136 @@
# :zap: zap [![GoDoc][doc-img]][doc] [![Build Status][ci-img]][ci] [![Coverage Status][cov-img]][cov]
Blazing fast, structured, leveled logging in Go.
## Installation
`go get -u go.uber.org/zap`
Note that zap only supports the two most recent minor versions of Go.
## Quick Start
In contexts where performance is nice, but not critical, use the
`SugaredLogger`. It's 4-10x faster than than other structured logging
packages and includes both structured and `printf`-style APIs.
```go
logger, _ := zap.NewProduction()
defer logger.Sync() // flushes buffer, if any
sugar := logger.Sugar()
sugar.Infow("failed to fetch URL",
// Structured context as loosely typed key-value pairs.
"url", url,
"attempt", 3,
"backoff", time.Second,
)
sugar.Infof("Failed to fetch URL: %s", url)
```
When performance and type safety are critical, use the `Logger`. It's even
faster than the `SugaredLogger` and allocates far less, but it only supports
structured logging.
```go
logger, _ := zap.NewProduction()
defer logger.Sync()
logger.Info("failed to fetch URL",
// Structured context as strongly typed Field values.
zap.String("url", url),
zap.Int("attempt", 3),
zap.Duration("backoff", time.Second),
)
```
See the [documentation][doc] and [FAQ](FAQ.md) for more details.
## Performance
For applications that log in the hot path, reflection-based serialization and
string formatting are prohibitively expensive &mdash; they're CPU-intensive
and make many small allocations. Put differently, using `encoding/json` and
`fmt.Fprintf` to log tons of `interface{}`s makes your application slow.
Zap takes a different approach. It includes a reflection-free, zero-allocation
JSON encoder, and the base `Logger` strives to avoid serialization overhead
and allocations wherever possible. By building the high-level `SugaredLogger`
on that foundation, zap lets users *choose* when they need to count every
allocation and when they'd prefer a more familiar, loosely typed API.
As measured by its own [benchmarking suite][], not only is zap more performant
than comparable structured logging packages &mdash; it's also faster than the
standard library. Like all benchmarks, take these with a grain of salt.<sup
id="anchor-versions">[1](#footnote-versions)</sup>
Log a message and 10 fields:
| Package | Time | Objects Allocated |
| :--- | :---: | :---: |
| :zap: zap | 3131 ns/op | 5 allocs/op |
| :zap: zap (sugared) | 4173 ns/op | 21 allocs/op |
| zerolog | 16154 ns/op | 90 allocs/op |
| lion | 16341 ns/op | 111 allocs/op |
| go-kit | 17049 ns/op | 126 allocs/op |
| logrus | 23662 ns/op | 142 allocs/op |
| log15 | 36351 ns/op | 149 allocs/op |
| apex/log | 42530 ns/op | 126 allocs/op |
Log a message with a logger that already has 10 fields of context:
| Package | Time | Objects Allocated |
| :--- | :---: | :---: |
| :zap: zap | 380 ns/op | 0 allocs/op |
| :zap: zap (sugared) | 564 ns/op | 2 allocs/op |
| zerolog | 321 ns/op | 0 allocs/op |
| lion | 7092 ns/op | 39 allocs/op |
| go-kit | 20226 ns/op | 115 allocs/op |
| logrus | 22312 ns/op | 130 allocs/op |
| log15 | 28788 ns/op | 79 allocs/op |
| apex/log | 42063 ns/op | 115 allocs/op |
Log a static string, without any context or `printf`-style templating:
| Package | Time | Objects Allocated |
| :--- | :---: | :---: |
| :zap: zap | 361 ns/op | 0 allocs/op |
| :zap: zap (sugared) | 534 ns/op | 2 allocs/op |
| zerolog | 323 ns/op | 0 allocs/op |
| standard library | 575 ns/op | 2 allocs/op |
| go-kit | 922 ns/op | 13 allocs/op |
| lion | 1413 ns/op | 10 allocs/op |
| logrus | 2291 ns/op | 27 allocs/op |
| apex/log | 3690 ns/op | 11 allocs/op |
| log15 | 5954 ns/op | 26 allocs/op |
## Development Status: Stable
All APIs are finalized, and no breaking changes will be made in the 1.x series
of releases. Users of semver-aware dependency management systems should pin
zap to `^1`.
## Contributing
We encourage and support an active, healthy community of contributors &mdash;
including you! Details are in the [contribution guide](CONTRIBUTING.md) and
the [code of conduct](CODE_OF_CONDUCT.md). The zap maintainers keep an eye on
issues and pull requests, but you can also report any negative conduct to
oss-conduct@uber.com. That email list is a private, safe space; even the zap
maintainers don't have access, so don't hesitate to hold us to a high
standard.
<hr>
Released under the [MIT License](LICENSE.txt).
<sup id="footnote-versions">1</sup> In particular, keep in mind that we may be
benchmarking against slightly older versions of other packages. Versions are
pinned in zap's [glide.lock][] file. [↩](#anchor-versions)
[doc-img]: https://godoc.org/go.uber.org/zap?status.svg
[doc]: https://godoc.org/go.uber.org/zap
[ci-img]: https://travis-ci.org/uber-go/zap.svg?branch=master
[ci]: https://travis-ci.org/uber-go/zap
[cov-img]: https://codecov.io/gh/uber-go/zap/branch/master/graph/badge.svg
[cov]: https://codecov.io/gh/uber-go/zap
[benchmarking suite]: https://github.com/uber-go/zap/tree/master/benchmarks
[glide.lock]: https://github.com/uber-go/zap/blob/master/glide.lock
+320
View File
@@ -0,0 +1,320 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"time"
"go.uber.org/zap/zapcore"
)
// Array constructs a field with the given key and ArrayMarshaler. It provides
// a flexible, but still type-safe and efficient, way to add array-like types
// to the logging context. The struct's MarshalLogArray method is called lazily.
func Array(key string, val zapcore.ArrayMarshaler) Field {
return Field{Key: key, Type: zapcore.ArrayMarshalerType, Interface: val}
}
// Bools constructs a field that carries a slice of bools.
func Bools(key string, bs []bool) Field {
return Array(key, bools(bs))
}
// ByteStrings constructs a field that carries a slice of []byte, each of which
// must be UTF-8 encoded text.
func ByteStrings(key string, bss [][]byte) Field {
return Array(key, byteStringsArray(bss))
}
// Complex128s constructs a field that carries a slice of complex numbers.
func Complex128s(key string, nums []complex128) Field {
return Array(key, complex128s(nums))
}
// Complex64s constructs a field that carries a slice of complex numbers.
func Complex64s(key string, nums []complex64) Field {
return Array(key, complex64s(nums))
}
// Durations constructs a field that carries a slice of time.Durations.
func Durations(key string, ds []time.Duration) Field {
return Array(key, durations(ds))
}
// Float64s constructs a field that carries a slice of floats.
func Float64s(key string, nums []float64) Field {
return Array(key, float64s(nums))
}
// Float32s constructs a field that carries a slice of floats.
func Float32s(key string, nums []float32) Field {
return Array(key, float32s(nums))
}
// Ints constructs a field that carries a slice of integers.
func Ints(key string, nums []int) Field {
return Array(key, ints(nums))
}
// Int64s constructs a field that carries a slice of integers.
func Int64s(key string, nums []int64) Field {
return Array(key, int64s(nums))
}
// Int32s constructs a field that carries a slice of integers.
func Int32s(key string, nums []int32) Field {
return Array(key, int32s(nums))
}
// Int16s constructs a field that carries a slice of integers.
func Int16s(key string, nums []int16) Field {
return Array(key, int16s(nums))
}
// Int8s constructs a field that carries a slice of integers.
func Int8s(key string, nums []int8) Field {
return Array(key, int8s(nums))
}
// Strings constructs a field that carries a slice of strings.
func Strings(key string, ss []string) Field {
return Array(key, stringArray(ss))
}
// Times constructs a field that carries a slice of time.Times.
func Times(key string, ts []time.Time) Field {
return Array(key, times(ts))
}
// Uints constructs a field that carries a slice of unsigned integers.
func Uints(key string, nums []uint) Field {
return Array(key, uints(nums))
}
// Uint64s constructs a field that carries a slice of unsigned integers.
func Uint64s(key string, nums []uint64) Field {
return Array(key, uint64s(nums))
}
// Uint32s constructs a field that carries a slice of unsigned integers.
func Uint32s(key string, nums []uint32) Field {
return Array(key, uint32s(nums))
}
// Uint16s constructs a field that carries a slice of unsigned integers.
func Uint16s(key string, nums []uint16) Field {
return Array(key, uint16s(nums))
}
// Uint8s constructs a field that carries a slice of unsigned integers.
func Uint8s(key string, nums []uint8) Field {
return Array(key, uint8s(nums))
}
// Uintptrs constructs a field that carries a slice of pointer addresses.
func Uintptrs(key string, us []uintptr) Field {
return Array(key, uintptrs(us))
}
// Errors constructs a field that carries a slice of errors.
func Errors(key string, errs []error) Field {
return Array(key, errArray(errs))
}
type bools []bool
func (bs bools) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range bs {
arr.AppendBool(bs[i])
}
return nil
}
type byteStringsArray [][]byte
func (bss byteStringsArray) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range bss {
arr.AppendByteString(bss[i])
}
return nil
}
type complex128s []complex128
func (nums complex128s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendComplex128(nums[i])
}
return nil
}
type complex64s []complex64
func (nums complex64s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendComplex64(nums[i])
}
return nil
}
type durations []time.Duration
func (ds durations) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range ds {
arr.AppendDuration(ds[i])
}
return nil
}
type float64s []float64
func (nums float64s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendFloat64(nums[i])
}
return nil
}
type float32s []float32
func (nums float32s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendFloat32(nums[i])
}
return nil
}
type ints []int
func (nums ints) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendInt(nums[i])
}
return nil
}
type int64s []int64
func (nums int64s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendInt64(nums[i])
}
return nil
}
type int32s []int32
func (nums int32s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendInt32(nums[i])
}
return nil
}
type int16s []int16
func (nums int16s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendInt16(nums[i])
}
return nil
}
type int8s []int8
func (nums int8s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendInt8(nums[i])
}
return nil
}
type stringArray []string
func (ss stringArray) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range ss {
arr.AppendString(ss[i])
}
return nil
}
type times []time.Time
func (ts times) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range ts {
arr.AppendTime(ts[i])
}
return nil
}
type uints []uint
func (nums uints) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendUint(nums[i])
}
return nil
}
type uint64s []uint64
func (nums uint64s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendUint64(nums[i])
}
return nil
}
type uint32s []uint32
func (nums uint32s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendUint32(nums[i])
}
return nil
}
type uint16s []uint16
func (nums uint16s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendUint16(nums[i])
}
return nil
}
type uint8s []uint8
func (nums uint8s) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendUint8(nums[i])
}
return nil
}
type uintptrs []uintptr
func (nums uintptrs) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range nums {
arr.AppendUintptr(nums[i])
}
return nil
}
+106
View File
@@ -0,0 +1,106 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
// Package buffer provides a thin wrapper around a byte slice. Unlike the
// standard library's bytes.Buffer, it supports a portion of the strconv
// package's zero-allocation formatters.
package buffer // import "go.uber.org/zap/buffer"
import "strconv"
const _size = 1024 // by default, create 1 KiB buffers
// Buffer is a thin wrapper around a byte slice. It's intended to be pooled, so
// the only way to construct one is via a Pool.
type Buffer struct {
bs []byte
pool Pool
}
// AppendByte writes a single byte to the Buffer.
func (b *Buffer) AppendByte(v byte) {
b.bs = append(b.bs, v)
}
// AppendString writes a string to the Buffer.
func (b *Buffer) AppendString(s string) {
b.bs = append(b.bs, s...)
}
// AppendInt appends an integer to the underlying buffer (assuming base 10).
func (b *Buffer) AppendInt(i int64) {
b.bs = strconv.AppendInt(b.bs, i, 10)
}
// AppendUint appends an unsigned integer to the underlying buffer (assuming
// base 10).
func (b *Buffer) AppendUint(i uint64) {
b.bs = strconv.AppendUint(b.bs, i, 10)
}
// AppendBool appends a bool to the underlying buffer.
func (b *Buffer) AppendBool(v bool) {
b.bs = strconv.AppendBool(b.bs, v)
}
// AppendFloat appends a float to the underlying buffer. It doesn't quote NaN
// or +/- Inf.
func (b *Buffer) AppendFloat(f float64, bitSize int) {
b.bs = strconv.AppendFloat(b.bs, f, 'f', -1, bitSize)
}
// Len returns the length of the underlying byte slice.
func (b *Buffer) Len() int {
return len(b.bs)
}
// Cap returns the capacity of the underlying byte slice.
func (b *Buffer) Cap() int {
return cap(b.bs)
}
// Bytes returns a mutable reference to the underlying byte slice.
func (b *Buffer) Bytes() []byte {
return b.bs
}
// String returns a string copy of the underlying byte slice.
func (b *Buffer) String() string {
return string(b.bs)
}
// Reset resets the underlying byte slice. Subsequent writes re-use the slice's
// backing array.
func (b *Buffer) Reset() {
b.bs = b.bs[:0]
}
// Write implements io.Writer.
func (b *Buffer) Write(bs []byte) (int, error) {
b.bs = append(b.bs, bs...)
return len(bs), nil
}
// Free returns the Buffer to its Pool.
//
// Callers must not retain references to the Buffer after calling Free.
func (b *Buffer) Free() {
b.pool.put(b)
}
+49
View File
@@ -0,0 +1,49 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package buffer
import "sync"
// A Pool is a type-safe wrapper around a sync.Pool.
type Pool struct {
p *sync.Pool
}
// NewPool constructs a new Pool.
func NewPool() Pool {
return Pool{p: &sync.Pool{
New: func() interface{} {
return &Buffer{bs: make([]byte, 0, _size)}
},
}}
}
// Get retrieves a Buffer from the pool, creating one if necessary.
func (p Pool) Get() *Buffer {
buf := p.p.Get().(*Buffer)
buf.Reset()
buf.pool = p
return buf
}
func (p Pool) put(buf *Buffer) {
p.p.Put(buf)
}
Generated Vendored Executable
+17
View File
@@ -0,0 +1,17 @@
#!/bin/bash -e
ERROR_COUNT=0
while read -r file
do
case "$(head -1 "${file}")" in
*"Copyright (c) "*" Uber Technologies, Inc.")
# everything's cool
;;
*)
echo "$file is missing license header."
(( ERROR_COUNT++ ))
;;
esac
done < <(git ls-files "*\.go")
exit $ERROR_COUNT
+243
View File
@@ -0,0 +1,243 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"sort"
"time"
"go.uber.org/zap/zapcore"
)
// SamplingConfig sets a sampling strategy for the logger. Sampling caps the
// global CPU and I/O load that logging puts on your process while attempting
// to preserve a representative subset of your logs.
//
// Values configured here are per-second. See zapcore.NewSampler for details.
type SamplingConfig struct {
Initial int `json:"initial" yaml:"initial"`
Thereafter int `json:"thereafter" yaml:"thereafter"`
}
// Config offers a declarative way to construct a logger. It doesn't do
// anything that can't be done with New, Options, and the various
// zapcore.WriteSyncer and zapcore.Core wrappers, but it's a simpler way to
// toggle common options.
//
// Note that Config intentionally supports only the most common options. More
// unusual logging setups (logging to network connections or message queues,
// splitting output between multiple files, etc.) are possible, but require
// direct use of the zapcore package. For sample code, see the package-level
// BasicConfiguration and AdvancedConfiguration examples.
//
// For an example showing runtime log level changes, see the documentation for
// AtomicLevel.
type Config struct {
// Level is the minimum enabled logging level. Note that this is a dynamic
// level, so calling Config.Level.SetLevel will atomically change the log
// level of all loggers descended from this config.
Level AtomicLevel `json:"level" yaml:"level"`
// Development puts the logger in development mode, which changes the
// behavior of DPanicLevel and takes stacktraces more liberally.
Development bool `json:"development" yaml:"development"`
// DisableCaller stops annotating logs with the calling function's file
// name and line number. By default, all logs are annotated.
DisableCaller bool `json:"disableCaller" yaml:"disableCaller"`
// DisableStacktrace completely disables automatic stacktrace capturing. By
// default, stacktraces are captured for WarnLevel and above logs in
// development and ErrorLevel and above in production.
DisableStacktrace bool `json:"disableStacktrace" yaml:"disableStacktrace"`
// Sampling sets a sampling policy. A nil SamplingConfig disables sampling.
Sampling *SamplingConfig `json:"sampling" yaml:"sampling"`
// Encoding sets the logger's encoding. Valid values are "json" and
// "console", as well as any third-party encodings registered via
// RegisterEncoder.
Encoding string `json:"encoding" yaml:"encoding"`
// EncoderConfig sets options for the chosen encoder. See
// zapcore.EncoderConfig for details.
EncoderConfig zapcore.EncoderConfig `json:"encoderConfig" yaml:"encoderConfig"`
// OutputPaths is a list of paths to write logging output to. See Open for
// details.
OutputPaths []string `json:"outputPaths" yaml:"outputPaths"`
// ErrorOutputPaths is a list of paths to write internal logger errors to.
// The default is standard error.
//
// Note that this setting only affects internal errors; for sample code that
// sends error-level logs to a different location from info- and debug-level
// logs, see the package-level AdvancedConfiguration example.
ErrorOutputPaths []string `json:"errorOutputPaths" yaml:"errorOutputPaths"`
// InitialFields is a collection of fields to add to the root logger.
InitialFields map[string]interface{} `json:"initialFields" yaml:"initialFields"`
}
// NewProductionEncoderConfig returns an opinionated EncoderConfig for
// production environments.
func NewProductionEncoderConfig() zapcore.EncoderConfig {
return zapcore.EncoderConfig{
TimeKey: "ts",
LevelKey: "level",
NameKey: "logger",
CallerKey: "caller",
MessageKey: "msg",
StacktraceKey: "stacktrace",
LineEnding: zapcore.DefaultLineEnding,
EncodeLevel: zapcore.LowercaseLevelEncoder,
EncodeTime: zapcore.EpochTimeEncoder,
EncodeDuration: zapcore.SecondsDurationEncoder,
EncodeCaller: zapcore.ShortCallerEncoder,
}
}
// NewProductionConfig is a reasonable production logging configuration.
// Logging is enabled at InfoLevel and above.
//
// It uses a JSON encoder, writes to standard error, and enables sampling.
// Stacktraces are automatically included on logs of ErrorLevel and above.
func NewProductionConfig() Config {
return Config{
Level: NewAtomicLevelAt(InfoLevel),
Development: false,
Sampling: &SamplingConfig{
Initial: 100,
Thereafter: 100,
},
Encoding: "json",
EncoderConfig: NewProductionEncoderConfig(),
OutputPaths: []string{"stderr"},
ErrorOutputPaths: []string{"stderr"},
}
}
// NewDevelopmentEncoderConfig returns an opinionated EncoderConfig for
// development environments.
func NewDevelopmentEncoderConfig() zapcore.EncoderConfig {
return zapcore.EncoderConfig{
// Keys can be anything except the empty string.
TimeKey: "T",
LevelKey: "L",
NameKey: "N",
CallerKey: "C",
MessageKey: "M",
StacktraceKey: "S",
LineEnding: zapcore.DefaultLineEnding,
EncodeLevel: zapcore.CapitalLevelEncoder,
EncodeTime: zapcore.ISO8601TimeEncoder,
EncodeDuration: zapcore.StringDurationEncoder,
EncodeCaller: zapcore.ShortCallerEncoder,
}
}
// NewDevelopmentConfig is a reasonable development logging configuration.
// Logging is enabled at DebugLevel and above.
//
// It enables development mode (which makes DPanicLevel logs panic), uses a
// console encoder, writes to standard error, and disables sampling.
// Stacktraces are automatically included on logs of WarnLevel and above.
func NewDevelopmentConfig() Config {
return Config{
Level: NewAtomicLevelAt(DebugLevel),
Development: true,
Encoding: "console",
EncoderConfig: NewDevelopmentEncoderConfig(),
OutputPaths: []string{"stderr"},
ErrorOutputPaths: []string{"stderr"},
}
}
// Build constructs a logger from the Config and Options.
func (cfg Config) Build(opts ...Option) (*Logger, error) {
enc, err := cfg.buildEncoder()
if err != nil {
return nil, err
}
sink, errSink, err := cfg.openSinks()
if err != nil {
return nil, err
}
log := New(
zapcore.NewCore(enc, sink, cfg.Level),
cfg.buildOptions(errSink)...,
)
if len(opts) > 0 {
log = log.WithOptions(opts...)
}
return log, nil
}
func (cfg Config) buildOptions(errSink zapcore.WriteSyncer) []Option {
opts := []Option{ErrorOutput(errSink)}
if cfg.Development {
opts = append(opts, Development())
}
if !cfg.DisableCaller {
opts = append(opts, AddCaller())
}
stackLevel := ErrorLevel
if cfg.Development {
stackLevel = WarnLevel
}
if !cfg.DisableStacktrace {
opts = append(opts, AddStacktrace(stackLevel))
}
if cfg.Sampling != nil {
opts = append(opts, WrapCore(func(core zapcore.Core) zapcore.Core {
return zapcore.NewSampler(core, time.Second, int(cfg.Sampling.Initial), int(cfg.Sampling.Thereafter))
}))
}
if len(cfg.InitialFields) > 0 {
fs := make([]Field, 0, len(cfg.InitialFields))
keys := make([]string, 0, len(cfg.InitialFields))
for k := range cfg.InitialFields {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
fs = append(fs, Any(k, cfg.InitialFields[k]))
}
opts = append(opts, Fields(fs...))
}
return opts
}
func (cfg Config) openSinks() (zapcore.WriteSyncer, zapcore.WriteSyncer, error) {
sink, closeOut, err := Open(cfg.OutputPaths...)
if err != nil {
return nil, nil, err
}
errSink, _, err := Open(cfg.ErrorOutputPaths...)
if err != nil {
closeOut()
return nil, nil, err
}
return sink, errSink, nil
}
func (cfg Config) buildEncoder() (zapcore.Encoder, error) {
return newEncoder(cfg.Encoding, cfg.EncoderConfig)
}
+113
View File
@@ -0,0 +1,113 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
// Package zap provides fast, structured, leveled logging.
//
// For applications that log in the hot path, reflection-based serialization
// and string formatting are prohibitively expensive - they're CPU-intensive
// and make many small allocations. Put differently, using json.Marshal and
// fmt.Fprintf to log tons of interface{} makes your application slow.
//
// Zap takes a different approach. It includes a reflection-free,
// zero-allocation JSON encoder, and the base Logger strives to avoid
// serialization overhead and allocations wherever possible. By building the
// high-level SugaredLogger on that foundation, zap lets users choose when
// they need to count every allocation and when they'd prefer a more familiar,
// loosely typed API.
//
// Choosing a Logger
//
// In contexts where performance is nice, but not critical, use the
// SugaredLogger. It's 4-10x faster than other structured logging packages and
// supports both structured and printf-style logging. Like log15 and go-kit,
// the SugaredLogger's structured logging APIs are loosely typed and accept a
// variadic number of key-value pairs. (For more advanced use cases, they also
// accept strongly typed fields - see the SugaredLogger.With documentation for
// details.)
// sugar := zap.NewExample().Sugar()
// defer sugar.Sync()
// sugar.Infow("failed to fetch URL",
// "url", "http://example.com",
// "attempt", 3,
// "backoff", time.Second,
// )
// sugar.Printf("failed to fetch URL: %s", "http://example.com")
//
// By default, loggers are unbuffered. However, since zap's low-level APIs
// allow buffering, calling Sync before letting your process exit is a good
// habit.
//
// In the rare contexts where every microsecond and every allocation matter,
// use the Logger. It's even faster than the SugaredLogger and allocates far
// less, but it only supports strongly-typed, structured logging.
// logger := zap.NewExample()
// defer logger.Sync()
// logger.Info("failed to fetch URL",
// zap.String("url", "http://example.com"),
// zap.Int("attempt", 3),
// zap.Duration("backoff", time.Second),
// )
//
// Choosing between the Logger and SugaredLogger doesn't need to be an
// application-wide decision: converting between the two is simple and
// inexpensive.
// logger := zap.NewExample()
// defer logger.Sync()
// sugar := logger.Sugar()
// plain := sugar.Desugar()
//
// Configuring Zap
//
// The simplest way to build a Logger is to use zap's opinionated presets:
// NewExample, NewProduction, and NewDevelopment. These presets build a logger
// with a single function call:
// logger, err := zap.NewProduction()
// if err != nil {
// log.Fatalf("can't initialize zap logger: %v", err)
// }
// defer logger.Sync()
//
// Presets are fine for small projects, but larger projects and organizations
// naturally require a bit more customization. For most users, zap's Config
// struct strikes the right balance between flexibility and convenience. See
// the package-level BasicConfiguration example for sample code.
//
// More unusual configurations (splitting output between files, sending logs
// to a message queue, etc.) are possible, but require direct use of
// go.uber.org/zap/zapcore. See the package-level AdvancedConfiguration
// example for sample code.
//
// Extending Zap
//
// The zap package itself is a relatively thin wrapper around the interfaces
// in go.uber.org/zap/zapcore. Extending zap to support a new encoding (e.g.,
// BSON), a new log sink (e.g., Kafka), or something more exotic (perhaps an
// exception aggregation service, like Sentry or Rollbar) typically requires
// implementing the zapcore.Encoder, zapcore.WriteSyncer, or zapcore.Core
// interfaces. See the zapcore documentation for details.
//
// Similarly, package authors can use the high-performance Encoder and Core
// implementations in the zapcore package to build their own loggers.
//
// Frequently Asked Questions
//
// An FAQ covering everything from installation errors to design decisions is
// available at https://github.com/uber-go/zap/blob/master/FAQ.md.
package zap // import "go.uber.org/zap"
+75
View File
@@ -0,0 +1,75 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"errors"
"fmt"
"sync"
"go.uber.org/zap/zapcore"
)
var (
errNoEncoderNameSpecified = errors.New("no encoder name specified")
_encoderNameToConstructor = map[string]func(zapcore.EncoderConfig) (zapcore.Encoder, error){
"console": func(encoderConfig zapcore.EncoderConfig) (zapcore.Encoder, error) {
return zapcore.NewConsoleEncoder(encoderConfig), nil
},
"json": func(encoderConfig zapcore.EncoderConfig) (zapcore.Encoder, error) {
return zapcore.NewJSONEncoder(encoderConfig), nil
},
}
_encoderMutex sync.RWMutex
)
// RegisterEncoder registers an encoder constructor, which the Config struct
// can then reference. By default, the "json" and "console" encoders are
// registered.
//
// Attempting to register an encoder whose name is already taken returns an
// error.
func RegisterEncoder(name string, constructor func(zapcore.EncoderConfig) (zapcore.Encoder, error)) error {
_encoderMutex.Lock()
defer _encoderMutex.Unlock()
if name == "" {
return errNoEncoderNameSpecified
}
if _, ok := _encoderNameToConstructor[name]; ok {
return fmt.Errorf("encoder already registered for name %q", name)
}
_encoderNameToConstructor[name] = constructor
return nil
}
func newEncoder(name string, encoderConfig zapcore.EncoderConfig) (zapcore.Encoder, error) {
_encoderMutex.RLock()
defer _encoderMutex.RUnlock()
if name == "" {
return nil, errNoEncoderNameSpecified
}
constructor, ok := _encoderNameToConstructor[name]
if !ok {
return nil, fmt.Errorf("no encoder registered for name %q", name)
}
return constructor(encoderConfig)
}
+80
View File
@@ -0,0 +1,80 @@
// Copyright (c) 2017 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"sync"
"go.uber.org/zap/zapcore"
)
var _errArrayElemPool = sync.Pool{New: func() interface{} {
return &errArrayElem{}
}}
// Error is shorthand for the common idiom NamedError("error", err).
func Error(err error) Field {
return NamedError("error", err)
}
// NamedError constructs a field that lazily stores err.Error() under the
// provided key. Errors which also implement fmt.Formatter (like those produced
// by github.com/pkg/errors) will also have their verbose representation stored
// under key+"Verbose". If passed a nil error, the field is a no-op.
//
// For the common case in which the key is simply "error", the Error function
// is shorter and less repetitive.
func NamedError(key string, err error) Field {
if err == nil {
return Skip()
}
return Field{Key: key, Type: zapcore.ErrorType, Interface: err}
}
type errArray []error
func (errs errArray) MarshalLogArray(arr zapcore.ArrayEncoder) error {
for i := range errs {
if errs[i] == nil {
continue
}
// To represent each error as an object with an "error" attribute and
// potentially an "errorVerbose" attribute, we need to wrap it in a
// type that implements LogObjectMarshaler. To prevent this from
// allocating, pool the wrapper type.
elem := _errArrayElemPool.Get().(*errArrayElem)
elem.error = errs[i]
arr.AppendObject(elem)
elem.error = nil
_errArrayElemPool.Put(elem)
}
return nil
}
type errArrayElem struct {
error
}
func (e *errArrayElem) MarshalLogObject(enc zapcore.ObjectEncoder) error {
// Re-use the error field's logic, which supports non-standard error types.
Error(e.error).AddTo(enc)
return nil
}
+310
View File
@@ -0,0 +1,310 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"fmt"
"math"
"time"
"go.uber.org/zap/zapcore"
)
// Field is an alias for Field. Aliasing this type dramatically
// improves the navigability of this package's API documentation.
type Field = zapcore.Field
// Skip constructs a no-op field, which is often useful when handling invalid
// inputs in other Field constructors.
func Skip() Field {
return Field{Type: zapcore.SkipType}
}
// Binary constructs a field that carries an opaque binary blob.
//
// Binary data is serialized in an encoding-appropriate format. For example,
// zap's JSON encoder base64-encodes binary blobs. To log UTF-8 encoded text,
// use ByteString.
func Binary(key string, val []byte) Field {
return Field{Key: key, Type: zapcore.BinaryType, Interface: val}
}
// Bool constructs a field that carries a bool.
func Bool(key string, val bool) Field {
var ival int64
if val {
ival = 1
}
return Field{Key: key, Type: zapcore.BoolType, Integer: ival}
}
// ByteString constructs a field that carries UTF-8 encoded text as a []byte.
// To log opaque binary blobs (which aren't necessarily valid UTF-8), use
// Binary.
func ByteString(key string, val []byte) Field {
return Field{Key: key, Type: zapcore.ByteStringType, Interface: val}
}
// Complex128 constructs a field that carries a complex number. Unlike most
// numeric fields, this costs an allocation (to convert the complex128 to
// interface{}).
func Complex128(key string, val complex128) Field {
return Field{Key: key, Type: zapcore.Complex128Type, Interface: val}
}
// Complex64 constructs a field that carries a complex number. Unlike most
// numeric fields, this costs an allocation (to convert the complex64 to
// interface{}).
func Complex64(key string, val complex64) Field {
return Field{Key: key, Type: zapcore.Complex64Type, Interface: val}
}
// Float64 constructs a field that carries a float64. The way the
// floating-point value is represented is encoder-dependent, so marshaling is
// necessarily lazy.
func Float64(key string, val float64) Field {
return Field{Key: key, Type: zapcore.Float64Type, Integer: int64(math.Float64bits(val))}
}
// Float32 constructs a field that carries a float32. The way the
// floating-point value is represented is encoder-dependent, so marshaling is
// necessarily lazy.
func Float32(key string, val float32) Field {
return Field{Key: key, Type: zapcore.Float32Type, Integer: int64(math.Float32bits(val))}
}
// Int constructs a field with the given key and value.
func Int(key string, val int) Field {
return Int64(key, int64(val))
}
// Int64 constructs a field with the given key and value.
func Int64(key string, val int64) Field {
return Field{Key: key, Type: zapcore.Int64Type, Integer: val}
}
// Int32 constructs a field with the given key and value.
func Int32(key string, val int32) Field {
return Field{Key: key, Type: zapcore.Int32Type, Integer: int64(val)}
}
// Int16 constructs a field with the given key and value.
func Int16(key string, val int16) Field {
return Field{Key: key, Type: zapcore.Int16Type, Integer: int64(val)}
}
// Int8 constructs a field with the given key and value.
func Int8(key string, val int8) Field {
return Field{Key: key, Type: zapcore.Int8Type, Integer: int64(val)}
}
// String constructs a field with the given key and value.
func String(key string, val string) Field {
return Field{Key: key, Type: zapcore.StringType, String: val}
}
// Uint constructs a field with the given key and value.
func Uint(key string, val uint) Field {
return Uint64(key, uint64(val))
}
// Uint64 constructs a field with the given key and value.
func Uint64(key string, val uint64) Field {
return Field{Key: key, Type: zapcore.Uint64Type, Integer: int64(val)}
}
// Uint32 constructs a field with the given key and value.
func Uint32(key string, val uint32) Field {
return Field{Key: key, Type: zapcore.Uint32Type, Integer: int64(val)}
}
// Uint16 constructs a field with the given key and value.
func Uint16(key string, val uint16) Field {
return Field{Key: key, Type: zapcore.Uint16Type, Integer: int64(val)}
}
// Uint8 constructs a field with the given key and value.
func Uint8(key string, val uint8) Field {
return Field{Key: key, Type: zapcore.Uint8Type, Integer: int64(val)}
}
// Uintptr constructs a field with the given key and value.
func Uintptr(key string, val uintptr) Field {
return Field{Key: key, Type: zapcore.UintptrType, Integer: int64(val)}
}
// Reflect constructs a field with the given key and an arbitrary object. It uses
// an encoding-appropriate, reflection-based function to lazily serialize nearly
// any object into the logging context, but it's relatively slow and
// allocation-heavy. Outside tests, Any is always a better choice.
//
// If encoding fails (e.g., trying to serialize a map[int]string to JSON), Reflect
// includes the error message in the final log output.
func Reflect(key string, val interface{}) Field {
return Field{Key: key, Type: zapcore.ReflectType, Interface: val}
}
// Namespace creates a named, isolated scope within the logger's context. All
// subsequent fields will be added to the new namespace.
//
// This helps prevent key collisions when injecting loggers into sub-components
// or third-party libraries.
func Namespace(key string) Field {
return Field{Key: key, Type: zapcore.NamespaceType}
}
// Stringer constructs a field with the given key and the output of the value's
// String method. The Stringer's String method is called lazily.
func Stringer(key string, val fmt.Stringer) Field {
return Field{Key: key, Type: zapcore.StringerType, Interface: val}
}
// Time constructs a Field with the given key and value. The encoder
// controls how the time is serialized.
func Time(key string, val time.Time) Field {
return Field{Key: key, Type: zapcore.TimeType, Integer: val.UnixNano(), Interface: val.Location()}
}
// Stack constructs a field that stores a stacktrace of the current goroutine
// under provided key. Keep in mind that taking a stacktrace is eager and
// expensive (relatively speaking); this function both makes an allocation and
// takes about two microseconds.
func Stack(key string) Field {
// Returning the stacktrace as a string costs an allocation, but saves us
// from expanding the zapcore.Field union struct to include a byte slice. Since
// taking a stacktrace is already so expensive (~10us), the extra allocation
// is okay.
return String(key, takeStacktrace())
}
// Duration constructs a field with the given key and value. The encoder
// controls how the duration is serialized.
func Duration(key string, val time.Duration) Field {
return Field{Key: key, Type: zapcore.DurationType, Integer: int64(val)}
}
// Object constructs a field with the given key and ObjectMarshaler. It
// provides a flexible, but still type-safe and efficient, way to add map- or
// struct-like user-defined types to the logging context. The struct's
// MarshalLogObject method is called lazily.
func Object(key string, val zapcore.ObjectMarshaler) Field {
return Field{Key: key, Type: zapcore.ObjectMarshalerType, Interface: val}
}
// Any takes a key and an arbitrary value and chooses the best way to represent
// them as a field, falling back to a reflection-based approach only if
// necessary.
//
// Since byte/uint8 and rune/int32 are aliases, Any can't differentiate between
// them. To minimize surprises, []byte values are treated as binary blobs, byte
// values are treated as uint8, and runes are always treated as integers.
func Any(key string, value interface{}) Field {
switch val := value.(type) {
case zapcore.ObjectMarshaler:
return Object(key, val)
case zapcore.ArrayMarshaler:
return Array(key, val)
case bool:
return Bool(key, val)
case []bool:
return Bools(key, val)
case complex128:
return Complex128(key, val)
case []complex128:
return Complex128s(key, val)
case complex64:
return Complex64(key, val)
case []complex64:
return Complex64s(key, val)
case float64:
return Float64(key, val)
case []float64:
return Float64s(key, val)
case float32:
return Float32(key, val)
case []float32:
return Float32s(key, val)
case int:
return Int(key, val)
case []int:
return Ints(key, val)
case int64:
return Int64(key, val)
case []int64:
return Int64s(key, val)
case int32:
return Int32(key, val)
case []int32:
return Int32s(key, val)
case int16:
return Int16(key, val)
case []int16:
return Int16s(key, val)
case int8:
return Int8(key, val)
case []int8:
return Int8s(key, val)
case string:
return String(key, val)
case []string:
return Strings(key, val)
case uint:
return Uint(key, val)
case []uint:
return Uints(key, val)
case uint64:
return Uint64(key, val)
case []uint64:
return Uint64s(key, val)
case uint32:
return Uint32(key, val)
case []uint32:
return Uint32s(key, val)
case uint16:
return Uint16(key, val)
case []uint16:
return Uint16s(key, val)
case uint8:
return Uint8(key, val)
case []byte:
return Binary(key, val)
case uintptr:
return Uintptr(key, val)
case []uintptr:
return Uintptrs(key, val)
case time.Time:
return Time(key, val)
case []time.Time:
return Times(key, val)
case time.Duration:
return Duration(key, val)
case []time.Duration:
return Durations(key, val)
case error:
return NamedError(key, val)
case []error:
return Errors(key, val)
case fmt.Stringer:
return Stringer(key, val)
default:
return Reflect(key, val)
}
}
+39
View File
@@ -0,0 +1,39 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"flag"
"go.uber.org/zap/zapcore"
)
// LevelFlag uses the standard library's flag.Var to declare a global flag
// with the specified name, default, and usage guidance. The returned value is
// a pointer to the value of the flag.
//
// If you don't want to use the flag package's global state, you can use any
// non-nil *Level as a flag.Value with your own *flag.FlagSet.
func LevelFlag(name string, defaultLevel zapcore.Level, usage string) *zapcore.Level {
lvl := defaultLevel
flag.Var(&lvl, name, usage)
return &lvl
}
+76
View File
@@ -0,0 +1,76 @@
hash: f073ba522c06c88ea3075bde32a8aaf0969a840a66cab6318a0897d141ffee92
updated: 2017-07-22T18:06:49.598185334-07:00
imports:
- name: go.uber.org/atomic
version: 4e336646b2ef9fc6e47be8e21594178f98e5ebcf
- name: go.uber.org/multierr
version: 3c4937480c32f4c13a875a1829af76c98ca3d40a
testImports:
- name: github.com/apex/log
version: d9b960447bfa720077b2da653cc79e533455b499
subpackages:
- handlers/json
- name: github.com/axw/gocov
version: 3a69a0d2a4ef1f263e2d92b041a69593d6964fe8
subpackages:
- gocov
- name: github.com/davecgh/go-spew
version: 04cdfd42973bb9c8589fd6a731800cf222fde1a9
subpackages:
- spew
- name: github.com/fatih/color
version: 62e9147c64a1ed519147b62a56a14e83e2be02c1
- name: github.com/go-kit/kit
version: e10f5bf035be9af21fd5b2fb4469d5716c6ab07d
subpackages:
- log
- name: github.com/go-logfmt/logfmt
version: 390ab7935ee28ec6b286364bba9b4dd6410cb3d5
- name: github.com/go-stack/stack
version: 54be5f394ed2c3e19dac9134a40a95ba5a017f7b
- name: github.com/golang/lint
version: c5fb716d6688a859aae56d26d3e6070808df29f7
subpackages:
- golint
- name: github.com/kr/logfmt
version: b84e30acd515aadc4b783ad4ff83aff3299bdfe0
- name: github.com/mattn/go-colorable
version: 3fa8c76f9daed4067e4a806fb7e4dc86455c6d6a
- name: github.com/mattn/go-isatty
version: fc9e8d8ef48496124e79ae0df75490096eccf6fe
- name: github.com/mattn/goveralls
version: 6efce81852ad1b7567c17ad71b03aeccc9dd9ae0
- name: github.com/pborman/uuid
version: e790cca94e6cc75c7064b1332e63811d4aae1a53
- name: github.com/pkg/errors
version: 645ef00459ed84a119197bfb8d8205042c6df63d
- name: github.com/pmezard/go-difflib
version: d8ed2627bdf02c080bf22230dbb337003b7aba2d
subpackages:
- difflib
- name: github.com/rs/zerolog
version: eed4c2b94d945e0b2456ad6aa518a443986b5f22
- name: github.com/satori/go.uuid
version: 5bf94b69c6b68ee1b541973bb8e1144db23a194b
- name: github.com/sirupsen/logrus
version: 7dd06bf38e1e13df288d471a57d5adbac106be9e
- name: github.com/stretchr/testify
version: f6abca593680b2315d2075e0f5e2a9751e3f431a
subpackages:
- assert
- require
- name: go.pedge.io/lion
version: 87958e8713f1fa138d993087133b97e976642159
- name: golang.org/x/sys
version: c4489faa6e5ab84c0ef40d6ee878f7a030281f0f
subpackages:
- unix
- name: golang.org/x/tools
version: 496819729719f9d07692195e0a94d6edd2251389
subpackages:
- cover
- name: gopkg.in/inconshreveable/log15.v2
version: b105bd37f74e5d9dc7b6ad7806715c7a2b83fd3f
subpackages:
- stack
- term
+35
View File
@@ -0,0 +1,35 @@
package: go.uber.org/zap
license: MIT
import:
- package: go.uber.org/atomic
version: ^1
- package: go.uber.org/multierr
version: ^1
testImport:
- package: github.com/satori/go.uuid
- package: github.com/sirupsen/logrus
- package: github.com/apex/log
subpackages:
- handlers/json
- package: github.com/go-kit/kit
subpackages:
- log
- package: github.com/stretchr/testify
subpackages:
- assert
- require
- package: gopkg.in/inconshreveable/log15.v2
- package: github.com/mattn/goveralls
- package: github.com/pborman/uuid
- package: github.com/pkg/errors
- package: go.pedge.io/lion
- package: github.com/rs/zerolog
- package: golang.org/x/tools
subpackages:
- cover
- package: github.com/golang/lint
subpackages:
- golint
- package: github.com/axw/gocov
subpackages:
- gocov
+169
View File
@@ -0,0 +1,169 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"bytes"
"fmt"
"log"
"os"
"sync"
"go.uber.org/zap/zapcore"
)
const (
_stdLogDefaultDepth = 2
_loggerWriterDepth = 2
_programmerErrorTemplate = "You've found a bug in zap! Please file a bug at " +
"https://github.com/uber-go/zap/issues/new and reference this error: %v"
)
var (
_globalMu sync.RWMutex
_globalL = NewNop()
_globalS = _globalL.Sugar()
)
// L returns the global Logger, which can be reconfigured with ReplaceGlobals.
// It's safe for concurrent use.
func L() *Logger {
_globalMu.RLock()
l := _globalL
_globalMu.RUnlock()
return l
}
// S returns the global SugaredLogger, which can be reconfigured with
// ReplaceGlobals. It's safe for concurrent use.
func S() *SugaredLogger {
_globalMu.RLock()
s := _globalS
_globalMu.RUnlock()
return s
}
// ReplaceGlobals replaces the global Logger and SugaredLogger, and returns a
// function to restore the original values. It's safe for concurrent use.
func ReplaceGlobals(logger *Logger) func() {
_globalMu.Lock()
prev := _globalL
_globalL = logger
_globalS = logger.Sugar()
_globalMu.Unlock()
return func() { ReplaceGlobals(prev) }
}
// NewStdLog returns a *log.Logger which writes to the supplied zap Logger at
// InfoLevel. To redirect the standard library's package-global logging
// functions, use RedirectStdLog instead.
func NewStdLog(l *Logger) *log.Logger {
logger := l.WithOptions(AddCallerSkip(_stdLogDefaultDepth + _loggerWriterDepth))
f := logger.Info
return log.New(&loggerWriter{f}, "" /* prefix */, 0 /* flags */)
}
// NewStdLogAt returns *log.Logger which writes to supplied zap logger at
// required level.
func NewStdLogAt(l *Logger, level zapcore.Level) (*log.Logger, error) {
logger := l.WithOptions(AddCallerSkip(_stdLogDefaultDepth + _loggerWriterDepth))
logFunc, err := levelToFunc(logger, level)
if err != nil {
return nil, err
}
return log.New(&loggerWriter{logFunc}, "" /* prefix */, 0 /* flags */), nil
}
// RedirectStdLog redirects output from the standard library's package-global
// logger to the supplied logger at InfoLevel. Since zap already handles caller
// annotations, timestamps, etc., it automatically disables the standard
// library's annotations and prefixing.
//
// It returns a function to restore the original prefix and flags and reset the
// standard library's output to os.Stderr.
func RedirectStdLog(l *Logger) func() {
f, err := redirectStdLogAt(l, InfoLevel)
if err != nil {
// Can't get here, since passing InfoLevel to redirectStdLogAt always
// works.
panic(fmt.Sprintf(_programmerErrorTemplate, err))
}
return f
}
// RedirectStdLogAt redirects output from the standard library's package-global
// logger to the supplied logger at the specified level. Since zap already
// handles caller annotations, timestamps, etc., it automatically disables the
// standard library's annotations and prefixing.
//
// It returns a function to restore the original prefix and flags and reset the
// standard library's output to os.Stderr.
func RedirectStdLogAt(l *Logger, level zapcore.Level) (func(), error) {
return redirectStdLogAt(l, level)
}
func redirectStdLogAt(l *Logger, level zapcore.Level) (func(), error) {
flags := log.Flags()
prefix := log.Prefix()
log.SetFlags(0)
log.SetPrefix("")
logger := l.WithOptions(AddCallerSkip(_stdLogDefaultDepth + _loggerWriterDepth))
logFunc, err := levelToFunc(logger, level)
if err != nil {
return nil, err
}
log.SetOutput(&loggerWriter{logFunc})
return func() {
log.SetFlags(flags)
log.SetPrefix(prefix)
log.SetOutput(os.Stderr)
}, nil
}
func levelToFunc(logger *Logger, lvl zapcore.Level) (func(string, ...Field), error) {
switch lvl {
case DebugLevel:
return logger.Debug, nil
case InfoLevel:
return logger.Info, nil
case WarnLevel:
return logger.Warn, nil
case ErrorLevel:
return logger.Error, nil
case DPanicLevel:
return logger.DPanic, nil
case PanicLevel:
return logger.Panic, nil
case FatalLevel:
return logger.Fatal, nil
}
return nil, fmt.Errorf("unrecognized level: %q", lvl)
}
type loggerWriter struct {
logFunc func(msg string, fields ...Field)
}
func (l *loggerWriter) Write(p []byte) (int, error) {
p = bytes.TrimSpace(p)
l.logFunc(string(p))
return len(p), nil
}
+81
View File
@@ -0,0 +1,81 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"encoding/json"
"fmt"
"net/http"
"go.uber.org/zap/zapcore"
)
// ServeHTTP is a simple JSON endpoint that can report on or change the current
// logging level.
//
// GET requests return a JSON description of the current logging level. PUT
// requests change the logging level and expect a payload like:
// {"level":"info"}
//
// It's perfectly safe to change the logging level while a program is running.
func (lvl AtomicLevel) ServeHTTP(w http.ResponseWriter, r *http.Request) {
type errorResponse struct {
Error string `json:"error"`
}
type payload struct {
Level *zapcore.Level `json:"level"`
}
enc := json.NewEncoder(w)
switch r.Method {
case "GET":
current := lvl.Level()
enc.Encode(payload{Level: &current})
case "PUT":
var req payload
if errmess := func() string {
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
return fmt.Sprintf("Request body must be well-formed JSON: %v", err)
}
if req.Level == nil {
return "Must specify a logging level."
}
return ""
}(); errmess != "" {
w.WriteHeader(http.StatusBadRequest)
enc.Encode(errorResponse{Error: errmess})
return
}
lvl.SetLevel(*req.Level)
enc.Encode(req)
default:
w.WriteHeader(http.StatusMethodNotAllowed)
enc.Encode(errorResponse{
Error: "Only GET and PUT are supported.",
})
}
}
+31
View File
@@ -0,0 +1,31 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
// Package bufferpool houses zap's shared internal buffer pool. Third-party
// packages can recreate the same functionality with buffers.NewPool.
package bufferpool
import "go.uber.org/zap/buffer"
var (
_pool = buffer.NewPool()
// Get retrieves a buffer from the pool, creating one if necessary.
Get = _pool.Get
)
+44
View File
@@ -0,0 +1,44 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
// Package color adds coloring functionality for TTY output.
package color
import "fmt"
// Foreground colors.
const (
Black Color = iota + 30
Red
Green
Yellow
Blue
Magenta
Cyan
White
)
// Color represents a text color.
type Color uint8
// Add adds the coloring to the given string.
func (c Color) Add(s string) string {
return fmt.Sprintf("\x1b[%dm%s\x1b[0m", uint8(c), s)
}
+64
View File
@@ -0,0 +1,64 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
// Package exit provides stubs so that unit tests can exercise code that calls
// os.Exit(1).
package exit
import "os"
var real = func() { os.Exit(1) }
// Exit normally terminates the process by calling os.Exit(1). If the package
// is stubbed, it instead records a call in the testing spy.
func Exit() {
real()
}
// A StubbedExit is a testing fake for os.Exit.
type StubbedExit struct {
Exited bool
prev func()
}
// Stub substitutes a fake for the call to os.Exit(1).
func Stub() *StubbedExit {
s := &StubbedExit{prev: real}
real = s.exit
return s
}
// WithStub runs the supplied function with Exit stubbed. It returns the stub
// used, so that users can test whether the process would have crashed.
func WithStub(f func()) *StubbedExit {
s := Stub()
defer s.Unstub()
f()
return s
}
// Unstub restores the previous exit function.
func (se *StubbedExit) Unstub() {
real = se.prev
}
func (se *StubbedExit) exit() {
se.Exited = true
}
+132
View File
@@ -0,0 +1,132 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"go.uber.org/atomic"
"go.uber.org/zap/zapcore"
)
const (
// DebugLevel logs are typically voluminous, and are usually disabled in
// production.
DebugLevel = zapcore.DebugLevel
// InfoLevel is the default logging priority.
InfoLevel = zapcore.InfoLevel
// WarnLevel logs are more important than Info, but don't need individual
// human review.
WarnLevel = zapcore.WarnLevel
// ErrorLevel logs are high-priority. If an application is running smoothly,
// it shouldn't generate any error-level logs.
ErrorLevel = zapcore.ErrorLevel
// DPanicLevel logs are particularly important errors. In development the
// logger panics after writing the message.
DPanicLevel = zapcore.DPanicLevel
// PanicLevel logs a message, then panics.
PanicLevel = zapcore.PanicLevel
// FatalLevel logs a message, then calls os.Exit(1).
FatalLevel = zapcore.FatalLevel
)
// LevelEnablerFunc is a convenient way to implement zapcore.LevelEnabler with
// an anonymous function.
//
// It's particularly useful when splitting log output between different
// outputs (e.g., standard error and standard out). For sample code, see the
// package-level AdvancedConfiguration example.
type LevelEnablerFunc func(zapcore.Level) bool
// Enabled calls the wrapped function.
func (f LevelEnablerFunc) Enabled(lvl zapcore.Level) bool { return f(lvl) }
// An AtomicLevel is an atomically changeable, dynamic logging level. It lets
// you safely change the log level of a tree of loggers (the root logger and
// any children created by adding context) at runtime.
//
// The AtomicLevel itself is an http.Handler that serves a JSON endpoint to
// alter its level.
//
// AtomicLevels must be created with the NewAtomicLevel constructor to allocate
// their internal atomic pointer.
type AtomicLevel struct {
l *atomic.Int32
}
// NewAtomicLevel creates an AtomicLevel with InfoLevel and above logging
// enabled.
func NewAtomicLevel() AtomicLevel {
return AtomicLevel{
l: atomic.NewInt32(int32(InfoLevel)),
}
}
// NewAtomicLevelAt is a convenience function that creates an AtomicLevel
// and then calls SetLevel with the given level.
func NewAtomicLevelAt(l zapcore.Level) AtomicLevel {
a := NewAtomicLevel()
a.SetLevel(l)
return a
}
// Enabled implements the zapcore.LevelEnabler interface, which allows the
// AtomicLevel to be used in place of traditional static levels.
func (lvl AtomicLevel) Enabled(l zapcore.Level) bool {
return lvl.Level().Enabled(l)
}
// Level returns the minimum enabled log level.
func (lvl AtomicLevel) Level() zapcore.Level {
return zapcore.Level(int8(lvl.l.Load()))
}
// SetLevel alters the logging level.
func (lvl AtomicLevel) SetLevel(l zapcore.Level) {
lvl.l.Store(int32(l))
}
// String returns the string representation of the underlying Level.
func (lvl AtomicLevel) String() string {
return lvl.Level().String()
}
// UnmarshalText unmarshals the text to an AtomicLevel. It uses the same text
// representations as the static zapcore.Levels ("debug", "info", "warn",
// "error", "dpanic", "panic", and "fatal").
func (lvl *AtomicLevel) UnmarshalText(text []byte) error {
if lvl.l == nil {
lvl.l = &atomic.Int32{}
}
var l zapcore.Level
if err := l.UnmarshalText(text); err != nil {
return err
}
lvl.SetLevel(l)
return nil
}
// MarshalText marshals the AtomicLevel to a byte slice. It uses the same
// text representation as the static zapcore.Levels ("debug", "info", "warn",
// "error", "dpanic", "panic", and "fatal").
func (lvl AtomicLevel) MarshalText() (text []byte, err error) {
return lvl.Level().MarshalText()
}
+305
View File
@@ -0,0 +1,305 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"fmt"
"io/ioutil"
"os"
"runtime"
"strings"
"time"
"go.uber.org/zap/zapcore"
)
// A Logger provides fast, leveled, structured logging. All methods are safe
// for concurrent use.
//
// The Logger is designed for contexts in which every microsecond and every
// allocation matters, so its API intentionally favors performance and type
// safety over brevity. For most applications, the SugaredLogger strikes a
// better balance between performance and ergonomics.
type Logger struct {
core zapcore.Core
development bool
name string
errorOutput zapcore.WriteSyncer
addCaller bool
addStack zapcore.LevelEnabler
callerSkip int
}
// New constructs a new Logger from the provided zapcore.Core and Options. If
// the passed zapcore.Core is nil, it falls back to using a no-op
// implementation.
//
// This is the most flexible way to construct a Logger, but also the most
// verbose. For typical use cases, the highly-opinionated presets
// (NewProduction, NewDevelopment, and NewExample) or the Config struct are
// more convenient.
//
// For sample code, see the package-level AdvancedConfiguration example.
func New(core zapcore.Core, options ...Option) *Logger {
if core == nil {
return NewNop()
}
log := &Logger{
core: core,
errorOutput: zapcore.Lock(os.Stderr),
addStack: zapcore.FatalLevel + 1,
}
return log.WithOptions(options...)
}
// NewNop returns a no-op Logger. It never writes out logs or internal errors,
// and it never runs user-defined hooks.
//
// Using WithOptions to replace the Core or error output of a no-op Logger can
// re-enable logging.
func NewNop() *Logger {
return &Logger{
core: zapcore.NewNopCore(),
errorOutput: zapcore.AddSync(ioutil.Discard),
addStack: zapcore.FatalLevel + 1,
}
}
// NewProduction builds a sensible production Logger that writes InfoLevel and
// above logs to standard error as JSON.
//
// It's a shortcut for NewProductionConfig().Build(...Option).
func NewProduction(options ...Option) (*Logger, error) {
return NewProductionConfig().Build(options...)
}
// NewDevelopment builds a development Logger that writes DebugLevel and above
// logs to standard error in a human-friendly format.
//
// It's a shortcut for NewDevelopmentConfig().Build(...Option).
func NewDevelopment(options ...Option) (*Logger, error) {
return NewDevelopmentConfig().Build(options...)
}
// NewExample builds a Logger that's designed for use in zap's testable
// examples. It writes DebugLevel and above logs to standard out as JSON, but
// omits the timestamp and calling function to keep example output
// short and deterministic.
func NewExample(options ...Option) *Logger {
encoderCfg := zapcore.EncoderConfig{
MessageKey: "msg",
LevelKey: "level",
NameKey: "logger",
EncodeLevel: zapcore.LowercaseLevelEncoder,
EncodeTime: zapcore.ISO8601TimeEncoder,
EncodeDuration: zapcore.StringDurationEncoder,
}
core := zapcore.NewCore(zapcore.NewJSONEncoder(encoderCfg), os.Stdout, DebugLevel)
return New(core).WithOptions(options...)
}
// Sugar wraps the Logger to provide a more ergonomic, but slightly slower,
// API. Sugaring a Logger is quite inexpensive, so it's reasonable for a
// single application to use both Loggers and SugaredLoggers, converting
// between them on the boundaries of performance-sensitive code.
func (log *Logger) Sugar() *SugaredLogger {
core := log.clone()
core.callerSkip += 2
return &SugaredLogger{core}
}
// Named adds a new path segment to the logger's name. Segments are joined by
// periods. By default, Loggers are unnamed.
func (log *Logger) Named(s string) *Logger {
if s == "" {
return log
}
l := log.clone()
if log.name == "" {
l.name = s
} else {
l.name = strings.Join([]string{l.name, s}, ".")
}
return l
}
// WithOptions clones the current Logger, applies the supplied Options, and
// returns the resulting Logger. It's safe to use concurrently.
func (log *Logger) WithOptions(opts ...Option) *Logger {
c := log.clone()
for _, opt := range opts {
opt.apply(c)
}
return c
}
// With creates a child logger and adds structured context to it. Fields added
// to the child don't affect the parent, and vice versa.
func (log *Logger) With(fields ...Field) *Logger {
if len(fields) == 0 {
return log
}
l := log.clone()
l.core = l.core.With(fields)
return l
}
// Check returns a CheckedEntry if logging a message at the specified level
// is enabled. It's a completely optional optimization; in high-performance
// applications, Check can help avoid allocating a slice to hold fields.
func (log *Logger) Check(lvl zapcore.Level, msg string) *zapcore.CheckedEntry {
return log.check(lvl, msg)
}
// Debug logs a message at DebugLevel. The message includes any fields passed
// at the log site, as well as any fields accumulated on the logger.
func (log *Logger) Debug(msg string, fields ...Field) {
if ce := log.check(DebugLevel, msg); ce != nil {
ce.Write(fields...)
}
}
// Info logs a message at InfoLevel. The message includes any fields passed
// at the log site, as well as any fields accumulated on the logger.
func (log *Logger) Info(msg string, fields ...Field) {
if ce := log.check(InfoLevel, msg); ce != nil {
ce.Write(fields...)
}
}
// Warn logs a message at WarnLevel. The message includes any fields passed
// at the log site, as well as any fields accumulated on the logger.
func (log *Logger) Warn(msg string, fields ...Field) {
if ce := log.check(WarnLevel, msg); ce != nil {
ce.Write(fields...)
}
}
// Error logs a message at ErrorLevel. The message includes any fields passed
// at the log site, as well as any fields accumulated on the logger.
func (log *Logger) Error(msg string, fields ...Field) {
if ce := log.check(ErrorLevel, msg); ce != nil {
ce.Write(fields...)
}
}
// DPanic logs a message at DPanicLevel. The message includes any fields
// passed at the log site, as well as any fields accumulated on the logger.
//
// If the logger is in development mode, it then panics (DPanic means
// "development panic"). This is useful for catching errors that are
// recoverable, but shouldn't ever happen.
func (log *Logger) DPanic(msg string, fields ...Field) {
if ce := log.check(DPanicLevel, msg); ce != nil {
ce.Write(fields...)
}
}
// Panic logs a message at PanicLevel. The message includes any fields passed
// at the log site, as well as any fields accumulated on the logger.
//
// The logger then panics, even if logging at PanicLevel is disabled.
func (log *Logger) Panic(msg string, fields ...Field) {
if ce := log.check(PanicLevel, msg); ce != nil {
ce.Write(fields...)
}
}
// Fatal logs a message at FatalLevel. The message includes any fields passed
// at the log site, as well as any fields accumulated on the logger.
//
// The logger then calls os.Exit(1), even if logging at FatalLevel is
// disabled.
func (log *Logger) Fatal(msg string, fields ...Field) {
if ce := log.check(FatalLevel, msg); ce != nil {
ce.Write(fields...)
}
}
// Sync calls the underlying Core's Sync method, flushing any buffered log
// entries. Applications should take care to call Sync before exiting.
func (log *Logger) Sync() error {
return log.core.Sync()
}
// Core returns the Logger's underlying zapcore.Core.
func (log *Logger) Core() zapcore.Core {
return log.core
}
func (log *Logger) clone() *Logger {
copy := *log
return &copy
}
func (log *Logger) check(lvl zapcore.Level, msg string) *zapcore.CheckedEntry {
// check must always be called directly by a method in the Logger interface
// (e.g., Check, Info, Fatal).
const callerSkipOffset = 2
// Create basic checked entry thru the core; this will be non-nil if the
// log message will actually be written somewhere.
ent := zapcore.Entry{
LoggerName: log.name,
Time: time.Now(),
Level: lvl,
Message: msg,
}
ce := log.core.Check(ent, nil)
willWrite := ce != nil
// Set up any required terminal behavior.
switch ent.Level {
case zapcore.PanicLevel:
ce = ce.Should(ent, zapcore.WriteThenPanic)
case zapcore.FatalLevel:
ce = ce.Should(ent, zapcore.WriteThenFatal)
case zapcore.DPanicLevel:
if log.development {
ce = ce.Should(ent, zapcore.WriteThenPanic)
}
}
// Only do further annotation if we're going to write this message; checked
// entries that exist only for terminal behavior don't benefit from
// annotation.
if !willWrite {
return ce
}
// Thread the error output through to the CheckedEntry.
ce.ErrorOutput = log.errorOutput
if log.addCaller {
ce.Entry.Caller = zapcore.NewEntryCaller(runtime.Caller(log.callerSkip + callerSkipOffset))
if !ce.Entry.Caller.Defined {
fmt.Fprintf(log.errorOutput, "%v Logger.check error: failed to get caller\n", time.Now().UTC())
log.errorOutput.Sync()
}
}
if log.addStack.Enabled(ce.Entry.Level) {
ce.Entry.Stack = Stack("").String
}
return ce
}
+109
View File
@@ -0,0 +1,109 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import "go.uber.org/zap/zapcore"
// An Option configures a Logger.
type Option interface {
apply(*Logger)
}
// optionFunc wraps a func so it satisfies the Option interface.
type optionFunc func(*Logger)
func (f optionFunc) apply(log *Logger) {
f(log)
}
// WrapCore wraps or replaces the Logger's underlying zapcore.Core.
func WrapCore(f func(zapcore.Core) zapcore.Core) Option {
return optionFunc(func(log *Logger) {
log.core = f(log.core)
})
}
// Hooks registers functions which will be called each time the Logger writes
// out an Entry. Repeated use of Hooks is additive.
//
// Hooks are useful for simple side effects, like capturing metrics for the
// number of emitted logs. More complex side effects, including anything that
// requires access to the Entry's structured fields, should be implemented as
// a zapcore.Core instead. See zapcore.RegisterHooks for details.
func Hooks(hooks ...func(zapcore.Entry) error) Option {
return optionFunc(func(log *Logger) {
log.core = zapcore.RegisterHooks(log.core, hooks...)
})
}
// Fields adds fields to the Logger.
func Fields(fs ...Field) Option {
return optionFunc(func(log *Logger) {
log.core = log.core.With(fs)
})
}
// ErrorOutput sets the destination for errors generated by the Logger. Note
// that this option only affects internal errors; for sample code that sends
// error-level logs to a different location from info- and debug-level logs,
// see the package-level AdvancedConfiguration example.
//
// The supplied WriteSyncer must be safe for concurrent use. The Open and
// zapcore.Lock functions are the simplest ways to protect files with a mutex.
func ErrorOutput(w zapcore.WriteSyncer) Option {
return optionFunc(func(log *Logger) {
log.errorOutput = w
})
}
// Development puts the logger in development mode, which makes DPanic-level
// logs panic instead of simply logging an error.
func Development() Option {
return optionFunc(func(log *Logger) {
log.development = true
})
}
// AddCaller configures the Logger to annotate each message with the filename
// and line number of zap's caller.
func AddCaller() Option {
return optionFunc(func(log *Logger) {
log.addCaller = true
})
}
// AddCallerSkip increases the number of callers skipped by caller annotation
// (as enabled by the AddCaller option). When building wrappers around the
// Logger and SugaredLogger, supplying this Option prevents zap from always
// reporting the wrapper code as the caller.
func AddCallerSkip(skip int) Option {
return optionFunc(func(log *Logger) {
log.callerSkip += skip
})
}
// AddStacktrace configures the Logger to record a stack trace for all messages at
// or above a given level.
func AddStacktrace(lvl zapcore.LevelEnabler) Option {
return optionFunc(func(log *Logger) {
log.addStack = lvl
})
}
+126
View File
@@ -0,0 +1,126 @@
// Copyright (c) 2016 Uber Technologies, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package zap
import (
"runtime"
"strings"
"sync"
"go.uber.org/zap/internal/bufferpool"
)
const _zapPackage = "go.uber.org/zap"
var (
_stacktracePool = sync.Pool{
New: func() interface{} {
return newProgramCounters(64)
},
}
// We add "." and "/" suffixes to the package name to ensure we only match
// the exact package and not any package with the same prefix.
_zapStacktracePrefixes = addPrefix(_zapPackage, ".", "/")
_zapStacktraceVendorContains = addPrefix("/vendor/", _zapStacktracePrefixes...)
)
func takeStacktrace() string {
buffer := bufferpool.Get()
defer buffer.Free()
programCounters := _stacktracePool.Get().(*programCounters)
defer _stacktracePool.Put(programCounters)
var numFrames int
for {
// Skip the call to runtime.Counters and takeStacktrace so that the
// program counters start at the caller of takeStacktrace.
numFrames = runtime.Callers(2, programCounters.pcs)
if numFrames < len(programCounters.pcs) {
break
}
// Don't put the too-short counter slice back into the pool; this lets
// the pool adjust if we consistently take deep stacktraces.
programCounters = newProgramCounters(len(programCounters.pcs) * 2)
}
i := 0
skipZapFrames := true // skip all consecutive zap frames at the beginning.
frames := runtime.CallersFrames(programCounters.pcs[:numFrames])
// Note: On the last iteration, frames.Next() returns false, with a valid
// frame, but we ignore this frame. The last frame is a a runtime frame which
// adds noise, since it's only either runtime.main or runtime.goexit.
for frame, more := frames.Next(); more; frame, more = frames.Next() {
if skipZapFrames && isZapFrame(frame.Function) {
continue
} else {
skipZapFrames = false
}
if i != 0 {
buffer.AppendByte('\n')
}
i++
buffer.AppendString(frame.Function)
buffer.AppendByte('\n')
buffer.AppendByte('\t')
buffer.AppendString(frame.File)
buffer.AppendByte(':')
buffer.AppendInt(int64(frame.Line))
}
return buffer.String()
}
func isZapFrame(function string) bool {
for _, prefix := range _zapStacktracePrefixes {
if strings.HasPrefix(function, prefix) {
return true
}
}
// We can't use a prefix match here since the location of the vendor
// directory affects the prefix. Instead we do a contains match.
for _, contains := range _zapStacktraceVendorContains {
if strings.Contains(function, contains) {
return true
}
}
return false
}
type programCounters struct {
pcs []uintptr
}
func newProgramCounters(size int) *programCounters {
return &programCounters{make([]uintptr, size)}
}
func addPrefix(prefix string, ss ...string) []string {
withPrefix := make([]string, len(ss))
for i, s := range ss {
withPrefix[i] = prefix + s
}
return withPrefix
}

Some files were not shown because too many files have changed in this diff Show More