Feature/support env hcl and interpolations (#1423)

* support HCL language for env variables

Signed-off-by: xtphate <65117176+XT-Phate@users.noreply.github.com>
This commit is contained in:
xtphate 2024-04-22 02:02:14 +02:00 committed by GitHub
parent b703b5e061
commit a15a1b0731
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
29 changed files with 1897 additions and 118 deletions

667
docs/hcl_funcs.md Normal file
View File

@ -0,0 +1,667 @@
# HCL Functions
## Standard Library
The following functions are all from the [go-cty](https://pkg.go.dev/github.com/zclconf/go-cty/cty/function/stdlib#pkg-functions), [go-cty-yaml](https://pkg.go.dev/github.com/zclconf/go-cty-yaml) and [hcl](https://pkg.go.dev/github.com/hashicorp/hcl/v2@v2.20.1/ext/tryfunc#section-readme) libraries
#### abs
`abs` returns the absolute value
```
abs(number)
```
```
abs(-1)
# 1
abs(2)
# 2
```
#### can
`can` evaluates an expression and returns a boolean if a result can be produced without any error
```
can(expr)
```
```
map = {
myvar = "myvar"
}
can1 = can(hv.map.myVar)
# true
can2 = can(hv.map.notMyVar)
# false
```
#### ceil
`ceil` returns the ceiling value of a given number
```
ceil(number)
```
```
ceil(1)
# 1
ceil(1.1)
# 2
```
#### chomp
`chomp` removes newline characters at the end of a string.
```
chomp(string)
```
```
chomp("myVar\n")
# myVar
```
#### coalesce
`coalesce` returns the first of the given arguments that is not null. If all arguments are null, an error is produced.
All arguments must be of the same type apart from some cases
```
coalesce(any...)
```
```
coalesce(null, 2)
# 2
coalesce(null, "value")
# value
```
Use the three dots notation `...` to expand a list
```
coalesce([null, "value"]...)
# value
```
#### coalescelist
`coalescelist` takes any number of list arguments and returns the first one that isn't empty.
```
coalescelist(list)
```
```
coalescelist([], ["value"])
# ["value"]
```
Use the three dots notation `...` when using list of lists
```
coalescelist([[], ["val1", "val2"]]...)
# ["val1", "val2"]
```
#### compact
`compact` returns a new list with any empty string elements removed.
```
compact(list)
```
```
compact(["", "val1", "val2"])
# ["val1", "val2"]
```
#### concat
`concat` takes one or more sequences (lists or tuples) and returns the single sequence that results from concatenating them together in order.
```
concat(list, list...)
```
```
concat(["val1"], ["val2", "val3"])
# ["val1", "val2", "val3"]
```
#### contains
`contains` returns a boolean if a list contains a given value
```
contains(list, value)
```
```
contains(["val1", "val2"], "val2")
# true
```
#### csvdecode
`csvdecode` decodes a CSV-formatted string into a list of maps
```
csvdecode(string)
```
```
csvdecode("col1,col2\nv1,v2\nv3,v4")
###
[
{
"col1" = "v1"
"col2" = "v2"
},
{
"col1" = "v3"
"col2" = "v4"
}
]
```
#### distinct
`distinct` returns a new list from another by removing all duplicates
```
distinct(list)
```
```
distinct(["v1","v1","v2"])
["v1", "v2"]
```
#### element
`element` returns a single element from a given list at the given index. If index is greater than the length of the list then it is wrapped modulo the list length
```
element(list, index)
```
```
element(["val1","val2"], 1)
# val2
```
#### chunklist
`chunklist` splits a single list into fixed-size chunks, returning a list of lists.
```
chunklist(list, size)
```
```
chunklist(["a","b"], 1)
# [["a"], ["b"]]
```
#### flatten
`flatten` takes a list and replaces any elements that are lists with a flattened sequence of the list contents.
```
flatten(list)
```
```
flatten([["a"], ["a","b"], ["c"]])
# ["a","a","b","c"]
```
#### floor
`floor` returns the closest whole number lesser than or equal to the given value.
```
floor(number)
```
```
floor(1)
# 1
floor(0.7)
# 0
```
#### format
`format` produces a string representation of zero or more values using a format string similar to the "printf" function in C.
[Verbs details](https://pkg.go.dev/github.com/zclconf/go-cty/cty/function/stdlib#Format)
```
format(format, values)
```
```
format("Hello %s", "world")
# Hello world
```
#### formatdate
`formatdate` reformats a timestamp given in RFC3339 syntax into another time syntax defined by a given format string.
[Syntax details](https://pkg.go.dev/github.com/zclconf/go-cty/cty/function/stdlib#FormatDate)
```
formatdate(string, timestampString)
```
```
formatdate("MMM DD YYYY", "2024-01-01T00:12:00Z")
# Jan 01 2024
```
#### formatlist
`formatlist` does the same as `format` but for a list of strings
```
formatlist(formatString, values...)
```
```
formatlist("%s", ["Hello", "World"])
###
[
"Hello",
"World"
]
formatlist("%s %s", "hello", ["World", "You"])
###
[
"hello World",
"hello You",
]
```
#### indent
`indent` adds a given number of spaces to the beginnings of all but the first line in a given multi-line string.
```
indent(number, string)
```
```
indent(4, "hello,\nWorld\n!")
###
hello
World
!
```
#### int
`int` removes the fractional component of the given number returning an integer representing the whole number component, rounding towards zero.
```
int(number)
```
```
int(6.2)
# 6
```
#### join
`join` concatenates together the string elements of one or more lists with a given separator.
```
join(listOfStrings, separator)
```
```
join(" ", ["hello", "world"])
# hello world
```
#### jsondecode
`jsondecode` parses the given JSON string and, if it is valid, returns the value it represents.
```
jsonencode(string)
```
Example :
```
jsonencode({"hello"="world"})
# {"hello": "world"}
```
#### jsonencode
`jsonencode` returns a JSON serialization of the given value.
```
jsondecode(string)
```
Example :
```
jsondecode("{\"hello\": \"world\"}")
# { hello = "world" }
```
#### keys
`keys` takes a map and returns a sorted list of the map keys.
```
keys(map)
```
```
keys({val1=1, val2=2, val3=3})
# ["val1","val2","val3"]
```
#### length
`length` returns the number of elements in the given __collection__.
See `strlen` for strings
```
length(list)
```
```
length([1,2,3])
# 3
```
#### log
`log` returns returns the logarithm of a given number in a given base.
```
log(number, base)
```
```
log(1, 10)
# 0
```
#### lookup
`lookup` performs a dynamic lookup into a map. There are three required arguments, inputMap and key, plus a defaultValue, which is a value to return if the given key is not found in the inputMap.
```
lookup(inputMap, key, defaultValue)
```
```
map = { "luke" = "skywalker"}
lookup(hv.maptest, "luke", "none")
# skywalker
lookup(hv.maptest, "leia", "none")
# none
```
#### lower
`lower` is a Function that converts a given string to lowercase.
```
lower(string)
```
```
lower("HELLO world")
# hello world
```
#### max
`max` returns the maximum number from the given numbers.
```
max(numbers)
```
```
max(1,128,70)
# 128
```
#### merge
`merge` takes an arbitrary number of maps and returns a single map that contains a merged set of elements from all of the maps.
```
merge(maps)
```
```
merge({a="1"}, {a=[1,2], c="world"}, {d=40})
# { a = [1,2], c = "world", d = 40}
```
#### min
`min` returns the minimum number from the given numbers.
```
min(numbers)
```
```
min(1,128,70)
# 1
```
#### parseint
`parseint` parses a string argument and returns an integer of the specified base.
```
parseint(string, base)
```
```
parseint("190", 10)
# 190
parseint("11001", 2)
# 25
```
#### pow
`pow` returns the logarithm of a given number in a given base.
```
pow(number, power)
```
```
pow(1, 10)
# 1
pow(3, 12)
# 531441
```
#### range
`range` creates a list of numbers by starting from the given starting value, then adding the given step value until the result is greater than or equal to the given stopping value. Each intermediate result becomes an element in the resulting list.
```
range(startingNumber, stoppingNumber, stepNumber)
```
```
range(1, 10, 3)
# [1, 4, 7]
```
#### regex
`regex` is a function that extracts one or more substrings from a given string by applying a regular expression pattern, describing the first match.
The return type depends on the composition of the capture groups (if any) in the pattern:
If there are no capture groups at all, the result is a single string representing the entire matched pattern.
If all of the capture groups are named, the result is an object whose keys are the named groups and whose values are their sub-matches, or null if a particular sub-group was inside another group that didn't match.
If none of the capture groups are named, the result is a tuple whose elements are the sub-groups in order and whose values are their sub-matches, or null if a particular sub-group was inside another group that didn't match.
It is invalid to use both named and un-named capture groups together in the same pattern.
If the pattern doesn't match, this function returns an error. To test for a match, call `regexall` and check if the length of the result is greater than zero.
```
regex(pattern, string)
```
```
regex("[0-9]+", "v1.2.3")
# 1
```
#### regexall
`regexall` is similar to Regex but it finds all of the non-overlapping matches in the given string and returns a list of them.
The result type is always a list, whose element type is deduced from the pattern in the same way as the return type for Regex is decided.
If the pattern doesn't match at all, this function returns an empty list.
```
regexall(pattern, string)
```
```
regexall("[0-9]+", "v1.2.3")
# [1 2 3]
```
#### setintersection
`setintersection` returns a new set containing the elements that exist in all of the given sets, which must have element types that can all be converted to some common type using the standard type unification rules. If conversion is not possible, an error is returned.
```
setintersection(sets...)
```
```
setintersection(["val1", "val2"], ["val1", "val3"], ["val1", "val2"])
# ["val1"]
```
#### setproduct
`setproduct` computes the Cartesian product of sets or sequences.
```
setproduct(sets...)
```
```
setproduct(["host1", "host2"], ["stg.domain", "prod.domain"])
###
[
[
"host1",
"stg.domain"
],
[
"host2",
"stg.domain"
],
[
"host1",
"prod.domain"
],
[
"host2",
"prod.domain"
],
]
```
#### setsubtract
`setsubtract` returns a new set containing the elements from the first set that are not present in the second set. The sets must have element types that can both be converted to some common type using the standard type unification rules. If conversion is not possible, an error is returned.
```
setsubtract(sets...)
```
```
setsubtract(["a", "b", "c"], ["a", "b"])
###
["c"]
```
#### setunion
`setunion` returns a new set containing all of the elements from the given sets, which must have element types that can all be converted to some common type using the standard type unification rules. If conversion is not possible, an error is returned.
```
setunion(sets...)
```
```
setunion(["a", "b"], ["b", "c"], ["a", "d"])
###
["a", "b", "c", "d"]
```
#### signum
`signum` determines the sign of a number, returning a number between -1 and 1 to represent the sign.
```
signum(number)
```
```
signum(-182)
# -1
```
#### slice
`slice` extracts some consecutive elements from within a list.
startIndex is inclusive, endIndex is exclusive
```
slice(list, startIndex, endIndex)
```
```
slice([{"a" = "b"}, {"c" = "d"}, , {"e" = "f"}], 1, 1)
# []
slice([{"a" = "b"}, {"c" = "d"}, {"e" = "f"}], 1, 2)
# [{"c" = "d"}]
```
#### sort
`sort` re-orders the elements of a given list of strings so that they are in ascending lexicographical order.
```
sort(list)
```
```
sort(["1", "h", "r", "p", "word"])
# ["1", "h", "p", "r", "word"]
```
#### split
`split` divides a given string by a given separator, returning a list of strings containing the characters between the separator sequences.
```
split(separatorString, string)
```
```
split(".", "host.domain")
# ["host", "domain"]
```
#### strlen
`strlen` is a Function that returns the length of the given string in characters.
```
strlen(string)
```
```
strlen("yes")
# 3
```
#### strrev
`strrev` is a Function that reverses the order of the characters in the given string.
```
strrev(string)
```
```
strrev("yes")
# "sey"
```
#### substr
`substr` is a Function that extracts a sequence of characters from another string and creates a new string.
```
substr(string, offsetNumber, length)
```
```
substr("host.domain", 0, 4)
# "host"
```
#### timeadd
`timeadd` adds a duration to a timestamp, returning a new timestamp.
Only units "inferior" or equal to `h` are supported.
The duration can be negative.
```
substr(timestamp, duration)
```
```
timeadd("2024-01-01T00:00:00Z", "-2600h10m")
# 2023-09-14T15:50:00Z
```
#### trim
`trim` removes the specified characters from the start and end of the given string.
```
trim(string, string)
```
```
trim("Can you do that ? Yes ?", "?")
# "Can you do that ? Yes"
```
#### trimprefix
`trimprefix` removes the specified prefix from the start of the given string.
```
trimprefix(stringToTrim, trimmingString)
```
```
trimprefix("please, do it", "please, ")
# "do it"
```
#### trimspace
`trimspace` removes any space characters from the start and end of the given string.
```
trimspace(string)
```
```
trimspace(" Hello World ")
# "Hello World"
```
#### trimsuffix
`trimsuffix` removes the specified suffix from the end of the given string.
```
trimsuffix(stringToTrim, trimmingString)
```
```
trimsuffix("Hello World", " World")
# "Hello"
```
#### try
`try` is a variadic function that tries to evaluate all of is arguments in sequence until one succeeds, in which case it returns that result, or returns an error if none of them succeed.
```
try(expressions...)
```
```
values {
map = {
hello = "you"
world = "us"
}
try(hv.map.do_not_exist, hv.map.world)
}
# "us"
```
#### upper
`upper` is a Function that converts a given string to uppercase.
```
upper(string)
```
```
upper("up")
# "UP"
```
#### values
`values` returns a list of the map values, in the order of the sorted keys. This function only works on flat maps.
```
values(map)
```
```
values({"a" = 1,"b" = 2})
# [1, 2]
```
#### yamldecode
`yamldecode` parses the given JSON string and, if it is valid, returns the value it represents.
```
yamldecode(string)
```
```
yamldecode("hello: world\narray: [1, 2, 3]")
###
{
array = [1, 2, 3]
hello = "world"
}
```
#### yamlencode
`yamlencode` returns a JSON serialization of the given value.
```
yamlencode({array = [1, 2, 3], hello = "world"})
```
```
yamlencode({array = [1, 2, 3], hello = "world"})
###
"array":
- 1
- 2
- 3
"hello": "world"
```
#### zipmap
`zipmap` constructs a map from a list of keys and a corresponding list of values.
The lenght of each list must be equal
```
zipmap(keysList, valuesList)
```
```
zipmap(["key1", "key2"], ["val1", "val2"])
###
{
"key1" = "val1"
"key2" = "val2"
}
```

View File

@ -403,6 +403,9 @@ environments:
# `{{ .Values.foo.bar }}` is evaluated to `1`. # `{{ .Values.foo.bar }}` is evaluated to `1`.
values: values:
- environments/default/values.yaml - environments/default/values.yaml
# Everything from the values.hcl in the `values` block is available via `{{ .Values.KEY }}`.
# More details in its dedicated section
- environments/default/values.hcl
# Each entry in values can be either a file path or inline values. # Each entry in values can be either a file path or inline values.
# The below is an example of inline values, which is merged to the `.Values` # The below is an example of inline values, which is merged to the `.Values`
- myChartVer: 1.0.0-dev - myChartVer: 1.0.0-dev
@ -896,7 +899,11 @@ releases:
# snip # snip
``` ```
## Environment Values ### Environment Values
Helmfile supports 3 values languages :
- Straight yaml
- Go templates to generate straight yaml
- HCL
Environment Values allows you to inject a set of values specific to the selected environment, into `values.yaml` templates. Environment Values allows you to inject a set of values specific to the selected environment, into `values.yaml` templates.
Use it to inject common values from the environment to multiple values files, to make your configuration DRY. Use it to inject common values from the environment to multiple values files, to make your configuration DRY.
@ -967,7 +974,45 @@ releases:
... ...
``` ```
### Note on Environment.Values vs Values #### HCL specifications
Since Helmfile v0.164.0, HCL language is supported for environment values only.
HCL values supports interpolations and sharing values across files
* Only `.hcl` suffixed files will be interpreted as is
* Helmfile supports 2 differents blocks: `values` and `locals`
* `values` block is a shared block where all values are accessible everywhere in all loaded files
* `locals` block can't reference external values apart from the ones in the block itself, and where its defined values are only accessible in its local file
* Only values in `values` blocks are made available to the final root `.Values` (e.g : ` values { myvar = "var" }` is accessed through `{{ .Values.myvar }}`)
* There can only be 1 `locals` block per file
* Helmfile hcl `values` are referenced using the `hv` accessor.
* Helmfile hcl `locals` are referenced using the `local` accessor.
* Duplicated variables across .hcl `values` blocks are forbidden (An error will pop up specifying where are the duplicates)
* All cty [standard library functions](`https://pkg.go.dev/github.com/zclconf/go-cty@v1.14.3/cty/function/stdlib`) are available and custom functions could be created in the future
Consider the following example :
```terraform
# values1.hcl
locals {
hostname = "host1"
}
values {
domain = "DEV.EXAMPLE.COM"
hostnameV1 = "${local.hostname}.${lower(hv.domain)}" # "host1.dev.example.com"
}
```
```terraform
# values2.hcl
locals {
hostname = "host2"
}
values {
hostnameV2 = "${local.hostname}.${hv.domain}" # "host2.DEV.EXAMPLE.COM"
}
```
#### Note on Environment.Values vs Values
The `{{ .Values.foo }}` syntax is the recommended way of using environment values. The `{{ .Values.foo }}` syntax is the recommended way of using environment values.
@ -976,46 +1021,7 @@ This is still working but is **deprecated** and the new `{{ .Values.foo }}` synt
You can read more infos about the feature proposal [here](https://github.com/roboll/helmfile/issues/640). You can read more infos about the feature proposal [here](https://github.com/roboll/helmfile/issues/640).
### Loading remote Environment values files ### Environment Secrets
Since Helmfile v0.118.8, you can use `go-getter`-style URLs to refer to remote values files:
```yaml
environments:
cluster-azure-us-west:
values:
- git::https://git.company.org/helmfiles/global/azure.yaml?ref=master
- git::https://git.company.org/helmfiles/global/us-west.yaml?ref=master
- git::https://gitlab.com/org/repository-name.git@/config/config.test.yaml?ref=main # Public Gilab Repo
cluster-gcp-europe-west:
values:
- git::https://git.company.org/helmfiles/global/gcp.yaml?ref=master
- git::https://git.company.org/helmfiles/global/europe-west.yaml?ref=master
- git::https://ci:{{ env "CI_JOB_TOKEN" }}@gitlab.com/org/repository-name.git@/config.dev.yaml?ref={{ env "APP_COMMIT_SHA" }} # Private Gitlab Repo
staging:
values:
- git::https://{{ env "GITHUB_PAT" }}@github.com/[$GITHUB_ORGorGITHUB_USER]/repository-name.git@/values.dev.yaml?ref=main #Github Private repo
- http://$HOSTNAME/artifactory/example-repo-local/test.tgz@values.yaml #Artifactory url
---
releases:
- ...
```
Since Helmfile v0.158.0, support more protocols, such as: s3, https, http
```
values:
- s3::https://helm-s3-values-example.s3.us-east-2.amazonaws.com/values.yaml
- s3://helm-s3-values-example/subdir/values.yaml
- https://john:doe@helm-s3-values-example.s3.us-east-2.amazonaws.com/values.yaml
- http://helm-s3-values-example.s3.us-east-2.amazonaws.com/values.yaml
```
For more information about the supported protocols see: [go-getter Protocol-Specific Options](https://github.com/hashicorp/go-getter#protocol-specific-options-1).
This is particularly useful when you co-locate helmfiles within your project repo but want to reuse the definitions in a global repo.
## Environment Secrets
Environment Secrets *(not to be confused with Kubernetes Secrets)* are encrypted versions of `Environment Values`. Environment Secrets *(not to be confused with Kubernetes Secrets)* are encrypted versions of `Environment Values`.
You can list any number of `secrets.yaml` files created using `helm secrets` or `sops`, so that You can list any number of `secrets.yaml` files created using `helm secrets` or `sops`, so that
@ -1056,7 +1062,7 @@ Then the environment secret `foo.bar` can be referenced by the below template ex
{{ .Values.foo.bar }} {{ .Values.foo.bar }}
``` ```
### Loading remote Environment secrets files #### Loading remote Environment secrets files
Since Helmfile v0.149.0, you can use `go-getter`-style URLs to refer to remote secrets files, the same way as in values files: Since Helmfile v0.149.0, you can use `go-getter`-style URLs to refer to remote secrets files, the same way as in values files:
```yaml ```yaml
@ -1071,6 +1077,91 @@ environments:
- http://$HOSTNAME/artifactory/example-repo-local/test.tgz@environments/production.secret.yaml - http://$HOSTNAME/artifactory/example-repo-local/test.tgz@environments/production.secret.yaml
``` ```
### Loading remote Environment values files
Since Helmfile v0.118.8, you can use `go-getter`-style URLs to refer to remote values files:
```yaml
environments:
cluster-azure-us-west:
values:
- git::https://git.company.org/helmfiles/global/azure.yaml?ref=master
- git::https://git.company.org/helmfiles/global/us-west.yaml?ref=master
- git::https://gitlab.com/org/repository-name.git@/config/config.test.yaml?ref=main # Public Gilab Repo
cluster-gcp-europe-west:
values:
- git::https://git.company.org/helmfiles/global/gcp.yaml?ref=master
- git::https://git.company.org/helmfiles/global/europe-west.yaml?ref=master
- git::https://ci:{{ env "CI_JOB_TOKEN" }}@gitlab.com/org/repository-name.git@/config.dev.yaml?ref={{ env "APP_COMMIT_SHA" }} # Private Gitlab Repo
staging:
values:
- git::https://{{ env "GITHUB_PAT" }}@github.com/[$GITHUB_ORGorGITHUB_USER]/repository-name.git@/values.dev.yaml?ref=main #Github Private repo
- http://$HOSTNAME/artifactory/example-repo-local/test.tgz@values.yaml #Artifactory url
---
releases:
- ...
```
Since Helmfile v0.158.0, support more protocols, such as: s3, https, http
```
values:
- s3::https://helm-s3-values-example.s3.us-east-2.amazonaws.com/values.yaml
- s3://helm-s3-values-example/subdir/values.yaml
- https://john:doe@helm-s3-values-example.s3.us-east-2.amazonaws.com/values.yaml
- http://helm-s3-values-example.s3.us-east-2.amazonaws.com/values.yaml
```
For more information about the supported protocols see: [go-getter Protocol-Specific Options](https://github.com/hashicorp/go-getter#protocol-specific-options-1).
This is particularly useful when you co-locate helmfiles within your project repo but want to reuse the definitions in a global repo.
### Environment values precedence
With the introduction of HCL, a new value precedence was introduced over environment values.
Here is the order of precedence from least to greatest (the last one overrides all others)
1. `yaml` / `yaml.gotmpl`
2. `hcl`
3. `yaml` secrets
Example:
---
```yaml
# values1.yaml
domain: "dev.example.com"
```
```terraform
# values2.hcl
values {
domain = "overdev.example.com"
willBeOverriden = "override_me"
}
```
```yaml
# secrets.yml (assuming this one has been encrypted)
willBeOverriden: overrided
```
```
# helmfile.yaml.gotmpl
environments:
default:
values:
- value1.yaml
- value2.hcl
secrets:
- secrets.yml
---
releases:
- name: random-release
[...]
values:
domain: "{{ .Values.domain }}" # == "overdev.example.com"
willBeOverriden: "{{ .Values.willBeOverriden }}" # == "overrided"
```
## DAG-aware installation/deletion ordering with `needs` ## DAG-aware installation/deletion ordering with `needs`
`needs` controls the order of the installation/deletion of the release: `needs` controls the order of the installation/deletion of the release:

11
go.mod
View File

@ -14,6 +14,7 @@ require (
github.com/google/go-cmp v0.6.0 github.com/google/go-cmp v0.6.0
github.com/gosuri/uitable v0.0.4 github.com/gosuri/uitable v0.0.4
github.com/hashicorp/go-getter v1.7.4 github.com/hashicorp/go-getter v1.7.4
github.com/hashicorp/hcl/v2 v2.19.1
github.com/helmfile/chartify v0.19.0 github.com/helmfile/chartify v0.19.0
github.com/helmfile/vals v0.37.0 github.com/helmfile/vals v0.37.0
github.com/imdario/mergo v0.3.16 github.com/imdario/mergo v0.3.16
@ -22,6 +23,8 @@ require (
github.com/stretchr/testify v1.9.0 github.com/stretchr/testify v1.9.0
github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939 github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939
github.com/variantdev/dag v1.1.0 github.com/variantdev/dag v1.1.0
github.com/zclconf/go-cty v1.14.3
github.com/zclconf/go-cty-yaml v1.0.3
go.szostok.io/version v1.2.0 go.szostok.io/version v1.2.0
go.uber.org/zap v1.27.0 go.uber.org/zap v1.27.0
golang.org/x/sync v0.7.0 golang.org/x/sync v0.7.0
@ -65,9 +68,9 @@ require (
github.com/hashicorp/go-multierror v1.1.1 // indirect github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-rootcerts v1.0.2 // indirect github.com/hashicorp/go-rootcerts v1.0.2 // indirect
github.com/hashicorp/go-safetemp v1.0.0 // indirect github.com/hashicorp/go-safetemp v1.0.0 // indirect
github.com/hashicorp/go-slug v0.8.1 // indirect github.com/hashicorp/go-slug v0.12.2 // indirect
github.com/hashicorp/go-sockaddr v1.0.2 // indirect github.com/hashicorp/go-sockaddr v1.0.2 // indirect
github.com/hashicorp/go-tfe v1.2.0 // indirect github.com/hashicorp/go-tfe v1.36.0 // indirect
github.com/hashicorp/golang-lru v1.0.2 // indirect github.com/hashicorp/golang-lru v1.0.2 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d // indirect github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d // indirect
@ -132,9 +135,11 @@ require (
github.com/DopplerHQ/cli v0.5.11-0.20230908185655-7aef4713e1a4 // indirect github.com/DopplerHQ/cli v0.5.11-0.20230908185655-7aef4713e1a4 // indirect
github.com/Microsoft/hcsshim v0.11.4 // indirect github.com/Microsoft/hcsshim v0.11.4 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20230923063757-afb1ddc0824c // indirect github.com/ProtonMail/go-crypto v0.0.0-20230923063757-afb1ddc0824c // indirect
github.com/agext/levenshtein v1.2.3 // indirect
github.com/alessio/shellescape v1.4.1 // indirect github.com/alessio/shellescape v1.4.1 // indirect
github.com/antchfx/jsonquery v1.3.3 // indirect github.com/antchfx/jsonquery v1.3.3 // indirect
github.com/antchfx/xpath v1.2.3 // indirect github.com/antchfx/xpath v1.2.3 // indirect
github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de // indirect github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/atotto/clipboard v0.1.4 // indirect github.com/atotto/clipboard v0.1.4 // indirect
@ -206,7 +211,7 @@ require (
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/gorilla/mux v1.8.0 // indirect github.com/gorilla/mux v1.8.0 // indirect
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 // indirect github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 // indirect
github.com/hashicorp/go-retryablehttp v0.7.1 // indirect github.com/hashicorp/go-retryablehttp v0.7.4 // indirect
github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 // indirect github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 // indirect
github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 // indirect github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 // indirect
github.com/hashicorp/go-version v1.6.0 // indirect github.com/hashicorp/go-version v1.6.0 // indirect

27
go.sum
View File

@ -266,6 +266,8 @@ github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d h1:UrqY+r/O
github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d/go.mod h1:HI8ITrYtUY+O+ZhtlqUnD8+KwNPOyugEhfP9fdUIaEQ= github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d/go.mod h1:HI8ITrYtUY+O+ZhtlqUnD8+KwNPOyugEhfP9fdUIaEQ=
github.com/a8m/envsubst v1.4.2 h1:4yWIHXOLEJHQEFd4UjrWDrYeYlV7ncFWJOCBRLOZHQg= github.com/a8m/envsubst v1.4.2 h1:4yWIHXOLEJHQEFd4UjrWDrYeYlV7ncFWJOCBRLOZHQg=
github.com/a8m/envsubst v1.4.2/go.mod h1:MVUTQNGQ3tsjOOtKCNd+fl8RzhsXcDvvAEzkhGtlsbY= github.com/a8m/envsubst v1.4.2/go.mod h1:MVUTQNGQ3tsjOOtKCNd+fl8RzhsXcDvvAEzkhGtlsbY=
github.com/agext/levenshtein v1.2.3 h1:YB2fHEn0UJagG8T1rrWknE3ZQzWM06O8AMAatNn7lmo=
github.com/agext/levenshtein v1.2.3/go.mod h1:JEDfjyjHDjOF/1e4FlBE/PkbqA9OfWu2ki2W0IB5558=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alessio/shellescape v1.4.1 h1:V7yhSDDn8LP4lc4jS8pFkt0zCnzVJlG5JXy9BVKJUX0= github.com/alessio/shellescape v1.4.1 h1:V7yhSDDn8LP4lc4jS8pFkt0zCnzVJlG5JXy9BVKJUX0=
@ -275,6 +277,8 @@ github.com/antchfx/jsonquery v1.3.3/go.mod h1:1JG4DqRlRCHgVYDPY1ioYFAGSXGfWHzNgr
github.com/antchfx/xpath v1.2.3 h1:CCZWOzv5bAqjVv0offZ2LVgVYFbeldKQVuLNbViZdes= github.com/antchfx/xpath v1.2.3 h1:CCZWOzv5bAqjVv0offZ2LVgVYFbeldKQVuLNbViZdes=
github.com/antchfx/xpath v1.2.3/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs= github.com/antchfx/xpath v1.2.3/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/apparentlymart/go-textseg/v15 v15.0.0 h1:uYvfpb3DyLSCGWnctWKGj857c6ew1u1fNQOlOtuGxQY=
github.com/apparentlymart/go-textseg/v15 v15.0.0/go.mod h1:K8XmNZdhEBkdlyDdvbmmsvpAG721bKi0joRfFdHIWJ4=
github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de h1:FxWPpzIjnTlhPwqqXc4/vE0f7GvRjuAsbW+HOIe8KnA= github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de h1:FxWPpzIjnTlhPwqqXc4/vE0f7GvRjuAsbW+HOIe8KnA=
github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de/go.mod h1:DCaWoUhZrYW9p1lxo/cm8EmUOOzAPSEZNGF2DK1dJgw= github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de/go.mod h1:DCaWoUhZrYW9p1lxo/cm8EmUOOzAPSEZNGF2DK1dJgw=
github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=
@ -638,7 +642,6 @@ github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFb
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I= github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ= github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48= github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
github.com/hashicorp/go-getter v1.7.4 h1:3yQjWuxICvSpYwqSayAdKRFcvBl1y/vogCxczWSmix0= github.com/hashicorp/go-getter v1.7.4 h1:3yQjWuxICvSpYwqSayAdKRFcvBl1y/vogCxczWSmix0=
@ -649,8 +652,8 @@ github.com/hashicorp/go-hclog v1.2.1/go.mod h1:W4Qnvbt70Wk/zYJryRzDRU/4r0kIg0PVH
github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk= github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo= github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM= github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
github.com/hashicorp/go-retryablehttp v0.7.1 h1:sUiuQAnLlbvmExtFQs72iFW/HXeUn8Z1aJLQ4LJJbTQ= github.com/hashicorp/go-retryablehttp v0.7.4 h1:ZQgVdpTdAL7WpMIwLzCfbalOcSUdkDZnpUv3/+BxzFA=
github.com/hashicorp/go-retryablehttp v0.7.1/go.mod h1:vAew36LZh98gCBJNLH42IQ1ER/9wtLZZ8meHqQvEYWY= github.com/hashicorp/go-retryablehttp v0.7.4/go.mod h1:Jy/gPYAdjqffZ/yFGCFV2doI5wjtH1ewM9u8iYVjtX8=
github.com/hashicorp/go-rootcerts v1.0.2 h1:jzhAVGtqPKbwpyCPELlgNWhE1znq+qwJtW5Oi2viEzc= github.com/hashicorp/go-rootcerts v1.0.2 h1:jzhAVGtqPKbwpyCPELlgNWhE1znq+qwJtW5Oi2viEzc=
github.com/hashicorp/go-rootcerts v1.0.2/go.mod h1:pqUvnprVnM5bf7AOirdbb01K4ccR319Vf4pU3K5EGc8= github.com/hashicorp/go-rootcerts v1.0.2/go.mod h1:pqUvnprVnM5bf7AOirdbb01K4ccR319Vf4pU3K5EGc8=
github.com/hashicorp/go-safetemp v1.0.0 h1:2HR189eFNrjHQyENnQMMpCiBAsRxzbTMIgBhEyExpmo= github.com/hashicorp/go-safetemp v1.0.0 h1:2HR189eFNrjHQyENnQMMpCiBAsRxzbTMIgBhEyExpmo=
@ -660,14 +663,14 @@ github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6/go.mod h1:QmrqtbKuxxSWTN3
github.com/hashicorp/go-secure-stdlib/strutil v0.1.1/go.mod h1:gKOamz3EwoIoJq7mlMIRBpVTAUn8qPCrEclOKKWhD3U= github.com/hashicorp/go-secure-stdlib/strutil v0.1.1/go.mod h1:gKOamz3EwoIoJq7mlMIRBpVTAUn8qPCrEclOKKWhD3U=
github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 h1:kes8mmyCpxJsI7FTwtzRqEy9CdjCtrXrXGuOpxEA7Ts= github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 h1:kes8mmyCpxJsI7FTwtzRqEy9CdjCtrXrXGuOpxEA7Ts=
github.com/hashicorp/go-secure-stdlib/strutil v0.1.2/go.mod h1:Gou2R9+il93BqX25LAKCLuM+y9U2T4hlwvT1yprcna4= github.com/hashicorp/go-secure-stdlib/strutil v0.1.2/go.mod h1:Gou2R9+il93BqX25LAKCLuM+y9U2T4hlwvT1yprcna4=
github.com/hashicorp/go-slug v0.8.1 h1:srN7ivgAjHfZddYY1DjBaihRCFy20+vCcOrlx1O2AfE= github.com/hashicorp/go-slug v0.12.2 h1:Gb6nxnV5GI1UVa3aLJGUj66J8AOZFnjIoYalNCp2Cbo=
github.com/hashicorp/go-slug v0.8.1/go.mod h1:Ib+IWBYfEfJGI1ZyXMGNbu2BU+aa3Dzu41RKLH301v4= github.com/hashicorp/go-slug v0.12.2/go.mod h1:JZVtycnZZbiJ4oxpJ/zfhyfBD8XxT4f0uOSyjNLCqFY=
github.com/hashicorp/go-sockaddr v1.0.2 h1:ztczhD1jLxIRjVejw8gFomI1BQZOe2WoVOu0SyteCQc= github.com/hashicorp/go-sockaddr v1.0.2 h1:ztczhD1jLxIRjVejw8gFomI1BQZOe2WoVOu0SyteCQc=
github.com/hashicorp/go-sockaddr v1.0.2/go.mod h1:rB4wwRAUzs07qva3c5SdrY/NEtAUjGlgmH/UkBUC97A= github.com/hashicorp/go-sockaddr v1.0.2/go.mod h1:rB4wwRAUzs07qva3c5SdrY/NEtAUjGlgmH/UkBUC97A=
github.com/hashicorp/go-tfe v1.2.0 h1:L29LCo/qIjOqBUjfiUsZSAzBdxmsOLzwnwZpA+68WW8= github.com/hashicorp/go-tfe v1.36.0 h1:Wq73gjjDo/f9gkKQ5MVSb+4NNJ6T7c5MVTivA0s/bZ0=
github.com/hashicorp/go-tfe v1.2.0/go.mod h1:tJF/OlAXzVbmjiimAPLplSLgwg6kZDUOy0MzHuMwvF4= github.com/hashicorp/go-tfe v1.36.0/go.mod h1:awOuTZ4K9F1EJsKBIoxonJlb7Axn3PIb8YeBLtm/G/0=
github.com/hashicorp/go-uuid v1.0.2 h1:cfejS+Tpcp13yd5nYHWDI6qVCny6wyX2Mt5SGur2IGE= github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
github.com/hashicorp/go-uuid v1.0.2/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-version v1.6.0 h1:feTTfFNnjP967rlCxM/I9g701jU+RN74YKx2mOkIeek= github.com/hashicorp/go-version v1.6.0 h1:feTTfFNnjP967rlCxM/I9g701jU+RN74YKx2mOkIeek=
github.com/hashicorp/go-version v1.6.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= github.com/hashicorp/go-version v1.6.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
@ -676,6 +679,8 @@ github.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iP
github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4= github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
github.com/hashicorp/hcl/v2 v2.19.1 h1://i05Jqznmb2EXqa39Nsvyan2o5XyMowW5fnCKW5RPI=
github.com/hashicorp/hcl/v2 v2.19.1/go.mod h1:ThLC89FV4p9MPW804KVbe/cEXoQ8NZEh+JtMeeGErHE=
github.com/hashicorp/hcp-sdk-go v0.91.0 h1:XFJuB/KWP7kh6u3c/ruUy/lSA272vRrHJ1kzusMx9zw= github.com/hashicorp/hcp-sdk-go v0.91.0 h1:XFJuB/KWP7kh6u3c/ruUy/lSA272vRrHJ1kzusMx9zw=
github.com/hashicorp/hcp-sdk-go v0.91.0/go.mod h1:vQ4fzdL1AmhIAbCw+4zmFe5Hbpajj3NvRWkJoVuxmAk= github.com/hashicorp/hcp-sdk-go v0.91.0/go.mod h1:vQ4fzdL1AmhIAbCw+4zmFe5Hbpajj3NvRWkJoVuxmAk=
github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d h1:9ARUJJ1VVynB176G1HCwleORqCaXm/Vx0uUi0dL26I0= github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d h1:9ARUJJ1VVynB176G1HCwleORqCaXm/Vx0uUi0dL26I0=
@ -950,6 +955,10 @@ github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f h1
github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f/go.mod h1:GlGEuHIJweS1mbCqG+7vt2nvWLzLLnRHbXz5JKd/Qbg= github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f/go.mod h1:GlGEuHIJweS1mbCqG+7vt2nvWLzLLnRHbXz5JKd/Qbg=
github.com/zalando/go-keyring v0.2.3-0.20230503081219-17db2e5354bd h1:D+eeEnOlWcMXbwZ5X3oy68nHafBtGcj1jMKFHtVdybY= github.com/zalando/go-keyring v0.2.3-0.20230503081219-17db2e5354bd h1:D+eeEnOlWcMXbwZ5X3oy68nHafBtGcj1jMKFHtVdybY=
github.com/zalando/go-keyring v0.2.3-0.20230503081219-17db2e5354bd/go.mod h1:sI3evg9Wvpw3+n4SqplGSJUMwtDeROfD4nsFz4z9PG0= github.com/zalando/go-keyring v0.2.3-0.20230503081219-17db2e5354bd/go.mod h1:sI3evg9Wvpw3+n4SqplGSJUMwtDeROfD4nsFz4z9PG0=
github.com/zclconf/go-cty v1.14.3 h1:1JXy1XroaGrzZuG6X9dt7HL6s9AwbY+l4UNL8o5B6ho=
github.com/zclconf/go-cty v1.14.3/go.mod h1:VvMs5i0vgZdhYawQNq5kePSpLAoz8u1xvZgrPIxfnZE=
github.com/zclconf/go-cty-yaml v1.0.3 h1:og/eOQ7lvA/WWhHGFETVWNduJM7Rjsv2RRpx1sdFMLc=
github.com/zclconf/go-cty-yaml v1.0.3/go.mod h1:9YLUH4g7lOhVWqUbctnVlZ5KLpg7JAprQNgxSZ1Gyxs=
go.mongodb.org/mongo-driver v1.14.0 h1:P98w8egYRjYe3XDjxhYJagTokP/H6HzlsnojRgZRd80= go.mongodb.org/mongo-driver v1.14.0 h1:P98w8egYRjYe3XDjxhYJagTokP/H6HzlsnojRgZRd80=
go.mongodb.org/mongo-driver v1.14.0/go.mod h1:Vzb0Mk/pa7e6cWw85R4F/endUC3u0U9jGcNU603k65c= go.mongodb.org/mongo-driver v1.14.0/go.mod h1:Vzb0Mk/pa7e6cWw85R4F/endUC3u0U9jGcNU603k65c=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=

View File

@ -18,6 +18,7 @@ nav:
- Getting Started: - Getting Started:
- Paths Overview: paths.md - Paths Overview: paths.md
- Templating Funcs: templating_funcs.md - Templating Funcs: templating_funcs.md
- HCL Funcs: hcl_funcs.md
- Built-in Objects: builtin-objects.md - Built-in Objects: builtin-objects.md
- Advanced Features: - Advanced Features:
- Best Practices Guide: writing-helmfile.md - Best Practices Guide: writing-helmfile.md

View File

@ -0,0 +1,82 @@
package hcllang
import (
"github.com/hashicorp/hcl/v2/ext/tryfunc"
"github.com/imdario/mergo"
ctyyaml "github.com/zclconf/go-cty-yaml"
"github.com/zclconf/go-cty/cty/function"
"github.com/zclconf/go-cty/cty/function/stdlib"
)
func HCLFunctions(additionnalFunctions map[string]function.Function) (map[string]function.Function, error) {
var hclFunctions = map[string]function.Function{
"abs": stdlib.AbsoluteFunc,
"can": tryfunc.CanFunc,
"ceil": stdlib.CeilFunc,
"chomp": stdlib.ChompFunc,
"coalesce": stdlib.CoalesceFunc,
"coalescelist": stdlib.CoalesceListFunc,
"compact": stdlib.CompactFunc,
"concat": stdlib.ConcatFunc,
"contains": stdlib.ContainsFunc,
"csvdecode": stdlib.CSVDecodeFunc,
"distinct": stdlib.DistinctFunc,
"element": stdlib.ElementFunc,
"chunklist": stdlib.ChunklistFunc,
"flatten": stdlib.FlattenFunc,
"floor": stdlib.FloorFunc,
"format": stdlib.FormatFunc,
"formatdate": stdlib.FormatDateFunc,
"formatlist": stdlib.FormatListFunc,
"indent": stdlib.IndentFunc,
"int": stdlib.IntFunc,
"join": stdlib.JoinFunc,
"jsondecode": stdlib.JSONDecodeFunc,
"jsonencode": stdlib.JSONEncodeFunc,
"keys": stdlib.KeysFunc,
"length": stdlib.LengthFunc,
"log": stdlib.LogFunc,
"lookup": stdlib.LookupFunc,
"lower": stdlib.LowerFunc,
"max": stdlib.MaxFunc,
"merge": stdlib.MergeFunc,
"min": stdlib.MinFunc,
"parseint": stdlib.ParseIntFunc,
"pow": stdlib.PowFunc,
"range": stdlib.RangeFunc,
"regex": stdlib.RegexFunc,
"regexall": stdlib.RegexAllFunc,
"reverselist": stdlib.ReverseListFunc,
"setintersection": stdlib.SetIntersectionFunc,
"setproduct": stdlib.SetProductFunc,
"setsubtract": stdlib.SetSubtractFunc,
"setunion": stdlib.SetUnionFunc,
"signum": stdlib.SignumFunc,
"slice": stdlib.SliceFunc,
"sort": stdlib.SortFunc,
"split": stdlib.SplitFunc,
"strrev": stdlib.ReverseFunc,
"strlen": stdlib.StrlenFunc,
"substr": stdlib.SubstrFunc,
"timeadd": stdlib.TimeAddFunc,
"title": stdlib.TitleFunc,
"trim": stdlib.TrimFunc,
"trimprefix": stdlib.TrimPrefixFunc,
"trimspace": stdlib.TrimSpaceFunc,
"trimsuffix": stdlib.TrimSuffixFunc,
"try": tryfunc.TryFunc,
"upper": stdlib.UpperFunc,
"values": stdlib.ValuesFunc,
"yamldecode": ctyyaml.YAMLDecodeFunc,
"yamlencode": ctyyaml.YAMLEncodeFunc,
"zipmap": stdlib.ZipmapFunc,
}
if additionnalFunctions != nil {
err := mergo.MapWithOverwrite(hclFunctions, additionnalFunctions)
if err != nil {
return nil, err
}
}
return hclFunctions, nil
}

378
pkg/hcllang/hcl_loader.go Normal file
View File

@ -0,0 +1,378 @@
package hcllang
import (
nativejson "encoding/json"
"fmt"
"slices"
"strings"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/hclparse"
"github.com/hashicorp/hcl/v2/hclsyntax"
"github.com/imdario/mergo"
"github.com/variantdev/dag/pkg/dag"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/json"
"go.uber.org/zap"
"github.com/helmfile/helmfile/pkg/filesystem"
)
const (
badIdentifierDetail = "A name must start with a letter or underscore and may contain only letters, digits, underscores, and dashes."
ValuesBlockIdentifier = "values"
LocalsBlockIdentifier = "locals"
valuesAccessorPrefix = "hv"
localsAccessorPrefix = "local"
)
// HelmfileHCLValue represents a single entry from a "values" or "locals" block file.
// The blocks itself is not represented, because it serves only to
// provide context for us to interpret its contents.
type HelmfileHCLValue struct {
Name string
Expr hcl.Expression
Range hcl.Range
}
type HCLLoader struct {
hclFilesPath []string
fs *filesystem.FileSystem
logger *zap.SugaredLogger
}
func NewHCLLoader(fs *filesystem.FileSystem, logger *zap.SugaredLogger) *HCLLoader {
return &HCLLoader{
fs: fs,
logger: logger,
}
}
func (hl *HCLLoader) AddFile(file string) {
hl.hclFilesPath = append(hl.hclFilesPath, file)
}
func (hl *HCLLoader) AddFiles(files []string) {
hl.hclFilesPath = append(hl.hclFilesPath, files...)
}
func (hl *HCLLoader) Length() int {
return len(hl.hclFilesPath)
}
func (hl *HCLLoader) HCLRender() (map[string]any, error) {
if hl.Length() == 0 {
return nil, fmt.Errorf("nothing to render")
}
HelmfileHCLValues, locals, diags := hl.readHCLs()
if len(diags) > 0 {
return nil, diags.Errs()[0]
}
// Decode all locals from all files first
// in order for them to be usable in values blocks
localsCty := map[string]map[string]cty.Value{}
for k, local := range locals {
dagPlan, err := hl.createDAGGraph(local, LocalsBlockIdentifier)
if err != nil {
return nil, err
}
localFileCty, err := hl.decodeGraph(dagPlan, LocalsBlockIdentifier, locals[k], nil)
if err != nil {
return nil, err
}
localsCty[k] = make(map[string]cty.Value)
localsCty[k][localsAccessorPrefix] = localFileCty[localsAccessorPrefix]
}
// Decode Values
dagHelmfileValuePlan, err := hl.createDAGGraph(HelmfileHCLValues, ValuesBlockIdentifier)
if err != nil {
return nil, err
}
helmfileVarCty, err := hl.decodeGraph(dagHelmfileValuePlan, ValuesBlockIdentifier, HelmfileHCLValues, localsCty)
if err != nil {
return nil, err
}
nativeGovals, err := hl.convertToGo(helmfileVarCty)
if err != nil {
return nil, err
}
return nativeGovals, nil
}
func (hl *HCLLoader) createDAGGraph(HelmfileHCLValues map[string]*HelmfileHCLValue, blockType string) (*dag.Topology, error) {
dagGraph := dag.New()
for _, hv := range HelmfileHCLValues {
var traversals []string
for _, tr := range hv.Expr.Variables() {
attr, diags := hl.parseSingleAttrRef(tr, blockType)
if diags != nil {
return nil, fmt.Errorf("%s", diags.Errs()[0])
}
if attr != "" && !slices.Contains(traversals, attr) {
traversals = append(traversals, attr)
}
}
hl.logger.Debugf("Adding Dependency : %s => [%s]", hv.Name, strings.Join(traversals, ", "))
dagGraph.Add(hv.Name, dag.Dependencies(traversals))
}
//Generate Dag Plan which will provide the order from which to interpolate vars
plan, err := dagGraph.Plan(dag.SortOptions{
WithDependencies: true,
})
if err == nil {
return &plan, nil
}
if ude, ok := err.(*dag.UndefinedDependencyError); ok {
var quotedVariableNames []string
for _, d := range ude.Dependents {
quotedVariableNames = append(quotedVariableNames, fmt.Sprintf("%q", d))
}
return nil, fmt.Errorf("variables %s depend(s) on undefined vars %q", strings.Join(quotedVariableNames, ", "), ude.UndefinedNode)
} else {
return nil, fmt.Errorf("error while building the DAG variable graph : %s", err.Error())
}
}
func (hl *HCLLoader) decodeGraph(dagTopology *dag.Topology, blocktype string, vars map[string]*HelmfileHCLValue, additionalLocalContext map[string]map[string]cty.Value) (map[string]cty.Value, error) {
values := map[string]cty.Value{}
helmfileHCLValuesValues := map[string]cty.Value{}
var diags hcl.Diagnostics
hclFunctions, err := HCLFunctions(nil)
if err != nil {
return nil, err
}
for groupIndex := 0; groupIndex < len(*dagTopology); groupIndex++ {
dagNodesInGroup := (*dagTopology)[groupIndex]
for _, node := range dagNodesInGroup {
v := vars[node.String()]
if blocktype != LocalsBlockIdentifier && additionalLocalContext[v.Range.Filename] != nil {
values[localsAccessorPrefix] = additionalLocalContext[v.Range.Filename][localsAccessorPrefix]
}
ctx := &hcl.EvalContext{
Variables: values,
Functions: hclFunctions,
}
// Decode Value
helmfileHCLValuesValues[node.String()], diags = v.Expr.Value(ctx)
if len(diags) > 0 {
return nil, fmt.Errorf("error when trying to evaluate variable %s : %s", v.Name, diags.Errs()[0])
}
switch blocktype {
case ValuesBlockIdentifier:
// Update the eval context for the next value evaluation iteration
values[valuesAccessorPrefix] = cty.ObjectVal(helmfileHCLValuesValues)
// Set back local to nil to avoid an unexpected behavior when the next iteration is in another file
values[localsAccessorPrefix] = cty.NilVal
case LocalsBlockIdentifier:
values[localsAccessorPrefix] = cty.ObjectVal(helmfileHCLValuesValues)
}
}
}
return values, nil
}
func (hl *HCLLoader) readHCLs() (map[string]*HelmfileHCLValue, map[string]map[string]*HelmfileHCLValue, hcl.Diagnostics) {
var variables map[string]*HelmfileHCLValue
var local map[string]*HelmfileHCLValue
locals := map[string]map[string]*HelmfileHCLValue{}
var diags hcl.Diagnostics
for _, file := range hl.hclFilesPath {
variables, local, diags = hl.readHCL(variables, file)
if diags != nil {
return nil, nil, diags
}
locals[file] = make(map[string]*HelmfileHCLValue)
locals[file] = local
}
return variables, locals, nil
}
func (hl *HCLLoader) readHCL(hvars map[string]*HelmfileHCLValue, file string) (map[string]*HelmfileHCLValue, map[string]*HelmfileHCLValue, hcl.Diagnostics) {
src, err := hl.fs.ReadFile(file)
if err != nil {
return nil, nil, hcl.Diagnostics{
{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("%s", err),
Detail: "could not read file",
Subject: &hcl.Range{},
},
}
}
// Parse file as HCL
p := hclparse.NewParser()
hclFile, diags := p.ParseHCL(src, file)
if hclFile == nil || hclFile.Body == nil || diags != nil {
return nil, nil, diags
}
HelmfileHCLValuesSchema := &hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: ValuesBlockIdentifier,
},
{
Type: LocalsBlockIdentifier,
},
},
}
// make sure content has a struct with helmfile_vars Schema defined
content, diags := hclFile.Body.Content(HelmfileHCLValuesSchema)
if diags != nil {
return nil, nil, diags
}
var helmfileLocalsVars map[string]*HelmfileHCLValue
// Decode blocks to return HelmfileHCLValue object => (each var with expr + Name )
if len(content.Blocks.OfType(LocalsBlockIdentifier)) > 1 {
return nil, nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "A file can only support exactly 1 `locals` block",
Subject: &content.Blocks[0].DefRange,
}}
}
for _, block := range content.Blocks {
var helmfileBlockVars map[string]*HelmfileHCLValue
if block.Type == ValuesBlockIdentifier {
helmfileBlockVars, diags = hl.decodeHelmfileHCLValuesBlock(block)
if diags != nil {
return nil, nil, diags
}
}
if block.Type == LocalsBlockIdentifier {
helmfileLocalsVars, diags = hl.decodeHelmfileHCLValuesBlock(block)
if diags != nil {
return nil, nil, diags
}
}
// make sure vars are unique across blocks
for k := range helmfileBlockVars {
if hvars[k] != nil {
var diags hcl.Diagnostics
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Duplicate helmfile_vars definition",
Detail: fmt.Sprintf("The helmfile_var %q was already defined at %s:%d",
k, hvars[k].Range.Filename, hvars[k].Range.Start.Line),
Subject: &helmfileBlockVars[k].Range,
})
return nil, nil, diags
}
}
err = mergo.Merge(&hvars, &helmfileBlockVars)
if err != nil {
var diags hcl.Diagnostics
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Merge failed",
Detail: err.Error(),
Subject: nil,
})
return nil, nil, diags
}
}
return hvars, helmfileLocalsVars, nil
}
func (hl *HCLLoader) decodeHelmfileHCLValuesBlock(block *hcl.Block) (map[string]*HelmfileHCLValue, hcl.Diagnostics) {
attrs, diags := block.Body.JustAttributes()
if len(attrs) == 0 || diags != nil {
return nil, diags
}
hfVars := map[string]*HelmfileHCLValue{}
for name, attr := range attrs {
if !hclsyntax.ValidIdentifier(name) {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid helmfile_vars variable name",
Detail: badIdentifierDetail,
Subject: &attr.NameRange,
})
}
hfVars[name] = &HelmfileHCLValue{
Name: name,
Expr: attr.Expr,
Range: attr.Range,
}
}
return hfVars, diags
}
func (hl *HCLLoader) parseSingleAttrRef(traversal hcl.Traversal, blockType string) (string, hcl.Diagnostics) {
if len(traversal) == 0 {
return "", hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "An empty traversal can't be parsed",
},
}
}
root := traversal.RootName()
// In `values` blocks, Locals are always precomputed, so they don't need to be in the graph
if root == localsAccessorPrefix && blockType != LocalsBlockIdentifier {
return "", nil
}
rootRange := traversal[0].SourceRange()
if len(traversal) < 2 {
return "", hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid reference",
Detail: fmt.Sprintf("The %q object cannot be accessed directly. Instead, access it from one of its root.", root),
Subject: &rootRange,
},
}
}
if attrTrav, ok := traversal[1].(hcl.TraverseAttr); ok {
return attrTrav.Name, nil
}
return "", hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid reference",
Detail: fmt.Sprintf("The %q object does not support this operation.", root),
Subject: traversal[1].SourceRange().Ptr(),
},
}
}
func (hl *HCLLoader) convertToGo(src map[string]cty.Value) (map[string]any, error) {
// Ugly workaround on value conversion
// CTY conversion to go natives requires much processing and complexity
// All of this, in our context, can go away because of the CTY capability to dump a cty.Value as Json
// The Json document outputs 2 keys : "type" and "value" which describe the mapping between the two
// We only care about the value
b, err := json.Marshal(src[valuesAccessorPrefix], cty.DynamicPseudoType)
if err != nil {
return nil, fmt.Errorf("could not marshal cty value : %s", err.Error())
}
var jsonunm map[string]any
err = nativejson.Unmarshal(b, &jsonunm)
if err != nil {
return nil, fmt.Errorf("could not unmarshall json : %s", err.Error())
}
if result, ok := jsonunm["value"].(map[string]any); ok {
return result, nil
} else {
return nil, fmt.Errorf("could extract a map object from json \"value\" key")
}
}

View File

@ -0,0 +1,163 @@
package hcllang
import (
"io"
"testing"
"github.com/google/go-cmp/cmp"
ffs "github.com/helmfile/helmfile/pkg/filesystem"
"github.com/helmfile/helmfile/pkg/helmexec"
)
func newHCLLoader() *HCLLoader {
log := helmexec.NewLogger(io.Discard, "debug")
return &HCLLoader{
fs: ffs.DefaultFileSystem(),
logger: log,
}
}
func TestHCL_localsTraversalsParser(t *testing.T) {
l := newHCLLoader()
files := []string{"testdata/values.1.hcl"}
l.AddFiles(files)
_, filesLocals, diags := l.readHCLs()
if diags != nil {
t.Errorf("Test file parsing error : %s", diags.Errs()[0].Error())
}
actual := make(map[string]map[string]int)
for file, locals := range filesLocals {
actual[file] = make(map[string]int)
for k, v := range locals {
actual[file][k] = len(v.Expr.Variables())
}
}
expected := map[string]map[string]int{
"testdata/values.1.hcl": {
"myLocal": 0,
"myLocalRef": 1,
},
}
if diff := cmp.Diff(expected, actual); diff != "" {
t.Errorf(diff)
}
}
func TestHCL_localsTraversalsAttrParser(t *testing.T) {
l := newHCLLoader()
files := []string{"testdata/values.1.hcl"}
l.AddFiles(files)
_, filesLocals, diags := l.readHCLs()
if diags != nil {
t.Errorf("Test file parsing error : %s", diags.Errs()[0].Error())
}
actual := make(map[string]map[string]string)
for file, locals := range filesLocals {
actual[file] = make(map[string]string)
for k, v := range locals {
str := ""
for _, v := range v.Expr.Variables() {
tr, _ := l.parseSingleAttrRef(v, LocalsBlockIdentifier)
str += tr
}
actual[file][k] = str
}
}
expected := map[string]map[string]string{
"testdata/values.1.hcl": {
"myLocal": "",
"myLocalRef": "myLocal",
},
}
if diff := cmp.Diff(expected, actual); diff != "" {
t.Errorf(diff)
}
}
func TestHCL_valuesTraversalsParser(t *testing.T) {
l := newHCLLoader()
files := []string{"testdata/values.1.hcl"}
l.AddFiles(files)
fileValues, _, diags := l.readHCLs()
if diags != nil {
t.Errorf("Test file parsing error : %s", diags.Errs()[0].Error())
}
actual := make(map[string]int)
for k, v := range fileValues {
actual[k] = len(v.Expr.Variables())
}
expected := map[string]int{
"val1": 0,
"val2": 1,
"val3": 2,
"val4": 1,
}
if diff := cmp.Diff(expected, actual); diff != "" {
t.Errorf(diff)
}
}
func TestHCL_valuesTraversalsAttrParser(t *testing.T) {
l := newHCLLoader()
files := []string{"testdata/values.1.hcl"}
l.AddFiles(files)
fileValues, _, diags := l.readHCLs()
if diags != nil {
t.Errorf("Test file parsing error : %s", diags.Errs()[0].Error())
}
actual := make(map[string]string)
for k, v := range fileValues {
str := ""
for _, v := range v.Expr.Variables() {
tr, _ := l.parseSingleAttrRef(v, ValuesBlockIdentifier)
str += tr
}
actual[k] = str
}
expected := map[string]string{
"val1": "",
"val2": "",
"val3": "val1",
"val4": "val1",
}
if diff := cmp.Diff(expected, actual); diff != "" {
t.Errorf(diff)
}
}
func TestHCL_resultValidate(t *testing.T) {
l := newHCLLoader()
files := []string{"testdata/values.1.hcl"}
l.AddFiles(files)
actual, err := l.HCLRender()
if err != nil {
t.Errorf("Render error : %s", err.Error())
}
expected := map[string]any{
"val1": float64(1),
"val2": "LOCAL",
"val3": "local1",
"val4": float64(-1),
}
if diff := cmp.Diff(expected, actual); diff != "" {
t.Errorf(diff)
}
}

11
pkg/hcllang/testdata/values.1.hcl vendored Normal file
View File

@ -0,0 +1,11 @@
locals {
myLocal = "local"
myLocalRef = local.myLocal
}
values {
val1 = 1
val2 = upper(local.myLocal)
val3 = "${local.myLocal}${hv.val1}"
val4 = min(hv.val1, 10, -1)
}

View File

@ -354,6 +354,22 @@ func (helm *execer) DecryptSecret(context HelmContext, name string, flags ...str
return "", err return "", err
} }
// When the source encrypted file is not a yaml file AND helm secrets < 4
// secrets plugin returns a yaml file with all the content in a yaml `data` key
// which isn't parsable from an hcl perspective
if strings.HasSuffix(name, ".hcl") && pluginVersion.Major() < 4 {
type helmSecretDataV3 struct {
Data string `yaml:"data"`
}
var data helmSecretDataV3
err := yaml.Unmarshal(secretBytes, &data)
if err != nil {
return "", fmt.Errorf("Could not unmarshall helm secret plugin V3 decrypted file to a yaml string\n"+
"You may consider upgrading your helm secrets plugin to >4.0.\n %s", err.Error())
}
secretBytes = []byte(data.Data)
}
secret.bytes = secretBytes secret.bytes = secretBytes
} else { } else {
// Cache hit // Cache hit

View File

@ -4,6 +4,8 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"slices"
"strings"
"github.com/helmfile/vals" "github.com/helmfile/vals"
"github.com/imdario/mergo" "github.com/imdario/mergo"
@ -20,6 +22,7 @@ import (
const ( const (
DefaultHelmBinary = "helm" DefaultHelmBinary = "helm"
DefaultKustomizeBinary = "kustomize" DefaultKustomizeBinary = "kustomize"
DefaultHCLFileExtension = ".hcl"
) )
type StateLoadError struct { type StateLoadError struct {
@ -233,17 +236,19 @@ func (c *StateCreator) loadBases(envValues, overrodeEnv *environment.Environment
// nolint: unparam // nolint: unparam
func (c *StateCreator) loadEnvValues(st *HelmState, name string, failOnMissingEnv bool, ctxEnv, overrode *environment.Environment) (*environment.Environment, error) { func (c *StateCreator) loadEnvValues(st *HelmState, name string, failOnMissingEnv bool, ctxEnv, overrode *environment.Environment) (*environment.Environment, error) {
envVals := map[string]any{} secretVals := map[string]any{}
valuesVals := map[string]any{}
envSpec, ok := st.Environments[name] envSpec, ok := st.Environments[name]
decryptedFiles := []string{}
if ok { if ok {
var err error var err error
envVals, err = st.loadValuesEntries(envSpec.MissingFileHandler, envSpec.Values, c.remote, ctxEnv, name) // To keep supporting the secrets entries having precedence over the values
if err != nil { // This require to be done in 2 steps for HCL encrypted file support :
return nil, err // 1. Get the Secrets
} // 2. Merge the secrets with the envValues after
// Also makes the fail +- faster as it's trying to decrypt before loading values
if len(envSpec.Secrets) > 0 {
var envSecretFiles []string var envSecretFiles []string
if len(envSpec.Secrets) > 0 {
for _, urlOrPath := range envSpec.Secrets { for _, urlOrPath := range envSpec.Secrets {
resolved, skipped, err := st.storage().resolveFile(envSpec.MissingFileHandler, "environment values", urlOrPath, envSpec.MissingFileHandlerConfig.resolveFileOptions()...) resolved, skipped, err := st.storage().resolveFile(envSpec.MissingFileHandler, "environment values", urlOrPath, envSpec.MissingFileHandlerConfig.resolveFileOptions()...)
if err != nil { if err != nil {
@ -252,18 +257,40 @@ func (c *StateCreator) loadEnvValues(st *HelmState, name string, failOnMissingEn
if skipped { if skipped {
continue continue
} }
envSecretFiles = append(envSecretFiles, resolved...) envSecretFiles = append(envSecretFiles, resolved...)
} }
if err = c.scatterGatherEnvSecretFiles(st, envSecretFiles, envVals); err != nil { keepSecretFilesExtensions := []string{DefaultHCLFileExtension}
decryptedFiles, err = c.scatterGatherEnvSecretFiles(st, envSecretFiles, secretVals, keepSecretFilesExtensions)
if err != nil {
return nil, err return nil, err
} }
defer func() {
for _, file := range decryptedFiles {
if err := c.fs.DeleteFile(file); err != nil {
c.logger.Warnf("failed removing decrypted file %s: %w", file, err)
}
}
}()
}
var valuesFiles []any
for _, f := range decryptedFiles {
valuesFiles = append(valuesFiles, f)
}
envValuesEntries := append(valuesFiles, envSpec.Values...)
valuesVals, err = st.loadValuesEntries(envSpec.MissingFileHandler, envValuesEntries, c.remote, ctxEnv, name)
if err != nil {
return nil, err
}
if err = mergo.Merge(&valuesVals, &secretVals, mergo.WithOverride); err != nil {
return nil, err
} }
} else if ctxEnv == nil && overrode == nil && name != DefaultEnv && failOnMissingEnv { } else if ctxEnv == nil && overrode == nil && name != DefaultEnv && failOnMissingEnv {
return nil, &UndefinedEnvError{Env: name} return nil, &UndefinedEnvError{Env: name}
} }
newEnv := &environment.Environment{Name: name, Values: envVals, KubeContext: envSpec.KubeContext} newEnv := &environment.Environment{Name: name, Values: valuesVals, KubeContext: envSpec.KubeContext}
if ctxEnv != nil { if ctxEnv != nil {
intCtxEnv := *ctxEnv intCtxEnv := *ctxEnv
@ -288,9 +315,14 @@ func (c *StateCreator) loadEnvValues(st *HelmState, name string, failOnMissingEn
return newEnv, nil return newEnv, nil
} }
func (c *StateCreator) scatterGatherEnvSecretFiles(st *HelmState, envSecretFiles []string, envVals map[string]any) error { // For all keepFileExtensions, the decrypted files will be retained
// with the specified extensions
// They will not be parsed nor added to the envVals.
// Only their decrypted filePath will be returned
// Up to the caller to remove them
func (c *StateCreator) scatterGatherEnvSecretFiles(st *HelmState, envSecretFiles []string, envVals map[string]any, keepFileExtensions []string) ([]string, error) {
var errs []error var errs []error
var decryptedFilesKeeper []string
helm := c.getHelm(st) helm := c.getHelm(st)
inputs := envSecretFiles inputs := envSecretFiles
inputsSize := len(inputs) inputsSize := len(inputs)
@ -325,14 +357,21 @@ func (c *StateCreator) scatterGatherEnvSecretFiles(st *HelmState, envSecretFiles
results <- secretResult{secret.id, nil, err, secret.path} results <- secretResult{secret.id, nil, err, secret.path}
continue continue
} }
for _, ext := range keepFileExtensions {
if strings.HasSuffix(secret.path, ext) {
decryptedFilesKeeper = append(decryptedFilesKeeper, decFile)
}
}
// nolint: staticcheck // nolint: staticcheck
defer func() { defer func() {
if !slices.Contains(decryptedFilesKeeper, decFile) {
if err := c.fs.DeleteFile(decFile); err != nil { if err := c.fs.DeleteFile(decFile); err != nil {
c.logger.Warnf("removing decrypted file %s: %w", decFile, err) c.logger.Warnf("removing decrypted file %s: %w", decFile, err)
} }
}
}() }()
var vals map[string]any
if !slices.Contains(decryptedFilesKeeper, decFile) {
bytes, err := c.fs.ReadFile(decFile) bytes, err := c.fs.ReadFile(decFile)
if err != nil { if err != nil {
results <- secretResult{secret.id, nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", secret.path, err), secret.path} results <- secretResult{secret.id, nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", secret.path, err), secret.path}
@ -346,11 +385,12 @@ func (c *StateCreator) scatterGatherEnvSecretFiles(st *HelmState, envSecretFiles
// All the nested map key should be string. Otherwise we get strange errors due to that // All the nested map key should be string. Otherwise we get strange errors due to that
// mergo or reflect is unable to merge map[any]any with map[string]any or vice versa. // mergo or reflect is unable to merge map[any]any with map[string]any or vice versa.
// See https://github.com/roboll/helmfile/issues/677 // See https://github.com/roboll/helmfile/issues/677
vals, err := maputil.CastKeysToStrings(m) vals, err = maputil.CastKeysToStrings(m)
if err != nil { if err != nil {
results <- secretResult{secret.id, nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", secret.path, err), secret.path} results <- secretResult{secret.id, nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", secret.path, err), secret.path}
continue continue
} }
}
results <- secretResult{secret.id, vals, nil, secret.path} results <- secretResult{secret.id, vals, nil, secret.path}
} }
}, },
@ -379,9 +419,9 @@ func (c *StateCreator) scatterGatherEnvSecretFiles(st *HelmState, envSecretFiles
for _, err := range errs { for _, err := range errs {
st.logger.Error(err) st.logger.Error(err)
} }
return fmt.Errorf("failed loading environment secrets with %d errors", len(errs)) return decryptedFilesKeeper, fmt.Errorf("failed loading environment secrets with %d errors", len(errs))
} }
return nil return decryptedFilesKeeper, nil
} }
func (st *HelmState) loadValuesEntries(missingFileHandler *string, entries []any, remote *remote.Remote, ctxEnv *environment.Environment, envName string) (map[string]any, error) { func (st *HelmState) loadValuesEntries(missingFileHandler *string, entries []any, remote *remote.Remote, ctxEnv *environment.Environment, envName string) (map[string]any, error) {

View File

@ -3,12 +3,14 @@ package state
import ( import (
"fmt" "fmt"
"path/filepath" "path/filepath"
"strings"
"github.com/imdario/mergo" "github.com/imdario/mergo"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/helmfile/helmfile/pkg/environment" "github.com/helmfile/helmfile/pkg/environment"
"github.com/helmfile/helmfile/pkg/filesystem" "github.com/helmfile/helmfile/pkg/filesystem"
"github.com/helmfile/helmfile/pkg/hcllang"
"github.com/helmfile/helmfile/pkg/maputil" "github.com/helmfile/helmfile/pkg/maputil"
"github.com/helmfile/helmfile/pkg/remote" "github.com/helmfile/helmfile/pkg/remote"
"github.com/helmfile/helmfile/pkg/tmpl" "github.com/helmfile/helmfile/pkg/tmpl"
@ -35,7 +37,11 @@ func NewEnvironmentValuesLoader(storage *Storage, fs *filesystem.FileSystem, log
} }
func (ld *EnvironmentValuesLoader) LoadEnvironmentValues(missingFileHandler *string, valuesEntries []any, ctxEnv *environment.Environment, envName string) (map[string]any, error) { func (ld *EnvironmentValuesLoader) LoadEnvironmentValues(missingFileHandler *string, valuesEntries []any, ctxEnv *environment.Environment, envName string) (map[string]any, error) {
result := map[string]any{} var (
result = map[string]any{}
hclLoader = hcllang.NewHCLLoader(ld.fs, ld.logger)
err error
)
for _, entry := range valuesEntries { for _, entry := range valuesEntries {
maps := []any{} maps := []any{}
@ -49,7 +55,6 @@ func (ld *EnvironmentValuesLoader) LoadEnvironmentValues(missingFileHandler *str
if skipped { if skipped {
continue continue
} }
for _, f := range files { for _, f := range files {
var env environment.Environment var env environment.Environment
if ctxEnv == nil { if ctxEnv == nil {
@ -57,7 +62,9 @@ func (ld *EnvironmentValuesLoader) LoadEnvironmentValues(missingFileHandler *str
} else { } else {
env = *ctxEnv env = *ctxEnv
} }
if strings.HasSuffix(f, ".hcl") {
hclLoader.AddFile(f)
} else {
tmplData := NewEnvironmentTemplateData(env, "", map[string]any{}) tmplData := NewEnvironmentTemplateData(env, "", map[string]any{})
r := tmpl.NewFileRenderer(ld.fs, filepath.Dir(f), tmplData) r := tmpl.NewFileRenderer(ld.fs, filepath.Dir(f), tmplData)
bytes, err := r.RenderToBytes(f) bytes, err := r.RenderToBytes(f)
@ -71,11 +78,34 @@ func (ld *EnvironmentValuesLoader) LoadEnvironmentValues(missingFileHandler *str
maps = append(maps, m) maps = append(maps, m)
ld.logger.Debugf("envvals_loader: loaded %s:%v", strOrMap, m) ld.logger.Debugf("envvals_loader: loaded %s:%v", strOrMap, m)
} }
}
case map[any]any, map[string]any: case map[any]any, map[string]any:
maps = append(maps, strOrMap) maps = append(maps, strOrMap)
default: default:
return nil, fmt.Errorf("unexpected type of value: value=%v, type=%T", strOrMap, strOrMap) return nil, fmt.Errorf("unexpected type of value: value=%v, type=%T", strOrMap, strOrMap)
} }
result, err = mapMerge(result, maps)
if err != nil {
return nil, err
}
}
maps := []any{}
if hclLoader.Length() > 0 {
m, err := hclLoader.HCLRender()
if err != nil {
return nil, err
}
maps = append(maps, m)
}
result, err = mapMerge(result, maps)
if err != nil {
return nil, err
}
return result, nil
}
func mapMerge(dest map[string]any, maps []any) (map[string]any, error) {
for _, m := range maps { for _, m := range maps {
// All the nested map key should be string. Otherwise we get strange errors due to that // All the nested map key should be string. Otherwise we get strange errors due to that
// mergo or reflect is unable to merge map[any]any with map[string]any or vice versa. // mergo or reflect is unable to merge map[any]any with map[string]any or vice versa.
@ -84,11 +114,9 @@ func (ld *EnvironmentValuesLoader) LoadEnvironmentValues(missingFileHandler *str
if err != nil { if err != nil {
return nil, err return nil, err
} }
if err := mergo.Merge(&result, &vals, mergo.WithOverride); err != nil { if err := mergo.Merge(&dest, &vals, mergo.WithOverride); err != nil {
return nil, fmt.Errorf("failed to merge %v: %v", m, err) return nil, fmt.Errorf("failed to merge %v: %v", m, err)
} }
} }
} return dest, nil
return result, nil
} }

View File

@ -187,3 +187,44 @@ func TestEnvValsLoad_OverwriteEmptyValue_Issue1168(t *testing.T) {
t.Errorf(diff) t.Errorf(diff)
} }
} }
func TestEnvValsLoad_MultiHCL(t *testing.T) {
l := newLoader()
actual, err := l.LoadEnvironmentValues(nil, []any{"testdata/values.7.hcl", "testdata/values.8.hcl"}, nil, "")
if err != nil {
t.Fatal(err)
}
expected := map[string]any{
"a": "a",
"b": "b",
"c": "ab",
"map": map[string]any{
"a": "a",
},
"list": []any{
"b",
},
"nestedmap": map[string]any{
"submap": map[string]any{
"subsubmap": map[string]any{
"hello": "ab",
},
},
},
"ternary": true,
"fromMap": "aab",
"expressionInText": "yes",
"insideFor": "b",
"multi_block": "block",
"block": "block",
"crossfile": "crossfile var",
"crossfile_var": "crossfile var",
"localRef": "localInValues7",
}
if diff := cmp.Diff(expected, actual); diff != "" {
t.Errorf(diff)
}
}

42
pkg/state/testdata/values.7.hcl vendored Normal file
View File

@ -0,0 +1,42 @@
locals {
localInValues7File = "localInValues7"
}
values {
a = "a"
b = "b"
c = "${hv.a}${hv.b}"
map = {
"a" = "${hv.a}"
}
list = [
hv.b
]
nestedmap = {
submap = {
subsubmap = {
hello = hv.c
}
}
}
ternary = true ? true : false
fromMap = "${hv.map.a}${hv.nestedmap.submap.subsubmap.hello}"
expressionInText = "%{if hv.ternary }yes%{else}no%{endif}"
insideFor = "%{for i in hv.list }${i}%{endfor}"
multi_block = hv.block
crossfile = hv.crossfile_var
localRef = local.localInValues7File
}
values {
block = "block"
}

3
pkg/state/testdata/values.8.hcl vendored Normal file
View File

@ -0,0 +1,3 @@
values {
crossfile_var = "crossfile var"
}

View File

@ -0,0 +1,4 @@
helmfileArgs:
- template
- --concurrency=1
- -q

View File

@ -0,0 +1,38 @@
environments:
default:
values:
- values1.hcl
- values1.yaml
- values2.hcl
---
helmDefaults:
kubeContext: minikube
releases:
- name: hcl-demo
chart: ../../charts/raw-0.1.0
values:
- templates:
- |
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap
data:
a: {{ .Values.a }}
b: {{ .Values.b }}
c: {{ .Values.c }}
crossfile: {{ .Values.crossfile }}
expressionInText: "{{ .Values.expressionInText }}"
fromMap: {{ .Values.fromMap }}
insideFor: {{ .Values.insideFor }}
list: "{{ .Values.list }}"
{{- .Values.locals | toYaml | nindent 10 -}}
multiBlock: {{ .Values.multiBlock }}
simpleCompute: "{{ .Values.simpleCompute }}"
yamlOverride: {{ .Values.yamlOverride }}

View File

@ -0,0 +1,20 @@
---
# Source: raw/templates/resources.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap
data:
a: a
b: b
c: ab
crossfile: values2.hcl
expressionInText: "yes"
fromMap: ab
insideFor: b
list: "[b]"
filename: values1.hcl
multiBlock: block
simpleCompute: "4"
yamlOverride: yaml_overrode

View File

@ -0,0 +1,53 @@
locals {
filename = "values1.hcl"
}
values {
a = "a"
b = "b"
c = "${hv.a}${hv.b}"
list = [
hv.b
]
locals = local
// Maps and some functions
nestedmap = {
submap = {
subsubmap = {
hello = hv.a
}
}
}
nestedmap2 = {
submap = {
subsubmap = {
hello = hv.c
}
}
}
merged = merge(hv.nestedmap, hv.nestedmap2)
fromMap = "${hv.merged.submap.subsubmap.hello}"
// precedence demo
yamlOverride = "yaml_overrode"
// Simple Expressions
ternary = true ? true : false
expressionInText = "%{if hv.ternary }yes%{else}no%{endif}"
insideFor = "%{for i in hv.list }${i}%{endfor}"
simpleCompute = 2 + 2
multiBlock = hv.block
crossfile = hv.crossfile_var
}
values {
block = "block"
}

View File

@ -0,0 +1 @@
yamlOverride: "will_be_overriden"

View File

@ -0,0 +1,7 @@
locals {
filename = "values2.hcl"
}
values {
crossfile_var = local.filename
}

View File

@ -94,6 +94,7 @@ ${kubectl} create namespace ${test_ns} || fail "Could not create namespace ${tes
. ${dir}/test-cases/chartify.sh . ${dir}/test-cases/chartify.sh
. ${dir}/test-cases/deps-mr-1011.sh . ${dir}/test-cases/deps-mr-1011.sh
. ${dir}/test-cases/deps-kustomization-i-1402.sh . ${dir}/test-cases/deps-kustomization-i-1402.sh
. ${dir}/test-cases/hcl-secrets.sh
# ALL DONE ----------------------------------------------------------------------------------------------------------- # ALL DONE -----------------------------------------------------------------------------------------------------------

View File

@ -0,0 +1,28 @@
export VAULT_ADDR=http://127.0.0.1:8200
export VAULT_TOKEN=toor
info "Inject sops key"
sops="sops --hc-vault-transit $VAULT_ADDR/v1/sops/keys/key"
hcl_secrets_case_input_dir="${cases_dir}/hcl-secrets/input"
hcl_secrets_case_output_dir="${cases_dir}/hcl-secrets/output"
mkdir -p ${hcl_secrets_case_input_dir}/tmp
info "Encrypt secrets"
${sops} -e ${hcl_secrets_case_input_dir}/secrets.hcl > ${hcl_secrets_case_input_dir}/tmp/secrets.hcl || fail "${sops} failed at ${hcl_secrets_case_input_dir}/secrets.hcl"
${sops} -e ${hcl_secrets_case_input_dir}/secrets.yaml > ${hcl_secrets_case_input_dir}/tmp/secrets.yaml || fail "${sops} failed at ${hcl_secrets_case_input_dir}/secrets.yaml"
info "values precedence order : yamlFile < hcl = hclSecrets < secretYamlFile"
test_start "hcl-yaml-mix - should output secrets with proper overrides"
hcl_secrets_tmp=$(mktemp -d)
result=${hcl_secrets_tmp}/result.yaml
info "Building output"
${helmfile} -f ${hcl_secrets_case_input_dir}/_helmfile.yaml.gotmpl template --skip-deps > ${result} || fail "\"helmfile template\" shouldn't fail"
diff -u ${hcl_secrets_case_output_dir}/output.yaml ${result} || fail "helmdiff should be consistent"
echo code=$?
test_pass "hcl-yaml-mix"

View File

@ -0,0 +1,28 @@
environments:
default:
values:
- values.hcl
- values.yaml
secrets:
- tmp/secrets.hcl
- tmp/secrets.yaml
---
helmDefaults:
kubeContext: minikube
releases:
- name: hcl-demo
chart: ../../../charts/raw
values:
- templates:
- |
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap
data:
secretOveriddenByPrecedence: {{ .Values.secretOveriddenByPrecedence }}
secretRef: {{ .Values.secretref }}
yamlOverride: {{ .Values.yamlOverride }}

View File

@ -0,0 +1,3 @@
values {
secret = "mySecretValue"
}

View File

@ -0,0 +1 @@
secretOveriddenByPrecedence: OverrodeHclVarByPrecedence

View File

@ -0,0 +1,6 @@
values {
// Secrets and precedence demo
secretref = upper(hv.secret)
yamlOverride = "yaml_overrode"
secretOveriddenByPrecedence = "will_be_overwrittten"
}

View File

@ -0,0 +1 @@
yamlOverride: "will_be_overriden"

View File

@ -0,0 +1,11 @@
---
# Source: raw/templates/resources.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap
data:
secretOveriddenByPrecedence: OverrodeHclVarByPrecedence
secretRef: MYSECRETVALUE
yamlOverride: yaml_overrode