Files
helmfile/pkg/maputil/maputil.go
T
Aditya Menon b91fd534ec Fix four critical bugs: array merging (#2281), AWS SDK logging (#2270), helmDefaults skip flags (#2269), and OCI chart versions (#2247) (#2288)
* fix: resolve issues #2281, #2270, #2269, and #2247

This commit addresses four critical bugs in helmfile:

1. **Issue #2281**: Fix array merging in --state-values-set
   - Problem: Arrays were being replaced entirely instead of merged element-by-element
   - Root cause: MergeMaps() didn't handle arrays, and mergo.Merge was used in some places
   - Solution:
     * Enhanced MergeMaps() with mergeSlices() and toInterfaceSlice() functions
     * Replaced mergo.Merge calls with MergeMaps in environment.go and create.go
     * Arrays now merge element-by-element, with nested maps merged recursively
   - Files changed:
     * pkg/maputil/maputil.go - Added array merging logic
     * pkg/maputil/maputil_test.go - Added comprehensive unit tests
     * pkg/environment/environment.go - Use MergeMaps instead of mergo.Merge
     * pkg/state/create.go - Use MergeMaps instead of mergo.Merge
     * test/integration/test-cases/issue-2281-array-merge/ - Integration test
     * test/integration/run.sh - Added new integration test

2. **Issue #2270**: Suppress AWS SDK debug logging
   - Problem: AWS SDK debug logs exposing sensitive information (tokens, auth headers)
   - Root cause: vals.New() called without LogOutput option
   - Solution: Set LogOutput to io.Discard in ValsInstance()
   - Files changed:
     * pkg/plugins/vals.go - Added LogOutput: io.Discard option

3. **Issue #2269**: Fix helmDefaults.skipDeps and helmDefaults.skipRefresh being ignored
   - Problem: skipRefresh only checked CLI flags, not helmDefaults or release settings
   - Root cause: Incomplete calculation at line 1559 in state.go
   - Solution: Added proper skipRefresh calculation mirroring skipDeps logic
   - Files changed:
     * pkg/state/state.go - Fixed skipRefresh calculation (lines 1522-1525, 1564)
     * pkg/state/skip_test.go - Added unit tests for skipDeps and skipRefresh

4. **Issue #2247**: Allow OCI charts without explicit version
   - Problem: OCI charts without version defaulted to "latest" which was then rejected
   - Root cause: getOCIQualifiedChartName() defaulted chartVersion to "latest"
   - Solution: Use release.Version directly without defaulting, only reject explicit "latest"
   - Files changed:
     * pkg/state/state.go - Remove default to "latest", use empty string
     * pkg/state/oci_chart_version_test.go - Added comprehensive unit tests
     * test/integration/test-cases/issue-2247/ - Integration test with registry
     * test/integration/run.sh - Added new integration test

Fixes #2281, #2270, #2269, #2247

Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

* fix: correct integration test for issue #2281 array merging

The helmfile template needed to pass the 'top' values to the chart
so that .Values.top is accessible in the template context.

Changes:
- Pass state values to chart values using toYaml
- Adjusted indentation for proper YAML structure
- Template now correctly accesses .Values.top for array data

Test output now matches expected output with proper element-by-element
array merging.

Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

* fix: make Helm version parsing more robust in issue-2247 test

Improved version parsing to handle edge cases in CI environments:
- Added fallback to 3.8 if version parsing fails
- Added default values for HELM_MAJOR and HELM_MINOR
- Prevents test failures due to version detection issues

This ensures the test runs correctly across different environments
and Helm versions.

Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

* debug: add diagnostic output for issue-2247 test failure

Added debug logging to show:
- helmfile command output when it succeeds unexpectedly
- Helm version being used by the test

This will help diagnose why the validation isn't triggering in CI.

Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

* fix: make OCI 'latest' validation work for all Helm versions

The validation for explicit 'latest' in OCI charts was depending on
helm.IsVersionAtLeast("3.8.0") which could fail if Helm version
detection has issues in CI environments.

Changes:
- Remove Helm version check from validation
- Always reject explicit 'latest' for OCI charts
- Remove Helm version check from integration test
- Update unit tests to expect 'latest' to fail for all Helm versions

This ensures consistent behavior across all environments and
Helm versions, fixing the CI failure where helm version detection
was problematic.

Fixes integration test failure in CI.

Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

* fix: remove unused helm parameter from getOCIQualifiedChartName

Since the Helm version check was removed from the OCI validation,
the helm parameter is no longer needed in getOCIQualifiedChartName.

Changes:
- Removed helm parameter from function signature
- Updated all callers to not pass helm argument
- Removed unused mockHelmExec test implementation
- Removed unused imports (testutil, helmexec, chart)

This resolves the golangci-lint unparam error.

Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

* test: update TestGetOCIQualifiedChartName to expect 'latest' rejection

Updated test case for Helm 3.7.0 to expect error when using 'latest'
since we now reject explicit 'latest' for all Helm versions, not just
>= 3.8.0.

This aligns the test with the updated validation logic that ensures
consistent behavior across all Helm versions.

Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

* fix: handle set -e in issue-2247 integration test

The integration test script is sourced by run.sh which has `set -e`
enabled. When helmfile commands fail (as expected for validation tests),
the script would exit immediately before capturing the exit code.

This fix temporarily disables `set -e` around each helmfile command that
may fail, allowing proper exit code capture and validation.

This resolves the persistent CI test failure where the test would exit
at Test 1.1 without showing any error message.

Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

* fix: add set -e handling for helm commands in issue-2247 test

Extends the previous set -e fix to cover helm package and push commands
in the registry tests (Test 2.2). These commands can fail and need proper
error handling without triggering immediate script exit.

This ensures:
- helm package failures are caught and handled gracefully
- helm push failures are caught and handled gracefully
- Test can skip registry tests and pass with validation-only results
- set -e is properly re-enabled after each command sequence

Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

---------

Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
2025-11-22 09:27:51 +08:00

311 lines
6.0 KiB
Go

package maputil
import (
"fmt"
"strconv"
"strings"
)
func CastKeysToStrings(s any) (map[string]any, error) {
new := map[string]any{}
switch src := s.(type) {
case map[any]any:
for k, v := range src {
var strK string
switch typedK := k.(type) {
case string:
strK = typedK
default:
return nil, fmt.Errorf("unexpected type of key in map: expected string, got %T: value=%v, map=%v", typedK, typedK, src)
}
castedV, err := RecursivelyStringifyMapKey(v)
if err != nil {
return nil, err
}
new[strK] = castedV
}
case map[string]any:
for k, v := range src {
castedV, err := RecursivelyStringifyMapKey(v)
if err != nil {
return nil, err
}
new[k] = castedV
}
}
return new, nil
}
func RecursivelyStringifyMapKey(v any) (any, error) {
var castedV any
switch typedV := v.(type) {
case map[any]any, map[string]any:
tmp, err := CastKeysToStrings(typedV)
if err != nil {
return nil, err
}
castedV = tmp
case []any:
a := []any{}
for i := range typedV {
res, err := RecursivelyStringifyMapKey(typedV[i])
if err != nil {
return nil, err
}
a = append(a, res)
}
castedV = a
default:
castedV = typedV
}
return castedV, nil
}
type arg interface {
getMap(map[string]any) map[string]any
set(map[string]any, any)
}
type keyArg struct {
key string
}
func (a keyArg) getMap(m map[string]any) map[string]any {
_, ok := m[a.key]
if !ok {
m[a.key] = map[string]any{}
}
switch t := m[a.key].(type) {
case map[string]any:
return t
default:
panic(fmt.Errorf("unexpected type: %v(%T)", t, t))
}
}
func (a keyArg) set(m map[string]any, value any) {
m[a.key] = value
}
type indexedKeyArg struct {
key string
index int
}
func (a indexedKeyArg) getArray(m map[string]any) []any {
_, ok := m[a.key]
if !ok {
m[a.key] = make([]any, a.index+1)
}
switch t := m[a.key].(type) {
case []any:
if len(t) <= a.index {
t2 := make([]any, a.index+1)
copy(t2, t)
t = t2
}
return t
default:
panic(fmt.Errorf("unexpected type: %v(%T)", t, t))
}
}
func (a indexedKeyArg) getMap(m map[string]any) map[string]any {
t := a.getArray(m)
if t[a.index] == nil {
t[a.index] = map[string]any{}
}
switch t := t[a.index].(type) {
case map[string]any:
return t
default:
panic(fmt.Errorf("unexpected type: %v(%T)", t, t))
}
}
func (a indexedKeyArg) set(m map[string]any, value any) {
t := a.getArray(m)
t[a.index] = value
m[a.key] = t
}
func getCursor(key string) arg {
key = strings.ReplaceAll(key, "[", " ")
key = strings.ReplaceAll(key, "]", " ")
k := key
idx := 0
n, err := fmt.Sscanf(key, "%s %d", &k, &idx)
if n == 2 && err == nil {
return indexedKeyArg{
key: k,
index: idx,
}
}
return keyArg{
key: key,
}
}
func ParseKey(key string) []string {
r := []string{}
part := ""
escaped := false
for _, rune := range key {
split := false
switch {
case !escaped && rune == '\\':
escaped = true
continue
case rune == '.':
split = !escaped
}
escaped = false
if split {
r = append(r, part)
part = ""
} else {
part += string(rune)
}
}
if len(part) > 0 {
r = append(r, part)
}
return r
}
func Set(m map[string]any, key []string, value string, stringBool bool) {
if len(key) == 0 {
panic(fmt.Errorf("bug: unexpected length of key: %d", len(key)))
}
for len(key) > 1 {
m, key = getCursor(key[0]).getMap(m), key[1:]
}
getCursor(key[0]).set(m, typedVal(value, stringBool))
}
func typedVal(val string, st bool) any {
// if st is true, directly return it without casting it
if st {
return val
}
if strings.EqualFold(val, "true") {
return true
}
if strings.EqualFold(val, "false") {
return false
}
if strings.EqualFold(val, "null") {
return nil
}
// handling of only zero, if val has zero prefix, it will be considered as string
if strings.EqualFold(val, "0") {
return int64(0)
}
// If this value does not start with zero, try parsing it to an int
if len(val) != 0 && val[0] != '0' {
if iv, err := strconv.ParseInt(val, 10, 64); err == nil {
return iv
}
}
return val
}
func MergeMaps(a, b map[string]interface{}) map[string]interface{} {
out := make(map[string]interface{}, len(a))
// fill the out map with the first map
for k, v := range a {
out[k] = v
}
for k, v := range b {
if v == nil {
continue
}
if v, ok := v.(map[string]interface{}); ok {
if bv, ok := out[k]; ok {
if bv, ok := bv.(map[string]interface{}); ok {
// if b and out map has a map value, merge it too
out[k] = MergeMaps(bv, v)
continue
}
}
}
// Handle array merging element-by-element
vSlice := toInterfaceSlice(v)
if vSlice != nil {
if outVal, exists := out[k]; exists {
outSlice := toInterfaceSlice(outVal)
if outSlice != nil {
// Both are slices - merge element by element
out[k] = mergeSlices(outSlice, vSlice)
continue
}
}
}
out[k] = v
}
return out
}
// toInterfaceSlice converts various slice types to []interface{}
func toInterfaceSlice(v any) []any {
if slice, ok := v.([]any); ok {
return slice
}
return nil
}
// mergeSlices merges two slices element by element
// Elements from override (b) take precedence, but we preserve elements from base (a)
// that don't exist in override
func mergeSlices(base, override []any) []any {
// Determine the maximum length
maxLen := len(base)
if len(override) > maxLen {
maxLen = len(override)
}
result := make([]interface{}, maxLen)
// First copy all elements from base
copy(result, base)
// Then merge/override with elements from override
for i := 0; i < len(override); i++ {
overrideVal := override[i]
// Skip nil values in override - they represent unset indices
if overrideVal == nil {
continue
}
// If both are maps, merge them recursively
if overrideMap, ok := overrideVal.(map[string]any); ok {
if i < len(base) {
if baseMap, ok := base[i].(map[string]any); ok {
result[i] = MergeMaps(baseMap, overrideMap)
continue
}
}
}
// Otherwise, override completely
result[i] = overrideVal
}
return result
}