Files
helmfile/test/integration/test-cases/chart-needs.sh
T
yxxheroandJavex 9964a2eacb feat: Ensure repo update is only run once (#2378)
* feat: Ensure repo update is only run once

Perform a single Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "glm-bitnami" chart repository
...Unable to get an update from the "fluent" chart repository (https://fluent.github.io/helm-charts):
	Get "https://fluent.github.io/helm-charts/index.yaml": read tcp 192.168.0.104:51893->185.199.108.153:443: read: connection reset by peer
...Unable to get an update from the "grafana" chart repository (https://grafana.github.io/helm-charts):
	Get "https://grafana.github.io/helm-charts/index.yaml": read tcp 192.168.0.104:51897->185.199.109.153:443: read: connection reset by peer
...Unable to get an update from the "ingress-nginx" chart repository (https://kubernetes.github.io/ingress-nginx):
	Get "https://kubernetes.github.io/ingress-nginx/index.yaml": read tcp 192.168.0.104:51894->185.199.110.153:443: read: connection reset by peer
...Unable to get an update from the "chartmuseum" chart repository (https://chartmuseum.github.io/charts):
	Get "https://chartmuseum.github.io/charts/index.yaml": read tcp 192.168.0.104:51896->185.199.110.153:443: read: connection reset by peer
...Successfully got an update from the "glm-chartmuseum" chart repository
...Successfully got an update from the "apollo" chart repository
...Successfully got an update from the "kyverno" chart repository
...Unable to get an update from the "mysql-operator" chart repository (https://mysql.github.io/mysql-operator/):
	Get "https://mysql.github.io/mysql-operator/index.yaml": read tcp 192.168.0.104:51903->185.199.111.153:443: read: connection reset by peer
...Unable to get an update from the "metallb" chart repository (https://metallb.github.io/metallb):
	Get "https://metallb.github.io/metallb/index.yaml": read tcp 192.168.0.104:51904->185.199.111.153:443: read: connection reset by peer
...Unable to get an update from the "dragonfly" chart repository (https://dragonflyoss.github.io/helm-charts/):
	Get "https://dragonflyoss.github.io/helm-charts/index.yaml": read tcp 192.168.0.104:51905->185.199.108.153:443: read: connection reset by peer
...Unable to get an update from the "openfga" chart repository (https://openfga.github.io/helm-charts):
	Get "https://openfga.github.io/helm-charts/index.yaml": read tcp 192.168.0.104:51907->185.199.111.153:443: read: connection reset by peer
...Unable to get an update from the "cnpg" chart repository (https://cloudnative-pg.github.io/charts):
	Get "https://cloudnative-pg.github.io/charts/index.yaml": read tcp 192.168.0.104:51910->185.199.111.153:443: read: connection reset by peer
...Unable to get an update from the "metrics-server" chart repository (https://kubernetes-sigs.github.io/metrics-server/):
	Get "https://kubernetes-sigs.github.io/metrics-server/index.yaml": read tcp 192.168.0.104:51913->185.199.111.153:443: read: connection reset by peer
...Unable to get an update from the "ot-helm" chart repository (https://ot-container-kit.github.io/helm-charts/):
	Get "https://ot-container-kit.github.io/helm-charts/index.yaml": read tcp 192.168.0.104:51914->185.199.111.153:443: read: connection reset by peer
...Unable to get an update from the "coredns" chart repository (https://coredns.github.io/helm):
	Get "https://coredns.github.io/helm/index.yaml": read tcp 192.168.0.104:51917->185.199.111.153:443: read: connection reset by peer
...Unable to get an update from the "redis-operator" chart repository (https://ot-container-kit.github.io/helm-charts/):
	Get "https://ot-container-kit.github.io/helm-charts/index.yaml": read tcp 192.168.0.104:51912->185.199.111.153:443: read: connection reset by peer
...Unable to get an update from the "andrcuns" chart repository (https://andrcuns.github.io/charts):
	Get "https://andrcuns.github.io/charts/index.yaml": read tcp 192.168.0.104:51915->185.199.111.153:443: read: connection reset by peer
...Successfully got an update from the "gitlab-jh" chart repository
...Successfully got an update from the "hashicorp" chart repository
...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "jenkins" chart repository
...Successfully got an update from the "nvidia" chart repository
...Successfully got an update from the "elastic" chart repository
...Successfully got an update from the "projectcalico" chart repository
...Unable to get an update from the "juicefs" chart repository (https://juicedata.github.io/charts/):
	Get "https://juicedata.github.io/charts/index.yaml": read tcp 192.168.0.104:51919->185.199.111.153:443: read: connection reset by peer
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈ before running any
commands, allowing us to safely pass --skip-refresh to avoid redundant
repo updates for each chart with external dependencies.

This reduces the number of repository refresh operations from O(n) to O(1)
where n is the number of charts with remote dependencies.

Co-authored-by: Javex <github@javex.eu>
Signed-off-by: yxxhero <aiopsclub@163.com>

* fix: ensure repo update only runs when repositories are configured

This fixes CI issues where tests fail with 'no repositories found' error.

The PR #2378 adds a single helm.UpdateRepo() call before running helm dep
build commands. However, when no repositories are configured, this call
fails. The fix adds a check for len(st.Repositories) > 0 before calling
UpdateRepo().

Additionally, updated snapshot files to reflect the new output ordering
where repo update happens before building dependencies.

Signed-off-by: yxxhero <aiopsclub@163.com>

* feat: Update test snapshots for single repo update

The code changes in PR #2378 ensure that helm repo update is only run once
before building dependencies. This requires updating test snapshots to include
the 'Updating repo' output that now appears before 'Building dependency' messages.

Updated snapshots:
- chart_need/output.yaml
- chart_need_enable_live_output/output.yaml
- release_template_inheritance/output.yaml
- environments_releases_without_same_yaml_part/output.yaml
- environment_missing_in_subhelmfile/output.yaml
- pr_560/output.yaml
- environments_values_gotmpl_with_environment_name/output.yaml
- postrenderer/output.yaml (fixed YAML structure)
- oci_need/output.yaml

Signed-off-by: yxxhero <aiopsclub@163.com>

* fix: Correctly update test snapshots based on repository configuration

Only update snapshots for tests that have repositories defined:
- chart_need/output.yaml (has repositories - shows 'Updating repo')
- chart_need_enable_live_output/output.yaml (has repositories - shows 'Updating repo')
- release_template_inheritance/output.yaml (has repositories - shows 'Updating repo')

Tests without repositories should NOT show 'Updating repo':
- environments_releases_without_same_yaml_part/output.yaml
- environments_values_gotmpl_with_environment_name/output.yaml
- pr_560/output.yaml
- environment_missing_in_subhelmfile/output.yaml
- postrenderer/output.yaml (uses OCI dependencies)
- oci_need/output.yaml (uses OCI dependencies)

This matches the conditional logic in the code that only runs
helm.UpdateRepo() when len(st.Repositories) > 0.

Signed-off-by: yxxhero <aiopsclub@163.com>

* fix: correct snapshot test expectations for repo update optimization

- Re-add trailing newlines to environment_missing_in_subhelmfile output
- Restore correct chart paths (/... instead of ../../...)
- Restore postrenderer output with cm2 ConfigMap and correct field order
- Fixes CI test failures introduced by incorrect snapshot updates

Signed-off-by: yxxhero <aiopsclub@163.com>

* fix: update integration test expected lint output for repo update

Include 'Updating repo' messages in expected lint output files
to match the new behavior where helm repo update is run once
before building dependencies.

Signed-off-by: yxxhero <aiopsclub@163.com>

* fix: remove extra blank line from lint output files

Integration test output files had an extra blank line that was
not present in the expected output, causing test failures.

Signed-off-by: yxxhero <aiopsclub@163.com>

* fix: update lint output for single repo update

With the repo update optimization, lint runs only once
with 'Updating repo' messages instead of running twice.
Update expected output to match new single-run behavior.

Signed-off-by: yxxhero <aiopsclub@163.com>

* fix: filter out repo update messages in lint test

Update test runner to filter out repo update messages that are
now generated by the single helm.UpdateRepo() call, keeping
the expected lint output consistent with the original behavior.

Signed-off-by: yxxhero <aiopsclub@163.com>

* fix: filter repo update messages from diff test

Filter out repo update messages in diff test output to
match new behavior where helm.UpdateRepo() is called once.

Signed-off-by: yxxhero <aiopsclub@163.com>

* Fix missing closing parenthesis in grep command

Signed-off-by: yxxhero <aiopsclub@163.com>

* fix: prevent --args flags from being passed to helm repo commands

When helmfile template --args is used, the extra flags were being
passed to helm repo update and helm repo add commands, which don't
support all flags that helm template/install support. This caused
failures when flags like --dry-run were passed via --args.

The fix saves the extra flags before executing helm repo commands,
clears them, and restores them afterwards to ensure repo commands
run without unsupported flags.

Fixes CI issue in PR #2378 where test issue-1749 fails
with "Error: unknown flag: --dry-run" during helm repo update.

Signed-off-by: yxxhero <aiopsclub@163.com>

---------

Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: Javex <github@javex.eu>
2026-01-29 19:15:44 -05:00

89 lines
5.0 KiB
Bash

chart_need_case_input_dir="${cases_dir}/chart-needs/input"
chart_need_case_output_dir="${cases_dir}/chart-needs/output"
config_file="helmfile.yaml.gotmpl"
chart_needs_tmp=$(mktemp -d)
chart_needs_template_reverse=${chart_needs_tmp}/chart.needs.template.log
chart_needs_lint_reverse=${chart_needs_tmp}/chart.needs.lint.log
chart_needs_diff_reverse=${chart_needs_tmp}/chart.needs.diff.log
lint_out_file=${chart_need_case_output_dir}/lint
diff_out_file=${chart_need_case_output_dir}/diff
if [[ $EXTRA_HELMFILE_FLAGS == *--enable-live-output* ]]; then
lint_out_file=${chart_need_case_output_dir}/lint-live
diff_out_file=${chart_need_case_output_dir}/diff-live
fi
# Use Helm 4 variant files for lint (diff output is identical between Helm 3 and 4)
if [ "${HELMFILE_HELM4}" = "1" ]; then
if [ -f "${lint_out_file}-helm4" ]; then
lint_out_file="${lint_out_file}-helm4"
fi
fi
test_start "chart prepare when helmfile template with needs"
info "https://github.com/helmfile/helmfile/issues/455"
for i in $(seq 10); do
info "Comparing template/chart-needs #$i"
${helmfile} -f ${chart_need_case_input_dir}/${config_file} template --include-needs > ${chart_needs_template_reverse} || fail "\"helmfile template\" shouldn't fail"
./dyff between -bs ${chart_need_case_output_dir}/template ${chart_needs_template_reverse} || fail "\"helmfile template\" should be consistent"
done
for i in $(seq 10); do
info "Comparing lint/chart-needs #$i"
# Remove azuredisk-csi-driver repo to ensure consistent output (repo addition message)
${helm} repo remove azuredisk-csi-driver &>/dev/null || true
${helmfile} -f ${chart_need_case_input_dir}/${config_file} lint --include-needs | grep -v Linting | grep -v "has been removed" | grep -Ev "(Updating repo|Hang tight while we grab|Update Complete|Successfully got an update from|Warning:.*is not a valid SemVerV2|\[WARNING\].*is not a valid SemVerV2|failed to load plugins)" > ${chart_needs_lint_reverse} || fail "\"helmfile lint\" shouldn't fail"
diff -u ${lint_out_file} ${chart_needs_lint_reverse} || fail "\"helmfile lint\" should be consistent"
done
for i in $(seq 10); do
info "Comparing diff/chart-needs #$i"
# Remove azuredisk-csi-driver repo to ensure consistent output (repo addition message)
${helm} repo remove azuredisk-csi-driver &>/dev/null || true
${helmfile} -f ${chart_need_case_input_dir}/${config_file} diff --include-needs | grep -Ev "(Updating repo|Hang tight while we grab|Update Complete|Successfully got an update from|Comparing release=azuredisk-csi-storageclass, chart=.*/chartify.*/azuredisk-csi-storageclass)" > ${chart_needs_diff_reverse}.tmp || fail "\"helmfile diff\" shouldn't fail"
cat ${chart_needs_diff_reverse}.tmp | sed -E '/\*{20}/,/\*{20}/d' > ${chart_needs_diff_reverse}
# With --enable-live-output, there's a race condition that can cause non-deterministic ordering
# Try both the primary expected output and the alternate ordering
if ! diff -u ${diff_out_file} ${chart_needs_diff_reverse} >/dev/null 2>&1; then
if [[ $EXTRA_HELMFILE_FLAGS == *--enable-live-output* ]] && [ -f "${diff_out_file}-alt" ]; then
info "Primary diff failed, trying alternate ordering (due to --enable-live-output race condition)"
diff -u ${diff_out_file}-alt ${chart_needs_diff_reverse} || fail "\"helmfile diff\" should match either expected output"
else
diff -u ${diff_out_file} ${chart_needs_diff_reverse} || fail "\"helmfile diff\" should be consistent"
fi
fi
done
info "Applying ${chart_need_case_input_dir}/${config_file}"
${helmfile} -f ${chart_need_case_input_dir}/${config_file} apply --include-needs
code=$?
[ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile apply: want 0, got ${code}"
${kubectl} get storageclass managed-csi -o yaml | grep -q "provisioner: disk.csi.azure.com" || fail "storageclass managed-csi should be created when applying helmfile.yaml"
info "Destroying ${chart_need_case_input_dir}/${config_file}"
${helmfile} -f ${chart_need_case_input_dir}/${config_file} destroy
code=$?
[ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile destroy: want 0, got ${code}"
info "Syncing ${chart_need_case_input_dir}/${config_file}"
${helmfile} -f ${chart_need_case_input_dir}/${config_file} sync --include-needs
code=$?
[ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile apply: want 0, got ${code}"
${kubectl} get storageclass managed-csi -o yaml | grep -q "provisioner: disk.csi.azure.com" || fail "storageclass managed-csi should be created when syncing helmfile.yaml"
info "Destroying ${chart_need_case_input_dir}/${config_file}"
${helmfile} -f ${chart_need_case_input_dir}/${config_file} destroy
code=$?
[ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile destroy: want 0, got ${code}"
# Clean up: remove azuredisk-csi-driver repo to avoid conflicts with subsequent tests
${helm} repo remove azuredisk-csi-driver &>/dev/null || true
test_pass "chart prepare when helmfile template with needs"