Bump github.com/helmfile/vals from v0.46.0 to v0.46.1.
vals v0.46.1 is a dependency-maintenance release with no API changes;
it also bumps its go directive to 1.26.6, so this updates the go
directive in go.mod accordingly (CI reads the Go version from go.mod).
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat(telemetry): add opt-in OpenTelemetry tracing (PR 1: lifecycle + root span)
Implements the first increment of docs/proposals/otel-tracing.md (#2767):
- pkg/telemetry: SDK setup from standard OTEL_* env vars (autoexport for
exporter selection, env-driven sampler/propagators, OTEL_SDK_DISABLED),
command-span lifecycle, no-op-by-default accessors
- --otel-tracing flag / HELMFILE_OTEL_TRACING env switch
- root span "helmfile <command>" with file/environment/selectors/exit_code
attributes; TRACEPARENT-based remote-parent extraction for CI correlation
- shutdown flush on both normal-exit and signal paths (nil-safe, 5s bound)
- app.New derives its context from telemetry.CommandContext()
(Background-identical when tracing is disabled)
- docs: otel.md user guide, experimental-features entry, design proposal
- tests: hermetic unit tests, app context-contract pinning, flag registration
Telemetry problems never fail a run: exporter misconfiguration and export
errors degrade to disabled with a warning. When disabled, behavior and
performance are identical to before (no-op tracer, no goroutines, no
network).
Refs: #2767, #2758
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat(telemetry): trace every external process + trace-context bridges (PR 2)
Implements the second increment of docs/proposals/otel-tracing.md (#2767):
- pkg/helmexec/span.go: one span per external process started by helmfile
(helm invocations, hooks, plugin execs) at the ShellRunner choke point —
helm.exec (with helm.subcommand) vs os.exec, with redacted exec.args,
exec.exit_code, and error status on failure
- pkg/helmexec/redact.go: shared argument redaction with two profiles;
legacy is byte-identical to the historical exit-error behavior (existing
goldens unchanged), strict (spans) additionally covers --set=k=v and
credential flags; exit_error.go now uses the shared helper
- orphan-trace bridges with bit-identical cancellation semantics
(context.WithoutCancel of the command context): both kubedog call sites
(state.go) and hook execution (event.Bus gains an optional Ctx consumed
by its default runner; state.go sets it, nil falls back to TODO as before)
- OTLP end-to-end test (in-process httptest receiver, no external
collector): span export, error status/exit code, redaction, and
parent-linkage to the command span
- docs/otel.md updated to the now-traced surface
Verified end-to-end with the console exporter: helmfile template on a
local chart yields the command span plus helm.exec spans for helm
version/dependency/template, all nested under it.
Refs: #2767
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat(telemetry): state-loading and hook spans (PR 3a)
Implements the third increment of docs/proposals/otel-tracing.md (#2767):
- helmfile.discover_states around findDesiredStateFiles and helmfile.load
around loadDesiredStateFromYamlWithBaseDir; both cover all callers
(incl. nested helmfiles) with no signature changes
- helmfile.render / helmfile.parse children per document part, parented
through a traceCtx field on the unexported desiredStateLoader struct
(set once at its single construction site)
- helmfile.hook span per hook execution: Trigger's per-hook body extracted
into runHook (readability win on its own), the hook's subprocess span
nests under it via a per-hook ctx-swapped ShellRunner clone
(cancellation unchanged — Bus.Ctx never carries cancellation by contract)
- pkg/telemetry/otlptest: shared in-process OTLP/HTTP receiver harness,
now used by helmexec, event, and app span tests
- golden span-tree test at the app layer (root -> discover -> load ->
render/parse, via the exectest fake helm) and a hook-span nesting test
- nil-ctx guard for App literals built directly by tests (App.spanParentCtx)
Verified end-to-end with the console exporter: a template run over a
gotmpl state file with a prepare hook yields the full tree with the hook's
os.exec nested under helmfile.hook.
Refs: #2767
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat(telemetry): per-release spans nested under the load span (PR 3b)
Implements the per-release increment of docs/proposals/otel-tracing.md
(#2767) — spans nest command -> load -> release -> helm exec:
- helmexec.HelmContext gains an optional Ctx carrying the per-release span
context; the execer's new execWithContext funnel consumes it via a
per-call runner clone (runnerWithCtx) so the shared, cached execer is
never mutated across concurrent workers. The seven Interface methods
that take a HelmContext (Sync/Diff/ReleaseStatus/List/DecryptSecret/
Delete/Test) route through it; nil Ctx behaves exactly as before.
exec() lost its always-nil override parameter on the way (unparam).
- pkg/state/span.go: SetTraceContext + startReleaseSpan/endReleaseSpan
helpers (release/namespace/chart/labels attributes, sorted for stable
output); a typed-nil guard (releaseErrAsError) avoids the classic
nil-pointer-in-interface trap on *ReleaseError.
- release spans in the worker loops: SyncReleases, DiffReleases,
DeleteReleasesForSync, PrepareCharts, and iterateOnReleases (status/
delete/test via a new verb parameter); their HelmContext is stamped with
the release span context where one is built.
- pkg/app sets st.SetTraceContext(loadCtx) right after loading a state
file, rooting all per-release spans under helmfile.load.
- bridged one more detached tracking call found on the way
(trackReleaseIfEnabled's context.Background in the sync worker).
- golden test: release span present, nested under load, correct
attributes; unit test for the runnerWithCtx clone semantics.
Verified with the console exporter: helmfile template yields
release.prepare(demo) under load with full attributes.
Refs: #2767
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat(telemetry): nest status/delete/test execs under their release spans
Completes the per-release exec nesting for the iterateOnReleases-based
loops (docs/proposals/otel-tracing.md §4.4 phase 2): the do closures now
receive the release span context and stamp it into their HelmContext, so
helm status/delete/test subprocess spans nest under
helmfile.release.<verb> like sync/diff already did.
- scatterGatherReleases/iterateOnReleases/doWithReleaseSpan: do gains a
context parameter (the release span context)
- ReleaseStatuses/DeleteReleases/TestReleases closures stamp
HelmContext.Ctx from it
- integration test with a real execer (version-probe shim binary): the
release's status subprocess nests under helmfile.release.status, same
trace, with helm.subcommand=status
This also makes the otel.md claim ("upgrade, diff, delete, status, test
nested under the release span") fully accurate.
Refs: #2767
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat(telemetry): OTel metrics — helm exec duration and release results (PR 4)
Implements the metrics increment of docs/proposals/otel-tracing.md
(#2767) on the same provider, switch, and resource as traces:
- pkg/telemetry/metrics.go: helmfile.helm.exec.duration histogram
(subcommand, success) and helmfile.release.count counter (verb,
result). Instruments come from the otel global meter, so recording at
call sites is branch-free no-op when telemetry is disabled.
- Setup builds the resource once and installs both providers; reader
selection delegates to autoexport (OTEL_METRICS_EXPORTER: otlp |
console | prometheus | none), the OTLP reader's interval honors
OTEL_METRIC_EXPORT_INTERVAL (read by the SDK). Shutdown flushes both
providers (errors.Join). StartCommandSpan now carries the meter
provider across state transitions (fixes a nil-shutdown panic).
- helmexec: finishExecSpan records exec duration for helm binaries;
state: endReleaseSpan counts release outcomes for sync/diff/delete/
status/test/prepare (diff counted as success when no hard error).
- otlptest: recorder routes by OTLP path (/v1/traces vs /v1/metrics)
and decodes metrics; new FindMetric helper.
- tests: metrics recorded as no-op when disabled, provider enabled with
the none exporter, degradation on an invalid metrics exporter, and an
integration assertion (status exec duration datapoint + one successful
release.count) in the shim-based state test.
Verified with the console exporter: helmfile template emits
helmfile.helm.exec.duration per subcommand (version/dependency/
template) and helmfile.release.count{verb=prepare,result=success}=1.
Refs: #2767
Signed-off-by: yxxhero <aiopsclub@163.com>
* docs: complete OTel documentation coverage
- docs/cli.md: --otel-tracing in the CLI reference help block (verbatim
from the cobra output)
- CHANGELOG.md: [Unreleased] Added entry for tracing + metrics
- docs/index.md: Observability highlight linking docs/otel.md
- docs/proposals/otel-tracing.md: add OTEL_METRICS_EXPORTER /
OTEL_METRIC_EXPORT_INTERVAL rows to the env-var table and note the
periodic reader + bounded metric cardinality in §7
Refs: #2767
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: drop unused id parameter from parsePart (unparam)
The id parameter was never used inside the span wrapper; the caller's id
variable is still used for the render calls and error messages.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix(telemetry): address review — redaction gaps, kubedog valve, phantom metrics
Addresses all Copilot review comments on #2769:
Security (span payloads):
- exec.args: positional arguments are additionally passed through
helmexec.RedactedURL, so credentials embedded in chart/repository URLs
(AddRepo, RegistryLogin, OCI refs) are masked exactly like log output
- release spans sanitize helmfile.chart the same way
- error statuses no longer embed raw errors (which contain rendered
commands, arguments, and subprocess output): the command span, release
spans, hook spans, and exec spans now use generic descriptions; the
concrete exit code remains an attribute, and RecordError on the root
span is dropped
Correctness:
- kubedog safety valve restored: execWithContext now attaches the
per-release span into the runner's own context instead of replacing it,
so trackHandle.Cancel() can interrupt a wedged helm again and app
cancellation semantics stay exactly as before the PR
- diff release spans/metrics: real failures are recorded (exit code 2
"changes detected" still counts as success); previously every diff was
exported as successful
- skipped releases no longer emit phantom spans and inflate
helmfile.release.count: iterateOnReleases callers pass a skip predicate
(skipUndesired for status/test; delete deletes undesired releases and
passes nil)
- Setup shuts down the already-constructed tracer provider (bounded) when
the metrics provider fails, instead of abandoning its batch goroutine
Tests: URL redaction cases (masked/untouched), spanAttachedContext
preserves the runner cancellation chain while attaching the caller's
span, skipUndesired, and a failing-hook span asserting the generic
message.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: lint — restore nolint placement and avoid nil context literal
- the skipUndesired insertion had displaced the // nolint: unparam
directive off iterateOnReleases (helm param is intentionally unused
there); also fixes a skipDesired/skipUndesired comment typo
- use a typed nil in TestSpanAttachedContext (staticcheck SA1012)
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix(telemetry): address review round 2 — remote-ref redaction, wrapper helm binaries, hook release attribution
Addresses all 6 new review comments on #2769:
Security (remote references):
- new helmexec.RedactedRef sanitizes go-getter style references for
telemetry: forced-form prefixes (git::, s3::) preserved, whole URL
userinfo masked (usernames carry tokens too), credential-bearing query
parameters masked using pkg/remote's heuristic (token/password/secret/
key/signature). Applied to helmfile.file (command span), helmfile.path
(discover_states), helmfile.chart (release spans), and exec.args —
log-time RedactedURL is untouched so log output is unchanged
Correctness:
- wrapper helm binaries (--helm-binary custom names) are now classified
as helm operations by an explicit context marker stamped in the execer
funnel, instead of the executable-basename heuristic; the same
classification gates helmfile.helm.exec.duration, so the metric no
longer misses wrapper invocations (classifyExec)
- release-scoped hooks (presync/postsync/preuninstall/postuninstall/
cleanup in the sync/delete/diff workers) now attach their helmfile.hook
spans to the active helmfile.release.* span via a variadic parent on
the trigger functions; global hooks keep the command context and all
29 existing call sites compile unchanged; hook cancellation stays
detached (WithoutCancel) as before
- signal-terminated runs (Shutdown with exitCode 130/143 and nil error)
now mark the command span with error status, consistent with their
nonzero exit code
Tests: RedactedRef table (forced forms, userinfo, s3/token query params,
untouched cases), classifyExec marker case, hookTraceContext parent
attribution + non-cancellability + fallback.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix(telemetry): address review round 3 — redaction corner cases, value runners
Addresses 5 of the 6 new review comments on #2769 (the sixth — an
unused strings import in exit_error.go — is a false positive: Indent
still uses strings.Split/Builder and the package compiles):
- RedactArgs read the previous token from the progressively redacted
output, so {--set, --set-string, secret} leaked the secret
(the masked value hid the following flag). Read the previous token
from the original input, restoring the legacy contract for adjacent
secret flags
- RedactedRef fails closed for malformed references: URL-like refs with
invalid percent escapes export a fully redacted value, and an
unparseable query is dropped entirely instead of exported verbatim
- ShellRunner has value receivers, so a ShellRunner VALUE satisfies the
Runner API; the helm marker stamping and the per-release span
attachment now handle both value and pointer forms (matching
WithContext), so value-runner callers keep release nesting and the
helm.exec classification/metric
Regression tests: adjacent secret flags (legacy + strict), malformed
URL-like ref, malformed query, value-runner marker + span attachment.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix(telemetry): stamp the helm marker on the stdin funnel too
execStdIn (registry login, repo add) called the runner directly, so
wrapper --helm-binary names were misclassified as os.exec and omitted
from helmfile.helm.exec.duration on that path. The marking now goes
through a shared markHelmRunner helper (value and pointer ShellRunner
forms) used by both execution funnels.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix(telemetry): redact helm's --kube-token in strict profile
Helm's global --kube-token carries a bearer token; both the
two-argument and inline forms are now masked in span exec.args
(legacy exit-error output is untouched, matching its historical
behavior).
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix(telemetry): OTel metrics best-practice alignment
- helmfile.helm.exec.duration now declares explicit bucket boundaries
tuned for seconds-scale helm invocations (5ms…600s); the SDK defaults
are millisecond-oriented and lumped every sub-5s invocation — the
common case — into the first bucket, defeating the histogram
- instruments are re-created under the installed provider with the
instrumentation scope version stamped (Setup-time, race-free)
- helmfile.release.count declares the {release} curly-annotation unit
per the metrics naming conventions
Tested end-to-end via the OTLP integration test: exported bounds are
the tuned set, units are asserted, and the scope carries the version.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat(telemetry): per-release duration metrics behind an opt-in switch
New helmfile.release.duration histogram (seconds, same tuned buckets)
with bounded dimensions by default (verb, result). Setting
HELMFILE_OTEL_METRICS_PER_RELEASE=true adds helmfile.release and
helmfile.namespace, answering "which release is slow" from dashboards:
- well-suited to bounded CI runs; long-lived centralized collection
needs a backend capacity/TTL story (documented in docs/otel.md)
- per-release timing remains available in traces without the flag
- env read per call (release operations are low-frequency, and tests
toggle it)
endReleaseSpan now takes the release and the operation start time; the
five worker-loop call sites pass them (doWithReleaseSpan, SyncReleases,
DeleteReleasesForSync, PrepareCharts, DiffReleases).
Verified end-to-end with the console exporter (default dims vs
per-release) and OTLP integration tests pinning both modes.
Refs: #2767, #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* refactor(telemetry): maintainability pass over the runner/metric plumbing
- StartCommandSpan copies the tracingState struct instead of enumerating
fields by hand — that pattern dropped the meter provider once already
- the two value/pointer ShellRunner switches (helm marker, span
attachment) are unified into one withRunnerCtx helper; the duplication
caused two review rounds of value-form misses
- classifyExec derives the helm classification from the span name
(helmExecSpanName constant) instead of returning a third parallel bool
- metrics: shared outcomeAttrs for the verb/result dimensions, and the
bucket slice renamed to durationBuckets with a comment covering both
histograms that use it
No behavior change; full -race suite green, lint clean.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* refactor(telemetry): consolidate test env lists, trace bridges, and hook prep; sync the design doc
Maintainability:
- HermeticEnvVars is now exported from pkg/telemetry (the owner of the
env surface) and used by both telemetry tests and otlptest — the two
copies had already drifted once (HELMFILE_OTEL_METRICS_PER_RELEASE
needed updating in both)
- kubedogTraceContext and hookTraceContext were the same concept written
twice; unified into traceOnlyContext(parent...) in span.go
Readability:
- runHook's nested kubectl rewrite extracted into prepareKubectlHook
with guard-clause structure
Accuracy (docs ↔ code, drifted over five review rounds):
- §4.4 now describes the implemented mechanism: the release span is
INJECTED into the runner's own context (preserving the kubedog safety
valve) rather than the runner context being replaced, and helm
classification is marker-based for wrapper binaries
- §5 exec span rows list the actual attributes incl. URL/query masking
- §6 strict profile documents RedactedRef, --kube-token, and the
adjacent-token guarantee
No behavior change; full -race suite green, lint clean.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* refactor(telemetry): drop the dead noop state, relocate skipUndesired, sync user-facing accuracy
- tracingState.noop was dead weight in the enabled state and a
copy-surface in every transition; a single package-level
noopTracerProvider now backs Tracer while disabled
- skipUndesired moved next to doWithReleaseSpan in span.go, its only
conceptual home (span/metric suppression, not run plumbing)
- accuracy: the package doc, --otel-tracing flag help,
experimental-features entry, and CHANGELOG now all say tracing AND
metrics and list the third instrument (helmfile.release.duration with
the HELMFILE_OTEL_METRICS_PER_RELEASE opt-in) — these had drifted
when the metric was added; the PR description's metric table is
updated to match as well
No behavior change; full -race suite green (except the pre-existing
network-dependent TestStorage_resolveFile flake), lint clean.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* refactor(telemetry): flatten Setup, name the prefix bound, dedupe test fake; fix instrument-count drift
Readability/maintainability:
- Setup drops from 56 to 39 lines: provider construction (including the
shutdown-tracer-on-meter-failure recovery) moves to newProviders in
exporter.go next to the constructors it composes
- refredact's magic 16 becomes maxForcedFormPrefix with a comment
- span_test's hand-rolled fakeRunner removed in favor of the existing
mockRunner (same package)
Accuracy:
- "Two instruments" wording survived in docs/otel.md and the design
proposal §7 after helmfile.release.duration was added; both now say
three and mention the per-release opt-in
No behavior change; full -race suite green (except the pre-existing
network flake), lint clean.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
* refactor(telemetry): co-locate span machinery, drop a dead export, fix docs nits
- the span plumbing helpers (markHelmExec, withRunnerCtx,
markHelmRunner, spanAttachedContext) move from exec.go to span.go,
next to the marker type and classifiers they serve — exec.go keeps
only the funnel call sites
- otlptest.SpanNames was never used outside the package; unexported
- isHelmBinary's comment now states it is the FALLBACK classifier
(funnel invocations are marker-classified), replacing the outdated
"cosmetic distinction" framing from before the marker existed
- docs/otel.md: release-scoped hooks nest under their release span
(added in review round 2, never documented)
No behavior change; full -race suite green, lint clean.
Refs: #2769
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
Bump github.com/helmfile/chartify to v0.28.2, which fixes the empty-render
crash upstream (helmfile/chartify#206, fixed in helmfile/chartify#207):
when a chart renders zero resources, chartify now treats it as a no-op
success itself - removing the chart's content dirs, cleaning Chart.yaml
dependencies and the lock file, and skipping the kustomize step - instead
of failing with:
assertion failed: unexpected dir entry "" it must be the abs path to the output directory
That makes the helmfile-side string-matching workaround from #2724 dead
code: the error it matched can no longer be produced. Remove
isChartifyEmptyRenderOutputError and the special-cased no-op branch in
processChartification, so the empty-render case simply flows through the
regular chartify path.
Also bump github.com/helmfile/vals to v0.46.0 and refresh transitive
dependencies.
The regression test for #1757 is updated to assert the new direct
behavior: processChartification succeeds, returns a chartified chart
whose Chart.yaml no longer declares dependencies (so a subsequent
"helm dep build"/"helm template" cannot fail with "found in Chart.yaml,
but missing in charts/ directory"), renders empty output, and is cleaned
up by CleanupChartifyTempDirs.
Fixes#1757 (follow-up to #2724)
Signed-off-by: yxxhero <aiopsclub@163.com>
dependabot[bot]