Fix all lint issues reported by golangci-lint-v2:
1. errcheck: Check error return from tracker.Close()
- Add defer function to check and log Close() error
- Previously just deferred tracker.Close() without checking return
2. unparam: Remove unused parameter 'ops' from shouldUseKubeDog()
- Parameter was accepted but never used in the function
- Changed to '_' to explicitly indicate it's unused
3. unparam: Remove unused parameter 'ctx' from getReleaseManifest()
- Context parameter was accepted but never used in the function
- Changed to '_' to explicitly indicate it's unused
- Context is still needed in getReleaseResources() which calls this function
All lint issues now resolved for pkg/state package.
Signed-off-by: yxxhero <aiopsclub@163.com>
Implement the getReleaseResources method to detect Kubernetes resources
from Helm chart manifests for kubedog tracking.
This implementation:
1. Creates a new pkg/cluster package with:
- Resource struct for resource metadata (kind, name, namespace)
- ReleaseResources struct for holding all resources in a release
- TrackConfig for custom tracking configuration
- GetReleaseResourcesFromManifest() to parse helm template output
- Helper functions for filtering trackable vs static resources
- Helper functions for getting Helm release labels/annotations
2. Implements getReleaseResources() method that:
- Runs helm template to generate Kubernetes manifests
- Reads all YAML files from template output directory
- Combines files into a single manifest with document separators
- Parses manifest using k8s.io/apimachinery unstructured decoder
- Converts cluster.Resource to kubedog.ResourceSpec for tracking
- Cleans up temporary files after parsing
3. Removes unused functions from helmx.go:
- parseResourceKindAndName() - replaced by manifest parsing
- isTrackableResourceKind() - moved to cluster package
Benefits:
- Works offline without Kubernetes cluster connection
- Faster than API-based resource detection
- More reliable and deterministic
- Enables kubedog tracking with proper resource filtering
Fixes: #2383
Signed-off-by: yxxhero <aiopsclub@163.com>
Remove the kubedog custom tracking implementation that requires changes
to exec.go and helmexec.go interface.
The feature requires modifying TemplateRelease to return (string, error) instead
of error, which conflicts with existing API. Since these files should
not be modified, temporarily simplify getReleaseResources to return empty
list to avoid compilation errors.
Signed-off-by: yxxhero <aiopsclub@163.com>
Enhance kubedog tracker with flexible resource filtering options, allowing
users to control which resources are tracked.
Key Features:
- TrackKinds: Only track resources of specified types
- SkipKinds: Skip resources of specified types
- CustomTrackableKinds: Define custom resource types to actively track
- CustomStaticKinds: Define custom resource types that don't need tracking
- pkg/cluster/release.go: Manifest-based resource detection with filtering
- Comprehensive documentation and examples
Changes:
- pkg/kubedog/options.go: Add new tracking configuration fields and methods
- pkg/kubedog/tracker.go: Add filterResources and shouldSkipResource methods
- pkg/cluster/release.go: New package for manifest parsing and resource filtering
- docs/: Complete guides for custom tracking configuration
- examples/: Working examples demonstrating all filtering options
Benefits:
- Fine-grained control over resource tracking
- Support for custom resource types (CRDs)
- Performance improvement by skipping unnecessary tracking
- Backward compatible (defaults unchanged when not configured)
Signed-off-by: yxxhero <aiopsclub@163.com>
Changes:
- Replace helm.Template with helm.TemplateRelease to get release manifest
- Parse manifest to detect all resources (Deployment, StatefulSet, DaemonSet, Job, Pod, ReplicaSet)
- Track all detected resources with kubedog instead of hardcoded deployment name
- Add parseResourceKindAndName helper to extract resource type and name
- Add isTrackableResourceKind helper to filter supported resource types
- Remove assumption that resource name equals release name
This approach is more elegant and follows helmfile conventions by using
helm.TemplateRelease instead of manual manifest parsing.
Resolves: #660
Signed-off-by: yxxhero <aiopsclub@163.com>
* feat: upgrade Helm version to v3.20.0 and v4.1.0
This commit updates the recommended Helm version from v3.19.5/v4.0.5 to
v3.20.0/v4.1.0 across all workflows, Dockerfiles, and application constants.
Changes:
- Update CI matrix to test with Helm v3.20.0 and v4.1.0
- Update .github/workflows/Makefile HELM_VERSION to v4.1.0
- Update Dockerfiles with new version and SHA256 checksums
- Update pkg/app/init.go HelmRecommendedVersion to v4.1.0
- Update go.mod helm.sh/helm/v3 to v3.20.0 and helm.sh/helm/v4 to v4.1.0
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: remove source field from e2e test helm plugin configs
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: remove source field from integration test helm plugin config
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: add --force-update flag for Helm 4 to prevent stale repository indexes
Fixes#2337
Problem:
Helmfile with Helm v4 doesn't update repository indexes when adding repos,
leading to stale indexes and errors like:
"chart matching version not found in example index. (try 'helm repo update')"
This happens because Helm 4 changed behavior compared to Helm 3:
- Helm 3: Always downloads index when running "helm repo add", even if repo exists
- Helm 4: Skips downloading index if repo already exists with same config
(see: https://github.com/helm/helm/blob/v4.0.4/pkg/cmd/repo_add.go#L200)
Without --force-update, helmfile only works initially because Helm 4
downloads index on fresh repo setup, but subsequent "helmfile repos"
commands result in stale indexes.
Root Cause:
The code only added --force-update for Helm 3.3.2+, but not for Helm 4,
since it was believed to be default behavior in Helm 4. However, Helm 4
requires explicit --force-update flag to update indexes for existing repos.
Solution:
Add --force-update flag for Helm 4 in AddRepo function to ensure
repository indexes are updated even when repository already exists.
Refactoring:
Simplified the conditional logic from nested if statements to a single
readable condition using existing IsVersionAtLeast() helper:
if !helm.options.DisableForceUpdate &&
(helm.IsHelm4() || helm.IsVersionAtLeast("3.3.2")) {
args = append(args, "--force-update")
}
Changes:
- pkg/helmexec/exec.go: Add --force-update for Helm 4
- pkg/helmexec/exec_test.go: Update test expectations for both Helm 3.3.2+ and Helm 4
- AGENTS.md: Add development guide for the repository
Testing:
- All helmexec package tests pass
- Verified build succeeds
- Tested against Helm 3.2.0 (no force-update)
- Tested against Helm 3.3.2+ (with force-update)
- Tested against Helm 4.0.1 (with force-update)
Signed-off-by: opencode <opencode@users.noreply.github.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
* test: update expected output for Helm 4 repo add message
Update integration test expectations to match Helm 4 behavior with --force-update flag.
When --force-update is used, Helm 4 now outputs "has been added to your
repositories" instead of "already exists with the same configuration, skipping",
because it forcibly updates the repository index.
Related to #2337
Signed-off-by: opencode <opencode@users.noreply.github.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
---------
Signed-off-by: opencode <opencode@users.noreply.github.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
* fix: array merge regression - layer arrays now replace defaults (#2353)
PR #2288 introduced element-by-element array merging to fix#2281, but this
caused a regression where layer/environment arrays were merged instead of
replacing base arrays entirely.
This fix uses automatic sparse array detection:
- Arrays with nil values (from --state-values-set) merge element-by-element
- Arrays without nils (from layer YAML) replace entirely
This follows Helm's documented behavior where arrays replace rather than merge.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: use separate CLIOverrides field for element-by-element array merging
The previous approach using ArrayMergeStrategySparse detection didn't work
for --state-values-set array[0]=value because setting index 0 produces no
nils in the array.
This fix adds a CLIOverrides field to Environment that keeps CLI values
separate from layer values. CLI overrides are merged last using
ArrayMergeStrategyMerge (always element-by-element), while layer values
use the default strategy (arrays replace).
This ensures:
- --state-values-set array[0]=x only changes index 0, preserving other elements
- Layer/environment file arrays still replace base arrays entirely
- Issue #2281 fix is preserved (--state-values-set array[1].field=x works)
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: correct comment about array merge strategy in test
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: propagate Defaults in multi-part helmfiles and fix merge order
- Add Defaults field merging from ctxEnv to preserve base values across
helmfile parts separated by ---
- Fix merge order: current part values now correctly override previous
parts (was reversed, causing older values to win)
- Update 147 snapshot test files for new Environment log format with
CLIOverrides field
This completes the fix for issue #2353 by ensuring:
1. Layer arrays replace entirely (not element-by-element merge)
2. CLI --state-values-set sparse arrays still merge element-by-element
3. Multi-part helmfiles properly inherit and override values
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: address Copilot review comments
- Initialize EmptyEnvironment with empty maps to match New() constructor
- Update test comment to accurately describe ArrayMergeStrategySparse
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: ensure templates access merged values via .Environment.Values
This commit fixes a regression in the CLIOverrides integration where
templates accessing .Environment.Values couldn't see CLI override values.
Changes:
- Remove CLIOverrides-into-Values merge from Merge() to keep proper
layering order (Defaults → Values → CLIOverrides) in GetMergedValues()
- Update NewEnvironmentTemplateData to set envCopy.Values to the merged
values, ensuring templates see the same values via both .Values and
.Environment.Values
This ensures:
- Issue #2353: Layer arrays still replace entirely (Sparse strategy)
- Issue #2281: CLI sparse arrays still merge element-by-element
- Templates can access CLI overrides via .Environment.Values
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* docs: improve mergeSlices documentation per Copilot review
Address Copilot review comments on PR #2367:
- Document empty array edge case: explicitly setting [] clears base array
- Document recursive strategy propagation for nested map merging
- Add comprehensive behavior description for all array merge strategies
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: use merged values when rendering environment value files
Environment value files (*.yaml.gotmpl) can reference CLI values via
.Values. Previously, only env.Values was passed to template rendering,
which didn't include CLIOverrides.
Now we call env.GetMergedValues() to get Defaults + Values + CLIOverrides
before rendering, so templates can access CLI values like:
--state-values-set foo=bar
This fixes the state-values-set-cli-args-in-environments integration test.
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
---------
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: resolve --validate flag conflict with kustomize in Helm 4
Fixes#2355
In Helm 4, the --validate and --dry-run flags are mutually exclusive.
When using kustomize/chartify charts with helmfile diff --validate,
the code was adding both --validate AND --dry-run=server to the
helm template command, causing the error:
Error: if any flags in the group [validate dry-run] are set none
of the others can be; [dry-run validate] were all set
The fix checks if --validate is already set before adding --dry-run=server.
Since --validate already provides server-side validation (it was deprecated
in favor of --dry-run=server in Helm 4), adding --dry-run=server is
redundant when --validate is present.
Changes:
- Add !opts.Validate condition to processChartification() in state.go
- Add comprehensive unit tests for validate/dry-run mutual exclusion
- Add integration test with kustomize chart to prevent regression
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* address review feedback from Copilot
- Add missing test cases for destroy, delete, test, status WITH --validate
- Update integration test to use 'diff' instead of 'template' to properly
exercise the cluster-requiring code path that triggers --dry-run=server
- Add sync warning comments to the test helper function noting it must be
kept in sync with processChartification() in state.go
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* add missing 'build with validate' test case
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* address additional review feedback from Copilot
- Fix integration test to capture output and exit code in single execution
instead of running helmfile twice (more efficient)
- Add detailed documentation explaining why test helper duplication is
intentional: extracting shared function would require exposing internal
API and complex refactoring of processChartification dependencies
- Note that integration test exercises actual code path end-to-end
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: correct go doc comment formatting for gci linter
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* fix: update line number reference from 1497-1523 to 1497-1524
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
---------
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
fix: pass --kube-context to helm template when using jsonPatches (#2309)
When using jsonPatches or strategicMergePatches in helmfile, the
`helm template` command was not receiving the `--kube-context` flag.
This caused issues when `--dry-run=server` was used (introduced in
PR #2271 to support lookup() functions), because helm would connect
to the wrong cluster context.
Root Cause:
1. `flagsForTemplate()` did not call `appendConnectionFlags()`, unlike
`flagsForUpgrade()` and `flagsForDiff()` which both include this call.
2. `processChartification()` did not include `--kube-context` when
setting `chartifyOpts.TemplateArgs` for internal helm template calls.
Fix:
1. Added `appendConnectionFlags()` call to `flagsForTemplate()` to ensure
kube-context and other connection flags are passed to helm template.
2. Added `getKubeContext()` helper function that resolves kube-context
with proper priority: release > environment > helmDefaults.
3. Modified `processChartification()` to include `--kube-context` in
chartifyOpts.TemplateArgs when chartify needs to run helm template.
4. Added compatibility check for `--validate` flag to avoid Helm 4
mutual exclusion error between --validate and --dry-run (Issue #2355).
Fixes#2309
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
* Initial plan
* Fix helmfile.d parallel processing to respect alphabetical order
Changed multiple helmfile.d file processing from parallel to sequential
to match the documented behavior. Files are now processed in alphabetical
order as stated in the documentation, ensuring predictable deployment order.
This fixes issues where deployments would break because parallel processing
did not respect the intended ordering (e.g., databases before applications).
- Modified visitStatesWithContext to process files sequentially
- Removed unused processStateFileParallel function
- Updated test snapshots to reflect deterministic ordering
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Add --sequential-helmfiles flag for opt-in sequential processing
Based on feedback, added a flag to control processing mode instead of
forcing sequential behavior. Parallel processing is now the default for
performance, with an opt-in --sequential-helmfiles flag for when order matters.
Changes:
- Added SequentialHelmfiles field to GlobalOptions and App
- Added --sequential-helmfiles CLI flag
- Modified visitStatesWithContext to check flag and choose parallel vs sequential
- Updated documentation to clarify parallel is default, sequential is opt-in
- Restored original parallel processing as default behavior
This gives users control over the behavior based on their needs.
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Add documentation and improve code readability
- Added documentation to processStateFileParallel function
- Extracted shouldProcessInParallel variable for clarity
- All tests pass, security scan clean
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
* Fix lint issue: remove trailing whitespace
Removed trailing whitespace from line 983 in pkg/app/app.go to fix
formatting lint error.
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
Fixes an issue where Chart.yaml dependencies with relative file:// paths
fail during chartification because the paths become invalid when the chart
is copied to chartify's temporary directory.
The rewriteChartDependencies function now converts relative file://
dependencies to absolute paths before chartification, then restores the
original Chart.yaml afterwards. Absolute file:// and other repository
types (https, oci) are left unchanged.
Includes comprehensive test coverage for various dependency scenarios.
Signed-off-by: Shane Starcher <shane.starcher@gmail.com>
Co-authored-by: Shane Starcher <shane.starcher@gmail.com>
yxxhero
yxxhero
Manetheren
Aditya Menon
dependabot[bot]
Copilot
Salvatore Mazzarino
Jean-Yves LENHOF
Shane Starcher
Ronaldo Umana