mirror of
https://github.com/helmfile/helmfile.git
synced 2026-10-18 04:45:42 +02:00
feat: kubedog integration with unified resource handling (#2383)
* feat: add kubedog-based resource tracking integration Add kubedog tracking as an alternative to Helm's --wait flag with: - Real-time deployment progress tracking - Container log streaming - Fine-grained resource filtering (trackKinds/skipKinds/trackResources) Features: - New pkg/resource package for unified manifest parsing and filtering - New pkg/kubedog package wrapping kubedog library - CLI flags: --track-mode, --track-timeout, --track-logs - Helmfile YAML support for trackMode, trackTimeout, trackLogs, trackKinds, skipKinds, trackResources - Case-insensitive kind matching for filtering - Multi-context support with proper kubeconfig/kubeContext handling Tracking supports: Deployment, StatefulSet, DaemonSet, Job Resource filtering priority (highest to lowest): 1. trackResources - explicit resource whitelist 2. skipKinds - blacklist specific kinds 3. trackKinds - whitelist specific kinds Integration: - Disable Helm --wait when using kubedog tracking - Track after successful Helm sync/apply - Respect release.Namespace as fallback for resources without namespace - Use getKubeContext() for correct cluster targeting Tests: - Unit tests for resource filtering and kubedog options - Integration test with httpbin chart - E2E snapshot tests for YAML serialization - Documentation in docs/advanced-features.md Signed-off-by: yxxhero <aiopsclub@163.com> * fix: address PR #2383 review comments (round 4) 1. resource/filter.go: Skip empty whitelist entries in matchWhitelist - At least one field (kind/name/namespace) must be specified - Prevents matching all resources with empty TrackResources entries 2. config/apply.go: Add ValidateConfig for track-mode validation - Validate --track-mode must be 'helm' or 'kubedog' - Reject invalid values like --track-mode foo 3. config/sync.go: Add ValidateConfig for track-mode validation - Same validation as apply command - Ensures consistent behavior across commands Signed-off-by: yxxhero <aiopsclub@163.com> --------- Signed-off-by: yxxhero <aiopsclub@163.com>
This commit is contained in:
yxxhero
+90
@@ -0,0 +1,90 @@
|
||||
---
|
||||
# Source: __workingdir__/testdata/snapshot/kubedog_tracking/input.yaml.gotmpl
|
||||
|
||||
filepath: input.yaml.gotmpl
|
||||
helmBinary: helm
|
||||
kustomizeBinary: kustomize
|
||||
releases:
|
||||
- chart: ../../charts/raw-0.1.0
|
||||
name: kubedog-baseline
|
||||
labels:
|
||||
chart: raw-0.1.0
|
||||
name: kubedog-baseline
|
||||
namespace: ""
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-baseline
|
||||
namespace: {{ .Release.Namespace }}
|
||||
data:
|
||||
baseline: value
|
||||
trackMode: kubedog
|
||||
- chart: ../../charts/raw-0.1.0
|
||||
name: kubedog-with-timeout
|
||||
labels:
|
||||
chart: raw-0.1.0
|
||||
name: kubedog-with-timeout
|
||||
namespace: ""
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-timeout
|
||||
namespace: {{ .Release.Namespace }}
|
||||
data:
|
||||
timeout: value
|
||||
trackMode: kubedog
|
||||
trackTimeout: 300
|
||||
- chart: ../../charts/raw-0.1.0
|
||||
name: kubedog-with-logs
|
||||
labels:
|
||||
chart: raw-0.1.0
|
||||
name: kubedog-with-logs
|
||||
namespace: ""
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-logs
|
||||
namespace: {{ .Release.Namespace }}
|
||||
data:
|
||||
logs: value
|
||||
trackMode: kubedog
|
||||
trackLogs: true
|
||||
- chart: ../../charts/raw-0.1.0
|
||||
name: kubedog-with-whitelist
|
||||
labels:
|
||||
chart: raw-0.1.0
|
||||
name: kubedog-with-whitelist
|
||||
namespace: ""
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-tracked
|
||||
namespace: {{ .Release.Namespace }}
|
||||
data:
|
||||
tracked: value
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-secret
|
||||
namespace: {{ .Release.Namespace }}
|
||||
type: Opaque
|
||||
data:
|
||||
secret: dmFsdWU=
|
||||
trackMode: kubedog
|
||||
trackKinds:
|
||||
- ConfigMap
|
||||
templates: {}
|
||||
renderedvalues: {}
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
---
|
||||
# Source: __workingdir__/testdata/snapshot/kubedog_tracking/input.yaml.gotmpl
|
||||
|
||||
filepath: input.yaml.gotmpl
|
||||
helmBinary: helm
|
||||
kustomizeBinary: kustomize
|
||||
releases:
|
||||
- chart: ../../charts/raw-0.1.0
|
||||
name: kubedog-baseline
|
||||
labels:
|
||||
chart: raw-0.1.0
|
||||
name: kubedog-baseline
|
||||
namespace: ""
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-baseline
|
||||
namespace: {{ .Release.Namespace }}
|
||||
data:
|
||||
baseline: value
|
||||
trackMode: kubedog
|
||||
- chart: ../../charts/raw-0.1.0
|
||||
name: kubedog-with-timeout
|
||||
labels:
|
||||
chart: raw-0.1.0
|
||||
name: kubedog-with-timeout
|
||||
namespace: ""
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-timeout
|
||||
namespace: {{ .Release.Namespace }}
|
||||
data:
|
||||
timeout: value
|
||||
trackMode: kubedog
|
||||
trackTimeout: 300
|
||||
- chart: ../../charts/raw-0.1.0
|
||||
name: kubedog-with-logs
|
||||
labels:
|
||||
chart: raw-0.1.0
|
||||
name: kubedog-with-logs
|
||||
namespace: ""
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-logs
|
||||
namespace: {{ .Release.Namespace }}
|
||||
data:
|
||||
logs: value
|
||||
trackMode: kubedog
|
||||
trackLogs: true
|
||||
- chart: ../../charts/raw-0.1.0
|
||||
name: kubedog-with-whitelist
|
||||
labels:
|
||||
chart: raw-0.1.0
|
||||
name: kubedog-with-whitelist
|
||||
namespace: ""
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-tracked
|
||||
namespace: {{ .Release.Namespace }}
|
||||
data:
|
||||
tracked: value
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-secret
|
||||
namespace: {{ .Release.Namespace }}
|
||||
type: Opaque
|
||||
data:
|
||||
secret: dmFsdWU=
|
||||
trackMode: kubedog
|
||||
trackKinds:
|
||||
- ConfigMap
|
||||
templates: {}
|
||||
renderedvalues: {}
|
||||
+69
@@ -0,0 +1,69 @@
|
||||
releases:
|
||||
- name: kubedog-baseline
|
||||
chart: ../../charts/raw-0.1.0
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{`{{ .Release.Name }}`}}-baseline
|
||||
namespace: {{`{{ .Release.Namespace }}`}}
|
||||
data:
|
||||
baseline: value
|
||||
trackMode: kubedog
|
||||
|
||||
- name: kubedog-with-timeout
|
||||
chart: ../../charts/raw-0.1.0
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{`{{ .Release.Name }}`}}-timeout
|
||||
namespace: {{`{{ .Release.Namespace }}`}}
|
||||
data:
|
||||
timeout: value
|
||||
trackMode: kubedog
|
||||
trackTimeout: 300
|
||||
|
||||
- name: kubedog-with-logs
|
||||
chart: ../../charts/raw-0.1.0
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{`{{ .Release.Name }}`}}-logs
|
||||
namespace: {{`{{ .Release.Namespace }}`}}
|
||||
data:
|
||||
logs: value
|
||||
trackMode: kubedog
|
||||
trackLogs: true
|
||||
|
||||
- name: kubedog-with-whitelist
|
||||
chart: ../../charts/raw-0.1.0
|
||||
values:
|
||||
- templates:
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{`{{ .Release.Name }}`}}-tracked
|
||||
namespace: {{`{{ .Release.Namespace }}`}}
|
||||
data:
|
||||
tracked: value
|
||||
- |
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: {{`{{ .Release.Name }}`}}-secret
|
||||
namespace: {{`{{ .Release.Namespace }}`}}
|
||||
type: Opaque
|
||||
data:
|
||||
secret: dmFsdWU=
|
||||
trackMode: kubedog
|
||||
trackKinds:
|
||||
- ConfigMap
|
||||
@@ -135,6 +135,7 @@ ${kubectl} create namespace ${test_ns} || fail "Could not create namespace ${tes
|
||||
. ${dir}/test-cases/issue-2418.sh
|
||||
. ${dir}/test-cases/issue-2424-sequential-values-paths.sh
|
||||
. ${dir}/test-cases/issue-2431.sh
|
||||
. ${dir}/test-cases/kubedog-tracking.sh
|
||||
|
||||
# ALL DONE -----------------------------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
test_start "kubedog-tracking - resource tracking with kubedog integration"
|
||||
|
||||
kubedog_case_dir="${cases_dir}/kubedog-tracking"
|
||||
config_file="helmfile.yaml.gotmpl"
|
||||
|
||||
info "Testing kubedog integration with httpbin chart"
|
||||
|
||||
# Test 1: Basic sync with kubedog tracking
|
||||
info "Syncing release with basic kubedog tracking"
|
||||
${helmfile} -f "${kubedog_case_dir}/${config_file}" -l name=httpbin-basic sync
|
||||
code=$?
|
||||
[ "${code}" -eq 0 ] || fail "unexpected exit code returned by helmfile sync: ${code}"
|
||||
|
||||
wait_deploy_ready httpbin-basic-httpbin
|
||||
info "Verifying httpbin-basic deployment is running"
|
||||
${kubectl} get deployment httpbin-basic-httpbin -n "${test_ns}" || fail "httpbin-basic deployment not found"
|
||||
|
||||
# Test 2: Sync with whitelist filtering
|
||||
info "Syncing release with whitelist filtering"
|
||||
${helmfile} -f "${kubedog_case_dir}/${config_file}" -l name=httpbin-with-whitelist sync
|
||||
code=$?
|
||||
[ "${code}" -eq 0 ] || fail "unexpected exit code returned by helmfile sync with whitelist: ${code}"
|
||||
|
||||
wait_deploy_ready httpbin-with-whitelist-httpbin
|
||||
info "Verifying httpbin-with-whitelist deployment is running"
|
||||
${kubectl} get deployment httpbin-with-whitelist-httpbin -n "${test_ns}" || fail "httpbin-with-whitelist deployment not found"
|
||||
|
||||
# Test 3: Sync with specific resource tracking
|
||||
info "Syncing release with specific resource tracking"
|
||||
${helmfile} -f "${kubedog_case_dir}/${config_file}" -l name=httpbin-with-resources sync
|
||||
code=$?
|
||||
[ "${code}" -eq 0 ] || fail "unexpected exit code returned by helmfile sync with resource tracking: ${code}"
|
||||
|
||||
wait_deploy_ready httpbin-with-resources-httpbin
|
||||
info "Verifying httpbin-with-resources deployment is running"
|
||||
${kubectl} get deployment httpbin-with-resources-httpbin -n "${test_ns}" || fail "httpbin-with-resources deployment not found"
|
||||
|
||||
# Test 4: Apply all releases with kubedog via CLI flag
|
||||
info "Testing apply with kubedog CLI flag"
|
||||
${helmfile} -f "${kubedog_case_dir}/${config_file}" apply --track-mode kubedog --track-timeout 60
|
||||
code=$?
|
||||
[ "${code}" -eq 0 ] || fail "unexpected exit code returned by helmfile apply: ${code}"
|
||||
|
||||
# Test 5: Cleanup
|
||||
info "Destroying all releases"
|
||||
${helmfile} -f "${kubedog_case_dir}/${config_file}" destroy
|
||||
code=$?
|
||||
[ "${code}" -eq 0 ] || fail "unexpected exit code returned by helmfile destroy: ${code}"
|
||||
|
||||
info "kubedog integration test completed successfully"
|
||||
|
||||
test_pass "kubedog-tracking"
|
||||
@@ -0,0 +1,52 @@
|
||||
# Kubedog Integration Test
|
||||
|
||||
This test validates the kubedog resource tracking integration with Helmfile.
|
||||
|
||||
## What it tests
|
||||
|
||||
1. **Basic kubedog tracking**: Deploys httpbin with `trackMode: kubedog` enabled
|
||||
2. **Whitelist filtering**: Uses `trackKinds` to only track Deployment resources
|
||||
3. **Specific resource tracking**: Uses `trackResources` to track specific resources by name
|
||||
4. **CLI flags usage**: Tests `--track-mode` and `--track-timeout` flags
|
||||
5. **Cleanup**: Ensures all releases are properly deleted
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Kubernetes cluster (minikube for local testing)
|
||||
- Helm 3.x installed
|
||||
- kubedog library integrated (built into Helmfile)
|
||||
- kubectl configured to access the cluster
|
||||
|
||||
## Test Cases
|
||||
|
||||
### httpbin-basic
|
||||
- Simple deployment with kubedog tracking enabled
|
||||
- `trackMode: kubedog`
|
||||
- `trackTimeout: 60` seconds
|
||||
- `trackLogs: false`
|
||||
|
||||
### httpbin-with-whitelist
|
||||
- Deployment with resource kind whitelist
|
||||
- Only tracks `Deployment` resources
|
||||
- Skips `ConfigMap` and `Secret` resources
|
||||
|
||||
### httpbin-with-resources
|
||||
- Deployment with specific resource tracking
|
||||
- Tracks only the deployment by name and kind
|
||||
|
||||
## Running the test
|
||||
|
||||
```bash
|
||||
# Run all integration tests including kubedog
|
||||
./test/integration/run.sh
|
||||
|
||||
# Run only kubedog test (if supported by your test framework)
|
||||
# Note: Currently all tests run together via run.sh
|
||||
```
|
||||
|
||||
## Expected behavior
|
||||
|
||||
1. All three httpbin deployments should be created successfully
|
||||
2. Kubedog should track the resources during deployment
|
||||
3. Deployments should reach ready state
|
||||
4. All releases should be cleaned up after tests
|
||||
@@ -0,0 +1,34 @@
|
||||
releases:
|
||||
- name: httpbin-basic
|
||||
namespace: {{ .Namespace }}
|
||||
chart: ../../charts/httpbin
|
||||
set:
|
||||
- name: ingress.enabled
|
||||
value: false
|
||||
trackMode: kubedog
|
||||
trackTimeout: 60
|
||||
trackLogs: false
|
||||
|
||||
- name: httpbin-with-whitelist
|
||||
namespace: {{ .Namespace }}
|
||||
chart: ../../charts/httpbin
|
||||
set:
|
||||
- name: ingress.enabled
|
||||
value: false
|
||||
trackMode: kubedog
|
||||
trackKinds:
|
||||
- Deployment
|
||||
skipKinds:
|
||||
- ConfigMap
|
||||
- Secret
|
||||
|
||||
- name: httpbin-with-resources
|
||||
namespace: {{ .Namespace }}
|
||||
chart: ../../charts/httpbin
|
||||
set:
|
||||
- name: ingress.enabled
|
||||
value: false
|
||||
trackMode: kubedog
|
||||
trackResources:
|
||||
- kind: Deployment
|
||||
name: httpbin-with-resources-httpbin
|
||||
Reference in New Issue
Block a user