public_git/helmfile
1
0
Fork 0
mirror of https://github.com/helmfile/helmfile.git synced 2026-10-18 15:27:03 +02:00
Code Issues Projects Releases Wiki Activity

feat: add per-release continue-on-error support (#2804)

Browse Source 
* feat: add per-release continue-on-error support

Work in progress for discussion #2799.

This introduces the initial release-level configuration needed to
continue processing independent releases after a deployment failure.
The DAG execution and failure propagation behavior are still being
implemented.

Refs #2799

Signed-off-by: Axel Delille <Axel.delille31@gmail.com>

* fix: address review findings for continueOnError

Review fixes for the per-release continueOnError feature:

1. Regenerate values-file ID hashes in pkg/state/temp_test.go — adding
   ContinueOnError to ReleaseSpec shifts generateValuesID hashes, which
   broke TestGenerateID (the test file notes these must be regenerated
   whenever ReleaseSpec changes).

2. Remove the dead skippedErrors variable in withBatches and instead log
   each skipped release with logger.Warnf at decision time, so users see
   why a release never ran instead of only learning from the final error
   list.

3. Aggregate all errors from a state file instead of returning only
   errs[0] in visitStatesWithContext/processStateFileParallel. With
   continueOnError, multiple releases can fail or be skipped in one run;
   reporting only the first error hid the skip errors (and other
   failures), contradicting the feature's contract. Single-error
   rendering is unchanged.

4. Gate tolerated errors on ReleaseErrorCodeFailure so that non-failure
   release errors (e.g. helm-diff's "changes detected" exit code 2 can
   never enable continuation or block dependents.

5. Report skipped-release messages with the dependency's plain release
   name instead of its kubeContext/namespace-qualified needs id
   (e.g. dependency database instead of default/default/database),
   matching the documented message format.

6. Restructure the withBatches helpers into guard-clause style
   (filterBlockedReleases, toleratesBatchErrors) and drop the test-only
   logger nil-guards in favor of a nop logger in tests.

7. Add end-to-end coverage through App.Sync with the exectest fake helm:
   independent releases continue after a tolerated failure, dependents
   are skipped with an explicit error, the exit code stays non-zero, and
   fail-fast remains the default without continueOnError. Also cover the
   non-failure error code case at the withBatches level.

8. Use new(true) instead of a boolPtr helper (CI-enforced check-modernize)
   and document the failure-handling interaction in docs/releases.md.

Signed-off-by: yxxhero <aiopsclub@163.com>
EOF
)
Signed-off-by: yxxhero <aiopsclub@163.com>

---------

Signed-off-by: Axel Delille <Axel.delille31@gmail.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Co-authored-by: yxxhero <aiopsclub@163.com>
This commit is contained in:
Tsukoyachi
2026-09-21 07:48:50 +08:00
committed by GitHub
co-authored by yxxhero
parent 58417fd068
commit ca58090af4
9 changed files with 527 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/configuration.md
+6
View File
@@ -473,6 +473,12 @@ The following `helmDefaults` fields are also available but not shown in the exam
`condition` controls whether a release is enabled. An empty condition enables the release. A direct `true` or `false` value is treated as a literal boolean and bypasses values lookup. Any other condition must be a values lookup path ending in `.enabled`, such as `vault.enabled`.
#### Continue On Error
`continueOnError` defaults to `false`. When set to `true`, Helmfile keeps processing independent releases after this release fails, but any release that depends on it is skipped with an explicit error such as `release "backend" was skipped because dependency "database" failed`.
This setting only changes Helmfile orchestration. It does not change Helm's per-release rollback behavior (`atomic`, `rollbackOnFailure`, `cleanupOnFail`, `wait`, `waitForJobs`, `trackMode`, hooks, or `reinstallIfForbidden`), and it does not turn a release failure into a successful command. `helmfile sync` and the sync phase of `helmfile apply` still exit with a non-zero status when any release fails. For `apply`, diff and chart-preparation failures remain fatal and are not converted into tolerated release errors.
`conditionTemplate` is evaluated before `condition` is checked. It must render to a boolean value; when both `condition` and `conditionTemplate` are set, the rendered `conditionTemplate` value replaces `condition`. Like other `*Template` fields, `conditionTemplate` is not evaluated by the `list` command.
The following per-release fields are also available:
docs/releases.md
+14
View File
@@ -46,6 +46,20 @@ On `helmfile [delete|destroy]`, deletions happen in the reverse order.
That is, `myapp1` and `myapp2` are deleted first, then `servicemesh`, and finally `logging`.
### Failure handling and `continueOnError`
By default, `helmfile [sync|apply]` stops on the first release failure (fail-fast). Set `continueOnError: true` on a release to keep processing independent releases after it fails:
```yaml
- name: myapp1
chart: charts/myapp
continueOnError: true
needs:
- servicemesh
```
When a release with `continueOnError` fails, releases in other branches of the DAG are still processed, while releases that (transitively) depend on the failed release are skipped with an error like `release "myapp2" was skipped because dependency "myapp1" failed`. The command still exits non-zero, and Helm's own rollback behavior (`atomic`, `rollbackOnFailure`, ...) is unaffected. See [Continue On Error](configuration.md#continue-on-error) for details.
### Selectors and `needs`
When using selectors/labels, `needs` are ignored by default. This behaviour can be overruled with a few parameters: