public_git/actions-runner-controller
1
0
Fork 0
mirror of https://github.com/actions-runner-controller/actions-runner-controller.git synced 2026-09-30 15:06:29 +02:00
Code Issues Projects Releases Wiki Activity

Guard the applied revision patch with an optimistic lock

Browse Source 
patchAppliedActionableRevisionStatus wrapped retry.RetryOnConflict around
a plain client.MergeFrom status patch. A merge patch carries no
resourceVersion precondition, so the API server can never reject the write
as conflicting and the retry wrapper could never fire.

Worse, the monotonicity check inside the retry was unsound. Both the target
revision and the re-fetched object come from the cache-backed client, so a
stale reconcile could compare a stale target against an equally stale read,
pass the guard, and patch AppliedActionableRevision backwards. That
re-satisfies Spec.ActionableRevision > Status.AppliedActionableRevision and
sends the controller through the idle and pending runner cleanup again.

MergeFromWithOptimisticLock stamps the re-fetched resourceVersion into the
patch, so the server accepts it only if that object is still live. A
successful patch therefore proves the guard was evaluated against live data,
which is what makes the applied marker monotonic. A stale target can now only
ever under-advance the marker, never regress it, and a later reconcile with
fresh data completes the advance.

At this layer the re-fetch inside the retry still uses the cached client, so
a genuine conflict may refetch stale data, exhaust the backoff and return the
conflict error. That requeues rather than writing a bad value, so it fails
closed; it is only noisier than necessary. A later change makes the refetch
authoritative so the retry can resolve the conflict in place.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Nikola Jokic
2026-09-11 10:28:10 +02:00
co-authored by Copilot App
parent 26e62ec8ba
commit ec199b6d64
1 changed files with 11 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
controllers/actions.github.com/ephemeralrunnerset_controller.go
+11 -1
View File
@@ -259,6 +259,16 @@ func (r *EphemeralRunnerSetReconciler) Reconcile(ctx context.Context, req ctrl.R
// reconciler already has, because the cleanup can take long enough for that copy
// to go stale, and a conflicting write must not be resolved by replaying an old
// status.
//
// The patch carries an optimistic lock so that the re-fetch actually means
// something. A plain merge patch has no resourceVersion precondition, so the API
// server can never reject it as conflicting: RetryOnConflict would never fire,
// and a patch computed from a stale read could move the applied revision
// backwards, re-satisfying the spec > applied comparison above and deleting the
// idle runners all over again. With the lock, the server accepts the write only
// if the re-fetched object is still the live one, so a successful patch proves
// the monotonicity check above was evaluated against live data. A stale attempt
// conflicts and is retried or requeued instead of regressing the marker.
func (r *EphemeralRunnerSetReconciler) patchAppliedActionableRevisionStatus(ctx context.Context, key types.NamespacedName, targetAppliedRevision int64) error {
return retry.RetryOnConflict(retry.DefaultBackoff, func() error {
var latest v1alpha1.EphemeralRunnerSet
@@ -273,7 +283,7 @@ func (r *EphemeralRunnerSetReconciler) patchAppliedActionableRevisionStatus(ctx
original := latest.DeepCopy()
latest.Status.AppliedActionableRevision = targetAppliedRevision
return r.Status().Patch(ctx, &latest, client.MergeFrom(original))
return r.Status().Patch(ctx, &latest, client.MergeFromWithOptions(original, client.MergeFromWithOptimisticLock{}))
})
}