Commit Graph
101 Commits
Author SHA1 Message Date
Nikola JokicandCopilot App e3656c69ff Defer scale up until the listener publishes a state that accounts for finished runners
Spec.Replicas is the count the listener asked for when it published
Spec.PatchID. When the EphemeralRunnerSet controller cleaned up finished
runners in the same reconcile, it then compared that count against a live
count the cleanup had just reduced, and created runners to replace jobs
that had already completed. Nobody asked for those runners.

Delete the finished runners, record the patch ID the cleanup belongs to in
Status.FinishedRunnerCleanupPatchID, and return, so the scaling decision is
made on the next reconcile against post-cleanup data. Scale up stays
suppressed while Spec.PatchID still equals that recorded patch ID: the gap
below Spec.Replicas is the one the cleanup opened, not new demand. The
listener marks itself dirty on every job completion and publishes a fresh
incrementing patch ID, so the suppression lifts as soon as it reports a
desired state that accounts for the completions.

Runners that are mid-deletion were not counted at all, which let the
controller over-create while deletions were still in flight. Count them
towards the scale-up total.

The cleanup helper is renamed to deleteTerminatedEphemeralRunners because
it is no longer specific to finished runners.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-10 22:56:48 +02:00
Nikola JokicandCopilot App 26e62ec8ba Regenerate CRDs for the actionable revision doc comment
The review suggestion that reworded ActionableRevision's doc comment changed
the generated CRD description too, but the manifests were not regenerated.
CI compares the chart CRDs against config/crd/bases, so leave them in sync.

Description text only; the schema is unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-10 22:56:48 +02:00
Nikola JokicandCopilot App ec793dfe9f Track runner spec updates with an actionable revision
When the AutoscalingRunnerSet's runner spec changes, the EphemeralRunnerSet
has to delete its idle and pending runners so they are rebuilt from the new
spec. That was detected by hashing Spec.EphemeralRunnerSpec into the
actions.github.com/integrity-hash annotation and comparing the annotation
against a freshly computed hash.

Replace it with a monotonic revision counter split across spec and status.
The AutoscalingRunnerSet controller bumps Spec.ActionableRevision when it
patches a new runner spec across; the EphemeralRunnerSet controller advances
Status.AppliedActionableRevision only after the cleanup has actually
succeeded. Because the applied marker lives in status and is written last, a
controller that crashes part-way through the cleanup comes back with the
applied revision still behind the spec revision and redoes the work, instead
of skipping runners that are still running the old spec.

The drift check uses apiequality.Semantic.DeepEqual rather than cmp.Equal or
reflect.DeepEqual. Most PodSpec collection fields carry omitempty, so a
template containing an explicitly empty value (`env: []`) is dropped when the
EphemeralRunnerSet is written and reads back as nil. A strict comparison would
report drift on every reconcile, bump the revision each time, and delete every
idle and pending runner forever. helpers_drift_test.go pins that behaviour with
a round-trip-through-JSON test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-10 22:56:48 +02:00
Nikola JokicandCopilot App b20df164d6 Track AutoscalingRunnerSet updates with metadata.generation
The AutoscalingRunnerSet controller detected changes by hashing the spec
and the labels on every reconcile and comparing the result against the
actions.github.com/integrity-hash annotation it had written on a previous
pass. That is a hand-rolled version of something the API server already
does for us: it bumps metadata.generation on every spec write, and nothing
else. Recording that value in the status as ObservedGeneration gives the
same "has this changed since I last applied it" signal without recomputing
a hash, without an extra non-status write to the object, and with a value a
user can read and reason about.

updateStatus now takes the observed generation and patches it next to the
phase, returning early only when both are already what we want. When the
generation is ahead of the observed generation we move to Pending but
deliberately leave the observed generation where it is; it only catches up
at the end of a reconcile that actually applied the change, so a reconcile
that fails part way through is retried as pending rather than being
mistaken for settled.

The old code returned immediately after stamping the hash. That was needed
because stamping the annotation was itself a write to the object, so
continuing would have worked from a stale copy. Recording the generation
only touches the status subresource, which does not bump generation, so the
rest of the reconcile can run on the same object and apply the change in
the same pass instead of waiting for the next event.

One user-visible difference: the old hash covered Labels as well as Spec,
and metadata.generation does not move when only labels change. Editing just
a label on an AutoscalingRunnerSet no longer forces it through Pending.
Labels still propagate to the EphemeralRunnerSet; they simply no longer
count as an update that has to be applied.

Hash, ListenerSpecHash and RunnerSetSpecHash are removed. The latter two
already had no callers, and the first has none now.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-10 22:56:47 +02:00
KR Ravindra 5e540b6c71 Add default and per-controller max-concurrent-reconciles flags (#4626)
Signed-off-by: KR Ravindra <42912207+KR-Ravindra@users.noreply.github.com>
2026-09-09 09:50:13 +02:00
Nikola Jokic 7c68e1d318 Set listener qps and burst (#4558) 2026-09-08 15:49:58 +02:00
Nikola Jokic 03463c051c Fix deprecated calls (#4570) 2026-07-14 19:34:13 +02:00
Nikola Jokic f6a3d738de Use metrics to display runner statuses instead of status field for EphemeralRunnerSet and AutoscalingRunnerSet (#4557) 2026-07-14 19:31:11 +02:00
Nikola Jokic 2fa72b510f Tag fields with optional allowing patches without issues (#4528) 2026-07-10 12:28:10 +02:00
Junya Okabe e06294f5de Make controller terminationGracePeriodSeconds configurable and align it with graceful shutdown timeout (#4556) 2026-07-07 00:28:57 +02:00
Nikola Jokic 9c50514160 Fix typo and rename status to phase (#4506) 2026-06-15 12:51:13 +02:00
Nikola Jokic 767e58e4b1 Upgrade resources in-place, causing 1-1 mapping between autoscaling runner set and ephemeral runner set (#4516) 2026-06-09 13:52:37 +02:00
dependabot[bot]andJiaren Wu 0dc5f8a0c2 Bump the gomod group across 1 directory with 9 updates (#4508)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jiaren Wu <jiaren-wu@github.com>
2026-05-30 00:55:53 +02:00
Francesco RenziandCopilot 9bb16ae49d Prepare 0.14.2 release (#4503)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-22 12:07:13 +02:00
Nikola Jokic 8ecfbc63bf Revert "Fix typo and rename status to phase" (#4505) 2026-05-22 11:58:48 +02:00
Nikola Jokic 8eb6cbe79f Fix typo and rename status to phase (#4466) 2026-05-13 00:17:08 +02:00
Junya Okabe 053b8f9ae5 Add health and readiness probes to controller manager (#4459) 2026-04-29 18:08:46 +02:00
Junya Okabe 13a03302c8 Add a flag for enabling pprof on the controller manager (#4449) 2026-04-24 10:03:26 +02:00
Junya Okabe a401686bd5 Add option to disable workqueue bucket rate limiter (#4451) 2026-04-22 23:26:39 +02:00
Francesco Renzi 74cfc3855e Prepare 0.14.1 release (#4448) 2026-04-14 17:03:22 +01:00
Nikola Jokic 8b7f232dc4 Prepare 0.14.0 release (#4413) 2026-03-19 18:53:37 +01:00
Nikola Jokic 802dc28d38 Add multi-label support to scalesets (#4408) 2026-03-19 15:29:40 +01:00
Nikola JokicandCopilot Autofix powered by AI 9bc1c9e53e Shutdown the scaleset when runner is deprecated (#4404)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-19 13:30:20 +01:00
Nikola Jokic 276717a04b Manually bump dependencies since it needs fixes related to the controller runtime API (#4406) 2026-03-16 10:09:36 +01:00
Nikola Jokic 1d9f626c53 Allow users to apply labels and annotations to internal resources (#4400) 2026-03-12 10:32:54 +01:00
Nikola Jokic a505fb5616 Prepare 0.13.1 release (#4341) 2025-12-23 14:57:05 +01:00
Nikola Jokic a0c30df25b Prepare 0.13.0 release (#4280) 2025-10-16 19:25:56 +02:00
Nikola Jokic 634e42c916 Bump all dependencies (#4266) 2025-10-14 13:24:25 +02:00
Nikola Jokic 088e2a3a90 Remove ephemeral runner when exit code != 0 and is patched with the job (#4239) 2025-09-17 21:40:37 +02:00
Nikola Jokic c27541140a Remove JIT config from ephemeral runner status field (#4191) 2025-08-04 12:35:04 +02:00
Alex Hatzenbuhler a07dce28bb Remove deprecated preserveUnknownFields from CRDs (#4135) 2025-07-24 08:47:34 +02:00
Nikola Jokic 349cc0835e Fix image pull secrets list arguments in the chart (#4164) 2025-07-01 15:28:18 +02:00
Nikola Jokic ded39bede6 Prepare 0.12.1 release (#4153) 2025-06-27 13:49:47 +02:00
Nikola Jokic d9826e5244 Prepare 0.12.0 release (#4122) 2025-06-13 14:23:26 +02:00
Nikola Jokic e46c929241 Azure Key Vault integration to resolve secrets (#4090) 2025-06-11 15:53:33 +02:00
Nikola Jokic cae7efa2c6 Create backoff mechanism for failed runners and allow re-creation of failed ephemeral runners (#4059) 2025-05-14 15:38:50 +02:00
Nikola Jokic 4ca37fbdf2 Prepare 0.11.0 release (#3992) 2025-03-25 11:09:03 +01:00
Nikola Jokic 5a960b5ebb Create configurable metrics (#3975) 2025-03-24 15:27:42 +01:00
Nikola Jokic 7033e299cd Add events role permission to leader_election_role (#3988) 2025-03-24 15:10:47 +01:00
J. FernándezandNikola Jokic 3c1a323381 feat: allow namespace overrides (#3797)
Signed-off-by: Jesús Fernández <7312236+fernandezcuesta@users.noreply.github.com>
Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com>
2025-03-18 21:41:04 +01:00
Nikola Jokic fb9b96bf75 Update all dependencies, conforming to the new controller-runtime API (#3949) 2025-03-11 15:52:52 +01:00
Mikey SmetandNikola Jokic 75c6a94010 Use gha-runner-scale-set-controller.chart instead of .Chart.Version (#3729)
Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com>
2025-03-10 11:48:30 +01:00
Nikola Jokic 7a5996f467 Remove old githubrunnerscalesetlistener, remove warning and fix config bug (#3937) 2025-03-07 11:58:16 +01:00
Nikola Jokic 66172ab0bd Fix template tests and add go test on gha-validate-chart (#3886) 2025-01-15 15:54:33 +01:00
Bassem Dghaidi 1e10417be8 Prepare 0.10.1 release (#3859) 2024-12-18 16:22:50 +01:00
Bassem Dghaidi 1ef7196115 Fix helm chart bug related to runnerMaxConcurrentReconciles (#3858) 2024-12-18 16:14:55 +01:00
Bassem Dghaidi 59cb1d2c8b Prepare 0.10.0 release (#3849) 2024-12-16 11:39:55 +01:00
Bassem DghaidiandTaketoshi Fujiwara 7e04027d19 Make k8s client rate limiter parameters configurable (#3848)
Co-authored-by: Taketoshi Fujiwara <t-b-fujiwara@mercari.com>
2024-12-13 15:37:01 +01:00
Yusuke KuokaandBassem Dghaidi 3998f6dee6 Make EphemeralRunnerController MaxConcurrentReconciles configurable (#3832)
Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com>
2024-12-11 21:19:43 +01:00
Nikola Jokic 80d848339e Prepare 0.9.3 release (#3624) 2024-06-25 12:35:39 +02:00