Commit Graph
414 Commits
Author SHA1 Message Date
Nikola JokicandCopilot App 7833438dd7 Handle scale set messages with parallel workers in the listener
The scaleset listener no longer dissects the message it polls. It owns
session management, polling and acking, and hands the whole message to a
single Scale call, so acquiring jobs and recording metrics move to the
only component that still reads them.

That handover is what makes the work parallelisable. The listener used to
replay a message one API call at a time, in a fixed order: every job
started patch, then every job completed, then the scale patch. The job
events touch distinct EphemeralRunners and carry no ordering between
them, so they now run across a bounded worker pool, with the worker that
patches the EphemeralRunnerSet running alongside them.

The one ordering that does matter is kept. deleteIdleEphemeralRunners
skips a runner only once it carries a job request ID, so a patch that
lowers the replica count could offer up a runner that just picked up a
job if it were published while job started patches were still in flight.
The scaling worker therefore waits for the event workers on a scale down,
and only then. A patch that scales up or holds cannot delete anything.

Kubernetes has no bulk write: get, create, update, patch and delete are
single-resource verbs, and deletecollection is the only collection-scoped
mutating verb there is, so N events cannot be collapsed into fewer
requests. Server side apply is a PATCH on one object URI and does not
change that. Issuing the N patches concurrently over the one HTTP/2
connection is the available win; the alternative is writing fewer
objects, which trades N cheap independent writes for one contended,
watch-amplifying, size-bounded write.

The pool size is configurable through listenerConfig.scaler.workers,
alongside the existing qps and burst, and defaults to 10: two calls per
event keeps a full pool well inside the default QPS budget.

Statistics are now cached by the scaler. The listener stopped tracking
them, and a long poll that times out carries no message at all, so
without the cache an idle scale set would stop converging.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-18 15:05:54 +02:00
dc12c2d417 Keep the outdated phase until the runner spec itself changes (#4660)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-09-17 17:03:21 +02:00
9ce3169df3 Let the listener own the EphemeralRunner Running phase transition (#4646)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-09-15 13:46:40 +02:00
484564e6d6 Defer scale up until the listener publishes a state that accounts for finished runners (#4642)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-09-11 15:29:02 +02:00
6f89d057c0 Track runner spec updates with an actionable revision (#4638)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-09-11 12:44:03 +00:00
Nikola JokicandCopilot App 5581cd0c54 Track AutoscalingRunnerSet updates with metadata.generation (#4636)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-11 14:38:30 +02:00
Nikola JokicandCopilot App 22046a2718 Validate and coerce listener metadata at render time (#4640)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-10 09:14:48 +00:00
Nikola JokicandCopilot App adf7d0026e Validate label and annotation metadata at chart render time (#4639)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-10 11:12:10 +02:00
b0e69a37bd Render label and annotation values as strings (#4637)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-09-10 10:22:05 +02:00
KR Ravindra 5e540b6c71 Add default and per-controller max-concurrent-reconciles flags (#4626)
Signed-off-by: KR Ravindra <42912207+KR-Ravindra@users.noreply.github.com>
2026-09-09 09:50:13 +02:00
Nikola Jokic 7c68e1d318 Set listener qps and burst (#4558) 2026-09-08 15:49:58 +02:00
Nikola Jokic 03463c051c Fix deprecated calls (#4570) 2026-07-14 19:34:13 +02:00
Nikola Jokic f6a3d738de Use metrics to display runner statuses instead of status field for EphemeralRunnerSet and AutoscalingRunnerSet (#4557) 2026-07-14 19:31:11 +02:00
Nikola Jokic 2fa72b510f Tag fields with optional allowing patches without issues (#4528) 2026-07-10 12:28:10 +02:00
Junya Okabe e06294f5de Make controller terminationGracePeriodSeconds configurable and align it with graceful shutdown timeout (#4556) 2026-07-07 00:28:57 +02:00
Nikola Jokic 9c50514160 Fix typo and rename status to phase (#4506) 2026-06-15 12:51:13 +02:00
Nikola Jokic 767e58e4b1 Upgrade resources in-place, causing 1-1 mapping between autoscaling runner set and ephemeral runner set (#4516) 2026-06-09 13:52:37 +02:00
dependabot[bot]andJiaren Wu 0dc5f8a0c2 Bump the gomod group across 1 directory with 9 updates (#4508)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jiaren Wu <jiaren-wu@github.com>
2026-05-30 00:55:53 +02:00
Francesco RenziandCopilot 9bb16ae49d Prepare 0.14.2 release (#4503)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-22 12:07:13 +02:00
Nikola Jokic 8ecfbc63bf Revert "Fix typo and rename status to phase" (#4505) 2026-05-22 11:58:48 +02:00
Nikola Jokic 79ddd38442 Port rate limiter to experimental charts (#4478) 2026-05-22 11:45:09 +02:00
Nikola Jokic dfcbd8344b Fix helm chart validation workflow (#4479) 2026-05-22 11:43:54 +02:00
Nikola Jokic 8eb6cbe79f Fix typo and rename status to phase (#4466) 2026-05-13 00:17:08 +02:00
Nikola Jokic ef48f429b3 Render empty arrays for kubernetes-novolume volumes fields (#4461) 2026-05-11 13:56:01 +02:00
Junya Okabe 053b8f9ae5 Add health and readiness probes to controller manager (#4459) 2026-04-29 18:08:46 +02:00
Junya Okabe 13a03302c8 Add a flag for enabling pprof on the controller manager (#4449) 2026-04-24 10:03:26 +02:00
Junya Okabe a401686bd5 Add option to disable workqueue bucket rate limiter (#4451) 2026-04-22 23:26:39 +02:00
Gleb Khaykin e0feb3b711 Fix orphan no-permission ServiceAccount in kubernetes-novolume mode (#4455) 2026-04-20 13:31:23 +02:00
Francesco Renzi 74cfc3855e Prepare 0.14.1 release (#4448) 2026-04-14 17:03:22 +01:00
Nikola Jokic 79e7b17b56 Fix null field for resource metadata fields in experimental chart (#4419) 2026-04-02 23:44:37 +02:00
Nikola Jokic 104bc6b0b0 Fix chart version for publishing (#4415) 2026-03-19 18:13:17 +00:00
Nikola Jokic 8b7f232dc4 Prepare 0.14.0 release (#4413) 2026-03-19 18:53:37 +01:00
Nikola Jokic 802dc28d38 Add multi-label support to scalesets (#4408) 2026-03-19 15:29:40 +01:00
Nikola JokicandCopilot Autofix powered by AI 9bc1c9e53e Shutdown the scaleset when runner is deprecated (#4404)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-19 13:30:20 +01:00
Nikola Jokic 40595d806f Add chart-level API to customize internal resources (#4410) 2026-03-18 14:44:30 +01:00
Nikola Jokic 2fc51aaf32 Regenerate manifests for experimental charts (#4407) 2026-03-16 10:42:07 +01:00
Nikola Jokic 276717a04b Manually bump dependencies since it needs fixes related to the controller runtime API (#4406) 2026-03-16 10:09:36 +01:00
Nikola Jokic aa031d3902 Introduce experimental chart release (#4373) 2026-03-16 10:09:05 +01:00
Nikola Jokic 1d9f626c53 Allow users to apply labels and annotations to internal resources (#4400) 2026-03-12 10:32:54 +01:00
Nikola Jokic a505fb5616 Prepare 0.13.1 release (#4341) 2025-12-23 14:57:05 +01:00
Marcus Ramberg cac7a40b70 Add support for giving kubernetes mode scaleset service account additional permissions (#4282) 2025-11-21 15:56:08 +01:00
Nikola Jokic a0c30df25b Prepare 0.13.0 release (#4280) 2025-10-16 19:25:56 +02:00
Nikola Jokic 634e42c916 Bump all dependencies (#4266) 2025-10-14 13:24:25 +02:00
Nikola JokicandCopilot e3ed1ba226 Introduce new kubernetes-novolume mode (#4250)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-10-03 12:03:38 +02:00
Yusuke Kuoka f731873df9 Add workflow name and target labels (#4240) 2025-09-30 16:01:51 +02:00
Nikola Jokic 088e2a3a90 Remove ephemeral runner when exit code != 0 and is patched with the job (#4239) 2025-09-17 21:40:37 +02:00
clechevalliandNikola Jokic 30abbe0cab Fix usage of underscore in Runner Scale Set name (#3545)
Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com>
2025-08-06 09:32:49 +01:00
Nikola Jokic c27541140a Remove JIT config from ephemeral runner status field (#4191) 2025-08-04 12:35:04 +02:00
Alex Hatzenbuhler a07dce28bb Remove deprecated preserveUnknownFields from CRDs (#4135) 2025-07-24 08:47:34 +02:00
Cory Calahan ad826725ce Update example GitHub URLs in values.yaml to include an example for enterprise account-level runners (#4181) 2025-07-16 10:40:38 +02:00