Nikola Jokic and Copilot App
7833438dd7
Handle scale set messages with parallel workers in the listener
...
The scaleset listener no longer dissects the message it polls. It owns
session management, polling and acking, and hands the whole message to a
single Scale call, so acquiring jobs and recording metrics move to the
only component that still reads them.
That handover is what makes the work parallelisable. The listener used to
replay a message one API call at a time, in a fixed order: every job
started patch, then every job completed, then the scale patch. The job
events touch distinct EphemeralRunners and carry no ordering between
them, so they now run across a bounded worker pool, with the worker that
patches the EphemeralRunnerSet running alongside them.
The one ordering that does matter is kept. deleteIdleEphemeralRunners
skips a runner only once it carries a job request ID, so a patch that
lowers the replica count could offer up a runner that just picked up a
job if it were published while job started patches were still in flight.
The scaling worker therefore waits for the event workers on a scale down,
and only then. A patch that scales up or holds cannot delete anything.
Kubernetes has no bulk write: get, create, update, patch and delete are
single-resource verbs, and deletecollection is the only collection-scoped
mutating verb there is, so N events cannot be collapsed into fewer
requests. Server side apply is a PATCH on one object URI and does not
change that. Issuing the N patches concurrently over the one HTTP/2
connection is the available win; the alternative is writing fewer
objects, which trades N cheap independent writes for one contended,
watch-amplifying, size-bounded write.
The pool size is configurable through listenerConfig.scaler.workers,
alongside the existing qps and burst, and defaults to 10: two calls per
event keeps a full pool well inside the default QPS budget.
Statistics are now cached by the scaler. The listener stopped tracking
them, and a long poll that times out carries no message at all, so
without the cache an idle scale set would stop converging.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-18 15:05:54 +02:00
Nikola Jokic and Copilot App
be44bb58cf
Pin the listener's optimistic lock to API server behavior ( #4648 )
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-17 17:02:45 +02:00
9ce3169df3
Let the listener own the EphemeralRunner Running phase transition ( #4646 )
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-09-15 13:46:40 +02:00
Nikola Jokic
7c68e1d318
Set listener qps and burst ( #4558 )
2026-09-08 15:49:58 +02:00
Nikola Jokic
f0d0b1d439
Add guards on timings so we don't publish incorrect metrics ( #4621 )
2026-09-08 14:32:24 +02:00
Nikola Jokic
e7b6482761
Fix job execution duration when runner assign time is not set ( #4472 )
2026-05-11 15:57:43 +02:00
Nikola Jokic and Copilot Autofix powered by AI
9bc1c9e53e
Shutdown the scaleset when runner is deprecated ( #4404 )
...
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-03-19 13:30:20 +01:00
Nikola Jokic
dc7c858e68
Remove actions client ( #4405 )
2026-03-16 14:39:55 +01:00
Nikola Jokic
f99c6eda0b
Moving to scaleset client for the controller ( #4390 )
2026-03-13 14:36:41 +01:00
Nikola Jokic
8b7fd9ffef
Switch client to scaleset library for the listener and update mocks ( #4383 )
2026-02-24 14:17:31 +01:00
Nikola Jokic
c6e4c94a6a
Fix tests and generate mocks ( #4384 )
2026-02-24 13:36:01 +01:00
Yusuke Kuoka
f731873df9
Add workflow name and target labels ( #4240 )
2025-09-30 16:01:51 +02:00
Nikola Jokic
088e2a3a90
Remove ephemeral runner when exit code != 0 and is patched with the job ( #4239 )
2025-09-17 21:40:37 +02:00
calx
e6e621a50a
Remove duplicate float64 call ( #4139 )
2025-06-24 11:26:20 +02:00
Nikola Jokic
e46c929241
Azure Key Vault integration to resolve secrets ( #4090 )
2025-06-11 15:53:33 +02:00
Tingluo Huang and Nikola Jokic
c359d14e69
Avoid nil point when config.Metrics is nil and expose all metrics if none are configured ( #4101 )
...
Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com >
2025-06-11 15:51:26 +02:00
Ryo Sakamoto
97697e80b4
Add job_workflow_ref label to listener metrics ( #4054 )
...
Signed-off-by: rskmm0chang <rskmm0chang@hatena.ne.jp >
2025-06-05 08:33:30 +02:00
Nikola Jokic
1dbb88cb9e
Allow use of client id as an app id ( #4057 )
2025-05-16 16:21:06 +02:00
Ryosei Karaki
f832b0b254
upgrade(golangci-lint): v2.1.2 ( #4023 )
...
Signed-off-by: karamaru-alpha <mrnk3078@gmail.com >
2025-04-17 16:14:31 +02:00
David Maxwell
ea27448da5
Fix busy runners metric ( #4016 )
2025-04-04 17:17:09 +02:00
Nikola Jokic
5a960b5ebb
Create configurable metrics ( #3975 )
2025-03-24 15:27:42 +01:00
Nikola Jokic
fb9b96bf75
Update all dependencies, conforming to the new controller-runtime API ( #3949 )
2025-03-11 15:52:52 +01:00
Nikola Jokic
7a5996f467
Remove old githubrunnerscalesetlistener, remove warning and fix config bug ( #3937 )
2025-03-07 11:58:16 +01:00
ddc872d3ee
metrics cardinality for ghalistener ( #3671 )
...
Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com >
Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com >
2025-02-26 15:34:17 +01:00
Han-Wen Nienhuys and Han-Wen Nienhuys
f673a085b0
cmd/ghalistener/config: export Validate ( #3870 )
...
Co-authored-by: Han-Wen Nienhuys <hanwenn@gmail.com >
2025-01-17 12:25:33 +01:00
Nikola Jokic
3bda9bb240
Refresh session if token expires during delete message ( #3529 )
2024-05-17 15:16:38 +02:00
Nikola Jokic
ab92e4edc3
Re-use the last desired patch on empty batch ( #3453 )
2024-05-17 15:12:16 +02:00
Nikola Jokic
f965dfef73
Shutdown metrics server when listener exits ( #3445 )
2024-04-16 21:29:03 +02:00
Nikola Jokic
4ee49fee14
Propagate max capacity information to the actions back-end ( #3431 )
2024-04-16 14:00:40 +02:00
Nikola Jokic
963ae48a3f
Include self correction on empty batch and avoid removing pending runners when cluster is busy ( #3426 )
2024-04-16 12:55:25 +02:00
Nikola Jokic and Francesco Renzi
7a643a5107
Fix overscaling when the controller is much faster then the listener ( #3371 )
...
Co-authored-by: Francesco Renzi <rentziass@gmail.com >
2024-03-20 15:36:12 +01:00
Nikola Jokic and Samuel Rats
814947c60e
Update metrics to include repository on job-based label ( #3310 )
...
Co-authored-by: Samuel Rats <samuel.rats@teads.com >
2024-03-18 12:45:52 +01:00
Nikola Jokic
7da2d7f96a
Fix acquire jobs after session refresh ghalistener ( #3307 )
2024-02-27 17:37:42 +01:00
Nikola Jokic
f7b6ad901d
Add listener graceful termination period and background context after the message is received ( #3187 )
2024-01-25 15:45:07 +01:00
Nikola Jokic
728f05c844
Delete message session when listener.Listen returns ( #3240 )
2024-01-25 15:12:19 +01:00
Nikola Jokic
c00465973e
Publish metrics in the new ghalistener ( #3193 )
2024-01-25 14:46:42 +01:00
Nikola Jokic
a029b705cd
Fix proxy issue in new listener client ( #3181 )
2023-12-21 15:35:36 +01:00
Nikola Jokic
f7eb88ce9c
Change minRunners behavior and fix the new listener min runners ( #3139 )
2023-12-13 19:39:21 +01:00
Nikola Jokic
0fd8eac305
Update user agent for new ghalistener ( #3138 )
2023-12-08 14:01:22 +01:00
Nikola Jokic
b78cadd901
Refactoring listener app with configurable fallback ( #3096 )
2023-12-08 13:41:06 +01:00
Nikola Jokic
202a97ab12
Modify user agent format with subsystem and is proxy configured information ( #3116 )
2023-12-08 13:16:29 +01:00
Nikola Jokic
65fd04540c
Bump go version and all direct dependencies to newest for k8s compatibility ( #2947 )
2023-11-14 16:19:43 +01:00
Nikola Jokic
16815230bb
Metrics: set max and min runners during startup time ( #3032 )
2023-11-07 14:20:10 +01:00
Nikola Jokic
b511953df7
Trim down metrics cardinality ( #3003 )
2023-10-20 12:20:30 +02:00
Nikola Jokic and Bassem Dghaidi
2117fd1892
Configure listener pod with the secret instead of env ( #2965 )
...
Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com >
2023-10-19 12:29:32 +02:00
Nikola Jokic
07bff8aa1e
Extend the user agent and fix the build version for the listener app ( #2892 )
2023-09-14 20:10:49 +02:00
a0a3916c80
Provide scale-set listener metrics ( #2559 )
...
Co-authored-by: Tingluo Huang <tingluohuang@github.com >
Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com >
2023-08-21 13:50:07 +02:00
Lukas Hauser
78271000c0
Logs - Add missing formatting ( #2780 )
2023-08-09 17:54:24 +09:00
Nikola Jokic
336e11a4e9
Fix scaling back to 0 after min runners were set to number > 0 ( #2742 )
2023-08-09 10:32:08 +02:00
Nikola Jokic
6fe8008640
Add configurable log format to values.yaml and propagate it to listener ( #2686 )
2023-07-05 21:06:42 +02:00
