Commit Graph
48 Commits
Author SHA1 Message Date
Nikola JokicandCopilot App 0c17a55075 Guard Running phase transition with optimistic locking
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-12 23:37:31 +02:00
Nikola JokicandCopilot App 03a2cdbdec Let the listener own the EphemeralRunner Running phase transition
The controller derived Status.Phase directly from the pod phase, so a runner
became Running as soon as its pod started, whether or not it had picked up a
job. That made Running mean "the pod is up" instead of "the runner is busy",
and it left the EphemeralRunnerSet scale-down path unable to tell an idle
runner from one that is executing a job.

The listener already knows when a job is assigned to a specific runner, so
move the transition there. HandleJobStarted now reads the runner first and
only promotes it to Running when it is not terminal (Failed, Succeeded or
Outdated) and not being deleted, then patches the phase alongside the job
fields it already writes. The listener role gains "get" on ephemeralrunners
for that read.

On the controller side updateRunStatusFromPod keeps publishing the initial
Pending phase while the pod is starting, and no longer promotes to Running.
Runners waiting for work now stay Pending, so scale-down picks them before
runners that are actually executing a job.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-12 23:37:31 +02:00
Nikola JokicandCopilot App 6b62a10639 Take the reconcile snapshot lazily, right before the first mutation
Reconcilers deep copied the object they had just fetched on every single
reconcile, purely so a merge patch could be computed on the rare pass
that actually changes something. The copy is a full recursive walk and
allocation of the object, and the overwhelming majority of reconciles
throw it away untouched.

Introduce lazyCopy, which takes the snapshot on the first call to
Mutate and hands back the live object. Because Mutate is the only way
to reach the object, the snapshot cannot be taken after the mutation it
is supposed to be diffed against, which is the way this optimization is
usually gotten wrong.

Apply it to the four Reconcile entry points, and move the two
EphemeralRunnerSet status copies inside the branch that patches, so they
are only paid for when the status really changed.

While here, drop the short circuit in the EphemeralRunner finalizer
block: `addedFinalizers || AddFinalizer(...)` skipped adding the actions
finalizer whenever the first finalizer was added.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-09-12 23:28:18 +02:00
Nikola Jokic 54147cfa5e Introduce cache for lookups of desired resource state, to reduce the amount of allocations in the controller (#4568) 2026-09-07 13:49:28 +02:00
Nikola Jokic f6a3d738de Use metrics to display runner statuses instead of status field for EphemeralRunnerSet and AutoscalingRunnerSet (#4557) 2026-07-14 19:31:11 +02:00
Nikola Jokic 767e58e4b1 Upgrade resources in-place, causing 1-1 mapping between autoscaling runner set and ephemeral runner set (#4516) 2026-06-09 13:52:37 +02:00
Junya Okabe 0f2a659878 Fix: Detect init container failure in EphemeralRunner controller (#4457) 2026-05-07 13:26:46 +02:00
Nikola JokicandCopilot Autofix powered by AI 9bc1c9e53e Shutdown the scaleset when runner is deprecated (#4404)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-19 13:30:20 +01:00
Nikola Jokic dc7c858e68 Remove actions client (#4405) 2026-03-16 14:39:55 +01:00
Nikola Jokic f99c6eda0b Moving to scaleset client for the controller (#4390) 2026-03-13 14:36:41 +01:00
Nikola Jokic bfe78ccd5d Make restart pod more flexible to different failure scenarios (#4340) 2025-12-19 15:49:42 +01:00
Nikola Jokic 50038fba61 Re-schedule if the failed reason starts with OutOf (#4336) 2025-12-16 13:26:44 +01:00
Nikola Jokic 95d2107a6a Code style changes on the controller (#4324) 2025-11-21 14:20:44 +01:00
Nikola Jokic 6d07b8d853 Add ephemeral runner finalizer during creation and check finalizer without requeue (#4320) 2025-11-20 23:06:27 +01:00
Nikola JokicandCopilot 9f9409a4c1 Handle resource quota on status forbidden by retrying (#4305)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-10 13:58:25 +01:00
Nikola Jokic 94a6f3cc3a Ensure ephemeral runner is deleted from the service on exit != 0 (#4260) 2025-10-06 11:38:56 +02:00
Nikola Jokic 088e2a3a90 Remove ephemeral runner when exit code != 0 and is patched with the job (#4239) 2025-09-17 21:40:37 +02:00
Nikola Jokic ddc2918a48 Requeue if create pod returns already exists error (#4201) 2025-08-14 17:00:48 +02:00
Nikola Jokic c27541140a Remove JIT config from ephemeral runner status field (#4191) 2025-08-04 12:35:04 +02:00
Nikola Jokic 9890c0592d Explicitly requeue during backoff ephemeral runner (#4152) 2025-06-27 12:05:43 +02:00
Nikola Jokic 3b5693eecb Remove check if runner exists after exit code 0 (#4142) 2025-06-27 11:11:39 +02:00
Nikola Jokic e46c929241 Azure Key Vault integration to resolve secrets (#4090) 2025-06-11 15:53:33 +02:00
Nikola Jokic cae7efa2c6 Create backoff mechanism for failed runners and allow re-creation of failed ephemeral runners (#4059) 2025-05-14 15:38:50 +02:00
Ryosei Karaki f832b0b254 upgrade(golangci-lint): v2.1.2 (#4023)
Signed-off-by: karamaru-alpha <mrnk3078@gmail.com>
2025-04-17 16:14:31 +02:00
Nikola Jokic d8f1a61ab6 Clean up as much as possible in a single pass for the EphemeralRunner reconciler (#3941) 2025-03-10 11:03:45 +01:00
Nikola Jokic 2dab45c373 Wrap errors in controller helper methods and swap logic in cleanups (#3960) 2025-03-07 11:58:53 +01:00
Nikola Jokic e122615553 Use Ready from the pod conditions when setting it to the EphemeralRunner (#3891) 2025-03-05 10:21:06 +01:00
Yusuke KuokaandBassem Dghaidi 32ae917937 Make EphemeralRunnerReconciler create runner pods earlier (#3831)
Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com>
2024-12-11 21:28:29 +01:00
Yusuke KuokaandBassem Dghaidi 3998f6dee6 Make EphemeralRunnerController MaxConcurrentReconciles configurable (#3832)
Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com>
2024-12-11 21:19:43 +01:00
Nikola Jokic a62ca3d853 Exclude label prefix propagation (#3607) 2024-06-21 12:12:14 +02:00
Nikola Jokic 2cc793a835 Remove .Named() from the ephemeral runner controller (#3596) 2024-06-17 10:36:08 +02:00
Serge e45ac190e2 Customize work directory (#3477) 2024-06-04 15:16:45 +02:00
Katarzyna d0fb7206a4 Fix problem with ephemeralRunner Succeeded state before build executed (#3528) 2024-06-03 10:49:45 +02:00
Nikola Jokic fa7a4f584e Extract single place to set up indexers (#3454) 2024-05-17 14:42:46 +02:00
Nikola JokicandFrancesco Renzi 8075e5ee74 Refactor actions client error to include request id (#3430)
Co-authored-by: Francesco Renzi <rentziass@gmail.com>
2024-04-16 12:57:44 +02:00
Nikola Jokic b6a95ae879 Change duplicate message key in logs while updating ephemeral runner status (#3380) 2024-03-26 12:57:46 +01:00
Nikola JokicandFrancesco Renzi 7a643a5107 Fix overscaling when the controller is much faster then the listener (#3371)
Co-authored-by: Francesco Renzi <rentziass@gmail.com>
2024-03-20 15:36:12 +01:00
Toru Komatsu b08d533105 Record the error when the creation pod fails (#3112)
Signed-off-by: utam0k <k0ma@utam0k.jp>
2023-12-07 21:11:52 +01:00
Toru Komatsu 7793e1974a Record a reason for pod failure in EphemeralRunner (#3074)
Signed-off-by: utam0k <k0ma@utam0k.jp>
2023-11-21 08:26:29 +01:00
Tingluo Huang 2bf83d0d7f Remove list/watch secrets permission from the manager cluster role. (#2276) 2023-03-14 09:23:14 -04:00
c569304271 Add support for self-signed CA certificates (#2268)
Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com>
Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-03-09 17:23:32 +00:00
6b4250ca90 Add support for proxy (#2286)
Co-authored-by: Nikola Jokic <jokicnikola07@gmail.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
Co-authored-by: Ferenc Hammerl <fhammerl@github.com>
2023-02-21 17:33:48 +00:00
Nikola JokicandTingluo Huang 8e52a6d2cf EphemeralRunner: On cleanup, if pod is pending, delete from service (#2255)
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2023-02-11 19:55:12 -05:00
Nikola Jokic 9990243520 Early return if finalizer does not exist to make it more readable (#2262) 2023-02-08 15:21:13 +01:00
Tingluo Huang facae69e0b Remove un-required permissions for the manager-role of the new AutoScalingRunnerSet (#2260) 2023-02-07 12:37:09 -05:00
dependabot[bot]andYusuke Kuoka 219ba5b477 chore(deps): bump sigs.k8s.io/controller-runtime from 0.13.1 to 0.14.1 (#2132)
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Yusuke Kuoka <ykuoka@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
2023-01-27 09:23:28 +09:00
Tingluo Huang 4932412cd6 Fix L0 test to make it more reliable. (#2178) 2023-01-19 07:33:04 -05:00
622eaa34f8 Introduce new preview auto-scaling mode for ARC. (#2153)
Co-authored-by: Cory Miller <cory-miller@github.com>
Co-authored-by: Nikola Jokic <nikola-jokic@github.com>
Co-authored-by: Ava Stancu <AvaStancu@github.com>
Co-authored-by: Ferenc Hammerl <fhammerl@github.com>
Co-authored-by: Francesco Renzi <rentziass@github.com>
Co-authored-by: Bassem Dghaidi <Link-@github.com>
2023-01-17 12:06:20 -05:00