public_git
/
oauth2-proxy
mirror of https://github.com/oauth2-proxy/oauth2-proxy.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

feat(oidc): keep /ready healthy under lazy discovery and make backoff configurable 

Signed-off-by: Orkhan Huseynli <orkhan.huseyn@outlook.com>
This commit is contained in:
Orkhan Huseynli 2026-08-12 08:46:48 +04:00
parent 857a764cef
commit c57b2c6282
10 changed files with 132 additions and 111 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
CHANGELOG.md
View File

@ -8,7 +8,7 @@
## Changes since v7.15.3
- [#3487](https://github.com/oauth2-proxy/oauth2-proxy/issues/3487) Add opt-in `--oidc-lazy-discovery` (default off): when the OIDC issuer is unreachable at startup, oauth2-proxy starts anyway and performs OIDC discovery in the background (retrying with backoff), so Basic Auth via `--htpasswd-file` and other provider-independent features keep working while discovery is pending. The readiness endpoint reports not-ready and OAuth2 login returns 503 until discovery succeeds.
- [#3487](https://github.com/oauth2-proxy/oauth2-proxy/issues/3487) Add opt-in `--oidc-lazy-discovery` (default off): when the OIDC issuer is unreachable at startup, oauth2-proxy starts anyway and performs OIDC discovery in the background (retrying with backoff), so Basic Auth via `--htpasswd-file` and other provider-independent features keep working while discovery is pending. The readiness endpoint stays healthy (keeping the pod in load-balancer rotation) while OAuth2 login returns 503 until discovery succeeds. The background backoff is configurable via `--oidc-lazy-discovery-initial-interval` (default `3s`) and `--oidc-lazy-discovery-max-interval` (default `30s`).
# V7.15.3

4
docs/docs/configuration/alpha_config.md
View File

@ -544,7 +544,9 @@ character.
| `insecureSkipIssuerVerification` | _bool_ | InsecureSkipIssuerVerification skips verification of ID token issuers. When false, ID Token Issuers must match the OIDC discovery URL<br/>default set to 'false' |
| `insecureSkipNonce` | _bool_ | InsecureSkipNonce skips verifying the ID Token's nonce claim that must match<br/>the random nonce sent in the initial OAuth flow. Otherwise, the nonce is checked<br/>after the initial OAuth redeem & subsequent token refreshes.<br/>default set to 'true'<br/>Warning: In a future release, this will change to 'false' by default for enhanced security. |
| `skipDiscovery` | _bool_ | SkipDiscovery allows to skip OIDC discovery and use manually supplied Endpoints<br/>default set to 'false' |
| `lazyDiscovery` | _bool_ | LazyDiscovery allows oauth2-proxy to start even when the OIDC issuer is<br/>unreachable. When enabled, OIDC discovery is performed in the background<br/>(retrying with backoff) instead of blocking startup, so features that do<br/>not depend on the provider - such as Basic Auth via htpasswdFile - remain<br/>available while discovery is pending. The readiness endpoint reports<br/>not-ready until discovery succeeds. Only applies when discovery is enabled<br/>(SkipDiscovery is false). Note that a configuration error (rather than an<br/>unreachable issuer) is retried indefinitely instead of failing startup.<br/>default set to 'false' |
| `lazyDiscovery` | _bool_ | LazyDiscovery lets oauth2-proxy start when the OIDC issuer is unreachable,<br/>retrying discovery in the background instead of failing startup. OAuth2<br/>login returns 503 until discovery succeeds; other features (e.g. Basic Auth)<br/>keep working. default set to 'false' |
| `lazyDiscoveryInitialInterval` | _duration_ | LazyDiscoveryInitialInterval is the backoff before the first background<br/>discovery retry, doubling up to LazyDiscoveryMaxInterval. default set to '3s' |
| `lazyDiscoveryMaxInterval` | _duration_ | LazyDiscoveryMaxInterval is the maximum backoff between background<br/>discovery retries. default set to '30s' |
| `jwksURL` | _string_ | JwksURL is the OpenID Connect JWKS URL<br/>eg: https://www.googleapis.com/oauth2/v3/certs |
| `publicKeyFiles` | _[]string_ | PublicKeyFiles is a list of paths pointing to public key files in PEM format to use<br/>for verifying JWT tokens |
| `emailClaim` | _string_ | EmailClaim indicates which claim contains the user email,<br/>default set to 'email' |

4
docs/docs/configuration/overview.md
View File

@ -98,7 +98,9 @@ Provider specific options can be found on their respective subpages.
| flag: `--oidc-extra-audience`<br/>toml: `oidc_extra_audiences` | string \| list | additional audiences which are allowed to pass verification | `"[]"` |
| flag: `--oidc-groups-claim`<br/>toml: `oidc_groups_claim` | string | which OIDC claim contains the user groups | `"groups"` |
| flag: `--oidc-issuer-url`<br/>toml: `oidc_issuer_url` | string | the OpenID Connect issuer URL, e.g. `"https://accounts.google.com"` | |
| flag: `--oidc-lazy-discovery`<br/>toml: `oidc_lazy_discovery` | bool | start oauth2-proxy even if the OIDC issuer is unreachable and perform discovery in the background (retrying with backoff) instead of failing startup. Provider-independent features such as Basic Auth via `--htpasswd-file` keep working while discovery is pending; the readiness endpoint reports not-ready and OAuth2 login returns `503` until it succeeds. Only applies when OIDC discovery is enabled. **Note:** a configuration error (e.g. issuer mismatch) is retried indefinitely instead of failing startup, so the provider never becomes ready - watch the readiness endpoint and logs | false |
| flag: `--oidc-lazy-discovery`<br/>toml: `oidc_lazy_discovery` | bool | start oauth2-proxy even when the OIDC issuer is unreachable, retrying discovery in the background instead of failing startup. OAuth2 login returns `503` until discovery succeeds; other features (e.g. Basic Auth) keep working | false |
| flag: `--oidc-lazy-discovery-initial-interval`<br/>toml: `oidc_lazy_discovery_initial_interval` | duration | backoff before the first background discovery retry, doubling up to `--oidc-lazy-discovery-max-interval` | `3s` |
| flag: `--oidc-lazy-discovery-max-interval`<br/>toml: `oidc_lazy_discovery_max_interval` | duration | maximum backoff between background discovery retries | `30s` |
| flag: `--oidc-jwks-url`<br/>toml: `oidc_jwks_url` | string | OIDC JWKS URI for token verification; required if OIDC discovery is disabled and public key files are not provided | |
| flag: `--oidc-public-key-file`<br/>toml: `oidc_public_key_files` | string | Path to public key file in PEM format to use for verifying JWT tokens (may be given multiple times). Required if OIDC discovery is disabled na JWKS URL isn't provided | |
| flag: `--oidc-enabled-signing-alg`<br/>toml: `oidc_enabled_signing_algs` | string \| list | List of allowed JWT signing algorithms. When oidc discovery is enabled, the effective set is the intersection between this list and the provider's discovered supported algorithms. | |

2
main_test.go
View File

@ -188,6 +188,8 @@ redirect_url="http://localhost:4180/oauth2/callback"
InsecureSkipIssuerVerification: ptr.To(false),
SkipDiscovery: ptr.To(false),
LazyDiscovery: ptr.To(false),
LazyDiscoveryInitialInterval: options.DefaultOIDCLazyDiscoveryInitialInterval,
LazyDiscoveryMaxInterval: options.DefaultOIDCLazyDiscoveryMaxInterval,
EnabledSigningAlgs: []string{},
},
MicrosoftEntraIDConfig: options.MicrosoftEntraIDOptions{

39
oauthproxy.go
View File

@ -213,8 +213,7 @@ func NewOAuthProxy(opts *options.Options, validator func(string) bool) (*OAuthPr
return nil, err
}
readinessVerifiable := readinessVerifiers{sessionStore, providerReadiness{provider: provider}}
preAuthChain, err := buildPreAuthChain(opts, readinessVerifiable, trustedProxies)
preAuthChain, err := buildPreAuthChain(opts, sessionStore, trustedProxies)
if err != nil {
return nil, fmt.Errorf("could not build pre-auth chain: %v", err)
}
@ -374,7 +373,7 @@ func (p *OAuthProxy) buildProxySubrouter(s *mux.Router) {
// buildPreAuthChain constructs a chain that should process every request before
// the OAuth2 Proxy authentication logic kicks in.
// For example forcing HTTPS or health checks.
func buildPreAuthChain(opts *options.Options, readiness middleware.Verifiable, trustedProxies *ip.NetSet) (alice.Chain, error) {
func buildPreAuthChain(opts *options.Options, sessionStore sessionsapi.SessionStore, trustedProxies *ip.NetSet) (alice.Chain, error) {
chain := alice.New(middleware.NewScope(opts.ReverseProxy, opts.Logging.RequestIDHeader, trustedProxies))
if opts.ForceHTTPS {
@ -398,14 +397,14 @@ func buildPreAuthChain(opts *options.Options, readiness middleware.Verifiable, t
if opts.Logging.SilencePing {
chain = chain.Append(
middleware.NewHealthCheck(healthCheckPaths, healthCheckUserAgents),
middleware.NewReadynessCheck(opts.ReadyPath, readiness),
middleware.NewReadynessCheck(opts.ReadyPath, sessionStore),
middleware.NewRequestLogger(),
)
} else {
chain = chain.Append(
middleware.NewRequestLogger(),
middleware.NewHealthCheck(healthCheckPaths, healthCheckUserAgents),
middleware.NewReadynessCheck(opts.ReadyPath, readiness),
middleware.NewReadynessCheck(opts.ReadyPath, sessionStore),
)
}
@ -536,22 +535,11 @@ func setupProvider(providerConfig options.Provider) (providers.Provider, error)
return lazy, nil
}
// providerReadiness reports the readiness of a LazyProvider for the /ready deep
// health check. Non-lazy providers are always considered ready.
type providerReadiness struct {
provider providers.Provider
}
func (p providerReadiness) VerifyConnection(_ context.Context) error {
if lazy, ok := p.provider.(*providers.LazyProvider); ok && !lazy.Ready() {
return errors.New("provider not ready: OIDC discovery is still pending")
}
return nil
}
// providerReady reports whether the identity provider can serve OAuth2 flows.
// A LazyProvider is only ready once background OIDC discovery has completed; all
// other providers are always ready.
// other providers are always ready. Note this does not gate the /ready endpoint
// (which stays healthy under lazy discovery so the pod remains in load-balancer
// rotation); it only gates the OAuth2 login flow.
func (p *OAuthProxy) providerReady() bool {
if p.lazyProvider != nil {
return p.lazyProvider.Ready()
@ -559,19 +547,6 @@ func (p *OAuthProxy) providerReady() bool {
return true
}
// readinessVerifiers combines multiple Verifiable checks into one; VerifyConnection
// fails if any of them fail.
type readinessVerifiers []middleware.Verifiable
func (rs readinessVerifiers) VerifyConnection(ctx context.Context) error {
for _, r := range rs {
if err := r.VerifyConnection(ctx); err != nil {
return err
}
}
return nil
}
// buildRoutesAllowlist builds an []allowedRoute list from either the legacy
// SkipAuthRegex option (paths only support) or newer SkipAuthRoutes option
// (method=path support)

58
oauthproxy_lazy_test.go
View File

@ -1,7 +1,6 @@
package main
import (
"context"
"net/http"
"net/http/httptest"
"testing"
@ -13,6 +12,27 @@ import (
. "github.com/onsi/gomega"
)
// newLazyPendingProxy builds a proxy whose OIDC issuer is unreachable with lazy
// discovery enabled. Start is not called, so background discovery never runs and
// the provider stays not-ready for the duration of the test.
func newLazyPendingProxy(g *WithT) *OAuthProxy {
opts := baseTestOptions()
opts.Providers[0].Type = "oidc"
opts.Providers[0].OIDCConfig = options.OIDCOptions{
IssuerURL: "http://127.0.0.1:1/realms/test",
SkipDiscovery: ptr.To(false),
LazyDiscovery: ptr.To(true),
EmailClaim: options.OIDCEmailClaim,
AudienceClaims: []string{"aud"},
}
g.Expect(validation.Validate(opts)).To(Succeed())
proxy, err := NewOAuthProxy(opts, func(string) bool { return true })
g.Expect(err).ToNot(HaveOccurred())
g.Expect(proxy.providerReady()).To(BeFalse())
return proxy
}
func unreachableOIDCProvider(lazy bool) options.Provider {
return options.Provider{
ID: "test-provider",
@ -46,18 +66,19 @@ func TestSetupProviderWithoutLazyFailsFast(t *testing.T) {
g.Expect(err).To(HaveOccurred())
}
func TestProviderReadinessVerifier(t *testing.T) {
// TestReadyEndpointOKWhenProviderNotReady ensures /ready stays healthy under
// lazy discovery so the pod remains in load-balancer rotation while discovery
// is still pending.
func TestReadyEndpointOKWhenProviderNotReady(t *testing.T) {
g := NewWithT(t)
// A non-lazy provider (google needs no discovery) is always considered ready.
nonLazy, err := providers.NewProvider(options.Provider{ID: "g", Type: "google", ClientID: "client-id"})
g.Expect(err).ToNot(HaveOccurred())
g.Expect(providerReadiness{provider: nonLazy}.VerifyConnection(context.Background())).To(Succeed())
proxy := newLazyPendingProxy(g)
// A lazy provider that has not completed discovery is not ready.
lazy, err := providers.NewLazyProvider(unreachableOIDCProvider(true))
g.Expect(err).ToNot(HaveOccurred())
g.Expect(providerReadiness{provider: lazy}.VerifyConnection(context.Background())).ToNot(Succeed())
rw := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodGet, "/ready", nil)
proxy.ServeHTTP(rw, req)
g.Expect(rw.Code).To(Equal(http.StatusOK))
}
// TestOAuthStartUnavailableWhenProviderNotReady ensures the OAuth2 login flow
@ -66,22 +87,7 @@ func TestProviderReadinessVerifier(t *testing.T) {
func TestOAuthStartUnavailableWhenProviderNotReady(t *testing.T) {
g := NewWithT(t)
opts := baseTestOptions()
opts.Providers[0].Type = "oidc"
opts.Providers[0].OIDCConfig = options.OIDCOptions{
IssuerURL: "http://127.0.0.1:1/realms/test",
SkipDiscovery: ptr.To(false),
LazyDiscovery: ptr.To(true),
EmailClaim: options.OIDCEmailClaim,
AudienceClaims: []string{"aud"},
}
g.Expect(validation.Validate(opts)).To(Succeed())
// Not calling Start, so background discovery never runs and the provider
// stays not-ready for the duration of the test.
proxy, err := NewOAuthProxy(opts, func(string) bool { return true })
g.Expect(err).ToNot(HaveOccurred())
g.Expect(proxy.providerReady()).To(BeFalse())
proxy := newLazyPendingProxy(g)
rw := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodGet, "/oauth2/start", nil)

6
pkg/apis/options/legacy_options.go
View File

@ -541,6 +541,8 @@ type LegacyProvider struct {
InsecureOIDCSkipNonce bool `flag:"insecure-oidc-skip-nonce" cfg:"insecure_oidc_skip_nonce"`
SkipOIDCDiscovery bool `flag:"skip-oidc-discovery" cfg:"skip_oidc_discovery"`
OIDCLazyDiscovery bool `flag:"oidc-lazy-discovery" cfg:"oidc_lazy_discovery"`
OIDCLazyDiscoveryInitialInterval time.Duration `flag:"oidc-lazy-discovery-initial-interval" cfg:"oidc_lazy_discovery_initial_interval"`
OIDCLazyDiscoveryMaxInterval time.Duration `flag:"oidc-lazy-discovery-max-interval" cfg:"oidc_lazy_discovery_max_interval"`
OIDCJwksURL string `flag:"oidc-jwks-url" cfg:"oidc_jwks_url"`
OIDCEmailClaim string `flag:"oidc-email-claim" cfg:"oidc_email_claim"`
OIDCGroupsClaim string `flag:"oidc-groups-claim" cfg:"oidc_groups_claim"`
@ -604,6 +606,8 @@ func legacyProviderFlagSet() *pflag.FlagSet {
flagSet.Bool("insecure-oidc-skip-nonce", true, "skip verifying the OIDC ID Token's nonce claim")
flagSet.Bool("skip-oidc-discovery", false, "Skip OIDC discovery and use manually supplied Endpoints")
flagSet.Bool("oidc-lazy-discovery", false, "Start oauth2-proxy even if the OIDC issuer is unreachable and perform discovery in the background, retrying with backoff. Features that do not depend on the provider (e.g. Basic Auth via htpasswd-file) remain available while discovery is pending")
flagSet.Duration("oidc-lazy-discovery-initial-interval", DefaultOIDCLazyDiscoveryInitialInterval, "Initial backoff before the first background OIDC discovery retry when oidc-lazy-discovery is enabled; doubles after each attempt up to oidc-lazy-discovery-max-interval")
flagSet.Duration("oidc-lazy-discovery-max-interval", DefaultOIDCLazyDiscoveryMaxInterval, "Maximum backoff between background OIDC discovery retries when oidc-lazy-discovery is enabled")
flagSet.String("oidc-jwks-url", "", "OpenID Connect JWKS URL (ie: https://www.googleapis.com/oauth2/v3/certs)")
flagSet.String("oidc-groups-claim", OIDCGroupsClaim, "which OIDC claim contains the user groups")
flagSet.String("oidc-email-claim", OIDCEmailClaim, "which OIDC claim contains the user's email")
@ -726,6 +730,8 @@ func (l *LegacyProvider) convert() (Providers, error) {
InsecureSkipNonce: &l.InsecureOIDCSkipNonce,
SkipDiscovery: &l.SkipOIDCDiscovery,
LazyDiscovery: &l.OIDCLazyDiscovery,
LazyDiscoveryInitialInterval: l.OIDCLazyDiscoveryInitialInterval,
LazyDiscoveryMaxInterval: l.OIDCLazyDiscoveryMaxInterval,
JwksURL: l.OIDCJwksURL,
UserIDClaim: l.UserIDClaim,
EmailClaim: l.OIDCEmailClaim,

3
pkg/apis/options/load_test.go
View File

@ -44,6 +44,9 @@ var _ = Describe("Load", func() {
OIDCGroupsClaim: "groups",
OIDCAudienceClaims: []string{"aud"},
InsecureOIDCSkipNonce: true,
OIDCLazyDiscoveryInitialInterval: DefaultOIDCLazyDiscoveryInitialInterval,
OIDCLazyDiscoveryMaxInterval: DefaultOIDCLazyDiscoveryMaxInterval,
},
Options: Options{

42
pkg/apis/options/providers.go
View File

@ -1,6 +1,10 @@
package options
import "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/util/ptr"
import (
"time"
"github.com/oauth2-proxy/oauth2-proxy/v7/pkg/util/ptr"
)
const (
// OIDCEmailClaim is the generic email claim used by the OIDC provider.
@ -17,6 +21,14 @@ const (
// When false, a failed OIDC discovery aborts startup (historical behaviour).
DefaultOIDCLazyDiscovery bool = false
// DefaultOIDCLazyDiscoveryInitialInterval is the default value for
// OIDCOptions.LazyDiscoveryInitialInterval.
DefaultOIDCLazyDiscoveryInitialInterval time.Duration = 3 * time.Second
// DefaultOIDCLazyDiscoveryMaxInterval is the default value for
// OIDCOptions.LazyDiscoveryMaxInterval.
DefaultOIDCLazyDiscoveryMaxInterval time.Duration = 30 * time.Second
// DefaultInsecureSkipNonce is the default value
// for OIDCOptions.InsecureSkipNonce
DefaultInsecureSkipNonce bool = true
@ -304,16 +316,17 @@ type OIDCOptions struct {
// SkipDiscovery allows to skip OIDC discovery and use manually supplied Endpoints
// default set to 'false'
SkipDiscovery *bool `yaml:"skipDiscovery,omitempty"`
// LazyDiscovery allows oauth2-proxy to start even when the OIDC issuer is
// unreachable. When enabled, OIDC discovery is performed in the background
// (retrying with backoff) instead of blocking startup, so features that do
// not depend on the provider - such as Basic Auth via htpasswdFile - remain
// available while discovery is pending. The readiness endpoint reports
// not-ready until discovery succeeds. Only applies when discovery is enabled
// (SkipDiscovery is false). Note that a configuration error (rather than an
// unreachable issuer) is retried indefinitely instead of failing startup.
// default set to 'false'
// LazyDiscovery lets oauth2-proxy start when the OIDC issuer is unreachable,
// retrying discovery in the background instead of failing startup. OAuth2
// login returns 503 until discovery succeeds; other features (e.g. Basic Auth)
// keep working. default set to 'false'
LazyDiscovery *bool `yaml:"lazyDiscovery,omitempty"`
// LazyDiscoveryInitialInterval is the backoff before the first background
// discovery retry, doubling up to LazyDiscoveryMaxInterval. default set to '3s'
LazyDiscoveryInitialInterval time.Duration `yaml:"lazyDiscoveryInitialInterval,omitempty"`
// LazyDiscoveryMaxInterval is the maximum backoff between background
// discovery retries. default set to '30s'
LazyDiscoveryMaxInterval time.Duration `yaml:"lazyDiscoveryMaxInterval,omitempty"`
// JwksURL is the OpenID Connect JWKS URL
// eg: https://www.googleapis.com/oauth2/v3/certs
JwksURL string `yaml:"jwksURL,omitempty"`
@ -412,6 +425,15 @@ func (o *OIDCOptions) EnsureDefaults() {
if o.LazyDiscovery == nil {
o.LazyDiscovery = ptr.To(DefaultOIDCLazyDiscovery)
}
if o.LazyDiscoveryInitialInterval <= 0 {
o.LazyDiscoveryInitialInterval = DefaultOIDCLazyDiscoveryInitialInterval
}
if o.LazyDiscoveryMaxInterval <= 0 {
o.LazyDiscoveryMaxInterval = DefaultOIDCLazyDiscoveryMaxInterval
}
if o.LazyDiscoveryMaxInterval < o.LazyDiscoveryInitialInterval {
o.LazyDiscoveryMaxInterval = o.LazyDiscoveryInitialInterval
}
if o.UserIDClaim == "" {
o.UserIDClaim = OIDCEmailClaim
}

21
providers/lazy_provider.go
View File

@ -18,12 +18,6 @@ import (
// keep working while discovery is pending.
var ErrProviderNotReady = errors.New("provider not ready: OIDC discovery has not completed yet")
// Background discovery backoff bounds.
const (
lazyDiscoveryInitialInterval = 3 * time.Second
lazyDiscoveryMaxInterval = 30 * time.Second
)
// LazyProvider wraps a Provider whose construction depends on OIDC discovery.
// It starts out not-ready, delegating only Data() to a discovery-independent
// placeholder, and returns ErrProviderNotReady from OAuth flow methods. Once
@ -78,7 +72,16 @@ func (l *LazyProvider) setInner(p Provider) {
// succeeds the real provider is swapped in and the LazyProvider becomes ready.
// It is intended to be run in a goroutine.
func (l *LazyProvider) InitWithRetry(ctx context.Context) {
interval := lazyDiscoveryInitialInterval
initialInterval := l.providerConfig.OIDCConfig.LazyDiscoveryInitialInterval
if initialInterval <= 0 {
initialInterval = options.DefaultOIDCLazyDiscoveryInitialInterval
}
maxInterval := l.providerConfig.OIDCConfig.LazyDiscoveryMaxInterval
if maxInterval < initialInterval {
maxInterval = initialInterval
}
interval := initialInterval
attempt := 0
for {
attempt++
@ -100,8 +103,8 @@ func (l *LazyProvider) InitWithRetry(ctx context.Context) {
}
interval *= 2
if interval > lazyDiscoveryMaxInterval {
interval = lazyDiscoveryMaxInterval
if interval > maxInterval {
interval = maxInterval
}
}
}