public_git/postgres-operator
1
0
Fork 0
mirror of https://github.com/zalando/postgres-operator.git synced 2026-10-07 04:51:28 +02:00
Code Issues Projects Releases Wiki Activity

Merge branch 'master' into fix/set-password-encryption-default-to-scram-sha-256

Browse Source
This commit is contained in:
Felix Kunde
2026-06-03 18:17:28 +02:00
committed by GitHub
parent 7bb4ecce42 a1bbfd9d9a
commit fb18cce0fe
157 changed files with 17422 additions and 4076 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/administrator.md
+18 -14
View File
@@ -65,7 +65,10 @@ the `PGVERSION` environment variable is set for the database pods. Since
In-place major version upgrades can be configured to be executed by the
operator with the `major_version_upgrade_mode` option. By default, it is
enabled (mode: `manual`). In any case, altering the version in the manifest
will trigger a rolling update of pods to update the `PGVERSION` env variable.
will update the desired `PGVERSION`. If `maintenanceWindows` are configured,
major-version-related pod rotation is deferred until the next maintenance
window. Without maintenance windows, the operator will trigger a rolling
update of pods to apply the new `PGVERSION`.
Spilo's [`configure_spilo`](https://github.com/zalando/spilo/blob/master/postgres-appliance/scripts/configure_spilo.py)
script will notice the version mismatch but start the current version again.
@@ -92,10 +95,11 @@ Thus, the `full` mode can create drift between desired and actual state.
### Upgrade during maintenance windows
When `maintenanceWindows` are defined in the Postgres manifest the operator
will trigger a major version upgrade only during these periods. Make sure they
are at least twice as long as your configured `resync_period` to guarantee
that operator actions can be triggered.
When `maintenanceWindows` are defined in the Postgres manifest or in the global
config the operator will trigger major-version-related pod rotation and the
major version upgrade only during these periods. Make sure they are at least
twice as long as your configured `resync_period` to guarantee that operator
actions can be triggered.
### Upgrade annotations
@@ -887,15 +891,13 @@ cluster manifest. In the case any of these variables are omitted from the
manifest, the operator configuration settings `enable_master_load_balancer` and
`enable_replica_load_balancer` apply. Note that the operator settings affect
all Postgresql services running in all namespaces watched by the operator.
If load balancing is enabled two default annotations will be applied to its
services:
If load balancing is enabled the following default annotation will be applied to
its services:
- `external-dns.alpha.kubernetes.io/hostname` with the value defined by the
operator configs `master_dns_name_format` and `replica_dns_name_format`.
This value can't be overwritten. If any changing in its value is needed, it
MUST be done changing the DNS format operator config parameters; and
- `service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout` with
a default value of "3600".
There are multiple options to specify service annotations that will be merged
with each other and override in the following order (where latter take
@@ -1312,7 +1314,7 @@ aws_or_gcp:
If cluster members have to be (re)initialized restoring physical backups
happens automatically either from the backup location or by running
[pg_basebackup](https://www.postgresql.org/docs/17/app-pgbasebackup.html)
[pg_basebackup](https://www.postgresql.org/docs/18/app-pgbasebackup.html)
on one of the other running instances (preferably replicas if they do not lag
behind). You can test restoring backups by [cloning](user.md#how-to-clone-an-existing-postgresql-cluster)
clusters.
@@ -1346,10 +1348,12 @@ If you are using [additional environment variables](#custom-pod-environment-vari
to access your backup location you have to copy those variables and prepend
the `STANDBY_` prefix for Spilo to find the backups and WAL files to stream.
Alternatively, standby clusters can also stream from a remote primary cluster.
Standby clusters can also stream from a remote primary cluster.
You have to specify the host address. Port is optional and defaults to 5432.
Note, that only one of the options (`s3_wal_path`, `gs_wal_path`,
`standby_host`) can be present under the `standby` top-level key.
You can combine `standby_host` with either `s3_wal_path` or `gs_wal_path`
for additional redundancy. Note that `s3_wal_path` and `gs_wal_path` are
mutually exclusive. At least one of `s3_wal_path`, `gs_wal_path`, or
`standby_host` must be specified under the `standby` top-level key.
## Logical backups
@@ -1498,7 +1502,7 @@ make docker
# build in image in minikube docker env
eval $(minikube docker-env)
docker build -t ghcr.io/zalando/postgres-operator-ui:v1.13.0 .
docker build -t ghcr.io/zalando/postgres-operator-ui:v1.15.1 .
# apply UI manifests next to a running Postgres Operator
kubectl apply -f manifests/
docs/developer.md
+7 -11
View File
@@ -33,17 +33,14 @@ by setting the `GO111MODULE` environment variable to `on`. The make targets do
this for you, so simply run
```bash
make deps
make
```
This would take a while to complete. You have to redo `make deps` every time
your dependencies list changes, i.e. after adding a new library dependency.
Build the operator with the `make docker` command. You may define the TAG
variable to assign an explicit tag to your Docker image and the IMAGE to set
the image name. By default, the tag is computed with
`git describe --tags --always --dirty` and the image is
`registry.opensource.zalan.do/acid/postgres-operator`
`ghcr.io/zalando/postgres-operator`
```bash
export TAG=$(git describe --tags --always --dirty)
@@ -72,7 +69,7 @@ make docker
# kind
make docker
kind load docker-image registry.opensource.zalan.do/acid/postgres-operator:${TAG} --name <kind-cluster-name>
kind load docker-image ghcr.io/zalando/postgres-operator:${TAG} --name <kind-cluster-name>
```
Then create a new Postgres Operator deployment.
@@ -223,14 +220,13 @@ dlv connect 127.0.0.1:DLV_PORT
Prerequisites:
```bash
make deps
make mocks
```
To run all unit tests, you can simply do:
```bash
go test ./pkg/...
make test
```
In case if you need to debug your unit test, it's possible to use delve:
@@ -276,10 +272,10 @@ Examples for fake K8s objects can be found in:
The operator provides reference end-to-end (e2e) tests to
ensure various infrastructure parts work smoothly together. The test code is available at `e2e/tests`.
The special `registry.opensource.zalan.do/acid/postgres-operator-e2e-tests-runner` image is used to run the tests. The container mounts the local `e2e/tests` directory at runtime, so whatever you modify in your local copy of the tests will be executed by a test runner. By maintaining a separate test runner image we avoid the need to re-build the e2e test image on every build.
The special `ghcr.io/zalando/postgres-operator-e2e-tests-runner` image is used to run the tests. The container mounts the local `e2e/tests` directory at runtime, so whatever you modify in your local copy of the tests will be executed by a test runner. By maintaining a separate test runner image we avoid the need to re-build the e2e test image on every build.
Each e2e execution tests a Postgres Operator image built from the current git branch. The test
runner creates a new local K8s cluster using [kind](https://kind.sigs.k8s.io/),
Each e2e execution tests a Postgres Operator image built from the current git branch.
The test runner creates a new local K8s cluster using [kind](https://kind.sigs.k8s.io/),
utilizes provided manifest examples, and runs e2e tests contained in the `tests`
folder. The K8s API client in the container connects to the `kind` cluster via
the standard Docker `bridge` network. The kind cluster is deleted if tests
docs/reference/cluster_manifest.md
+22 -7
View File
@@ -85,6 +85,10 @@ These parameters are grouped directly under the `spec` key in the manifest.
requires a custom Spilo image. Note the FSGroup of a Pod cannot be changed
without recreating a new Pod. Optional.
* **livenessProbe**
Allows for adding a liveness probe to the Spilo container to detect if it's
running properly.
* **enableMasterLoadBalancer**
boolean flag to override the operator defaults (set by the
`enable_master_load_balancer` parameter) to define whether to enable the load
@@ -116,9 +120,11 @@ These parameters are grouped directly under the `spec` key in the manifest.
* **maintenanceWindows**
a list which defines specific time frames when certain maintenance operations
such as automatic major upgrades or master pod migration. Accepted formats
are "01:00-06:00" for daily maintenance windows or "Sat:00:00-04:00" for specific
days, with all times in UTC.
such as automatic major upgrades or master pod migration are allowed to happen.
Accepted formats are "01:00-06:00" for daily maintenance windows or
"Sat:00:00-04:00" for specific days, with all times in UTC. Note, when the
global config option `enable_maintenance_windows` is false, the specified
windows will be ignored.
* **users**
a map of usernames to user flags for the users that should be created in the
@@ -457,22 +463,31 @@ under the `clone` top-level key and do not affect the already running cluster.
On startup, an existing `standby` top-level key creates a standby Postgres
cluster streaming from a remote location - either from a S3 or GCS WAL
archive or a remote primary. Only one of options is allowed and required
if the `standby` key is present.
archive, a remote primary, or a combination of both. At least one of
`s3_wal_path`, `gs_wal_path`, or `standby_host` must be specified.
Note that `s3_wal_path` and `gs_wal_path` are mutually exclusive.
* **s3_wal_path**
the url to S3 bucket containing the WAL archive of the remote primary.
Can be combined with `standby_host` for additional redundancy.
* **gs_wal_path**
the url to GS bucket containing the WAL archive of the remote primary.
Can be combined with `standby_host` for additional redundancy.
* **standby_host**
hostname or IP address of the primary to stream from.
Can be specified alone or combined with either `s3_wal_path` or `gs_wal_path`.
* **standby_port**
TCP port on which the primary is listening for connections. Patroni will
use `"5432"` if not set.
* **standby_primary_slot_name**
name of the replication slot to use on the primary server when streaming
from a remote primary. See the Patroni documentation
[here](https://patroni.readthedocs.io/en/latest/standby_cluster.html) for more details. Optional.
## Volume properties
Those parameters are grouped under the `volume` top-level key and define the
@@ -638,7 +653,7 @@ the global configuration before adding the `tls` section'.
## Change data capture streams
This sections enables change data capture (CDC) streams via Postgres'
[logical decoding](https://www.postgresql.org/docs/17/logicaldecoding.html)
[logical decoding](https://www.postgresql.org/docs/18/logicaldecoding.html)
feature and `pgoutput` plugin. While the Postgres operator takes responsibility
for providing the setup to publish change events, it relies on external tools
to consume them. At Zalando, we are using a workflow based on
@@ -671,7 +686,7 @@ can have the following properties:
The CDC operator is following the [outbox pattern](https://debezium.io/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/).
The application is responsible for putting events into a (JSON/B or VARCHAR)
payload column of the outbox table in the structure of the specified target
event type. The operator will create a [PUBLICATION](https://www.postgresql.org/docs/17/logical-replication-publication.html)
event type. The operator will create a [PUBLICATION](https://www.postgresql.org/docs/18/logical-replication-publication.html)
in Postgres for all tables specified for one `database` and `applicationId`.
The CDC operator will consume from it shortly after transactions are
committed to the outbox table. The `idColumn` will be used in telemetry for
docs/reference/command_line_and_environment.md
+6
View File
@@ -23,6 +23,12 @@ The following command-line options are supported for the operator:
off can can be overridden by the aforementioned operator configuration
option.
* **-kubeqps**
set the maximum number of Kubernetes API requests per second. Default is 10.
* **-kubeburst**
set the burst limit for Kubernetes API requests, allowing temporary spikes beyond the configured QPS. Default is 20.
In addition to that, standard [glog
flags](https://godoc.org/github.com/golang/glog) are also supported. For
instance, one may want to add `-alsologtostderr` and `-v=8` to debug the
docs/reference/operator_parameters.md
+41 -5
View File
@@ -163,7 +163,26 @@ Those are top-level keys, containing both leaf keys and groups.
for some clusters it might be required to scale beyond the limits that can be
configured with `min_instances` and `max_instances` options. You can define
an annotation key that can be used as a toggle in cluster manifests to ignore
globally configured instance limits. The default is empty.
globally configured instance limits. The value must be `"true"` to be
effective. The default is empty which means the feature is disabled.
* **ignore_resources_limits_annotation_key**
for some clusters it might be required to request resources beyond the globally
configured thresholds for maximum requests and minimum limits. You can define
an annotation key that can be used as a toggle in cluster manifests to ignore
the thresholds. The value must be `"true"` to be effective. The default is empty
which means the feature is disabled.
* **enable_maintenance_windows**
toggle for using the maintenance windows feature. Default is `"true"`.
* **maintenance_windows**
a list which defines specific time frames when certain maintenance
operations such as automatic major upgrades or master pod migration are
allowed to happen for all database clusters. Accepted formats are
"01:00-06:00" for daily maintenance windows or "Sat:00:00-04:00" for
specific days, with all times in UTC. Locally defined maintenance
windows take precedence over globally configured ones.
* **resync_period**
period between consecutive sync requests. The default is `30m`.
@@ -252,12 +271,12 @@ CRD-configuration, they are grouped under the `major_version_upgrade` key.
* **minimal_major_version**
The minimal Postgres major version that will not automatically be upgraded
when `major_version_upgrade_mode` is set to `"full"`. The default is `"13"`.
when `major_version_upgrade_mode` is set to `"full"`. The default is `"14"`.
* **target_major_version**
The target Postgres major version when upgrading clusters automatically
which violate the configured allowed `minimal_major_version` when
`major_version_upgrade_mode` is set to `"full"`. The default is `"17"`.
`major_version_upgrade_mode` is set to `"full"`. The default is `"18"`.
## Kubernetes resources
@@ -315,6 +334,10 @@ configuration they are grouped under the `kubernetes` key.
Postgres pods are [terminated forcefully](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination)
after this timeout. The default is `5m`.
* **liveness_probe**
Allows for adding a liveness probe to the Spilo container to detect if it's
running properly. Cannot be configured via ConfigMap. Default is empty.
* **custom_pod_annotations**
This key/value map provides a list of annotations that get attached to each pod
of a database created by the operator. If the annotation key is also provided
@@ -824,7 +847,7 @@ grouped under the `logical_backup` key.
runs `pg_dumpall` on a replica if possible and uploads compressed results to
an S3 bucket under the key `/<configured-s3-bucket-prefix>/<pg_cluster_name>/<cluster_k8s_uuid>/logical_backups`.
The default image is the same image built with the Zalando-internal CI
pipeline. Default: "ghcr.io/zalando/postgres-operator/logical-backup:v1.13.0"
pipeline. Default: "ghcr.io/zalando/postgres-operator/logical-backup:v1.15.1"
* **logical_backup_google_application_credentials**
Specifies the path of the google cloud service account json file. Default is empty.
@@ -881,6 +904,19 @@ grouped under the `logical_backup` key.
* **logical_backup_cronjob_environment_secret**
Reference to a Kubernetes secret, which keys will be added as environment variables to the cronjob. Default: ""
The following environment variables can be passed to the logical backup
cronjob via `logical_backup_cronjob_environment_secret` to control
connectivity checks before the backup starts:
* **LOGICAL_BACKUP_CONNECT_RETRIES**
Number of times to retry connecting to the target PostgreSQL pod before
giving up. This is useful when NetworkPolicy enforcement introduces a
short delay before a newly-created pod's IP is allowed through ingress
rules on the destination node. Default: "10"
* **LOGICAL_BACKUP_CONNECT_RETRY_DELAY**
Delay in seconds between connectivity retries. Default: "2"
## Debugging the operator
Options to aid debugging of the operator itself. Grouped under the `debug` key.
@@ -1043,7 +1079,7 @@ operator being able to provide some reasonable defaults.
* **connection_pooler_image**
Docker image to use for connection pooler deployment.
Default: "registry.opensource.zalan.do/acid/pgbouncer"
Default: "ghcr.io/zalando/postgres-operator/pgbouncer:latest"
* **connection_pooler_max_db_connections**
How many connections the pooler can max hold. This value is divided among the
docs/user.md
+40 -12
View File
@@ -30,7 +30,7 @@ spec:
databases:
foo: zalando
postgresql:
version: "17"
version: "18"
```
Once you cloned the Postgres Operator [repository](https://github.com/zalando/postgres-operator)
@@ -110,7 +110,7 @@ metadata:
spec:
[...]
postgresql:
version: "17"
version: "18"
parameters:
password_encryption: scram-sha-256
```
@@ -518,7 +518,7 @@ Postgres Operator will create the following NOLOGIN roles:
The `<dbname>_owner` role is the database owner and should be used when creating
new database objects. All members of the `admin` role, e.g. teams API roles, can
become the owner with the `SET ROLE` command. [Default privileges](https://www.postgresql.org/docs/17/sql-alterdefaultprivileges.html)
become the owner with the `SET ROLE` command. [Default privileges](https://www.postgresql.org/docs/18/sql-alterdefaultprivileges.html)
are configured for the owner role so that the `<dbname>_reader` role
automatically gets read-access (SELECT) to new tables and sequences and the
`<dbname>_writer` receives write-access (INSERT, UPDATE, DELETE on tables,
@@ -595,7 +595,7 @@ spec:
### Schema `search_path` for default roles
The schema [`search_path`](https://www.postgresql.org/docs/17/ddl-schemas.html#DDL-SCHEMAS-PATH)
The schema [`search_path`](https://www.postgresql.org/docs/18/ddl-schemas.html#DDL-SCHEMAS-PATH)
for each role will include the role name and the schemas, this role should have
access to. So `foo_bar_writer` does not have to schema-qualify tables from
schemas `foo_bar_writer, bar`, while `foo_writer` can look up `foo_writer` and
@@ -696,7 +696,7 @@ handle it.
### HugePages support
The operator supports [HugePages](https://www.postgresql.org/docs/17/kernel-resources.html#LINUX-HUGEPAGES).
The operator supports [HugePages](https://www.postgresql.org/docs/18/kernel-resources.html#LINUX-HUGEPAGES).
To enable HugePages, set the matching resource requests and/or limits in the manifest:
```yaml
@@ -715,7 +715,7 @@ but Kubernetes will not spin up the pod if the requested HugePages cannot be all
For more information on HugePages in Kubernetes, see also
[https://kubernetes.io/docs/tasks/manage-hugepages/scheduling-hugepages/](https://kubernetes.io/docs/tasks/manage-hugepages/scheduling-hugepages/)
## Use taints, tolerations and node affinity for dedicated PostgreSQL nodes
## Use taints, tolerations, node affinity and topology spread constraint for dedicated PostgreSQL nodes
To ensure Postgres pods are running on nodes without any other application pods,
you can use [taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
@@ -756,9 +756,26 @@ spec:
If you need to define a `nodeAffinity` for all your Postgres clusters use the
`node_readiness_label` [configuration](administrator.md#node-readiness-labels).
If you need PostgreSQL Pods to run on separate nodes, you can use the
[topologySpreadConstraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/) to control how they are distributed across your cluster.
This ensures they are spread among failure domains such as
regions, zones, nodes, or other user-defined topology domains.
```yaml
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
name: acid-minimal-cluster
spec:
topologySpreadConstraints:
- maxskew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
```
## In-place major version upgrade
Starting with Spilo 13, operator supports in-place major version upgrade to a
Starting with Spilo 14, operator supports in-place major version upgrade to a
higher major version (e.g. from PG 14 to PG 16). To trigger the upgrade,
simply increase the version in the manifest. It is your responsibility to test
your applications against the new version before the upgrade; downgrading is
@@ -793,7 +810,7 @@ spec:
clone:
uid: "efd12e58-5786-11e8-b5a7-06148230260c"
cluster: "acid-minimal-cluster"
timestamp: "2017-12-19T12:40:33+01:00"
timestamp: "2025-12-19T12:40:33+01:00"
```
Here `cluster` is a name of a source cluster that is going to be cloned. A new
@@ -828,7 +845,7 @@ spec:
clone:
uid: "efd12e58-5786-11e8-b5a7-06148230260c"
cluster: "acid-minimal-cluster"
timestamp: "2017-12-19T12:40:33+01:00"
timestamp: "2025-12-19T12:40:33+01:00"
s3_wal_path: "s3://custom/path/to/bucket"
s3_endpoint: https://s3.acme.org
s3_access_key_id: 0123456789abcdef0123456789abcdef
@@ -839,7 +856,7 @@ spec:
### Clone directly
Another way to get a fresh copy of your source DB cluster is via
[pg_basebackup](https://www.postgresql.org/docs/17/app-pgbasebackup.html). To
[pg_basebackup](https://www.postgresql.org/docs/18/app-pgbasebackup.html). To
use this feature simply leave out the timestamp field from the clone section.
The operator will connect to the service of the source cluster by name. If the
cluster is called test, then the connection string will look like host=test
@@ -901,8 +918,9 @@ the PostgreSQL version between source and target cluster has to be the same.
To start a cluster as standby, add the following `standby` section in the YAML
file. You can stream changes from archived WAL files (AWS S3 or Google Cloud
Storage) or from a remote primary. Only one option can be specified in the
manifest:
Storage), from a remote primary, or combine a remote primary with a WAL archive.
At least one of `s3_wal_path`, `gs_wal_path`, or `standby_host` must be specified.
Note that `s3_wal_path` and `gs_wal_path` are mutually exclusive.
```yaml
spec:
@@ -930,6 +948,16 @@ spec:
standby_port: "5433"
```
You can also combine a remote primary with a WAL archive for additional redundancy:
```yaml
spec:
standby:
standby_host: "acid-minimal-cluster.default"
standby_port: "5433"
s3_wal_path: "s3://<bucketname>/spilo/<source_db_cluster>/<UID>/wal/<PGVERSION>"
```
Note, that the pods and services use the same role labels like for normal clusters:
The standby leader is labeled as `master`. When using the `standby_host` option
you have to copy the credentials from the source cluster's secrets to successfully