Merge branch 'master' into gh-pages

This commit is contained in:
Felix Kunde
2024-03-15 15:52:54 +01:00
145 changed files with 2824 additions and 1377 deletions
+20 -1
View File
@@ -355,6 +355,23 @@ This would be the recommended option to enable rotation in secrets of database
owners, but only if they are not used as application users for regular read
and write operations.
### Ignore rotation for certain users
If you wish to globally enable password rotation but need certain users to
opt out from it there are two ways. First, you can remove the user from the
manifest's `users` section. The corresponding secret to this user will no
longer be synced by the operator then.
Secondly, if you want the operator to continue syncing the secret (e.g. to
recreate if it got accidentally removed) but cannot allow it being rotated,
add the user to the following list in your manifest:
```
spec:
usersIgnoringSecretRotation:
- bar_user
```
### Turning off password rotation
When password rotation is turned off again the operator will check if the
@@ -1200,7 +1217,7 @@ aws_or_gcp:
If cluster members have to be (re)initialized restoring physical backups
happens automatically either from the backup location or by running
[pg_basebackup](https://www.postgresql.org/docs/15/app-pgbasebackup.html)
[pg_basebackup](https://www.postgresql.org/docs/16/app-pgbasebackup.html)
on one of the other running instances (preferably replicas if they do not lag
behind). You can test restoring backups by [cloning](user.md#how-to-clone-an-existing-postgresql-cluster)
clusters.
@@ -1348,6 +1365,8 @@ You can also expose the operator API through a [service](https://github.com/zala
Some displayed options can be disabled from UI using simple flags under the
`OPERATOR_UI_CONFIG` field in the deployment.
The viewing and creation of clusters within the UI is limited to the namespace specified by the `TARGET_NAMESPACE` option. To allow the creation and viewing of clusters in all namespaces, set `TARGET_NAMESPACE` to `*`.
### Deploy the UI on K8s
Now, apply all manifests from the `ui/manifests` folder to deploy the Postgres
Binary file not shown.

After

Width:  |  Height:  |  Size: 922 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 887 KiB

+42 -2
View File
@@ -142,6 +142,14 @@ These parameters are grouped directly under the `spec` key in the manifest.
database, like a flyway user running a migration on Pod start. See more
details in the [administrator docs](https://github.com/zalando/postgres-operator/blob/master/docs/administrator.md#password-replacement-without-extra-users).
* **usersIgnoringSecretRotation**
if you have secret rotation enabled globally you can define a list of
of users that should opt out from it, for example if you store credentials
outside of K8s, too, and corresponding deployments cannot dynamically
reference secrets. Note, you can also opt out from the rotation by removing
users from the manifest's `users` section. The operator will not drop them
from the database. Optional.
* **databases**
a map of database names to database owners for the databases that should be
created by the operator. The owner users should already exist on the cluster
@@ -359,6 +367,14 @@ CPU and memory requests for the Postgres container.
memory requests for the Postgres container. Optional, overrides the
`default_memory_request` operator configuration parameter.
* **hugepages-2Mi**
hugepages-2Mi requests for the sidecar container.
Optional, defaults to not set.
* **hugepages-1Gi**
1Gi hugepages requests for the sidecar container.
Optional, defaults to not set.
### Limits
CPU and memory limits for the Postgres container.
@@ -371,6 +387,14 @@ CPU and memory limits for the Postgres container.
memory limits for the Postgres container. Optional, overrides the
`default_memory_limits` operator configuration parameter.
* **hugepages-2Mi**
hugepages-2Mi requests for the sidecar container.
Optional, defaults to not set.
* **hugepages-1Gi**
1Gi hugepages requests for the sidecar container.
Optional, defaults to not set.
## Parameters defining how to clone the cluster from another one
Those parameters are applied when the cluster should be a clone of another one
@@ -500,6 +524,14 @@ CPU and memory requests for the sidecar container.
memory requests for the sidecar container. Optional, overrides the
`default_memory_request` operator configuration parameter. Optional.
* **hugepages-2Mi**
hugepages-2Mi requests for the sidecar container.
Optional, defaults to not set.
* **hugepages-1Gi**
1Gi hugepages requests for the sidecar container.
Optional, defaults to not set.
### Limits
CPU and memory limits for the sidecar container.
@@ -512,6 +544,14 @@ CPU and memory limits for the sidecar container.
memory limits for the sidecar container. Optional, overrides the
`default_memory_limits` operator configuration parameter. Optional.
* **hugepages-2Mi**
hugepages-2Mi requests for the sidecar container.
Optional, defaults to not set.
* **hugepages-1Gi**
1Gi hugepages requests for the sidecar container.
Optional, defaults to not set.
## Connection pooler
Parameters are grouped under the `connectionPooler` top-level key and specify
@@ -581,7 +621,7 @@ the global configuration before adding the `tls` section'.
## Change data capture streams
This sections enables change data capture (CDC) streams via Postgres'
[logical decoding](https://www.postgresql.org/docs/15/logicaldecoding.html)
[logical decoding](https://www.postgresql.org/docs/16/logicaldecoding.html)
feature and `pgoutput` plugin. While the Postgres operator takes responsibility
for providing the setup to publish change events, it relies on external tools
to consume them. At Zalando, we are using a workflow based on
@@ -613,7 +653,7 @@ can have the following properties:
and `payloadColumn`). The CDC operator is following the [outbox pattern](https://debezium.io/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/).
The application is responsible for putting events into a (JSON/B or VARCHAR)
payload column of the outbox table in the structure of the specified target
event type. The operator will create a [PUBLICATION](https://www.postgresql.org/docs/15/logical-replication-publication.html)
event type. The operator will create a [PUBLICATION](https://www.postgresql.org/docs/16/logical-replication-publication.html)
in Postgres for all tables specified for one `database` and `applicationId`.
The CDC operator will consume from it shortly after transactions are
committed to the outbox table. The `idColumn` will be used in telemetry for
+93 -48
View File
@@ -3,33 +3,46 @@
There are two mutually-exclusive methods to set the Postgres Operator
configuration.
* ConfigMaps-based, the legacy one. The configuration is supplied in a
key-value configmap, defined by the `CONFIG_MAP_NAME` environment variable.
Non-scalar values, i.e. lists or maps, are encoded in the value strings using
the comma-based syntax for lists and coma-separated `key:value` syntax for
maps. String values containing ':' should be enclosed in quotes. The
configuration is flat, parameter group names below are not reflected in the
configuration structure. There is an
[example](https://github.com/zalando/postgres-operator/blob/master/manifests/configmap.yaml)
* ConfigMaps-based, the legacy one
* CRD-based configuration
* CRD-based configuration. The configuration is stored in a custom YAML
manifest. The manifest is an instance of the custom resource definition (CRD)
called `OperatorConfiguration`. The operator registers this CRD during the
start and uses it for configuration if the [operator deployment manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L36)
sets the `POSTGRES_OPERATOR_CONFIGURATION_OBJECT` env variable to a non-empty
value. The variable should point to the `postgresql-operator-configuration`
object in the operator's namespace.
Variable names are underscore-separated words.
The CRD-based configuration is a regular YAML document; non-scalar keys are
simply represented in the usual YAML way. There are no default values built-in
in the operator, each parameter that is not supplied in the configuration
receives an empty value. In order to create your own configuration just copy
the [default one](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml)
and change it.
### ConfigMaps-based
The configuration is supplied in a
key-value configmap, defined by the `CONFIG_MAP_NAME` environment variable.
Non-scalar values, i.e. lists or maps, are encoded in the value strings using
the comma-based syntax for lists and coma-separated `key:value` syntax for
maps. String values containing ':' should be enclosed in quotes. The
configuration is flat, parameter group names below are not reflected in the
configuration structure. There is an
[example](https://github.com/zalando/postgres-operator/blob/master/manifests/configmap.yaml)
To test the CRD-based configuration locally, use the following
For the configmap configuration, the [default parameter values](https://github.com/zalando/postgres-operator/blob/master/pkg/util/config/config.go#L14)
mentioned here are likely to be overwritten in your local operator installation
via your local version of the operator configmap. In the case you use the
operator CRD, all the CRD defaults are provided in the
[operator's default configuration manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml)
```bash
### CRD-based configuration
The configuration is stored in a custom YAML
manifest. The manifest is an instance of the custom resource definition (CRD)
called `OperatorConfiguration`. The operator registers this CRD during the
start and uses it for configuration if the [operator deployment manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L36)
sets the `POSTGRES_OPERATOR_CONFIGURATION_OBJECT` env variable to a non-empty
value. The variable should point to the `postgresql-operator-configuration`
object in the operator's namespace.
The CRD-based configuration is a regular YAML document; non-scalar keys are
simply represented in the usual YAML way. There are no default values built-in
in the operator, each parameter that is not supplied in the configuration
receives an empty value. In order to create your own configuration just copy
the [default one](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml)
and change it.
To test the CRD-based configuration locally, use the following
```bash
kubectl create -f manifests/operatorconfiguration.crd.yaml # registers the CRD
kubectl create -f manifests/postgresql-operator-default-configuration.yaml
@@ -37,7 +50,7 @@ configuration.
kubectl create -f manifests/postgres-operator.yaml # set the env var as mentioned above
kubectl get operatorconfigurations postgresql-operator-default-configuration -o yaml
```
```
The CRD-based configuration is more powerful than the one based on ConfigMaps
and should be used unless there is a compatibility requirement to use an already
@@ -58,15 +71,6 @@ parameters, those parameters have no effect and are replaced by the
`CRD_READY_WAIT_INTERVAL` and `CRD_READY_WAIT_TIMEOUT` environment variables.
They will be deprecated and removed in the future.
For the configmap configuration, the [default parameter values](https://github.com/zalando/postgres-operator/blob/master/pkg/util/config/config.go#L14)
mentioned here are likely to be overwritten in your local operator installation
via your local version of the operator configmap. In the case you use the
operator CRD, all the CRD defaults are provided in the
[operator's default configuration manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml)
Variable names are underscore-separated words.
## General
Those are top-level keys, containing both leaf keys and groups.
@@ -246,12 +250,12 @@ CRD-configuration, they are grouped under the `major_version_upgrade` key.
* **minimal_major_version**
The minimal Postgres major version that will not automatically be upgraded
when `major_version_upgrade_mode` is set to `"full"`. The default is `"11"`.
when `major_version_upgrade_mode` is set to `"full"`. The default is `"12"`.
* **target_major_version**
The target Postgres major version when upgrading clusters automatically
which violate the configured allowed `minimal_major_version` when
`major_version_upgrade_mode` is set to `"full"`. The default is `"15"`.
`major_version_upgrade_mode` is set to `"full"`. The default is `"16"`.
## Kubernetes resources
@@ -323,6 +327,45 @@ configuration they are grouped under the `kubernetes` key.
replaced by the cluster name. Only the `{cluster}` placeholders is allowed in
the template.
* **pdb_master_label_selector**
By default the PDB will match the master role hence preventing nodes to be
drained if the node_readiness_label is not used. This option if set to `false`
will not add the `spilo-role=master` selector to the PDB.
* **enable_finalizers**
By default, a deletion of the Postgresql resource will trigger an event
that leads to a cleanup of all child resources. However, if the database
cluster is in a broken state (e.g. failed initialization) and the operator
cannot fully sync it, there can be leftovers. By enabling finalizers the
operator will ensure all managed resources are deleted prior to the
Postgresql resource. There is a trade-off though: The deletion is only
performed after the next two SYNC cycles with the first one updating the
internal spec and the latter reacting on the `deletionTimestamp` while
processing the SYNC event. The final removal of the custom resource will
add a DELETE event to the worker queue but the child resources are already
gone at this point.
The default is `false`.
* **persistent_volume_claim_retention_policy**
The operator tries to protect volumes as much as possible. If somebody
accidentally deletes the statefulset or scales in the `numberOfInstances` the
Persistent Volume Claims and thus Persistent Volumes will be retained.
However, this can have some consequences when you scale out again at a much
later point, for example after the cluster's Postgres major version has been
upgraded, because the old volume runs the old Postgres version with stale data.
Even if the version has not changed the replication lag could be massive. In
this case a reinitialization of the re-added member would make sense. You can
also modify the [retention policy of PVCs](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention) in the operator configuration.
The behavior can be changed for two scenarios: `when_deleted` - default is
`"retain"` - or `when_scaled` - default is also `"retain"`. The other possible
option is `delete`.
* **enable_persistent_volume_claim_deletion**
By default, the operator deletes PersistentVolumeClaims when removing the
Postgres cluster manifest, no matter if `persistent_volume_claim_retention_policy`
on the statefulset is set to `retain`. To keep PVCs set this option to `false`.
The default is `true`.
* **enable_pod_disruption_budget**
PDB is enabled by default to protect the cluster from voluntarily disruptions
and hence unwanted DB downtime. However, on some cloud providers it could be
@@ -431,7 +474,7 @@ configuration they are grouped under the `kubernetes` key.
environment if they not if conflict with the environment variables generated
by the operator. The WAL location (bucket path) can be overridden, though.
The default is empty.
* **pod_environment_secret**
similar to pod_environment_configmap but referencing a secret with custom
environment variables. Because the secret is not allowed to exist in a
@@ -527,19 +570,19 @@ CRD-based configuration.
* **default_cpu_request**
CPU request value for the Postgres containers, unless overridden by
cluster-specific settings. The default is `100m`.
cluster-specific settings. Empty string or `0` disables the default.
* **default_memory_request**
memory request value for the Postgres containers, unless overridden by
cluster-specific settings. The default is `100Mi`.
cluster-specific settings. Empty string or `0` disables the default.
* **default_cpu_limit**
CPU limits for the Postgres containers, unless overridden by cluster-specific
settings. The default is `1`.
settings. Empty string or `0` disables the default.
* **default_memory_limit**
memory limits for the Postgres containers, unless overridden by cluster-specific
settings. The default is `500Mi`.
settings. Empty string or `0` disables the default.
* **max_cpu_request**
optional upper boundary for CPU request
@@ -549,11 +592,11 @@ CRD-based configuration.
* **min_cpu_limit**
hard CPU minimum what we consider to be required to properly run Postgres
clusters with Patroni on Kubernetes. The default is `250m`.
clusters with Patroni on Kubernetes.
* **min_memory_limit**
hard memory minimum what we consider to be required to properly run Postgres
clusters with Patroni on Kubernetes. The default is `250Mi`.
clusters with Patroni on Kubernetes.
## Patroni options
@@ -577,7 +620,7 @@ effect, and the parameters are grouped under the `timeouts` key in the
CRD-based configuration.
* **PatroniAPICheckInterval**
the interval between consecutive attempts waiting for the return of
the interval between consecutive attempts waiting for the return of
Patroni Api. The default is `1s`.
* **PatroniAPICheckTimeout**
@@ -651,7 +694,7 @@ In the CRD-based configuration they are grouped under the `load_balancer` key.
balancers. Allowed values are `Cluster` (default) and `Local`.
* **master_dns_name_format**
defines the DNS name string template for the master load balancer cluster.
defines the DNS name string template for the master load balancer cluster.
The default is `{cluster}.{namespace}.{hostedzone}`, where `{cluster}` is
replaced by the cluster name, `{namespace}` is replaced with the namespace
and `{hostedzone}` is replaced with the hosted zone (the value of the
@@ -774,7 +817,7 @@ grouped under the `logical_backup` key.
runs `pg_dumpall` on a replica if possible and uploads compressed results to
an S3 bucket under the key `/spilo/pg_cluster_name/cluster_k8s_uuid/logical_backups`.
The default image is the same image built with the Zalando-internal CI
pipeline. Default: "registry.opensource.zalan.do/acid/logical-backup:v1.10.1"
pipeline. Default: "registry.opensource.zalan.do/acid/logical-backup:v1.11.0"
* **logical_backup_google_application_credentials**
Specifies the path of the google cloud service account json file. Default is empty.
@@ -816,7 +859,7 @@ grouped under the `logical_backup` key.
is specified, no argument will be passed to `aws s3` command. Default: "AES256".
* **logical_backup_s3_retention_time**
Specify a retention time for logical backups stored in S3. Backups older than the specified retention
Specify a retention time for logical backups stored in S3. Backups older than the specified retention
time will be deleted after a new backup was uploaded. If empty, all backups will be kept. Example values are
"3 days", "2 weeks", or "1 month". The default is empty.
@@ -825,6 +868,9 @@ grouped under the `logical_backup` key.
[reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule)
into account. Default: "30 00 \* \* \*"
* **logical_backup_cronjob_environment_secret**
Reference to a Kubernetes secret, which keys will be added as environment variables to the cronjob. Default: ""
## Debugging the operator
Options to aid debugging of the operator itself. Grouped under the `debug` key.
@@ -1001,5 +1047,4 @@ operator being able to provide some reasonable defaults.
**connection_pooler_default_memory_reques**
**connection_pooler_default_cpu_limit**
**connection_pooler_default_memory_limit**
Default resource configuration for connection pooler deployment. The internal
default for memory request and limit is `100Mi`, for CPU it is `500m` and `1`.
Default resource configuration for connection pooler deployment.
+48 -30
View File
@@ -30,7 +30,7 @@ spec:
databases:
foo: zalando
postgresql:
version: "15"
version: "16"
```
Once you cloned the Postgres Operator [repository](https://github.com/zalando/postgres-operator)
@@ -109,7 +109,7 @@ metadata:
spec:
[...]
postgresql:
version: "15"
version: "16"
parameters:
password_encryption: scram-sha-256
```
@@ -517,7 +517,7 @@ Postgres Operator will create the following NOLOGIN roles:
The `<dbname>_owner` role is the database owner and should be used when creating
new database objects. All members of the `admin` role, e.g. teams API roles, can
become the owner with the `SET ROLE` command. [Default privileges](https://www.postgresql.org/docs/15/sql-alterdefaultprivileges.html)
become the owner with the `SET ROLE` command. [Default privileges](https://www.postgresql.org/docs/16/sql-alterdefaultprivileges.html)
are configured for the owner role so that the `<dbname>_reader` role
automatically gets read-access (SELECT) to new tables and sequences and the
`<dbname>_writer` receives write-access (INSERT, UPDATE, DELETE on tables,
@@ -580,7 +580,9 @@ For all LOGIN roles the operator will create K8s secrets in the namespace
specified in `secretNamespace`, if `enable_cross_namespace_secret` is set to
`true` in the config. Otherwise, they are created in the same namespace like
the Postgres cluster. Unlike roles specified with `namespace.username` under
`users`, the namespace will not be part of the role name here.
`users`, the namespace will not be part of the role name here. Keep in mind
that the underscores in a role name are replaced with dashes in the K8s
secret name.
```yaml
spec:
@@ -592,7 +594,7 @@ spec:
### Schema `search_path` for default roles
The schema [`search_path`](https://www.postgresql.org/docs/15/ddl-schemas.html#DDL-SCHEMAS-PATH)
The schema [`search_path`](https://www.postgresql.org/docs/16/ddl-schemas.html#DDL-SCHEMAS-PATH)
for each role will include the role name and the schemas, this role should have
access to. So `foo_bar_writer` does not have to schema-qualify tables from
schemas `foo_bar_writer, bar`, while `foo_writer` can look up `foo_writer` and
@@ -687,6 +689,30 @@ The minimum limits to properly run the `postgresql` resource are configured to
manifest the operator will raise the limits to the configured minimum values.
If no resources are defined in the manifest they will be obtained from the
configured [default requests](reference/operator_parameters.md#kubernetes-resource-requests).
If neither defaults nor minimum limits are configured the operator will not
specify any resources and it's up to K8s (or your own) admission hooks to
handle it.
### HugePages support
The operator supports [HugePages](https://www.postgresql.org/docs/16/kernel-resources.html#LINUX-HUGEPAGES).
To enable HugePages, set the matching resource requests and/or limits in the manifest:
```yaml
spec:
resources:
requests:
hugepages-2Mi: 250Mi
hugepages-1Gi: 1Gi
limits:
hugepages-2Mi: 500Mi
hugepages-1Gi: 2Gi
```
There are no minimums or maximums and the default is 0 for both HugePage sizes,
but Kubernetes will not spin up the pod if the requested HugePages cannot be allocated.
For more information on HugePages in Kubernetes, see also
[https://kubernetes.io/docs/tasks/manage-hugepages/scheduling-hugepages/](https://kubernetes.io/docs/tasks/manage-hugepages/scheduling-hugepages/)
## Use taints, tolerations and node affinity for dedicated PostgreSQL nodes
@@ -732,7 +758,7 @@ If you need to define a `nodeAffinity` for all your Postgres clusters use the
## In-place major version upgrade
Starting with Spilo 13, operator supports in-place major version upgrade to a
higher major version (e.g. from PG 10 to PG 13). To trigger the upgrade,
higher major version (e.g. from PG 11 to PG 13). To trigger the upgrade,
simply increase the version in the manifest. It is your responsibility to test
your applications against the new version before the upgrade; downgrading is
not supported. The easiest way to do so is to try the upgrade on the cloned
@@ -812,7 +838,7 @@ spec:
### Clone directly
Another way to get a fresh copy of your source DB cluster is via
[pg_basebackup](https://www.postgresql.org/docs/15/app-pgbasebackup.html). To
[pg_basebackup](https://www.postgresql.org/docs/16/app-pgbasebackup.html). To
use this feature simply leave out the timestamp field from the clone section.
The operator will connect to the service of the source cluster by name. If the
cluster is called test, then the connection string will look like host=test
@@ -938,33 +964,25 @@ established between standby replica(s).
One big advantage of standby clusters is that they can be promoted to a proper
database cluster. This means it will stop replicating changes from the source,
and start accept writes itself. This mechanism makes it possible to move
databases from one place to another with minimal downtime. Currently, the
operator does not support promoting a standby cluster. It has to be done
manually using `patronictl edit-config` inside the postgres container of the
standby leader pod. Remove the following lines from the YAML structure and the
leader promotion happens immediately. Before doing so, make sure that the
standby is not behind the source database.
databases from one place to another with minimal downtime.
```yaml
standby_cluster:
create_replica_methods:
- bootstrap_standby_with_wale
- basebackup_fast_xlog
restore_command: envdir "/home/postgres/etc/wal-e.d/env-standby" /scripts/restore_command.sh
"%f" "%p"
```
Before promoting a standby cluster, make sure that the standby is not behind
the source database. You should ideally stop writes to your source cluster and
then create a dummy database object that you check for being replicated in the
target to verify all data has been copied.
Finally, remove the `standby` section from the postgres cluster manifest.
To promote, remove the `standby` section from the postgres cluster manifest.
A rolling update will be triggered removing the `STANDBY_*` environment
variables from the pods, followed by a Patroni config update that promotes the
cluster.
### Turn a normal cluster into a standby
### Adding standby section after promotion
There is no way to transform a non-standby cluster to a standby cluster through
the operator. Adding the `standby` section to the manifest of a running
Postgres cluster will have no effect. But, as explained in the previous
paragraph it can be done manually through `patronictl edit-config`. This time,
by adding the `standby_cluster` section to the Patroni configuration. However,
the transformed standby cluster will not be doing any streaming. It will be in
standby mode and allow read-only transactions only.
Turning a running cluster into a standby is not easily possible and should be
avoided. The best way is to remove the cluster and resubmit the manifest
after a short wait of a few minutes. Adding the `standby` section would turn
the database cluster in read-only mode on next operator SYNC cycle but it
does not sync automatically with the source cluster again.
## Sidecar Support