public_git/postgres-operator
1
0
Fork 0
mirror of https://github.com/zalando/postgres-operator.git synced 2026-10-04 10:11:59 +02:00
Code Issues Projects Releases Wiki Activity

merge with master

Browse Source
This commit is contained in:
Felix Kunde
2021-01-26 17:11:58 +01:00
parent c3c413f0d1 43168ca622
commit 6deac03f32
190 changed files with 21094 additions and 3588 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/administrator.md
+297 -22
View File
@@ -11,17 +11,29 @@ switchover (planned failover) of the master to the Pod with new minor version.
The switch should usually take less than 5 seconds, still clients have to
reconnect.
Major version upgrades are supported via [cloning](user.md#how-to-clone-an-existing-postgresql-cluster).
The new cluster manifest must have a higher `version` string than the source
cluster and will be created from a basebackup. Depending of the cluster size,
downtime in this case can be significant as writes to the database should be
stopped and all WAL files should be archived first before cloning is started.
Major version upgrades are supported either via [cloning](user.md#how-to-clone-an-existing-postgresql-cluster)
or in-place.
Note, that simply changing the version string in the `postgresql` manifest does
not work at present and leads to errors. Neither Patroni nor Postgres Operator
can do in place `pg_upgrade`. Still, it can be executed manually in the Postgres
container, which is tricky (i.e. systems need to be stopped, replicas have to be
synced) but of course faster than cloning.
With cloning, the new cluster manifest must have a higher `version` string than
the source cluster and will be created from a basebackup. Depending of the
cluster size, downtime in this case can be significant as writes to the database
should be stopped and all WAL files should be archived first before cloning is
started.
Starting with Spilo 13, Postgres Operator can do in-place major version upgrade,
which should be faster than cloning. However, it is not fully automatic yet.
First, you need to make sure, that setting the `PGVERSION` environment variable
is enabled in the configuration. Since `v1.6.0`, `enable_pgversion_env_var` is
enabled by default.
To trigger the upgrade, increase the version in the cluster manifest. After
Pods are rotated `configure_spilo` will notice the version mismatch and start
the old version again. You can then exec into the Postgres container of the
master instance and call `python3 /scripts/inplace_upgrade.py N` where `N`
is the number of members of your cluster (see [`numberOfInstances`](https://github.com/zalando/postgres-operator/blob/50cb5898ea715a1db7e634de928b2d16dc8cd969/manifests/minimal-postgres-manifest.yaml#L10)).
The upgrade is usually fast, well under one minute for most DBs. Note, that
changes become irrevertible once `pg_upgrade` is called. To understand the
upgrade procedure, refer to the [corresponding PR in Spilo](https://github.com/zalando/spilo/pull/488).
## CRD Validation
@@ -44,7 +56,7 @@ Once the validation is enabled it can only be disabled manually by editing or
patching the CRD manifest:
```bash
zk8 patch crd postgresqls.acid.zalan.do -p '{"spec":{"validation": null}}'
kubectl patch crd postgresqls.acid.zalan.do -p '{"spec":{"validation": null}}'
```
## Non-default cluster domain
@@ -95,6 +107,96 @@ lacks access rights to any of them (except K8s system namespaces like
'list pods' execute at the cluster scope and fail at the first violation of
access rights.
## Operators with defined ownership of certain Postgres clusters
By default, multiple operators can only run together in one K8s cluster when
isolated into their [own namespaces](administrator.md#specify-the-namespace-to-watch).
But, it is also possible to define ownership between operator instances and
Postgres clusters running all in the same namespace or K8s cluster without
interfering.
First, define the [`CONTROLLER_ID`](../../manifests/postgres-operator.yaml#L38)
environment variable in the operator deployment manifest. Then specify the ID
in every Postgres cluster manifest you want this operator to watch using the
`"acid.zalan.do/controller"` annotation:
```yaml
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
name: demo-cluster
annotations:
"acid.zalan.do/controller": "second-operator"
spec:
...
```
Every other Postgres cluster which lacks the annotation will be ignored by this
operator. Conversely, operators without a defined `CONTROLLER_ID` will ignore
clusters with defined ownership of another operator.
## Delete protection via annotations
To avoid accidental deletes of Postgres clusters the operator can check the
manifest for two existing annotations containing the cluster name and/or the
current date (in YYYY-MM-DD format). The name of the annotation keys can be
defined in the configuration. By default, they are not set which disables the
delete protection. Thus, one could choose to only go with one annotation.
**postgres-operator ConfigMap**
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-operator
data:
delete_annotation_date_key: "delete-date"
delete_annotation_name_key: "delete-clustername"
```
**OperatorConfiguration**
```yaml
apiVersion: "acid.zalan.do/v1"
kind: OperatorConfiguration
metadata:
name: postgresql-operator-configuration
configuration:
kubernetes:
delete_annotation_date_key: "delete-date"
delete_annotation_name_key: "delete-clustername"
```
Now, every cluster manifest must contain the configured annotation keys to
trigger the delete process when running `kubectl delete pg`. Note, that the
`Postgresql` resource would still get deleted as K8s' API server does not
block it. Only the operator logs will tell, that the delete criteria wasn't
met.
**cluster manifest**
```yaml
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
name: demo-cluster
annotations:
delete-date: "2020-08-31"
delete-clustername: "demo-cluster"
spec:
...
```
In case, the resource has been deleted accidentally or the annotations were
simply forgotten, it's safe to recreate the cluster with `kubectl create`.
Existing Postgres cluster are not replaced by the operator. But, as the
original cluster still exists the status will show `CreateFailed` at first.
On the next sync event it should change to `Running`. However, as it is in
fact a new resource for K8s, the UID will differ which can trigger a rolling
update of the pods because the UID is used as part of backup path to S3.
## Role-based access control for the operator
The manifest [`operator-service-account-rbac.yaml`](../manifests/operator-service-account-rbac.yaml)
@@ -292,13 +394,21 @@ spec:
## Custom Pod Environment Variables
It is possible to configure a ConfigMap as well as a Secret which are used by the Postgres pods as
an additional provider for environment variables. One use case is to customize
the Spilo image and configure it with environment variables. Another case could be to provide custom
cloud provider or backup settings.
It is possible to configure a ConfigMap which is used by the Postgres pods as
an additional provider for environment variables.
In general the Operator will give preference to the globally configured variables, to not have the custom
ones interfere with core functionality. Variables with the 'WAL_' and 'LOG_' prefix can be overwritten though, to allow
backup and logshipping to be specified differently.
One use case is to customize the Spilo image and configure it with environment
variables. The ConfigMap with the additional settings is configured in the
operator's main ConfigMap:
### Via ConfigMap
The ConfigMap with the additional settings is referenced in the operator's main configuration.
A namespace can be specified along with the name. If left out, the configured
default namespace of your K8s client will be used and if the ConfigMap is not
found there, the Postgres cluster's namespace is taken when different:
**postgres-operator ConfigMap**
@@ -309,7 +419,7 @@ metadata:
name: postgres-operator
data:
# referencing config map with custom settings
pod_environment_configmap: postgres-pod-config
pod_environment_configmap: default/postgres-pod-config
```
**OperatorConfiguration**
@@ -322,7 +432,7 @@ metadata:
configuration:
kubernetes:
# referencing config map with custom settings
pod_environment_configmap: postgres-pod-config
pod_environment_configmap: default/postgres-pod-config
```
**referenced ConfigMap `postgres-pod-config`**
@@ -337,7 +447,54 @@ data:
MY_CUSTOM_VAR: value
```
This ConfigMap is then added as a source of environment variables to the
The key-value pairs of the ConfigMap are then added as environment variables to the
Postgres StatefulSet/pods.
### Via Secret
The Secret with the additional variables is referenced in the operator's main configuration.
To protect the values of the secret from being exposed in the pod spec they are each referenced
as SecretKeyRef.
This does not allow for the secret to be in a different namespace as the pods though
**postgres-operator ConfigMap**
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-operator
data:
# referencing secret with custom environment variables
pod_environment_secret: postgres-pod-secrets
```
**OperatorConfiguration**
```yaml
apiVersion: "acid.zalan.do/v1"
kind: OperatorConfiguration
metadata:
name: postgresql-operator-configuration
configuration:
kubernetes:
# referencing secret with custom environment variables
pod_environment_secret: postgres-pod-secrets
```
**referenced Secret `postgres-pod-secrets`**
```yaml
apiVersion: v1
kind: Secret
metadata:
name: postgres-pod-secrets
namespace: default
data:
MY_CUSTOM_VAR: dmFsdWU=
```
The key-value pairs of the Secret are all accessible as environment variables to the
Postgres StatefulSet/pods.
## Limiting the number of min and max instances in clusters
@@ -417,9 +574,12 @@ database.
* **Human users** originate from the [Teams API](user.md#teams-api-roles) that
returns a list of the team members given a team id. The operator differentiates
between (a) product teams that own a particular Postgres cluster and are granted
admin rights to maintain it, and (b) Postgres superuser teams that get the
superuser access to all Postgres databases running in a K8s cluster for the
purposes of maintaining and troubleshooting.
admin rights to maintain it, (b) Postgres superuser teams that get superuser
access to all Postgres databases running in a K8s cluster for the purposes of
maintaining and troubleshooting, and (c) additional teams, superuser teams or
members associated with the owning team. The latter is managed via the
[PostgresTeam CRD](user.md#additional-teams-and-members-per-cluster).
## Understanding rolling update of Spilo pods
@@ -430,6 +590,17 @@ from numerous escape characters in the latter log entry, view it in CLI with
`PodTemplate` used by the operator is yet to be updated with the default values
used internally in K8s.
The operator also support lazy updates of the Spilo image. That means the pod
template of a PG cluster's stateful set is updated immediately with the new
image, but no rolling update follows. This feature saves you a switchover - and
hence downtime - when you know pods are re-started later anyway, for instance
due to the node rotation. To force a rolling update, disable this mode by
setting the `enable_lazy_spilo_upgrade` to `false` in the operator configuration
and restart the operator pod. With the standard eager rolling updates the
operator checks during Sync all pods run images specified in their respective
statefulsets. The operator triggers a rolling upgrade for PG clusters that
violate this condition.
## Logical backups
The operator can manage K8s cron jobs to run logical backups of Postgres
@@ -479,6 +650,110 @@ A secret can be pre-provisioned in different ways:
* Automatically provisioned via a custom K8s controller like
[kube-aws-iam-controller](https://github.com/mikkeloscar/kube-aws-iam-controller)
## Google Cloud Platform setup
To configure the operator on GCP there are some prerequisites that are needed:
* A service account with the proper IAM setup to access the GCS bucket for the WAL-E logs
* The credentials file for the service account.
The configuration paramaters that we will be using are:
* `additional_secret_mount`
* `additional_secret_mount_path`
* `gcp_credentials`
* `wal_gs_bucket`
### Generate a K8s secret resource
Generate the K8s secret resource that will contain your service account's
credentials. It's highly recommended to use a service account and limit its
scope to just the WAL-E bucket.
```yaml
apiVersion: v1
kind: Secret
metadata:
name: psql-wale-creds
namespace: default
type: Opaque
stringData:
key.json: |-
<GCP .json credentials>
```
### Setup your operator configuration values
With the `psql-wale-creds` resource applied to your cluster, ensure that
the operator's configuration is set up like the following:
```yml
...
aws_or_gcp:
additional_secret_mount: "pgsql-wale-creds"
additional_secret_mount_path: "/var/secrets/google" # or where ever you want to mount the file
# aws_region: eu-central-1
# kube_iam_role: ""
# log_s3_bucket: ""
# wal_s3_bucket: ""
wal_gs_bucket: "postgres-backups-bucket-28302F2" # name of bucket on where to save the WAL-E logs
gcp_credentials: "/var/secrets/google/key.json" # combination of the mount path & key in the K8s resource. (i.e. key.json)
...
```
### Setup pod environment configmap
To make postgres-operator work with GCS, use following configmap:
```yml
apiVersion: v1
kind: ConfigMap
metadata:
name: pod-env-overrides
namespace: postgres-operator-system
data:
# Any env variable used by spilo can be added
USE_WALG_BACKUP: "true"
USE_WALG_RESTORE: "true"
CLONE_USE_WALG_RESTORE: "true"
```
This configmap will instruct operator to use WAL-G, instead of WAL-E, for backup and restore.
Then provide this configmap in postgres-operator settings:
```yml
...
# namespaced name of the ConfigMap with environment variables to populate on every pod
pod_environment_configmap: "postgres-operator-system/pod-env-overrides"
...
```
## Sidecars for Postgres clusters
A list of sidecars is added to each cluster created by the operator. The default
is empty.
```yaml
kind: OperatorConfiguration
configuration:
sidecars:
- image: image:123
name: global-sidecar
ports:
- containerPort: 80
volumeMounts:
- mountPath: /custom-pgdata-mountpoint
name: pgdata
- ...
```
In addition to any environment variables you specify, the following environment
variables are always passed to sidecars:
- `POD_NAME` - field reference to `metadata.name`
- `POD_NAMESPACE` - field reference to `metadata.namespace`
- `POSTGRES_USER` - the superuser that can be used to connect to the database
- `POSTGRES_PASSWORD` - the password for the superuser
## Setting up the Postgres Operator UI
Since the v1.2 release the Postgres Operator is shipped with a browser-based
docs/developer.md
+24 -4
View File
@@ -235,11 +235,31 @@ Then you can for example check the Patroni logs:
kubectl logs acid-minimal-cluster-0
```
## Unit tests with Mocks and K8s Fake API
Whenever possible you should rely on leveraging proper mocks and K8s fake client that allows full fledged testing of K8s objects in your unit tests.
To enable mocks, a code annotation is needed:
[Mock code gen annotation](https://github.com/zalando/postgres-operator/blob/master/pkg/util/volumes/volumes.go#L3)
To generate mocks run:
```bash
make mocks
```
Examples for mocks can be found in:
[Example mock usage](https://github.com/zalando/postgres-operator/blob/master/pkg/cluster/volumes_test.go#L248)
Examples for fake K8s objects can be found in:
[Example fake K8s client usage](https://github.com/zalando/postgres-operator/blob/master/pkg/cluster/volumes_test.go#L166)
## End-to-end tests
The operator provides reference end-to-end tests (e2e) (as Docker image) to
ensure various infrastructure parts work smoothly together. Each e2e execution
tests a Postgres Operator image built from the current git branch. The test
The operator provides reference end-to-end (e2e) tests to
ensure various infrastructure parts work smoothly together. The test code is available at `e2e/tests`.
The special `registry.opensource.zalan.do/acid/postgres-operator-e2e-tests-runner` image is used to run the tests. The container mounts the local `e2e/tests` directory at runtime, so whatever you modify in your local copy of the tests will be executed by a test runner. By maintaining a separate test runner image we avoid the need to re-build the e2e test image on every build.
Each e2e execution tests a Postgres Operator image built from the current git branch. The test
runner creates a new local K8s cluster using [kind](https://kind.sigs.k8s.io/),
utilizes provided manifest examples, and runs e2e tests contained in the `tests`
folder. The K8s API client in the container connects to the `kind` cluster via
@@ -284,7 +304,7 @@ manifest files:
Postgres manifest parameters are defined in the [api package](../pkg/apis/acid.zalan.do/v1/postgresql_type.go).
The operator behavior has to be implemented at least in [k8sres.go](../pkg/cluster/k8sres.go).
Validation of CRD parameters is controlled in [crd.go](../pkg/apis/acid.zalan.do/v1/crds.go).
Validation of CRD parameters is controlled in [crds.go](../pkg/apis/acid.zalan.do/v1/crds.go).
Please, reflect your changes in tests, for example in:
* [config_test.go](../pkg/util/config/config_test.go)
* [k8sres_test.go](../pkg/cluster/k8sres_test.go)
docs/diagrams/neutral_operator.excalidraw
+3499
View File
File diff suppressed because it is too large Load Diff
docs/diagrams/neutral_operator.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 849 KiB

docs/gsoc-2019/ideas.md
-63
View File
@@ -1,63 +0,0 @@
<h1>Google Summer of Code 2019</h1>
## Applications steps
1. Please carefully read the official [Google Summer of Code Student Guide](https://google.github.io/gsocguides/student/)
2. Join the #postgres-operator slack channel under [Postgres Slack](https://postgres-slack.herokuapp.com) to introduce yourself to the community and get quick feedback on your application.
3. Select a project from the list of ideas below or propose your own.
4. Write a proposal draft. Please open an issue with the label `gsoc2019_application` in the [operator repository](https://github.com/zalando/postgres-operator/issues) so that the community members can publicly review it. See proposal instructions below for details.
5. Submit proposal and the proof of enrollment before April 9 2019 18:00 UTC through the web site of the Program.
## Project ideas
### Place database pods into the "Guaranteed" Quality-of-Service class
* **Description**: Kubernetes runtime does not kill pods in this class on condition they stay within their resource limits, which is desirable for the DB pods serving production workloads. To be assigned to that class, pod's resources must equal its limits. The task is to add the `enableGuaranteedQoSClass` or the like option to the Postgres manifest and the operator configmap that forcibly re-write pod resources to match the limits.
* **Recommended skills**: golang, basic Kubernetes abstractions
* **Difficulty**: moderate
* **Mentor(s)**: Felix Kunde [@FxKu](https://github.com/fxku), Sergey Dudoladov [@sdudoladov](https://github.com/sdudoladov)
### Implement the kubectl plugin for the Postgres CustomResourceDefinition
* **Description**: [kubectl plugins](https://kubernetes.io/docs/tasks/extend-kubectl/kubectl-plugins/) enable extending the Kubernetes command-line client `kubectl` with commands to manage custom resources. The task is to design and implement a plugin for the `kubectl postgres` command,
that can enable, for example, correct deletion or major version upgrade of Postgres clusters.
* **Recommended skills**: golang, shell scripting, operational experience with Kubernetes
* **Difficulty**: moderate to medium, depending on the plugin design
* **Mentor(s)**: Felix Kunde [@FxKu](https://github.com/fxku), Sergey Dudoladov [@sdudoladov](https://github.com/sdudoladov)
### Implement the openAPIV3Schema for the Postgres CRD
* **Description**: at present the operator validates a database manifest on its own.
It will be helpful to reject erroneous manifests before they reach the operator using the [native Kubernetes CRD validation](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#validation). It is up to the student to decide whether to write the schema manually or to adopt existing [schema generator developed for the Prometheus project](https://github.com/ant31/crd-validation).
* **Recommended skills**: golang, JSON schema
* **Difficulty**: medium
* **Mentor(s)**: Sergey Dudoladov [@sdudoladov](https://github.com/sdudoladov)
* **Issue**: [#388](https://github.com/zalando/postgres-operator/issues/388)
### Design a solution for the local testing of the operator
* **Description**: The current way of testing is to run minikube, either manually or with some tooling around it like `/run-operator_locally.sh` or Vagrant. This has at least three problems:
First, minikube is a single node cluster, so it is unsuitable for testing vital functions such as pod migration between nodes. Second, minikube starts slowly; that prolongs local testing.
Third, every contributor needs to come up with their own solution for local testing. The task is to come up with a better option which will enable us to conveniently and uniformly run e2e tests locally / potentially in Travis CI.
A promising option is the Kubernetes own [kind](https://github.com/kubernetes-sigs/kind)
* **Recommended skills**: Docker, shell scripting, basic Kubernetes abstractions
* **Difficulty**: medium to hard depending on the selected desing
* **Mentor(s)**: Dmitry Dolgov [@erthalion](https://github.com/erthalion), Sergey Dudoladov [@sdudoladov](https://github.com/sdudoladov)
* **Issue**: [#475](https://github.com/zalando/postgres-operator/issues/475)
### Detach a Postgres cluster from the operator for maintenance
* **Description**: sometimes a Postgres cluster requires manual maintenance. During such maintenance the operator should ignore all the changes manually applied to the cluster.
Currently the only way to achieve this behavior is to shutdown the operator altogether, for instance by scaling down the operator's own deployment to zero pods. That approach evidently affects all Postgres databases under the operator control and thus is highly undesirable in production Kubernetes clusters. It would be much better to be able to detach only the desired Postgres cluster from the operator for the time being and re-attach it again after maintenance.
* **Recommended skills**: golang, architecture of a Kubernetes operator
* **Difficulty**: hard - requires significant modification of the operator's internals and careful consideration of the corner cases.
* **Mentor(s)**: Dmitry Dolgov [@erthalion](https://github.com/erthalion), Sergey Dudoladov [@sdudoladov](https://github.com/sdudoladov)
* **Issue**: [#421](https://github.com/zalando/postgres-operator/issues/421)
### Propose your own idea
Feel free to come up with your own ideas. For inspiration,
see [our bug tracker](https://github.com/zalando/postgres-operator/issues),
the [official `CustomResouceDefinition` docs](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/)
and [other operators](https://github.com/operator-framework/awesome-operators).
docs/index.md
+23 -8
View File
@@ -37,9 +37,10 @@ in some overarching orchestration, like rolling updates to improve the user
experience.
Monitoring or tuning Postgres is not in scope of the operator in the current
state. Other tools like [ZMON](https://opensource.zalando.com/zmon/),
[Prometheus](https://prometheus.io/) or more Postgres specific options can be
used to complement it.
state. However, with globally configurable sidecars we provide enough
flexibility to complement it with other tools like [ZMON](https://opensource.zalando.com/zmon/),
[Prometheus](https://prometheus.io/) or more Postgres specific options.
## Overview of involved entities
@@ -70,12 +71,26 @@ Please, report any issues discovered to https://github.com/zalando/postgres-oper
## Talks
1. "Building your own PostgreSQL-as-a-Service on Kubernetes" talk by Alexander Kukushkin, KubeCon NA 2018: [video](https://www.youtube.com/watch?v=G8MnpkbhClc) | [slides](https://static.sched.com/hosted_files/kccna18/1d/Building%20your%20own%20PostgreSQL-as-a-Service%20on%20Kubernetes.pdf)
- "PostgreSQL on K8S at Zalando: Two years in production" talk by Alexander Kukushkin, FOSSDEM 2020: [video](https://fosdem.org/2020/schedule/event/postgresql_postgresql_on_k8s_at_zalando_two_years_in_production/) | [slides](https://fosdem.org/2020/schedule/event/postgresql_postgresql_on_k8s_at_zalando_two_years_in_production/attachments/slides/3883/export/events/attachments/postgresql_postgresql_on_k8s_at_zalando_two_years_in_production/slides/3883/PostgreSQL_on_K8s_at_Zalando_Two_years_in_production.pdf)
2. "PostgreSQL and Kubernetes: DBaaS without a vendor-lock" talk by Oleksii Kliukin, PostgreSQL Sessions 2018: [video](https://www.youtube.com/watch?v=q26U2rQcqMw) | [slides](https://speakerdeck.com/alexeyklyukin/postgresql-and-kubernetes-dbaas-without-a-vendor-lock)
- "Postgres as a Service at Zalando" talk by Jan Mußler, DevOpsDays Poznań 2019: [video](https://www.youtube.com/watch?v=FiWS5m72XI8)
3. "PostgreSQL High Availability on Kubernetes with Patroni" talk by Oleksii Kliukin, Atmosphere 2018: [video](https://www.youtube.com/watch?v=cFlwQOPPkeg) | [slides](https://speakerdeck.com/alexeyklyukin/postgresql-high-availability-on-kubernetes-with-patroni)
- "Building your own PostgreSQL-as-a-Service on Kubernetes" talk by Alexander Kukushkin, KubeCon NA 2018: [video](https://www.youtube.com/watch?v=G8MnpkbhClc) | [slides](https://static.sched.com/hosted_files/kccna18/1d/Building%20your%20own%20PostgreSQL-as-a-Service%20on%20Kubernetes.pdf)
4. "Blue elephant on-demand: Postgres + Kubernetes" talk by Oleksii Kliukin and Jan Mussler, FOSDEM 2018: [video](https://fosdem.org/2018/schedule/event/blue_elephant_on_demand_postgres_kubernetes/) | [slides (pdf)](https://www.postgresql.eu/events/fosdem2018/sessions/session/1735/slides/59/FOSDEM%202018_%20Blue_Elephant_On_Demand.pdf)
- "PostgreSQL and Kubernetes: DBaaS without a vendor-lock" talk by Oleksii Kliukin, PostgreSQL Sessions 2018: [video](https://www.youtube.com/watch?v=q26U2rQcqMw) | [slides](https://speakerdeck.com/alexeyklyukin/postgresql-and-kubernetes-dbaas-without-a-vendor-lock)
5. "Kube-Native Postgres" talk by Josh Berkus, KubeCon 2017: [video](https://www.youtube.com/watch?v=Zn1vd7sQ_bc)
- "PostgreSQL High Availability on Kubernetes with Patroni" talk by Oleksii Kliukin, Atmosphere 2018: [video](https://www.youtube.com/watch?v=cFlwQOPPkeg) | [slides](https://speakerdeck.com/alexeyklyukin/postgresql-high-availability-on-kubernetes-with-patroni)
- "Blue elephant on-demand: Postgres + Kubernetes" talk by Oleksii Kliukin and Jan Mussler, FOSDEM 2018: [video](https://fosdem.org/2018/schedule/event/blue_elephant_on_demand_postgres_kubernetes/) | [slides (pdf)](https://www.postgresql.eu/events/fosdem2018/sessions/session/1735/slides/59/FOSDEM%202018_%20Blue_Elephant_On_Demand.pdf)
- "Kube-Native Postgres" talk by Josh Berkus, KubeCon 2017: [video](https://www.youtube.com/watch?v=Zn1vd7sQ_bc)
## Posts
- "How to set up continuous backups and monitoring" by Pål Kristensen on [GitHub](https://github.com/zalando/postgres-operator/issues/858#issuecomment-608136253), Mar. 2020.
- "Postgres on Kubernetes with the Zalando operator" by Vito Botta on [has_many :code](https://vitobotta.com/2020/02/05/postgres-kubernetes-zalando-operator/), Feb. 2020.
- "Running PostgreSQL in Google Kubernetes Engine" by Kenneth Rørvik on [Repill Linpro](https://www.redpill-linpro.com/techblog/2019/09/28/postgres-in-kubernetes.html), Sep. 2019.
- "Zalando Postgres Operator: One Year Later" by Sergey Dudoladov on [Open Source Zalando](https://opensource.zalando.com/blog/2018/11/postgres-operator/), Nov. 2018
docs/quickstart.md
+5 -16
View File
@@ -34,8 +34,8 @@ Postgres cluster. This can work in two ways: via a ConfigMap or a custom
The Postgres Operator can be deployed in the following ways:
* Manual deployment
* Kustomization
* Helm chart
* Operator Lifecycle Manager (OLM)
### Manual deployment setup
@@ -91,20 +91,6 @@ The chart works with both Helm 2 and Helm 3. The `crd-install` hook from v2 will
be skipped with warning when using v3. Documentation for installing applications
with Helm 2 can be found in the [v2 docs](https://v2.helm.sh/docs/).
### Operator Lifecycle Manager (OLM)
The [Operator Lifecycle Manager (OLM)](https://github.com/operator-framework/operator-lifecycle-manager)
has been designed to facilitate management of K8s operators. It has to be
installed in your K8s environment. When OLM is set up simply download and deploy
the Postgres Operator with the following command:
```bash
kubectl create -f https://operatorhub.io/install/postgres-operator.yaml
```
This installs the operator in the `operators` namespace. More information can be
found on [operatorhub.io](https://operatorhub.io/operator/postgres-operator).
## Check if Postgres Operator is running
Starting the operator may take a few seconds. Check if the operator pod is
@@ -142,6 +128,9 @@ To deploy the UI simply apply all its manifests files or use the UI helm chart:
# manual deployment
kubectl apply -f ui/manifests/
# or kustomization
kubectl apply -k github.com/zalando/postgres-operator/ui/manifests
# or helm chart
helm install postgres-operator-ui ./charts/postgres-operator-ui
```
@@ -160,7 +149,7 @@ You can now access the web interface by port forwarding the UI pod (mind the
label selector) and enter `localhost:8081` in your browser:
```bash
kubectl port-forward "$(kubectl get pod -l name=postgres-operator-ui --output='name')" 8081
kubectl port-forward svc/postgres-operator-ui 8081:80
```
Available option are explained in detail in the [UI docs](operator-ui.md).
docs/reference/cluster_manifest.md
+125 -11
View File
@@ -65,6 +65,20 @@ These parameters are grouped directly under the `spec` key in the manifest.
custom Docker image that overrides the **docker_image** operator parameter.
It should be a [Spilo](https://github.com/zalando/spilo) image. Optional.
* **schedulerName**
specifies the scheduling profile for database pods. If no value is provided
K8s' `default-scheduler` will be used. Optional.
* **spiloRunAsUser**
sets the user ID which should be used in the container to run the process.
This must be set to run the container without root. By default the container
runs with root. This option only works for Spilo versions >= 1.6-p3.
* **spiloRunAsGroup**
sets the group ID which should be used in the container to run the process.
This must be set to run the container without root. By default the container
runs with root. This option only works for Spilo versions >= 1.6-p3.
* **spiloFSGroup**
the Persistent Volumes for the Spilo pods in the StatefulSet will be owned and
writable by the group ID specified. This will override the **spilo_fsgroup**
@@ -111,12 +125,12 @@ These parameters are grouped directly under the `spec` key in the manifest.
value overrides the `pod_toleration` setting from the operator. Optional.
* **podPriorityClassName**
a name of the [priority
class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
that should be assigned to the cluster pods. When not specified, the value
is taken from the `pod_priority_class_name` operator parameter, if not set
then the default priority class is taken. The priority class itself must be
defined in advance. Optional.
a name of the [priority
class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
that should be assigned to the cluster pods. When not specified, the value
is taken from the `pod_priority_class_name` operator parameter, if not set
then the default priority class is taken. The priority class itself must be
defined in advance. Optional.
* **podAnnotations**
A map of key value pairs that gets attached as [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/)
@@ -140,6 +154,16 @@ These parameters are grouped directly under the `spec` key in the manifest.
is `false`, then no volume will be mounted no matter how operator was
configured (so you can override the operator configuration). Optional.
* **enableConnectionPooler**
Tells the operator to create a connection pooler with a database for the master
service. If this field is true, a connection pooler deployment will be created even if
`connectionPooler` section is empty. Optional, not set by default.
* **enableReplicaConnectionPooler**
Tells the operator to create a connection pooler with a database for the replica
service. If this field is true, a connection pooler deployment for replica
will be created even if `connectionPooler` section is empty. Optional, not set by default.
* **enableLogicalBackup**
Determines if the logical backup of this cluster should be taken and uploaded
to S3. Default: false. Optional.
@@ -149,6 +173,18 @@ These parameters are grouped directly under the `spec` key in the manifest.
[the reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule)
into account. Optional. Default is: "30 00 \* \* \*"
* **additionalVolumes**
List of additional volumes to mount in each container of the statefulset pod.
Each item must contain a `name`, `mountPath`, and `volumeSource` which is a
[kubernetes volumeSource](https://godoc.org/k8s.io/api/core/v1#VolumeSource).
It allows you to mount existing PersistentVolumeClaims, ConfigMaps and Secrets inside the StatefulSet.
Also an `emptyDir` volume can be shared between initContainer and statefulSet.
Additionaly, you can provide a `SubPath` for volume mount (a file in a configMap source volume, for example).
You can also specify in which container the additional Volumes will be mounted with the `targetContainers` array option.
If `targetContainers` is empty, additional volumes will be mounted only in the `postgres` container.
If you set the `all` special item, it will be mounted in all containers (postgres + sidecars).
Else you can set the list of target containers in which the additional volumes will be mounted (eg : postgres, telegraf)
## Postgres parameters
Those parameters are grouped under the `postgresql` top-level key, which is
@@ -184,9 +220,9 @@ explanation of `ttl` and `loop_wait` parameters.
```
hostssl all +pamrole all pam
```
, where pamrole is the name of the role for the pam authentication; any
custom `pg_hba` should include the pam line to avoid breaking pam
authentication. Optional.
where pamrole is the name of the role for the pam authentication; any
custom `pg_hba` should include the pam line to avoid breaking pam
authentication. Optional.
* **ttl**
Patroni `ttl` parameter value, optional. The default is set by the Spilo
@@ -212,6 +248,12 @@ explanation of `ttl` and `loop_wait` parameters.
automatically created by Patroni for cluster members and permanent replication
slots. Optional.
* **synchronous_mode**
Patroni `synchronous_mode` parameter value. The default is set to `false`. Optional.
* **synchronous_mode_strict**
Patroni `synchronous_mode_strict` parameter value. Can be used in addition to `synchronous_mode`. The default is set to `false`. Optional.
## Postgres container resources
Those parameters define [CPU and memory requests and limits](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
@@ -296,13 +338,13 @@ archive is supported.
the url to S3 bucket containing the WAL archive of the remote primary.
Required when the `standby` section is present.
## EBS volume resizing
## Volume properties
Those parameters are grouped under the `volume` top-level key and define the
properties of the persistent storage that stores Postgres data.
* **size**
the size of the target EBS volume. Usual Kubernetes size modifiers, i.e. `Gi`
the size of the target volume. Usual Kubernetes size modifiers, i.e. `Gi`
or `Mi`, apply. Required.
* **storageClass**
@@ -314,6 +356,14 @@ properties of the persistent storage that stores Postgres data.
* **subPath**
Subpath to use when mounting volume into Spilo container. Optional.
* **iops**
When running the operator on AWS the latest generation of EBS volumes (`gp3`)
allows for configuring the number of IOPS. Maximum is 16000. Optional.
* **throughput**
When running the operator on AWS the latest generation of EBS volumes (`gp3`)
allows for configuring the throughput in MB/s. Maximum is 1000. Optional.
## Sidecar definitions
Those parameters are defined under the `sidecars` key. They consist of a list
@@ -359,3 +409,67 @@ CPU and memory limits for the sidecar container.
* **memory**
memory limits for the sidecar container. Optional, overrides the
`default_memory_limits` operator configuration parameter. Optional.
## Connection pooler
Parameters are grouped under the `connectionPooler` top-level key and specify
configuration for connection pooler. If this section is not empty, a connection
pooler will be created for master service only even if `enableConnectionPooler`
is not present. But if this section is present then it defines the configuration
for both master and replica pooler services (if `enableReplicaConnectionPooler`
is enabled).
* **numberOfInstances**
How many instances of connection pooler to create.
* **schema**
Database schema to create for credentials lookup function.
* **user**
User to create for connection pooler to be able to connect to a database.
You can also choose a role from the `users` section or a system user role.
* **dockerImage**
Which docker image to use for connection pooler deployment.
* **maxDBConnections**
How many connections the pooler can max hold. This value is divided among the
pooler pods.
* **mode**
In which mode to run connection pooler, transaction or session.
* **resources**
Resource configuration for connection pooler deployment.
## Custom TLS certificates
Those parameters are grouped under the `tls` top-level key.
* **secretName**
By setting the `secretName` value, the cluster will switch to load the given
Kubernetes Secret into the container as a volume and uses that as the
certificate instead. It is up to the user to create and manage the
Kubernetes Secret either by hand or using a tool like the CertManager
operator.
* **certificateFile**
Filename of the certificate. Defaults to "tls.crt".
* **privateKeyFile**
Filename of the private key. Defaults to "tls.key".
* **caFile**
Optional filename to the CA certificate (e.g. "ca.crt"). Useful when the
client connects with `sslmode=verify-ca` or `sslmode=verify-full`.
Default is empty.
* **caSecretName**
By setting the `caSecretName` value, the ca certificate file defined by the
`caFile` will be fetched from this secret instead of `secretName` above.
This secret has to hold a file with that name in its root.
Optionally one can provide full path for any of them. By default it is
relative to the "/tls/", which is mount path of the tls secret.
If `caSecretName` is defined, the ca.crt path is relative to "/tlsca/",
otherwise to the same "/tls/".
docs/reference/command_line_and_environment.md
+5 -1
View File
@@ -45,7 +45,7 @@ The following environment variables are accepted by the operator:
all namespaces. Empty value defaults to the operator namespace. Overrides the
`watched_namespace` operator parameter.
* **SCALYR_API_KEY**
* **SCALYR_API_KEY** (*deprecated*)
the value of the Scalyr API key to supply to the pods. Overrides the
`scalyr_api_key` operator parameter.
@@ -56,3 +56,7 @@ The following environment variables are accepted by the operator:
* **CRD_READY_WAIT_INTERVAL**
defines the interval between consecutive attempts waiting for the
`postgresql` CRD to be created. The default is 5s.
* **ENABLE_JSON_LOGGING**
Set to `true` for JSON formatted logging output.
The default is false.
docs/reference/operator_parameters.md
+223 -57
View File
@@ -75,11 +75,27 @@ Those are top-level keys, containing both leaf keys and groups.
[OpenAPI v3 schema validation](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#validation)
The default is `true`.
* **enable_lazy_spilo_upgrade**
Instruct operator to update only the statefulsets with new images (Spilo and InitContainers) without immediately doing the rolling update. The assumption is pods will be re-started later with new images, for example due to the node rotation.
The default is `false`.
* **enable_pgversion_env_var**
With newer versions of Spilo, it is preferable to use `PGVERSION` pod environment variable instead of the setting `postgresql.bin_dir` in the `SPILO_CONFIGURATION` env variable. When this option is true, the operator sets `PGVERSION` and omits `postgresql.bin_dir` from `SPILO_CONFIGURATION`. When false, the `postgresql.bin_dir` is set. This setting takes precedence over `PGVERSION`; see PR 222 in Spilo. The default is `true`.
* **enable_spilo_wal_path_compat**
enables backwards compatible path between Spilo 12 and Spilo 13 images. The default is `false`.
* **etcd_host**
Etcd connection string for Patroni defined as `host:port`. Not required when
Patroni native Kubernetes support is used. The default is empty (use
Kubernetes-native DCS).
* **kubernetes_use_configmaps**
Select if setup uses endpoints (default), or configmaps to manage leader when
DCS is kubernetes (not etcd or similar). In OpenShift it is not possible to
use endpoints option, and configmaps is required. By default,
`kubernetes_use_configmaps: false`, meaning endpoints will be used.
* **docker_image**
Spilo Docker image for Postgres instances. For production, don't rely on the
default image, as it might be not the most up-to-date one. Instead, build
@@ -87,9 +103,18 @@ Those are top-level keys, containing both leaf keys and groups.
repository](https://github.com/zalando/spilo).
* **sidecar_docker_images**
a map of sidecar names to Docker images to run with Spilo. In case of the name
conflict with the definition in the cluster manifest the cluster-specific one
is preferred.
*deprecated*: use **sidecars** instead. A map of sidecar names to Docker
images to run with Spilo. In case of the name conflict with the definition in
the cluster manifest the cluster-specific one is preferred.
* **sidecars**
a list of sidecars to run with Spilo, for any cluster (i.e. globally defined
sidecars). Each item in the list is of type
[Container](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#container-v1-core).
Globally defined sidecars can be overwritten by specifying a sidecar in the
Postgres manifest with the same name.
Note: This field is not part of the schema validation. If the container
specification is invalid, then the operator fails to create the statefulset.
* **enable_shm_volume**
Instruct operator to start any new database pod without limitations on shm
@@ -101,7 +126,7 @@ Those are top-level keys, containing both leaf keys and groups.
* **workers**
number of working routines the operator spawns to process requests to
create/update/delete/sync clusters concurrently. The default is `4`.
create/update/delete/sync clusters concurrently. The default is `8`.
* **max_instances**
operator will cap the number of instances in any managed Postgres cluster up
@@ -127,8 +152,9 @@ Those are top-level keys, containing both leaf keys and groups.
at the cost of overprovisioning memory and potential scheduling problems for
containers with high memory limits due to the lack of memory on Kubernetes
cluster nodes. This affects all containers created by the operator (Postgres,
Scalyr sidecar, and other sidecars); to set resources for the operator's own
container, change the [operator deployment manually](../../manifests/postgres-operator.yaml#L20).
Scalyr sidecar, and other sidecars except **sidecars** defined in the operator
configuration); to set resources for the operator's own container, change the
[operator deployment manually](../../manifests/postgres-operator.yaml#L20).
The default is `false`.
## Postgres users
@@ -187,6 +213,22 @@ configuration they are grouped under the `kubernetes` key.
of a database created by the operator. If the annotation key is also provided
by the database definition, the database definition value is used.
* **delete_annotation_date_key**
key name for annotation that compares manifest value with current date in the
YYYY-MM-DD format. Allowed pattern: `'([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]'`.
The default is empty which also disables this delete protection check.
* **delete_annotation_name_key**
key name for annotation that compares manifest value with Postgres cluster name.
Allowed pattern: `'([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]'`. The default is
empty which also disables this delete protection check.
* **downscaler_annotations**
An array of annotations that should be passed from Postgres CRD on to the
statefulset and, if exists, to the connection pooler deployment as well.
Regular expressions like `downscaler/*` etc. are also accepted. Can be used
with [kube-downscaler](https://github.com/hjacobs/kube-downscaler).
* **watched_namespace**
The operator watches for Postgres objects in the given namespace. If not
specified, the value is taken from the operator namespace. A special `*`
@@ -207,12 +249,13 @@ configuration they are grouped under the `kubernetes` key.
Default is true.
* **enable_init_containers**
global option to allow for creating init containers to run actions before
Spilo is started. Default is true.
global option to allow for creating init containers in the cluster manifest to
run actions before Spilo is started. Default is true.
* **enable_sidecars**
global option to allow for creating sidecar containers to run alongside Spilo
on the same pod. Default is true.
global option to allow for creating sidecar containers in the cluster manifest
to run alongside Spilo on the same pod. Globally defined sidecars are always
enabled. Default is true.
* **secret_name_template**
a template for the name of the database user secrets generated by the
@@ -228,11 +271,24 @@ configuration they are grouped under the `kubernetes` key.
to the Postgres clusters after creation.
* **oauth_token_secret_name**
a name of the secret containing the `OAuth2` token to pass to the teams API.
The default is `postgresql-operator`.
namespaced name of the secret containing the `OAuth2` token to pass to the
teams API. The default is `postgresql-operator`.
* **infrastructure_roles_secret_name**
name of the secret containing infrastructure roles names and passwords.
*deprecated*: namespaced name of the secret containing infrastructure roles
with user names, passwords and role membership.
* **infrastructure_roles_secrets**
array of infrastructure role definitions which reference existing secrets
and specify the key names from which user name, password and role membership
are extracted. For the ConfigMap this has to be a string which allows
referencing only one infrastructure roles secret. The default is empty.
* **inherited_annotations**
list of annotation keys that can be inherited from the cluster manifest, and
added to each child objects (`Deployment`, `StatefulSet`, `Pod`, `PDB` and
`Services`) created by the operator incl. the ones from the connection
pooler deployment. The default is empty.
* **pod_role_label**
name of the label assigned to the Postgres pods (and services/endpoints) by
@@ -243,15 +299,16 @@ configuration they are grouped under the `kubernetes` key.
objects. The default is `application:spilo`.
* **inherited_labels**
list of labels that can be inherited from the cluster manifest, and added to
each child objects (`StatefulSet`, `Pod`, `Service` and `Endpoints`) created
by the operator. Typical use case is to dynamically pass labels that are
specific to a given Postgres cluster, in order to implement `NetworkPolicy`.
The default is empty.
list of label keys that can be inherited from the cluster manifest, and
added to each child objects (`Deployment`, `StatefulSet`, `Pod`, `PVCs`,
`PDB`, `Service`, `Endpoints` and `Secrets`) created by the operator.
Typical use case is to dynamically pass labels that are specific to a
given Postgres cluster, in order to implement `NetworkPolicy`. The default
is empty.
* **cluster_name_label**
name of the label assigned to Kubernetes objects created by the operator that
indicates which cluster a given object belongs to. The default is
name of the label assigned to Kubernetes objects created by the operator
that indicates which cluster a given object belongs to. The default is
`cluster-name`.
* **node_readiness_label**
@@ -269,17 +326,27 @@ configuration they are grouped under the `kubernetes` key.
for details on taints and tolerations. The default is empty.
* **pod_environment_configmap**
a name of the ConfigMap with environment variables to populate on every pod.
Right now this ConfigMap is searched in the namespace of the Postgres cluster.
All variables from that ConfigMap are injected to the pod's environment, on
conflicts they are overridden by the environment variables generated by the
operator. The default is empty.
namespaced name of the ConfigMap with environment variables to populate on
every pod. Right now this ConfigMap is searched in the namespace of the
Postgres cluster. All variables from that ConfigMap are injected to the pod's
environment, on conflicts they are overridden by the environment variables
generated by the operator. The default is empty.
* **pod_priority_class_name**
a name of the [priority class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
that should be assigned to the Postgres pods. The priority class itself must
be defined in advance. Default is empty (use the default priority class).
* **spilo_runasuser**
sets the user ID which should be used in the container to run the process.
This must be set to run the container without root. By default the container
runs with root. This option only works for Spilo versions >= 1.6-p3.
* **spilo_runasgroup**
sets the group ID which should be used in the container to run the process.
This must be set to run the container without root. By default the container
runs with root. This option only works for Spilo versions >= 1.6-p3.
* **spilo_fsgroup**
the Persistent Volumes for the Spilo pods in the StatefulSet will be owned and
writable by the group ID specified. This is required to run Spilo as a
@@ -291,12 +358,12 @@ configuration they are grouped under the `kubernetes` key.
used for AWS volume resizing and not required if you don't need that
capability. The default is `false`.
* **master_pod_move_timeout**
The period of time to wait for the success of migration of master pods from
an unschedulable node. The migration includes Patroni switchovers to
respective replicas on healthy nodes. The situation where master pods still
exist on the old node after this timeout expires has to be fixed manually.
The default is 20 minutes.
* **master_pod_move_timeout**
The period of time to wait for the success of migration of master pods from
an unschedulable node. The migration includes Patroni switchovers to
respective replicas on healthy nodes. The situation where master pods still
exist on the old node after this timeout expires has to be fixed manually.
The default is 20 minutes.
* **enable_pod_antiaffinity**
toggles [pod anti affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/)
@@ -312,6 +379,15 @@ configuration they are grouped under the `kubernetes` key.
of stateful sets of PG clusters. The default is `ordered_ready`, the second
possible value is `parallel`.
* **storage_resize_mode**
defines how operator handles the difference between the requested volume size and
the actual size. Available options are:
1. `ebs` : operator resizes EBS volumes directly and executes `resizefs` within a pod
2. `pvc` : operator only changes PVC definition
3. `off` : disables resize of the volumes.
4. `mixed` :operator uses AWS API to adjust size, throughput, and IOPS, and calls pvc change for file system resize
Default is "pvc".
## Kubernetes resource requests
This group allows you to configure resource requests for the Postgres pods.
@@ -381,6 +457,12 @@ CRD-based configuration.
Those options affect the behavior of load balancers created by the operator.
In the CRD-based configuration they are grouped under the `load_balancer` key.
* **custom_service_annotations**
This key/value map provides a list of annotations that get attached to each
service of a cluster created by the operator. If the annotation key is also
provided by the cluster definition, the manifest value is used.
Optional.
* **db_hosted_zone**
DNS zone for the cluster DNS name when the load balancer is configured for
the cluster. Only used when combined with
@@ -397,11 +479,8 @@ In the CRD-based configuration they are grouped under the `load_balancer` key.
cluster. Can be overridden by individual cluster settings. The default is
`false`.
* **custom_service_annotations**
This key/value map provides a list of annotations that get attached to each
service of a cluster created by the operator. If the annotation key is also
provided by the cluster definition, the manifest value is used.
Optional.
* **external_traffic_policy** defines external traffic policy for load
balancers. Allowed values are `Cluster` (default) and `Local`.
* **master_dns_name_format** defines the DNS name string template for the
master load balancer cluster. The default is
@@ -430,6 +509,20 @@ yet officially supported.
present and accessible by Postgres pods. At the moment, supported services by
Spilo are S3 and GCS. The default is empty.
* **wal_gs_bucket**
GCS bucket to use for shipping WAL segments with WAL-E. A bucket has to be
present and accessible by Postgres pods. Note, only the name of the bucket is
required. At the moment, supported services by Spilo are S3 and GCS.
The default is empty.
* **gcp_credentials**
Used to set the GOOGLE_APPLICATION_CREDENTIALS environment variable for the pods.
This is used in with conjunction with the `additional_secret_mount` and
`additional_secret_mount_path` to properly set the credentials for the spilo
containers. This will allow users to use specific
[service accounts](https://cloud.google.com/kubernetes-engine/docs/tutorials/authenticating-to-cloud-platform).
The default is empty
* **log_s3_bucket**
S3 bucket to use for shipping Postgres daily logs. Works only with S3 on AWS.
The bucket has to be present and accessible by Postgres pods. The default is
@@ -445,10 +538,22 @@ yet officially supported.
AWS region used to store EBS volumes. The default is `eu-central-1`.
* **additional_secret_mount**
Additional Secret (aws or gcp credentials) to mount in the pod. The default is empty.
Additional Secret (aws or gcp credentials) to mount in the pod.
The default is empty.
* **additional_secret_mount_path**
Path to mount the above Secret in the filesystem of the container(s). The default is empty.
Path to mount the above Secret in the filesystem of the container(s).
The default is empty.
* **enable_ebs_gp3_migration**
enable automatic migration on AWS from gp2 to gp3 volumes, that are smaller
than the configured max size (see below). This ignores that EBS gp3 is by
default only 125 MB/sec vs 250 MB/sec for gp2 >= 333GB.
The default is `false`.
* **enable_ebs_gp3_migration_max_size**
defines the maximum volume size in GB until which auto migration happens.
Default is 1000 (1TB) which matches 3000 IOPS.
## Logical backup
@@ -456,38 +561,48 @@ These parameters configure a K8s cron job managed by the operator to produce
Postgres logical backups. In the CRD-based configuration those parameters are
grouped under the `logical_backup` key.
* **logical_backup_schedule**
Backup schedule in the cron format. Please take the
[reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule)
into account. Default: "30 00 \* \* \*"
* **logical_backup_docker_image**
An image for pods of the logical backup job. The [example image](../../docker/logical-backup/Dockerfile)
runs `pg_dumpall` on a replica if possible and uploads compressed results to
an S3 bucket under the key `/spilo/pg_cluster_name/cluster_k8s_uuid/logical_backups`.
The default image is the same image built with the Zalando-internal CI
pipeline. Default: "registry.opensource.zalan.do/acid/logical-backup"
pipeline. Default: "registry.opensource.zalan.do/acid/logical-backup:v1.6.0"
* **logical_backup_google_application_credentials**
Specifies the path of the google cloud service account json file. Default is empty.
* **logical_backup_job_prefix**
The prefix to be prepended to the name of a k8s CronJob running the backups. Beware the prefix counts towards the name length restrictions imposed by k8s. Empty string is a legitimate value. Operator does not do the actual renaming: It simply creates the job with the new prefix. You will have to delete the old cron job manually. Default: "logical-backup-".
* **logical_backup_provider**
Specifies the storage provider to which the backup should be uploaded (`s3` or `gcs`).
Default: "s3"
* **logical_backup_s3_access_key_id**
When set, value will be in AWS_ACCESS_KEY_ID env variable. The Default is empty.
* **logical_backup_s3_bucket**
S3 bucket to store backup results. The bucket has to be present and
accessible by Postgres pods. Default: empty.
* **logical_backup_s3_region**
Specifies the region of the bucket which is required with some non-AWS S3 storage services. The default is empty.
* **logical_backup_s3_endpoint**
When using non-AWS S3 storage, endpoint can be set as a ENV variable. The default is empty.
* **logical_backup_s3_sse**
Specify server side encription that S3 storage is using. If empty string
is specified, no argument will be passed to `aws s3` command. Default: "AES256".
* **logical_backup_s3_access_key_id**
When set, value will be in AWS_ACCESS_KEY_ID env variable. The Default is empty.
* **logical_backup_s3_region**
Specifies the region of the bucket which is required with some non-AWS S3 storage services. The default is empty.
* **logical_backup_s3_secret_access_key**
When set, value will be in AWS_SECRET_ACCESS_KEY env variable. The Default is empty.
* **logical_backup_s3_sse**
Specify server side encryption that S3 storage is using. If empty string
is specified, no argument will be passed to `aws s3` command. Default: "AES256".
* **logical_backup_schedule**
Backup schedule in the cron format. Please take the
[reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule)
into account. Default: "30 00 \* \* \*"
## Debugging the operator
Options to aid debugging of the operator itself. Grouped under the `debug` key.
@@ -528,8 +643,8 @@ key.
The default is `"log_statement:all"`
* **enable_team_superuser**
whether to grant superuser to team members created from the Teams API.
The default is `false`.
whether to grant superuser to members of the cluster's owning team created
from the Teams API. The default is `false`.
* **team_admin_role**
role name to grant to team members created from the Teams API. The default is
@@ -562,6 +677,16 @@ key.
cluster to administer Postgres and maintain infrastructure built around it.
The default is empty.
* **enable_postgres_team_crd**
toggle to make the operator watch for created or updated `PostgresTeam` CRDs
and create roles for specified additional teams and members.
The default is `false`.
* **enable_postgres_team_crd_superusers**
in a `PostgresTeam` CRD additional superuser teams can assigned to teams that
own clusters. With this flag set to `false`, it will be ignored.
The default is `false`.
## Logging and REST API
Parameters affecting logging and REST API listener. In the CRD-based
@@ -576,11 +701,12 @@ configuration they are grouped under the `logging_rest_api` key.
* **cluster_history_entries**
number of entries in the cluster history ring buffer. The default is `1000`.
## Scalyr options
## Scalyr options (*deprecated*)
Those parameters define the resource requests/limits and properties of the
scalyr sidecar. In the CRD-based configuration they are grouped under the
`scalyr` key.
`scalyr` key. Note, that this section is deprecated. Instead, define Scalyr as
a global sidecar under the `sidecars` key in the configuration.
* **scalyr_api_key**
API key for the Scalyr sidecar. The default is empty.
@@ -602,3 +728,43 @@ scalyr sidecar. In the CRD-based configuration they are grouped under the
* **scalyr_memory_limit**
Memory limit value for the Scalyr sidecar. The default is `500Mi`.
## Connection pooler configuration
Parameters are grouped under the `connection_pooler` top-level key and specify
default configuration for connection pooler, if a postgres manifest requests it
but do not specify some of the parameters. All of them are optional with the
operator being able to provide some reasonable defaults.
* **connection_pooler_number_of_instances**
How many instances of connection pooler to create. Default is 2 which is also
the required minimum.
* **connection_pooler_schema**
Database schema to create for credentials lookup function to be used by the
connection pooler. Is is created in every database of the Postgres cluster.
You can also choose an existing schema. Default schema is `pooler`.
* **connection_pooler_user**
User to create for connection pooler to be able to connect to a database.
You can also choose an existing role, but make sure it has the `LOGIN`
privilege. Default role is `pooler`.
* **connection_pooler_image**
Docker image to use for connection pooler deployment.
Default: "registry.opensource.zalan.do/acid/pgbouncer"
* **connection_pooler_max_db_connections**
How many connections the pooler can max hold. This value is divided among the
pooler pods. Default is 60 which will make up 30 connections per pod for the
default setup with two instances.
* **connection_pooler_mode**
Default pooler mode, `session` or `transaction`. Default is `transaction`.
* **connection_pooler_default_cpu_request**
**connection_pooler_default_memory_reques**
**connection_pooler_default_cpu_limit**
**connection_pooler_default_memory_limit**
Default resource configuration for connection pooler deployment. The internal
default for memory request and limit is `100Mi`, for CPU it is `500m` and `1`.
docs/user.md
+541 -25
View File
@@ -30,7 +30,7 @@ spec:
databases:
foo: zalando
postgresql:
version: "11"
version: "12"
```
Once you cloned the Postgres Operator [repository](https://github.com/zalando/postgres-operator)
@@ -49,37 +49,48 @@ Note, that the name of the cluster must start with the `teamId` and `-`. At
Zalando we use team IDs (nicknames) to lower the chance of duplicate cluster
names and colliding entities. The team ID would also be used to query an API to
get all members of a team and create [database roles](#teams-api-roles) for
them.
them. Besides, the maximum cluster name length is 53 characters.
## Watch pods being created
Check if the database pods are coming up. Use the label `application=spilo` to
filter and list the label `spilo-role` to see when the master is promoted and
replicas get their labels.
```bash
kubectl get pods -w --show-labels
kubectl get pods -l application=spilo -L spilo-role -w
```
The operator also emits K8s events to the Postgresql CRD which can be inspected
in the operator logs or with:
```bash
kubectl describe postgresql acid-minimal-cluster
```
## Connect to PostgreSQL
With a `port-forward` on one of the database pods (e.g. the master) you can
connect to the PostgreSQL database. Use labels to filter for the master pod of
our test cluster.
connect to the PostgreSQL database from your machine. Use labels to filter for
the master pod of our test cluster.
```bash
# get name of master pod of acid-minimal-cluster
export PGMASTER=$(kubectl get pods -o jsonpath={.items..metadata.name} -l application=spilo,cluster-name=acid-minimal-cluster,spilo-role=master)
export PGMASTER=$(kubectl get pods -o jsonpath={.items..metadata.name} -l application=spilo,cluster-name=acid-minimal-cluster,spilo-role=master -n default)
# set up port forward
kubectl port-forward $PGMASTER 6432:5432
kubectl port-forward $PGMASTER 6432:5432 -n default
```
Open another CLI and connect to the database. Use the generated secret of the
`postgres` robot user to connect to our `acid-minimal-cluster` master running
in Minikube. As non-encrypted connections are rejected by default set the SSL
mode to require:
Open another CLI and connect to the database using e.g. the psql client.
When connecting with the `postgres` user read its password from the K8s secret
which was generated when creating the `acid-minimal-cluster`. As non-encrypted
connections are rejected by default set the SSL mode to `require`:
```bash
export PGPASSWORD=$(kubectl get secret postgres.acid-minimal-cluster.credentials -o 'jsonpath={.data.password}' | base64 -d)
export PGSSLMODE=require
psql -U postgres -p 6432
psql -U postgres -h localhost -p 6432
```
## Defining database roles in the operator
@@ -94,7 +105,10 @@ created on every cluster managed by the operator.
* `teams API roles`: automatically create users for every member of the team
owning the database cluster.
In the next sections, we will cover those use cases in more details.
In the next sections, we will cover those use cases in more details. Note, that
the Postgres Operator can also create databases with pre-defined owner, reader
and writer roles which saves you the manual setup. Read more in the next
chapter.
### Manifest roles
@@ -136,23 +150,62 @@ user. There are two ways to define them:
#### Infrastructure roles secret
The infrastructure roles secret is specified by the `infrastructure_roles_secret_name`
parameter. The role definition looks like this (values are base64 encoded):
Infrastructure roles can be specified by the `infrastructure_roles_secrets`
parameter where you can reference multiple existing secrets. Prior to `v1.6.0`
the operator could only reference one secret with the
`infrastructure_roles_secret_name` option. However, this secret could contain
multiple roles using the same set of keys plus incrementing index.
```yaml
user1: ZGJ1c2Vy
password1: c2VjcmV0
inrole1: b3BlcmF0b3I=
apiVersion: v1
kind: Secret
metadata:
name: postgresql-infrastructure-roles
data:
user1: ZGJ1c2Vy
password1: c2VjcmV0
inrole1: b3BlcmF0b3I=
user2: ...
```
The block above describes the infrastructure role 'dbuser' with password
'secret' that is a member of the 'operator' role. For the following definitions
one must increase the index, i.e. the next role will be defined as 'user2' and
so on. The resulting role will automatically be a login role.
'secret' that is a member of the 'operator' role. The resulting role will
automatically be a login role.
Note that with definitions that solely use the infrastructure roles secret
there is no way to specify role options (like superuser or nologin) or role
memberships. This is where the ConfigMap comes into play.
With the new option users can configure the names of secret keys that contain
the user name, password etc. The secret itself is referenced by the
`secretname` key. If the secret uses a template for multiple roles as described
above list them separately.
```yaml
apiVersion: v1
kind: OperatorConfiguration
metadata:
name: postgresql-operator-configuration
configuration:
kubernetes:
infrastructure_roles_secrets:
- secretname: "postgresql-infrastructure-roles"
userkey: "user1"
passwordkey: "password1"
rolekey: "inrole1"
- secretname: "postgresql-infrastructure-roles"
userkey: "user2"
...
```
Note, only the CRD-based configuration allows for referencing multiple secrets.
As of now, the ConfigMap is restricted to either one or the existing template
option with `infrastructure_roles_secret_name`. Please, refer to the example
manifests to understand how `infrastructure_roles_secrets` has to be configured
for the [configmap](../manifests/configmap.yaml) or [CRD configuration](../manifests/postgresql-operator-default-configuration.yaml).
If both `infrastructure_roles_secret_name` and `infrastructure_roles_secrets`
are defined the operator will create roles for both of them. So make sure,
they do not collide. Note also, that with definitions that solely use the
infrastructure roles secret there is no way to specify role options (like
superuser or nologin) or role memberships. This is where the additional
ConfigMap comes into play.
#### Secret plus ConfigMap
@@ -216,6 +269,304 @@ to choose superusers, group roles, [PAM configuration](https://github.com/CyberD
etc. An OAuth2 token can be passed to the Teams API via a secret. The name for
this secret is configurable with the `oauth_token_secret_name` parameter.
### Additional teams and members per cluster
Postgres clusters are associated with one team by providing the `teamID` in
the manifest. Additional superuser teams can be configured as mentioned in
the previous paragraph. However, this is a global setting. To assign
additional teams, superuser teams and single users to clusters of a given
team, use the [PostgresTeam CRD](../manifests/postgresteam.yaml).
Note, by default the `PostgresTeam` support is disabled in the configuration.
Switch `enable_postgres_team_crd` flag to `true` and the operator will start to
watch for this CRD. Make sure, the cluster role is up to date and contains a
section for [PostgresTeam](../manifests/operator-service-account-rbac.yaml#L30).
#### Additional teams
To assign additional teams and single users to clusters of a given team,
define a mapping with the `PostgresTeam` Kubernetes resource. The Postgres
Operator will read such team mappings each time it syncs all Postgres clusters.
```yaml
apiVersion: "acid.zalan.do/v1"
kind: PostgresTeam
metadata:
name: custom-team-membership
spec:
additionalTeams:
a-team:
- "b-team"
```
With the example above the operator will create login roles for all members
of `b-team` in every cluster owned by `a-team`. It's possible to do vice versa
for clusters of `b-team` in one manifest:
```yaml
spec:
additionalTeams:
a-team:
- "b-team"
b-team:
- "a-team"
```
You see, the `PostgresTeam` CRD is a global team mapping and independent from
the Postgres manifests. It is possible to define multiple mappings, even with
redundant content - the Postgres operator will create one internal cache from
it. Additional teams are resolved transitively, meaning you will also add
users for their `additionalTeams`, e.g.:
```yaml
spec:
additionalTeams:
a-team:
- "b-team"
- "c-team"
b-team:
- "a-team"
```
This creates roles for members of the `c-team` team not only in all clusters
owned by `a-team`, but as well in cluster owned by `b-team`, as `a-team` is
an `additionalTeam` to `b-team`
Not, you can also define `additionalSuperuserTeams` in the `PostgresTeam`
manifest. By default, this option is disabled and must be configured with
`enable_postgres_team_crd_superusers` to make it work.
#### Virtual teams
There can be "virtual teams" that do not exist in the Teams API. It can make
it easier to map a group of teams to many other teams:
```yaml
spec:
additionalTeams:
a-team:
- "virtual-team"
b-team:
- "virtual-team"
virtual-team:
- "c-team"
- "d-team"
```
This example would create roles for members of `c-team` and `d-team` plus
additional `virtual-team` members in clusters owned by `a-team` or `b-team`.
#### Teams changing their names
With `PostgresTeams` it is also easy to cover team name changes. Just add
the mapping between old and new team name and the rest can stay the same.
E.g. if team `a-team`'s name would change to `f-team` in the teams API it
could be reflected in a `PostgresTeam` mapping with just two lines:
```yaml
spec:
additionalTeams:
a-team:
- "f-team"
```
This is helpful, because Postgres cluster names are immutable and can not
be changed. Only via cloning it could get a different name starting with the
new `teamID`.
#### Additional members
Single members might be excluded from teams although they continue to work
with the same people. However, the teams API would not reflect this anymore.
To still add a database role for former team members list their role under
the `additionalMembers` section of the `PostgresTeam` resource:
```yaml
apiVersion: "acid.zalan.do/v1"
kind: PostgresTeam
metadata:
name: custom-team-membership
spec:
additionalMembers:
a-team:
- "tia"
```
This will create the login role `tia` in every cluster owned by `a-team`.
The user can connect to databases like the other team members.
The `additionalMembers` map can also be used to define users of virtual
teams, e.g. for `virtual-team` we used above:
```yaml
spec:
additionalMembers:
virtual-team:
- "flynch"
- "rdecker"
- "briggs"
```
## Prepared databases with roles and default privileges
The `users` section in the manifests only allows for creating database roles
with global privileges. Fine-grained data access control or role membership can
not be defined and must be set up by the user in the database. But, the Postgres
Operator offers a separate section to specify `preparedDatabases` that will be
created with pre-defined owner, reader and writer roles for each individual
database and, optionally, for each database schema, too. `preparedDatabases`
also enable users to specify PostgreSQL extensions that shall be created in a
given database schema.
### Default database and schema
A prepared database is already created by adding an empty `preparedDatabases`
section to the manifest. The database will then be called like the Postgres
cluster manifest (`-` are replaced with `_`) and will also contain a schema
called `data`.
```yaml
spec:
preparedDatabases: {}
```
### Default NOLOGIN roles
Given an example with a specified database and schema:
```yaml
spec:
preparedDatabases:
foo:
schemas:
bar: {}
```
Postgres Operator will create the following NOLOGIN roles:
| Role name | Member of | Admin |
| -------------- | -------------- | ------------- |
| foo_owner | | admin |
| foo_reader | | foo_owner |
| foo_writer | foo_reader | foo_owner |
| foo_bar_owner | | foo_owner |
| foo_bar_reader | | foo_bar_owner |
| foo_bar_writer | foo_bar_reader | foo_bar_owner |
The `<dbname>_owner` role is the database owner and should be used when creating
new database objects. All members of the `admin` role, e.g. teams API roles, can
become the owner with the `SET ROLE` command. [Default privileges](https://www.postgresql.org/docs/12/sql-alterdefaultprivileges.html)
are configured for the owner role so that the `<dbname>_reader` role
automatically gets read-access (SELECT) to new tables and sequences and the
`<dbname>_writer` receives write-access (INSERT, UPDATE, DELETE on tables,
USAGE and UPDATE on sequences). Both get USAGE on types and EXECUTE on
functions.
The same principle applies for database schemas which are owned by the
`<dbname>_<schema>_owner` role. `<dbname>_<schema>_reader` is read-only,
`<dbname>_<schema>_writer` has write access and inherit reading from the reader
role. Note, that the `<dbname>_*` roles have access incl. default privileges on
all schemas, too. If you don't need the dedicated schema roles - i.e. you only
use one schema - you can disable the creation like this:
```yaml
spec:
preparedDatabases:
foo:
schemas:
bar:
defaultRoles: false
```
Then, the schemas are owned by the database owner, too.
### Default LOGIN roles
The roles described in the previous paragraph can be granted to LOGIN roles from
the `users` section in the manifest. Optionally, the Postgres Operator can also
create default LOGIN roles for the database an each schema individually. These
roles will get the `_user` suffix and they inherit all rights from their NOLOGIN
counterparts.
| Role name | Member of | Admin |
| ------------------- | -------------- | ------------- |
| foo_owner_user | foo_owner | admin |
| foo_reader_user | foo_reader | foo_owner |
| foo_writer_user | foo_writer | foo_owner |
| foo_bar_owner_user | foo_bar_owner | foo_owner |
| foo_bar_reader_user | foo_bar_reader | foo_bar_owner |
| foo_bar_writer_user | foo_bar_writer | foo_bar_owner |
These default users are enabled in the manifest with the `defaultUsers` flag:
```yaml
spec:
preparedDatabases:
foo:
defaultUsers: true
schemas:
bar:
defaultUsers: true
```
### Database extensions
Prepared databases also allow for creating Postgres extensions. They will be
created by the database owner in the specified schema.
```yaml
spec:
preparedDatabases:
foo:
extensions:
pg_partman: public
postgis: data
```
Some extensions require SUPERUSER rights on creation unless they are not
whitelisted by the [pgextwlist](https://github.com/dimitri/pgextwlist)
extension, that is shipped with the Spilo image. To see which extensions are
on the list check the `extwlist.extension` parameter in the postgresql.conf
file.
```bash
SHOW extwlist.extensions;
```
Make sure that `pgextlist` is also listed under `shared_preload_libraries` in
the PostgreSQL configuration. Then the database owner should be able to create
the extension specified in the manifest.
### From `databases` to `preparedDatabases`
If you wish to create the role setup described above for databases listed under
the `databases` key, you have to make sure that the owner role follows the
`<dbname>_owner` naming convention of `preparedDatabases`. As roles are synced
first, this can be done with one edit:
```yaml
# before
spec:
databases:
foo: db_owner
# after
spec:
databases:
foo: foo_owner
preparedDatabases:
foo:
schemas:
my_existing_schema: {}
```
Adding existing database schemas to the manifest to create roles for them as
well is up the user and not done by the operator. Remember that if you don't
specify any schema a new database schema called `data` will be created. When
everything got synced (roles, schemas, extensions), you are free to remove the
database from the `databases` section. Note, that the operator does not delete
database objects or revoke privileges when removed from the manifest.
## Resource definition
The compute resources to be used for the Postgres containers in the pods can be
@@ -238,7 +589,7 @@ manifest the operator will raise the limits to the configured minimum values.
If no resources are defined in the manifest they will be obtained from the
configured [default requests](reference/operator_parameters.md#kubernetes-resource-requests).
## Use taints and tolerations for dedicated PostgreSQL nodes
## Use taints, tolerations and node affinity for dedicated PostgreSQL nodes
To ensure Postgres pods are running on nodes without any other application pods,
you can use [taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
@@ -252,6 +603,28 @@ spec:
effect: NoSchedule
```
If you need the pods to be scheduled on specific nodes you may use [node affinity](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/)
to specify a set of label(s), of which a prospective host node must have at least one. This could be used to
place nodes with certain hardware capabilities (e.g. SSD drives) in certain environments or network segments,
e.g. for PCI compliance.
```yaml
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
name: acid-minimal-cluster
spec:
teamId: "ACID"
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: environment
operator: In
values:
- pci
```
## How to clone an existing PostgreSQL cluster
You can spin up a new cluster as a clone of the existing one, using a `clone`
@@ -263,6 +636,10 @@ section in the spec. There are two options here:
Note, that cloning can also be used for [major version upgrades](administrator.md#minor-and-major-version-upgrade)
of PostgreSQL.
## In-place major version upgrade
Starting with Spilo 13, operator supports in-place major version upgrade to a higher major version (e.g. from PG 10 to PG 12). To trigger the upgrade, simply increase the version in the manifest. It is your responsibility to test your applications against the new version before the upgrade; downgrading is not supported. The easiest way to do so is to try the upgrade on the cloned cluster first. For details of how Spilo does the upgrade [see here](https://github.com/zalando/spilo/pull/488), operator implementation is described [in the admin docs](administrator.md#minor-and-major-version-upgrade).
### Clone from S3
Cloning from S3 has the advantage that there is no impact on your production
@@ -442,6 +819,8 @@ The PostgreSQL volume is shared with sidecars and is mounted at
specified but globally disabled in the configuration. The `enable_sidecars`
option must be set to `true`.
If you want to add a sidecar to every cluster managed by the operator, you can specify it in the [operator configuration](administrator.md#sidecars-for-postgres-clusters) instead.
## InitContainers Support
Each cluster can specify arbitrary init containers to run. These containers can
@@ -511,3 +890,140 @@ monitoring is outside the scope of operator responsibilities. See
[configuration reference](reference/cluster_manifest.md) and
[administrator documentation](administrator.md) for details on how backups are
executed.
## Connection pooler
The operator can create a database side connection pooler for those applications
where an application side pooler is not feasible, but a number of connections is
high. To create a connection pooler together with a database, modify the
manifest:
```yaml
spec:
enableConnectionPooler: true
enableReplicaConnectionPooler: true
```
This will tell the operator to create a connection pooler with default
configuration, through which one can access the master via a separate service
`{cluster-name}-pooler`. With the first option, connection pooler for master service
is created and with the second option, connection pooler for replica is created.
Note that both of these flags are independent of each other and user can set or
unset any of them as per their requirements without any effect on the other.
In most of the cases the
[default configuration](reference/operator_parameters.md#connection-pooler-configuration)
should be good enough. To configure a new connection pooler individually for
each Postgres cluster, specify:
```
spec:
connectionPooler:
# how many instances of connection pooler to create
numberOfInstances: 2
# in which mode to run, session or transaction
mode: "transaction"
# schema, which operator will create in each database
# to install credentials lookup function for connection pooler
schema: "pooler"
# user, which operator will create for connection pooler
user: "pooler"
# resources for each instance
resources:
requests:
cpu: 500m
memory: 100Mi
limits:
cpu: "1"
memory: 100Mi
```
The `enableConnectionPooler` flag is not required when the `connectionPooler`
section is present in the manifest. But, it can be used to disable/remove the
pooler while keeping its configuration.
By default, [`PgBouncer`](https://www.pgbouncer.org/) is used as connection pooler.
To find out about pool modes read the `PgBouncer` [docs](https://www.pgbouncer.org/config.html#pooler_mode)
(but it should be the general approach between different implementation).
Note, that using `PgBouncer` a meaningful resource CPU limit should be 1 core
or less (there is a way to utilize more than one, but in K8s it's easier just to
spin up more instances).
## Custom TLS certificates
By default, the Spilo image generates its own TLS certificate during startup.
However, this certificate cannot be verified and thus doesn't protect from
active MITM attacks. In this section we show how to specify a custom TLS
certificate which is mounted in the database pods via a K8s Secret.
Before applying these changes, in k8s the operator must also be configured with
the `spilo_fsgroup` set to the GID matching the postgres user group. If you
don't know the value, use `103` which is the GID from the default Spilo image
(`spilo_fsgroup=103` in the cluster request spec).
OpenShift allocates the users and groups dynamically (based on scc), and their
range is different in every namespace. Due to this dynamic behaviour, it's not
trivial to know at deploy time the uid/gid of the user in the cluster.
Therefore, instead of using a global `spilo_fsgroup` setting, use the
`spiloFSGroup` field per Postgres cluster.
Upload the cert as a kubernetes secret:
```sh
kubectl create secret tls pg-tls \
--key pg-tls.key \
--cert pg-tls.crt
```
When doing client auth, CA can come optionally from the same secret:
```sh
kubectl create secret generic pg-tls \
--from-file=tls.crt=server.crt \
--from-file=tls.key=server.key \
--from-file=ca.crt=ca.crt
```
Then configure the postgres resource with the TLS secret:
```yaml
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
name: acid-test-cluster
spec:
tls:
secretName: "pg-tls"
caFile: "ca.crt" # add this if the secret is configured with a CA
```
Optionally, the CA can be provided by a different secret:
```sh
kubectl create secret generic pg-tls-ca \
--from-file=ca.crt=ca.crt
```
Then configure the postgres resource with the TLS secret:
```yaml
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
name: acid-test-cluster
spec:
tls:
secretName: "pg-tls" # this should hold tls.key and tls.crt
caSecretName: "pg-tls-ca" # this should hold ca.crt
caFile: "ca.crt" # add this if the secret is configured with a CA
```
Alternatively, it is also possible to use
[cert-manager](https://cert-manager.io/docs/) to generate these secrets.
Certificate rotation is handled in the Spilo image which checks every 5
minutes if the certificates have changed and reloads postgres accordingly.