public_git/postgres-operator
1
0
Fork 0
mirror of https://github.com/zalando/postgres-operator.git synced 2026-10-01 14:57:37 +02:00
Code Issues Projects Releases Wiki Activity

Merge branch 'master' into lazy-update

Browse Source
This commit is contained in:
Felix Kunde
2020-03-27 09:41:52 +01:00
committed by GitHub
parent d1830f3ab3 ba9cf68650
commit 8f88fe7338
58 changed files with 3696 additions and 222 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/administrator.md
+36 -7
View File
@@ -95,6 +95,34 @@ lacks access rights to any of them (except K8s system namespaces like
'list pods' execute at the cluster scope and fail at the first violation of
access rights.
## Operators with defined ownership of certain Postgres clusters
By default, multiple operators can only run together in one K8s cluster when
isolated into their [own namespaces](administrator.md#specify-the-namespace-to-watch).
But, it is also possible to define ownership between operator instances and
Postgres clusters running all in the same namespace or K8s cluster without
interfering.
First, define the [`CONTROLLER_ID`](../../manifests/postgres-operator.yaml#L38)
environment variable in the operator deployment manifest. Then specify the ID
in every Postgres cluster manifest you want this operator to watch using the
`"acid.zalan.do/controller"` annotation:
```yaml
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
name: demo-cluster
annotations:
"acid.zalan.do/controller": "second-operator"
spec:
...
```
Every other Postgres cluster which lacks the annotation will be ignored by this
operator. Conversely, operators without a defined `CONTROLLER_ID` will ignore
clusters with defined ownership of another operator.
## Role-based access control for the operator
The manifest [`operator-service-account-rbac.yaml`](../manifests/operator-service-account-rbac.yaml)
@@ -293,11 +321,12 @@ spec:
## Custom Pod Environment Variables
It is possible to configure a ConfigMap which is used by the Postgres pods as
an additional provider for environment variables.
One use case is to customize the Spilo image and configure it with environment
variables. The ConfigMap with the additional settings is configured in the
operator's main ConfigMap:
an additional provider for environment variables. One use case is to customize
the Spilo image and configure it with environment variables. The ConfigMap with
the additional settings is referenced in the operator's main configuration.
A namespace can be specified along with the name. If left out, the configured
default namespace of your K8s client will be used and if the ConfigMap is not
found there, the Postgres cluster's namespace is taken when different:
**postgres-operator ConfigMap**
@@ -308,7 +337,7 @@ metadata:
name: postgres-operator
data:
# referencing config map with custom settings
pod_environment_configmap: postgres-pod-config
pod_environment_configmap: default/postgres-pod-config
```
**OperatorConfiguration**
@@ -321,7 +350,7 @@ metadata:
configuration:
kubernetes:
# referencing config map with custom settings
pod_environment_configmap: postgres-pod-config
pod_environment_configmap: default/postgres-pod-config
```
**referenced ConfigMap `postgres-pod-config`**
docs/reference/cluster_manifest.md
+64 -9
View File
@@ -111,12 +111,12 @@ These parameters are grouped directly under the `spec` key in the manifest.
value overrides the `pod_toleration` setting from the operator. Optional.
* **podPriorityClassName**
a name of the [priority
class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
that should be assigned to the cluster pods. When not specified, the value
is taken from the `pod_priority_class_name` operator parameter, if not set
then the default priority class is taken. The priority class itself must be
defined in advance. Optional.
a name of the [priority
class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
that should be assigned to the cluster pods. When not specified, the value
is taken from the `pod_priority_class_name` operator parameter, if not set
then the default priority class is taken. The priority class itself must be
defined in advance. Optional.
* **podAnnotations**
A map of key value pairs that gets attached as [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/)
@@ -140,6 +140,11 @@ These parameters are grouped directly under the `spec` key in the manifest.
is `false`, then no volume will be mounted no matter how operator was
configured (so you can override the operator configuration). Optional.
* **enableConnectionPool**
Tells the operator to create a connection pool with a database. If this
field is true, a connection pool deployment will be created even if
`connectionPool` section is empty. Optional, not set by default.
* **enableLogicalBackup**
Determines if the logical backup of this cluster should be taken and uploaded
to S3. Default: false. Optional.
@@ -184,9 +189,9 @@ explanation of `ttl` and `loop_wait` parameters.
```
hostssl all +pamrole all pam
```
, where pamrole is the name of the role for the pam authentication; any
custom `pg_hba` should include the pam line to avoid breaking pam
authentication. Optional.
where pamrole is the name of the role for the pam authentication; any
custom `pg_hba` should include the pam line to avoid breaking pam
authentication. Optional.
* **ttl**
Patroni `ttl` parameter value, optional. The default is set by the Spilo
@@ -359,3 +364,53 @@ CPU and memory limits for the sidecar container.
* **memory**
memory limits for the sidecar container. Optional, overrides the
`default_memory_limits` operator configuration parameter. Optional.
## Connection pool
Parameters are grouped under the `connectionPool` top-level key and specify
configuration for connection pool. If this section is not empty, a connection
pool will be created for a database even if `enableConnectionPool` is not
present.
* **numberOfInstances**
How many instances of connection pool to create.
* **schema**
Schema to create for credentials lookup function.
* **user**
User to create for connection pool to be able to connect to a database.
* **dockerImage**
Which docker image to use for connection pool deployment.
* **maxDBConnections**
How many connections the pooler can max hold. This value is divided among the
pooler pods.
* **mode**
In which mode to run connection pool, transaction or session.
* **resources**
Resource configuration for connection pool deployment.
## Custom TLS certificates
Those parameters are grouped under the `tls` top-level key.
* **secretName**
By setting the `secretName` value, the cluster will switch to load the given
Kubernetes Secret into the container as a volume and uses that as the
certificate instead. It is up to the user to create and manage the
Kubernetes Secret either by hand or using a tool like the CertManager
operator.
* **certificateFile**
Filename of the certificate. Defaults to "tls.crt".
* **privateKeyFile**
Filename of the private key. Defaults to "tls.key".
* **caFile**
Optional filename to the CA certificate. Useful when the client connects
with `sslmode=verify-ca` or `sslmode=verify-full`. Default is empty.
docs/reference/operator_parameters.md
+53 -15
View File
@@ -225,11 +225,12 @@ configuration they are grouped under the `kubernetes` key.
to the Postgres clusters after creation.
* **oauth_token_secret_name**
a name of the secret containing the `OAuth2` token to pass to the teams API.
The default is `postgresql-operator`.
namespaced name of the secret containing the `OAuth2` token to pass to the
teams API. The default is `postgresql-operator`.
* **infrastructure_roles_secret_name**
name of the secret containing infrastructure roles names and passwords.
namespaced name of the secret containing infrastructure roles names and
passwords.
* **pod_role_label**
name of the label assigned to the Postgres pods (and services/endpoints) by
@@ -266,11 +267,11 @@ configuration they are grouped under the `kubernetes` key.
for details on taints and tolerations. The default is empty.
* **pod_environment_configmap**
a name of the ConfigMap with environment variables to populate on every pod.
Right now this ConfigMap is searched in the namespace of the Postgres cluster.
All variables from that ConfigMap are injected to the pod's environment, on
conflicts they are overridden by the environment variables generated by the
operator. The default is empty.
namespaced name of the ConfigMap with environment variables to populate on
every pod. Right now this ConfigMap is searched in the namespace of the
Postgres cluster. All variables from that ConfigMap are injected to the pod's
environment, on conflicts they are overridden by the environment variables
generated by the operator. The default is empty.
* **pod_priority_class_name**
a name of the [priority class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
@@ -288,12 +289,12 @@ configuration they are grouped under the `kubernetes` key.
used for AWS volume resizing and not required if you don't need that
capability. The default is `false`.
* **master_pod_move_timeout**
The period of time to wait for the success of migration of master pods from
an unschedulable node. The migration includes Patroni switchovers to
respective replicas on healthy nodes. The situation where master pods still
exist on the old node after this timeout expires has to be fixed manually.
The default is 20 minutes.
* **master_pod_move_timeout**
The period of time to wait for the success of migration of master pods from
an unschedulable node. The migration includes Patroni switchovers to
respective replicas on healthy nodes. The situation where master pods still
exist on the old node after this timeout expires has to be fixed manually.
The default is 20 minutes.
* **enable_pod_antiaffinity**
toggles [pod anti affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/)
@@ -476,7 +477,7 @@ grouped under the `logical_backup` key.
When using non-AWS S3 storage, endpoint can be set as a ENV variable. The default is empty.
* **logical_backup_s3_sse**
Specify server side encription that S3 storage is using. If empty string
Specify server side encryption that S3 storage is using. If empty string
is specified, no argument will be passed to `aws s3` command. Default: "AES256".
* **logical_backup_s3_access_key_id**
@@ -599,3 +600,40 @@ scalyr sidecar. In the CRD-based configuration they are grouped under the
* **scalyr_memory_limit**
Memory limit value for the Scalyr sidecar. The default is `500Mi`.
## Connection pool configuration
Parameters are grouped under the `connection_pool` top-level key and specify
default configuration for connection pool, if a postgres manifest requests it
but do not specify some of the parameters. All of them are optional with the
operator being able to provide some reasonable defaults.
* **connection_pool_number_of_instances**
How many instances of connection pool to create. Default is 2 which is also
the required minimum.
* **connection_pool_schema**
Schema to create for credentials lookup function. Default is `pooler`.
* **connection_pool_user**
User to create for connection pool to be able to connect to a database.
Default is `pooler`.
* **connection_pool_image**
Docker image to use for connection pool deployment.
Default: "registry.opensource.zalan.do/acid/pgbouncer"
* **connection_pool_max_db_connections**
How many connections the pooler can max hold. This value is divided among the
pooler pods. Default is 60 which will make up 30 connections per pod for the
default setup with two instances.
* **connection_pool_mode**
Default pool mode, `session` or `transaction`. Default is `transaction`.
* **connection_pool_default_cpu_request**
**connection_pool_default_memory_reques**
**connection_pool_default_cpu_limit**
**connection_pool_default_memory_limit**
Default resource configuration for connection pool deployment. The internal
default for memory request and limit is `100Mi`, for CPU it is `500m` and `1`.
docs/user.md
+101 -1
View File
@@ -30,7 +30,7 @@ spec:
databases:
foo: zalando
postgresql:
version: "11"
version: "12"
```
Once you cloned the Postgres Operator [repository](https://github.com/zalando/postgres-operator)
@@ -511,3 +511,103 @@ monitoring is outside the scope of operator responsibilities. See
[configuration reference](reference/cluster_manifest.md) and
[administrator documentation](administrator.md) for details on how backups are
executed.
## Connection pool
The operator can create a database side connection pool for those applications,
where an application side pool is not feasible, but a number of connections is
high. To create a connection pool together with a database, modify the
manifest:
```yaml
spec:
enableConnectionPool: true
```
This will tell the operator to create a connection pool with default
configuration, through which one can access the master via a separate service
`{cluster-name}-pooler`. In most of the cases provided default configuration
should be good enough.
To configure a new connection pool, specify:
```
spec:
connectionPool:
# how many instances of connection pool to create
number_of_instances: 2
# in which mode to run, session or transaction
mode: "transaction"
# schema, which operator will create to install credentials lookup
# function
schema: "pooler"
# user, which operator will create for connection pool
user: "pooler"
# resources for each instance
resources:
requests:
cpu: 500m
memory: 100Mi
limits:
cpu: "1"
memory: 100Mi
```
By default `pgbouncer` is used to create a connection pool. To find out about
pool modes see [docs](https://www.pgbouncer.org/config.html#pool_mode) (but it
should be general approach between different implementation).
Note, that using `pgbouncer` means meaningful resource CPU limit should be less
than 1 core (there is a way to utilize more than one, but in K8S it's easier
just to spin up more instances).
## Custom TLS certificates
By default, the spilo image generates its own TLS certificate during startup.
However, this certificate cannot be verified and thus doesn't protect from
active MITM attacks. In this section we show how to specify a custom TLS
certificate which is mounted in the database pods via a K8s Secret.
Before applying these changes, the operator must also be configured with the
`spilo_fsgroup` set to the GID matching the postgres user group. If the value
is not provided, the cluster will default to `103` which is the GID from the
default spilo image.
Upload the cert as a kubernetes secret:
```sh
kubectl create secret tls pg-tls \
--key pg-tls.key \
--cert pg-tls.crt
```
Or with a CA:
```sh
kubectl create secret generic pg-tls \
--from-file=tls.crt=server.crt \
--from-file=tls.key=server.key \
--from-file=ca.crt=ca.crt
```
Alternatively it is also possible to use
[cert-manager](https://cert-manager.io/docs/) to generate these secrets.
Then configure the postgres resource with the TLS secret:
```yaml
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
name: acid-test-cluster
spec:
tls:
secretName: "pg-tls"
caFile: "ca.crt" # add this if the secret is configured with a CA
```
Certificate rotation is handled in the spilo image which checks every 5
minutes if the certificates have changed and reloads postgres accordingly.