public_git/postgres-operator
1
0
Fork 0
mirror of https://github.com/zalando/postgres-operator.git synced 2026-10-06 18:53:44 +02:00
Code Issues Projects Releases Wiki Activity

Merge branch 'master' of https://github.com/zalando/postgres-operator into standby

Browse Source
This commit is contained in:
Rafia Sabih
2019-06-14 12:14:17 +02:00
parent 70c0e59cdb ad5fec9bee
commit 14570dbc02
45 changed files with 1756 additions and 245 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/administrator.md
+21 -50
View File
@@ -1,47 +1,3 @@
## Create ConfigMap
A ConfigMap is used to store the configuration of the operator.
```bash
$ kubectl create -f manifests/configmap.yaml
```
## Deploying the operator
First you need to install the service account definition in your Minikube cluster.
```bash
$ kubectl create -f manifests/operator-service-account-rbac.yaml
```
Next deploy the postgres-operator from the docker image Zalando is using:
```bash
$ kubectl create -f manifests/postgres-operator.yaml
```
If you prefer to build the image yourself follow up down below.
### - Helm chart
You can install postgres-operator also with a [Helm](https://helm.sh/) chart.
This requires installing the Helm CLI first and then initializing it in the
cluster.
```bash
$ helm init
$ helm install --name my-release ./charts/postgres-operator
```
## Check if CustomResourceDefinition has been registered
```bash
$ kubectl get crd
NAME KIND
postgresqls.acid.zalan.do CustomResourceDefinition.v1beta1.apiextensions.k8s.io
```
# How to configure PostgreSQL operator
## Select the namespace to deploy to
@@ -103,6 +59,12 @@ In this definition, the operator overwrites the account's name to match
`pod_service_account_name` and the `default` namespace to match the target
namespace. The operator performs **no** further syncing of this account.
## Non-default cluster domain
If your cluster uses a different dns domain than `cluster.local`, this needs
to be set in the operator ConfigMap. This is used by the operator to connect
to the clusters after creation.
## Role-based access control for the operator
The `manifests/operator-service-account-rbac.yaml` defines cluster roles and
@@ -340,9 +302,18 @@ Postgres database cluster:
## Understanding rolling update of Spilo pods
The operator logs reasons for a rolling update with the `info` level and
a diff between the old and new StatefulSet specs with the `debug` level.
To read the latter log entry with the escaped characters rendered, view it
in CLI with `echo -e`. Note that the resultant message will contain some
noise because the `PodTemplate` used by the operator is yet to be updated
with the default values used internally in Kubernetes.
The operator logs reasons for a rolling update with the `info` level and a diff between the old and new StatefulSet specs with the `debug` level. To benefit from numerous escape characters in the latter log entry, view it in CLI with `echo -e`. Note that the resultant message will contain some noise because the `PodTemplate` used by the operator is yet to be updated with the default values used internally in Kubernetes.
## Logical backups
The operator can manage k8s cron jobs to run logical backups of Postgres clusters. The cron job periodically spawns a batch job that runs a single pod. The backup script within this pod's container can connect to a DB for a logical backup. The operator updates cron jobs during Sync if the job schedule changes; the job name acts as the job identifier. These jobs are to be enabled for each indvidual Postgres cluster by setting `enableLogicalBackup: true` in its manifest. Notes:
1. The [example image](../docker/logical-backup/Dockerfile) implements the backup via `pg_dumpall` and upload of compressed and encrypted results to an S3 bucket; the default image ``registry.opensource.zalan.do/acid/logical-backup`` is the same image built with the Zalando-internal CI pipeline. `pg_dumpall` requires a `superuser` access to a DB and runs on the replica when possible.
2. Due to the [limitation of Kubernetes cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) it is highly advisable to set up additional monitoring for this feature; such monitoring is outside of the scope of operator responsibilities.
3. The operator does not remove old backups.
4. You may use your own image by overwriting the relevant field in the operator configuration. Any such image must ensure the logical backup is able to finish [in presence of pod restarts](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#handling-pod-and-container-failures) and [simultaneous invocations](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) of the backup cron job.
5. For that feature to work, your RBAC policy must enable operations on the `cronjobs` resource from the `batch` API group for the operator service account. See [example RBAC](../manifests/operator-service-account-rbac.yaml)
docs/developer.md
+32 -12
View File
@@ -20,18 +20,17 @@ that your setup is working.
Note: if you use multiple Kubernetes clusters, you can switch to Minikube with
`kubectl config use-context minikube`
## Create ConfigMap
## Deploying the operator
ConfigMap is used to store the configuration of the operator
### Kubernetes manifest
A ConfigMap is used to store the configuration of the operator. Alternatively,
a CRD-based configuration can be used, as described [here](reference/operator_parameters).
```bash
$ kubectl --context minikube create -f manifests/configmap.yaml
```
## Deploying the operator
### - Kubernetes manifest
First you need to install the service account definition in your Minikube cluster.
```bash
@@ -46,15 +45,23 @@ Next deploy the postgres-operator from the docker image Zalando is using:
If you prefer to build the image yourself follow up down below.
### - Helm chart
### Helm chart
You can install postgres-operator also with a [Helm](https://helm.sh/) chart.
This requires installing the Helm CLI first and then initializing it in the
cluster.
Alternatively, the operator can be installed by using the provided [Helm](https://helm.sh/)
chart which saves you the manual steps. Therefore, you would need to install
the helm CLI on your machine. After initializing helm (and its server
component Tiller) in your local cluster you can install the operator chart.
You can define a release name that is prepended to the operator resource's
names.
Use `--name zalando` to match with the default service account name as older
operator versions do not support custom names for service accounts. When relying
solely on the CRD-based configuration edit the `serviceAccount` section in the
[values yaml file](../charts/values.yaml) by setting the name to `"operator"`.
```bash
$ helm init
$ helm install --name my-release ./charts/postgres-operator
$ helm install --name zalando ./charts/postgres-operator
```
## Check if CustomResourceDefinition has been registered
@@ -203,7 +210,7 @@ localhost:8080 by doing:
The inner 'query' gets the name of the postgres operator pod, and the outer
enables port forwarding. Afterwards, you can access the operator API with:
$ curl http://127.0.0.1:8080/$endpoint| jq .
$ curl --location http://127.0.0.1:8080/$endpoint | jq .
The available endpoints are listed below. Note that the worker ID is an integer
from 0 up to 'workers' - 1 (value configured in the operator configuration and
@@ -315,6 +322,16 @@ Then you can for example check the Patroni logs:
kubectl logs acid-minimal-cluster-0
```
## End-to-end tests
The operator provides reference e2e (end-to-end) tests to ensure various infra parts work smoothly together.
Each e2e execution tests a Postgres operator image built from the current git branch. The test runner starts a [kind](https://kind.sigs.k8s.io/) (local k8s) cluster and Docker container with tests. The k8s API client from within the container connects to the `kind` cluster using the standard Docker `bridge` network.
The tests utilize examples from `/manifests` (ConfigMap is used for the operator configuration) to avoid maintaining yet another set of configuration files. The kind cluster is deleted if tests complete successfully.
End-to-end tests are executed automatically during builds; to invoke them locally use `make e2e-run` from the project's top directory. Run `make e2e-tools e2e-build` to install `kind` and build the tests' image locally before the first run.
End-to-end tests are written in Python and use `flake8` for code quality. Please run flake8 [before submitting a PR](http://flake8.pycqa.org/en/latest/user/using-hooks.html).
## Introduce additional configuration parameters
In the case you want to add functionality to the operator that shall be
@@ -323,6 +340,9 @@ be updated. As explained [here](reference/operator_parameters.md), it's possible
to configure the operator either with a ConfigMap or CRD, but currently we aim
to synchronize parameters everywhere.
When choosing a parameter name for a new option in a PG manifest, keep in mind
the naming conventions there. The `snake_case` variables come from the Patroni/Postgres world, while the `camelCase` from the k8s world.
Note: If one option is defined in the operator configuration and in the cluster
[manifest](../manifests/complete-postgres-manifest.yaml), the latter takes
precedence.
docs/quickstart.md
+12 -2
View File
@@ -20,6 +20,12 @@ cd postgres-operator
minikube start
```
If you want to configure the Postgres Operator it must happen before deploying a
Postgres cluster. This can happen in two ways: Via a ConfigMap or a
`OperatorConfiguration` object, which adheres a CustomResourceDefinition (CRD).
More details on configuration can be found [here](reference/operator_parameters.md).
## Manual deployment setup
The Postgres Operator can be installed simply by applying yaml manifests.
@@ -37,8 +43,12 @@ chart which saves you the manual steps. Therefore, you would need to install
the helm CLI on your machine. After initializing helm (and its server
component Tiller) in your local cluster you can install the operator chart.
You can define a release name that is prepended to the operator resource's
names. Use `--name zalando` to match with the default service account name
as older operator versions do not support custom names for service accounts.
names.
Use `--name zalando` to match with the default service account name as older
operator versions do not support custom names for service accounts. When relying
solely on the CRD-based configuration edit the `serviceAccount` section in the
[values yaml file](../charts/values.yaml) by setting the name to `"operator"`.
```bash
# 1) initialize helm
docs/reference/cluster_manifest.md
+61 -4
View File
@@ -1,4 +1,3 @@
Individual postgres clusters are described by the Kubernetes *cluster manifest*
that has the structure defined by the `postgres CRD` (custom resource
definition). The following section describes the structure of the manifest and
@@ -14,6 +13,10 @@ measurements. Please, refer to the [Kubernetes
documentation](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
for the possible values of those.
:exclamation: If both operator configmap/CRD and a Postgres cluster manifest
define the same parameter, the value from the Postgres cluster manifest is
applied.
## Manifest structure
A postgres manifest is a `YAML` document. On the top level both individual
@@ -45,7 +48,7 @@ Those parameters are grouped under the `metadata` top-level key.
## Top-level parameters
Those are parameters grouped directly under the `spec` key in the manifest.
These parameters are grouped directly under the `spec` key in the manifest.
* **teamId**
name of the team the cluster belongs to. Changing it after the cluster
@@ -55,6 +58,13 @@ Those are parameters grouped directly under the `spec` key in the manifest.
custom docker image that overrides the **docker_image** operator parameter.
It should be a [Spilo](https://github.com/zalando/spilo) image. Optional.
* **spiloFSGroup**
the Persistent Volumes for the spilo pods in the StatefulSet will be owned
and writable by the group ID specified. This will override the **spilo_fsgroup**
operator parameter. This is required to run Spilo as a non-root process, but
requires a custom spilo image. Note the FSGroup of a Pod cannot be changed
without recreating a new Pod.
* **enableMasterLoadBalancer**
boolean flag to override the operator defaults (set by the
`enable_master_load_balancer` parameter) to define whether to enable the load
@@ -103,7 +113,8 @@ Those are parameters grouped directly under the `spec` key in the manifest.
class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
that should be assigned to the cluster pods. When not specified, the value
is taken from the `pod_priority_class_name` operator parameter, if not set
then the default priority class is taken. The priority class itself must be defined in advance.
then the default priority class is taken. The priority class itself must be
defined in advance.
* **enableShmVolume**
Start a database pod without limitations on shm memory. By default docker
@@ -117,6 +128,14 @@ Those are parameters grouped directly under the `spec` key in the manifest.
is `false`, then no volume will be mounted no matter how operator was
configured (so you can override the operator configuration).
* **enableLogicalBackup**
Determines if the logical backup of this cluster should be taken and uploaded
to S3. Default: false.
* **logicalBackupSchedule**
Schedule for the logical backup k8s cron job. Please take [the reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule)
into account. Default: "30 00 \* \* \*"
## Postgres parameters
Those parameters are grouped under the `postgresql` top-level key.
@@ -173,7 +192,12 @@ explanation of `ttl` and `loop_wait` parameters.
set by the Spilo docker image. Optional.
* **slots**
permanent replication slots that Patroni preserves after failover by re-creating them on the new primary immediately after doing a promote. Slots could be reconfigured with the help of `patronictl edit-config`. It is the responsibility of a user to avoid clashes in names between replication slots automatically created by Patroni for cluster members and permanent replication slots. Optional.
permanent replication slots that Patroni preserves after failover by
re-creating them on the new primary immediately after doing a promote. Slots
could be reconfigured with the help of `patronictl edit-config`. It is the
responsibility of a user to avoid clashes in names between replication slots
automatically created by Patroni for cluster members and permanent replication
slots. Optional.
* **standby_cluster**
initializes cluster as a standby creating a cascading replication, where standby leader is streaming from specified remote location
@@ -265,3 +289,36 @@ defined in the sidecar dictionary:
a dictionary of environment variables. Use usual Kubernetes definition
(https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/)
for environment variables. Optional.
* **resources** see below. Optional.
#### Sidecar container resources
Those parameters define [CPU and memory requests and
limits](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
for the sidecar container. They are grouped under the `resources` key for each sidecar.
There are two subgroups, `requests` and `limits`.
##### Requests
CPU and memory requests for the sidecar container.
* **cpu**
CPU requests for the sidecar container. Optional, overrides the
`default_cpu_requests` operator configuration parameter. Optional.
* **memory**
memory requests for the sidecar container. Optional, overrides the
`default_memory_request` operator configuration parameter. Optional.
##### Limits
CPU and memory limits for the sidecar container.
* **cpu**
CPU limits for the sidecar container. Optional, overrides the
`default_cpu_limits` operator configuration parameter. Optional.
* **memory**
memory limits for the sidecar container. Optional, overrides the
`default_memory_limits` operator configuration parameter. Optional.
docs/reference/operator_parameters.md
+114 -56
View File
@@ -11,17 +11,18 @@ configuration.
[example](https://github.com/zalando/postgres-operator/blob/master/manifests/configmap.yaml)
* CRD-based configuration. The configuration is stored in a custom YAML
manifest. The manifest is an instance of the custom resource definition (CRD) called
`OperatorConfiguration`. The operator registers this CRD
during the start and uses it for configuration if the [operator deployment manifest ](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L21) sets the `POSTGRES_OPERATOR_CONFIGURATION_OBJECT` env variable to a non-empty value. The variable should point to the
`postgresql-operator-configuration` object in the operator's namespace.
manifest. The manifest is an instance of the custom resource definition (CRD)
called `OperatorConfiguration`. The operator registers this CRD during the
start and uses it for configuration if the [operator deployment manifest ](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L21)
sets the `POSTGRES_OPERATOR_CONFIGURATION_OBJECT` env variable to a non-empty
value. The variable should point to the `postgresql-operator-configuration`
object in the operator's namespace.
The CRD-based configuration is a regular YAML
document; non-scalar keys are simply represented in the usual YAML way.
There are no default values built-in in the operator, each parameter that is
not supplied in the configuration receives an empty value. In order to
create your own configuration just copy the [default
one](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml)
The CRD-based configuration is a regular YAML document; non-scalar keys are
simply represented in the usual YAML way. There are no default values built-in
in the operator, each parameter that is not supplied in the configuration
receives an empty value. In order to create your own configuration just copy
the [default one](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml)
and change it.
To test the CRD-based configuration locally, use the following
@@ -31,19 +32,23 @@ configuration.
kubectl create -f manifests/postgresql-operator-default-configuration.yaml
kubectl get operatorconfigurations postgresql-operator-default-configuration -o yaml
```
Note that the operator first registers the definition of the CRD `OperatorConfiguration` and then waits for an instance of the CRD to be created. In between these two event the operator pod may be failing since it cannot fetch the not-yet-existing `OperatorConfiguration` instance.
Note that the operator first registers the CRD of the `OperatorConfiguration`
and then waits for an instance to be created. In between these two event the
operator pod may be failing since it cannot fetch the not-yet-existing
`OperatorConfiguration` instance.
The CRD-based configuration is more powerful than the one based on
ConfigMaps and should be used unless there is a compatibility requirement to
use an already existing configuration. Even in that case, it should be rather
straightforward to convert the configmap based configuration into the CRD-based
one and restart the operator. The ConfigMaps-based configuration will be
deprecated and subsequently removed in future releases.
The CRD-based configuration is more powerful than the one based on ConfigMaps
and should be used unless there is a compatibility requirement to use an already
existing configuration. Even in that case, it should be rather straightforward
to convert the configmap based configuration into the CRD-based one and restart
the operator. The ConfigMaps-based configuration will be deprecated and
subsequently removed in future releases.
Note that for the CRD-based configuration groups of configuration options below correspond
to the non-leaf keys in the target YAML (i.e. for the Kubernetes resources the
key is `kubernetes`). The key is mentioned alongside the group description. The
ConfigMap-based configuration is flat and does not allow non-leaf keys.
Note that for the CRD-based configuration groups of configuration options below
correspond to the non-leaf keys in the target YAML (i.e. for the Kubernetes
resources the key is `kubernetes`). The key is mentioned alongside the group
description. The ConfigMap-based configuration is flat and does not allow
non-leaf keys.
Since in the CRD-based case the operator needs to create a CRD first, which is
controlled by the `resource_check_interval` and `resource_check_timeout`
@@ -51,6 +56,12 @@ parameters, those parameters have no effect and are replaced by the
`CRD_READY_WAIT_INTERVAL` and `CRD_READY_WAIT_TIMEOUT` environment variables.
They will be deprecated and removed in the future.
For the configmap operator configuration, the [default parameter values](https://github.com/zalando-incubator/postgres-operator/blob/master/pkg/util/config/config.go#L14)
mentioned here are likely to be overwritten in your local operator installation
via your local version of the operator configmap. In the case you use the
operator CRD, all the CRD defaults are provided in the
[operator's default configuration manifest](https://github.com/zalando-incubator/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml)
Variable names are underscore-separated words.
@@ -85,8 +96,8 @@ Those are top-level keys, containing both leaf keys and groups.
* **min_instances**
operator will run at least the number of instances for any given postgres
cluster equal to the value of this parameter. When `-1` is specified, no limits
are applied. The default is `-1`.
cluster equal to the value of this parameter. When `-1` is specified, no
limits are applied. The default is `-1`.
* **resync_period**
period between consecutive sync requests. The default is `30m`.
@@ -122,7 +133,8 @@ configuration they are grouped under the `kubernetes` key.
* **pod_service_account_definition**
The operator tries to create the pod Service Account in the namespace that
doesn't define such an account using the YAML definition provided by this
option. If not defined, a simple definition that contains only the name will be used. The default is empty.
option. If not defined, a simple definition that contains only the name will
be used. The default is empty.
* **pod_service_account_role_binding_definition**
This definition must bind pod service account to a role with permission
@@ -140,8 +152,8 @@ configuration they are grouped under the `kubernetes` key.
* **watched_namespace**
The operator watches for postgres objects in the given namespace. If not
specified, the value is taken from the operator namespace. A special `*`
value makes it watch all namespaces. The default is empty (watch the operator pod
namespace).
value makes it watch all namespaces. The default is empty (watch the operator
pod namespace).
* **pdb_name_format**
defines the template for PDB (Pod Disruption Budget) names created by the
@@ -157,6 +169,11 @@ configuration they are grouped under the `kubernetes` key.
allowed. The default is
`{username}.{cluster}.credentials.{tprkind}.{tprgroup}`.
* **cluster_domain**
defines the default dns domain for the kubernetes cluster the operator is
running in. The default is `cluster.local`. Used by the operator to connect
to the postgres clusters after creation.
* **oauth_token_secret_name**
a name of the secret containing the `OAuth2` token to pass to the teams API.
The default is `postgresql-operator`.
@@ -174,8 +191,8 @@ configuration they are grouped under the `kubernetes` key.
* **inherited_labels**
list of labels that can be inherited from the cluster manifest, and added to
each child objects (`StatefulSet`, `Pod`, `Service` and `Endpoints`) created by
the opertor.
each child objects (`StatefulSet`, `Pod`, `Service` and `Endpoints`) created
by the opertor.
Typical use case is to dynamically pass labels that are specific to a given
postgres cluster, in order to implement `NetworkPolicy`.
The default is empty.
@@ -196,8 +213,7 @@ configuration they are grouped under the `kubernetes` key.
* **toleration**
a dictionary that should contain `key`, `operator`, `value` and
`effect` keys. In that case, the operator defines a pod toleration
according to the values of those keys. See [kubernetes
documentation](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
according to the values of those keys. See [kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
for details on taints and tolerations. The default is empty.
* **pod_environment_configmap**
@@ -208,31 +224,40 @@ configuration they are grouped under the `kubernetes` key.
operator. The default is empty.
* **pod_priority_class_name**
a name of the [priority
class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
that should be assigned to the Postgres pods. The priority class itself must be defined in advance.
Default is empty (use the default priority class).
a name of the [priority class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
that should be assigned to the Postgres pods. The priority class itself must
be defined in advance. Default is empty (use the default priority class).
* **spilo_fsgroup**
the Persistent Volumes for the spilo pods in the StatefulSet will be owned and writable by the group ID specified.
This is required to run Spilo as a non-root process, but requires a custom spilo image. Note the FSGroup of a Pod
cannot be changed without recreating a new Pod.
* **spilo_privileged**
whether the Spilo container should run in privileged mode. Privileged mode is used for AWS volume resizing and not required if you don't need that capability. The default is `false`.
whether the Spilo container should run in privileged mode. Privileged mode is
used for AWS volume resizing and not required if you don't need that
capability. The default is `false`.
* **master_pod_move_timeout**
The period of time to wait for the success of migration of master pods from an unschedulable node.
The migration includes Patroni switchovers to respective replicas on healthy nodes. The situation where master pods still exist on the old node after this timeout expires has to be fixed manually. The default is 20 minutes.
The period of time to wait for the success of migration of master pods from
an unschedulable node. The migration includes Patroni switchovers to
respective replicas on healthy nodes. The situation where master pods still
exist on the old node after this timeout expires has to be fixed manually.
The default is 20 minutes.
* **enable_pod_antiaffinity**
toggles [pod anti affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) on the Postgres pods, to avoid multiple pods
of the same Postgres cluster in the same topology , e.g. node. The default is `false`.
toggles [pod anti affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/)
on the Postgres pods, to avoid multiple pods of the same Postgres cluster in
the same topology , e.g. node. The default is `false`.
* **pod_antiaffinity_topology_key**
override
[topology key](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#interlude-built-in-node-labels)
override [topology key](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#interlude-built-in-node-labels)
for pod anti affinity. The default is `kubernetes.io/hostname`.
* **pod_management_policy**
specify the
[pod management policy](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-management-policies)
of stateful sets of PG clusters. The default is `ordered_ready`, the second possible value is `parallel`.
specify the [pod management policy](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-management-policies)
of stateful sets of PG clusters. The default is `ordered_ready`, the second
possible value is `parallel`.
## Kubernetes resource requests
@@ -257,7 +282,14 @@ CRD-based configuration.
settings. The default is `1Gi`.
* **set_memory_request_to_limit**
Set `memory_request` to `memory_limit` for all Postgres clusters (the default value is also increased). This prevents certain cases of memory overcommitment at the cost of overprovisioning memory and potential scheduling problems for containers with high memory limits due to the lack of memory on Kubernetes cluster nodes. This affects all containers created by the operator (Postgres, Scalyr sidecar, and other sidecars); to set resources for the operator's own container, change the [operator deployment manually](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L13). The default is `false`.
Set `memory_request` to `memory_limit` for all Postgres clusters (the default
value is also increased). This prevents certain cases of memory overcommitment
at the cost of overprovisioning memory and potential scheduling problems for
containers with high memory limits due to the lack of memory on Kubernetes
cluster nodes. This affects all containers created by the operator (Postgres,
Scalyr sidecar, and other sidecars); to set resources for the operator's own
container, change the [operator deployment manually](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L13).
The default is `false`.
* **enable_shm_volume**
Instruct operator to start any new database pod without limitations on shm
@@ -344,9 +376,10 @@ In the CRD-based configuration they are grouped under the `load_balancer` key.
## AWS or GCP interaction
The options in this group configure operator interactions with non-Kubernetes
objects from Amazon Web Services (AWS) or Google Cloud Platform (GCP). They have no effect unless you are using
either. In the CRD-based configuration those options are grouped under the
`aws_or_gcp` key. Note the GCP integration is not yet officially supported.
objects from Amazon Web Services (AWS) or Google Cloud Platform (GCP). They have
no effect unless you are using either. In the CRD-based configuration those
options are grouped under the `aws_or_gcp` key. Note the GCP integration is not
yet officially supported.
* **wal_s3_bucket**
S3 bucket to use for shipping WAL segments with WAL-E. A bucket has to be
@@ -355,7 +388,8 @@ either. In the CRD-based configuration those options are grouped under the
* **log_s3_bucket**
S3 bucket to use for shipping postgres daily logs. Works only with S3 on AWS.
The bucket has to be present and accessible by Postgres pods. The default is empty.
The bucket has to be present and accessible by Postgres pods. The default is
empty.
* **kube_iam_role**
AWS IAM role to supply in the `iam.amazonaws.com/role` annotation of Postgres
@@ -376,8 +410,8 @@ Options to aid debugging of the operator itself. Grouped under the `debug` key.
* **enable_database_access**
boolean parameter that toggles the functionality of the operator that require
access to the postgres database, i.e. creating databases and users. The default
is `true`.
access to the postgres database, i.e. creating databases and users. The
default is `true`.
## Automatic creation of human users in the database
@@ -414,7 +448,10 @@ key.
`admin`, that role is created by Spilo as a `NOLOGIN` role.
* **enable_admin_role_for_users**
if `true`, the `team_admin_role` will have the rights to grant roles coming from PG manifests. Such roles will be created as in "CREATE ROLE 'role_from_manifest' ... ADMIN 'team_admin_role'". The default is `true`.
if `true`, the `team_admin_role` will have the rights to grant roles coming
from PG manifests. Such roles will be created as in
"CREATE ROLE 'role_from_manifest' ... ADMIN 'team_admin_role'".
The default is `true`.
* **pam_role_name**
when set, the operator will add all team member roles to this group and add a
@@ -433,11 +470,14 @@ key.
infrastructure role. The default is `admin`.
* **postgres_superuser_teams**
List of teams which members need the superuser role in each PG database cluster to administer Postgres and maintain infrastructure built around it. The default is empty.
List of teams which members need the superuser role in each PG database
cluster to administer Postgres and maintain infrastructure built around it.
The default is empty.
## Logging and REST API
Parameters affecting logging and REST API listener. In the CRD-based configuration they are grouped under the `logging_rest_api` key.
Parameters affecting logging and REST API listener. In the CRD-based
configuration they are grouped under the `logging_rest_api` key.
* **api_port**
REST API listener listens to this port. The default is `8080`.
@@ -476,4 +516,22 @@ scalyr sidecar. In the CRD-based configuration they are grouped under the
Memory limit value for the Scalyr sidecar. The default is `1Gi`.
For the configmap operator configuration, the [default parameter values](https://github.com/zalando/postgres-operator/blob/master/pkg/util/config/config.go#L14) mentioned here are likely to be overwritten in your local operator installation via your local version of the operator configmap. In the case you use the operator CRD, all the CRD defaults are provided in the [operator's default configuration manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml)
## Logical backup
These parameters configure a k8s cron job managed by the operator to produce
Postgres logical backups. In the CRD-based configuration those parameters are
grouped under the `logical_backup` key.
* **logical_backup_schedule**
Backup schedule in the cron format. Please take [the reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) into account. Default: "30 00 \* \* \*"
* **logical_backup_docker_image**
An image for pods of the logical backup job. The [example image](../../docker/logical-backup/Dockerfile)
runs `pg_dumpall` on a replica if possible and uploads compressed results to
an S3 bucket under the key `/spilo/pg_cluster_name/cluster_k8s_uuid/logical_backups`.
The default image is the same image built with the Zalando-internal CI
pipeline. Default: "registry.opensource.zalan.do/acid/logical-backup"
* **logical_backup_s3_bucket**
S3 bucket to store backup results. The bucket has to be present and
accessible by Postgres pods. Default: empty.
docs/user.md
+15
View File
@@ -287,6 +287,13 @@ spec:
sidecars:
- name: "container-name"
image: "company/image:tag"
resources:
limits:
cpu: 500m
memory: 500Mi
requests:
cpu: 100m
memory: 100Mi
env:
- name: "ENV_VAR_NAME"
value: "any-k8s-env-things"
@@ -364,3 +371,11 @@ every 6 hours.
Note that if the statefulset is scaled down before resizing the size changes
are only applied to the volumes attached to the running pods. The size of the
volumes that correspond to the previously running pods is not changed.
## Logical backups
If you add
```
enableLogicalBackup: true
```
to the cluster manifest, the operator will create and sync a k8s cron job to do periodic logical backups of this particular Postgres cluster. Due to the [limitation of Kubernetes cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) it is highly advisable to set up additional monitoring for this feature; such monitoring is outside of the scope of operator responsibilities. See [configuration reference](reference/cluster_manifest.md) and [administrator documentation](administrator.md) for details on how backups are executed.