public_git
/
postgres-operator
mirror of https://github.com/zalando/postgres-operator.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Update docs for v1.2 (#609) 

* update docs and move parts from README to index.md
* fix typos, headings and code alignment in docs
This commit is contained in:
Felix Kunde 2019-07-11 17:19:27 +02:00 committed by GitHub
parent 1c340c68c6
commit 4fc5822b24
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 689 additions and 618 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
CONTRIBUTING.md
View File

@ -4,15 +4,19 @@ Wanna contribute to the Postgres Operator? Yay - here is how!
## Reporting issues ## Reporting issues
If you have a question about patroni or have a problem using it, please read the`README` before filing an issue. Before filing an issue, if you have a question about Postgres Operator or have
Also double check with the current issues on our [Issues Tracker](https://github.com/zalando/postgres-operator/issues). a problem using it, please read the [concepts](docs/index.md) page or use the
different guides that we provide for [users](docs/user.md),
[developers](docs/developer.md) or [admins](docs/administrator). Also double
check with the current issues on our [Issues Tracker](https://github.com/zalando/postgres-operator/issues).
## Contributing a pull request ## Contributing a pull request
1. Submit a comment to the relevant issue or create a new issue describing your proposed change. 1. Submit a comment to the relevant issue or create a new issue describing your
1. Do a fork, develop and test your code changes. proposed change.
1. Include documentation 2. Do a fork, develop and test your code changes.
1. Submit a pull request. 3. Include documentation
4. Submit a pull request.
You'll get feedback about your pull request as soon as possible. You'll get feedback about your pull request as soon as possible.

76
README.md
View File

@ -8,67 +8,37 @@
<img src="docs/diagrams/logo.png" width="200"> <img src="docs/diagrams/logo.png" width="200">
## Introduction to the Postgres Operator The Postgres Operator enables highly-available [PostgreSQL](https://www.postgresql.org/)
clusters on Kubernetes (K8s) powered by [Patroni](https://github.com/zalando/spilo).
It is configured only through manifests to ease integration into automated CI/CD
pipelines with no access to Kubernetes directly.
The Postgres [operator](https://coreos.com/blog/introducing-operators.html) The Postgres Operator has been developed at Zalando and is being used in
manages PostgreSQL clusters on Kubernetes: production for over two years.
1. The operator watches additions, updates, and deletions of PostgreSQL cluster ## Getting started
manifests and changes the running clusters accordingly. For example, when a
user submits a new manifest, the operator spawns a new Postgres cluster with
necessary entities such as StatefulSets, Services, and also Postgres roles. See this
[Postgres cluster manifest](manifests/complete-postgres-manifest.yaml)
for settings that a manifest may contain.
2. The operator also watches updates to [its own configuration](manifests/configmap.yaml) For a quick first impression follow the instructions of this
and alters running Postgres clusters if necessary. For instance, if a pod [tutorial](docs/quickstart.md).
Docker image is changed, the operator carries out the rolling update. That
is, the operator re-spawns one-by-one pods of each StatefulSet it manages
with the new Docker image.
3. Finally, the operator periodically synchronizes the actual state of each ## Documentation
Postgres cluster with the desired state defined in the cluster's manifest.
4. The operator aims to be hands free and configuration happens only via manifests and its own config. There is a browser-friendly version of this documentation at
This enables easy integration in automated deploy pipelines with no access to Kubernetes directly. [postgres-operator.readthedocs.io](https://postgres-operator.readthedocs.io)
# Getting started * [How it works](docs/index.md)
* [The Postgres experience on K8s](docs/user.md)
* [DBA options - from RBAC to backup](docs/administrator.md)
* [Debug and extend the operator](docs/developer.md)
* [Configuration options](docs/reference/operator_parameters.md)
* [Postgres manifest reference](docs/reference/cluster_manifest.md)
* [Command-line options and environment variables](docs/reference/command_line_and_environment.md)
For a quick first impression follow the instructions of [this](docs/quickstart.md) ## Google Summer of Code
tutorial.
# Google Summer of Code The Postgres Operator made it to the [Google Summer of Code 2019](https://summerofcode.withgoogle.com/organizations/5429926902104064/)!
Check [our ideas](docs/gsoc-2019/ideas.md#google-summer-of-code-2019)
The Postgres Operator made it to the [Google Summer of Code 2019](https://summerofcode.withgoogle.com/)! As a brand new mentoring organization, we are now looking for our first mentees. Check [our ideas](https://github.com/zalando/postgres-operator/blob/master/docs/gsoc-2019/ideas.md#google-summer-of-code-2019) and start discussion in [the issue tracker](https://github.com/zalando/postgres-operator/issues). And don't forget to spread a word about our GSoC participation to attract even more students. and start discussions in [the issue tracker](https://github.com/zalando/postgres-operator/issues).
## Table of contents
* [concepts](docs/index.md)
* [user documentation](docs/user.md)
* [administrator documentation](docs/administrator.md)
* [developer documentation](docs/developer.md)
* [operator configuration reference](docs/reference/operator_parameters.md)
* [cluster manifest reference](docs/reference/cluster_manifest.md)
* [command-line options and environment variables](docs/reference/command_line_and_environment.md)
The rest of this document is a tutorial to get you up and running locally with the operator on Minikube.
## Overview of involved entities
Here is a diagram, that summarizes what would be created by the operator, when a
new Postgres cluster CRD is submitted:
![postgresql-operator](docs/diagrams/operator.png "K8S resources, created by operator")
This picture is not complete without an overview of what is inside a single cluster pod, so
let's zoom in:
![pod](docs/diagrams/pod.png "Database pod components")
These two diagrams should help you to understand the basics of what kind of
functionality the operator provides.
There is a browser-friendly version of this documentation at [postgres-operator.readthedocs.io](https://postgres-operator.readthedocs.io)
## Community ## Community

4
charts/postgres-operator/Chart.yaml
View File

@ -3,7 +3,7 @@ name: postgres-operator
version: 1.2.0 version: 1.2.0
appVersion: 1.2.0 appVersion: 1.2.0
home: https://github.com/zalando/postgres-operator home: https://github.com/zalando/postgres-operator
description: Postgres operator creates and manages PostgreSQL clusters running in Kubernetes description: Postgres Operator creates and manages PostgreSQL clusters running in Kubernetes
keywords: keywords:
- postgres - postgres
- operator - operator
@ -16,5 +16,5 @@ maintainers:
- name: kimxogus - name: kimxogus
email: kgyoo8232@gmail.com email: kgyoo8232@gmail.com
sources: sources:
- https://github.com/zalando-incubator/postgres-operator - https://github.com/zalando/postgres-operator
engine: gotpl engine: gotpl

2
charts/postgres-operator/values-crd.yaml
View File

@ -243,7 +243,7 @@ serviceAccount:
create: true create: true
# The name of the ServiceAccount to use. # The name of the ServiceAccount to use.
# If not set and create is true, a name is generated using the fullname template # If not set and create is true, a name is generated using the fullname template
# When relying solely on the OperatorConfiguration CRD, set this value to "operator" # When relying solely on the OperatorConfiguration CRD, this value has to be "operator"
# Otherwise, the operator tries to use the "default" service account which is forbidden # Otherwise, the operator tries to use the "default" service account which is forbidden
name: operator name: operator

2
charts/postgres-operator/values.yaml
View File

@ -224,8 +224,6 @@ serviceAccount:
create: true create: true
# The name of the ServiceAccount to use. # The name of the ServiceAccount to use.
# If not set and create is true, a name is generated using the fullname template # If not set and create is true, a name is generated using the fullname template
# When relying solely on the OperatorConfiguration CRD, set this value to "operator"
# Otherwise, the operator tries to use the "default" service account which is forbidden
name: name:
priorityClassName: "" priorityClassName: ""

2
docker/Dockerfile
View File

@ -7,7 +7,7 @@ RUN apk --no-cache add ca-certificates
COPY build/* / COPY build/* /
RUN addgroup -g 1000 pgo RUN addgroup -g 1000 pgo
RUN adduser -D -u 1000 -G pgo -g 'Postgres operator' pgo RUN adduser -D -u 1000 -G pgo -g 'Postgres Operator' pgo
USER 1000:1000 USER 1000:1000

2
docker/logical-backup/dump.sh
View File

@ -35,7 +35,7 @@ function aws_upload {
# mimic bucket setup from Spilo # mimic bucket setup from Spilo
# to keep logical backups at the same path as WAL # to keep logical backups at the same path as WAL
# NB: $LOGICAL_BACKUP_S3_BUCKET_SCOPE_SUFFIX already contains the leading "/" when set by the Postgres operator # NB: $LOGICAL_BACKUP_S3_BUCKET_SCOPE_SUFFIX already contains the leading "/" when set by the Postgres Operator
PATH_TO_BACKUP=s3://$LOGICAL_BACKUP_S3_BUCKET"/spilo/"$SCOPE$LOGICAL_BACKUP_S3_BUCKET_SCOPE_SUFFIX"/logical_backups/"$(date +%s).sql.gz PATH_TO_BACKUP=s3://$LOGICAL_BACKUP_S3_BUCKET"/spilo/"$SCOPE$LOGICAL_BACKUP_S3_BUCKET_SCOPE_SUFFIX"/logical_backups/"$(date +%s).sql.gz
if [ -z "$EXPECTED_SIZE" ]; then if [ -z "$EXPECTED_SIZE" ]; then

236
docs/administrator.md
View File

@ -1,14 +1,19 @@
# How to configure PostgreSQL operator # Administrator Guide
## Select the namespace to deploy to Learn how to configure and manage the Postgres Operator in your Kubernetes (K8s)
environment.
## Namespaces
### Select the namespace to deploy to
The operator can run in a namespace other than `default`. For example, to use The operator can run in a namespace other than `default`. For example, to use
the `test` namespace, run the following before deploying the operator's the `test` namespace, run the following before deploying the operator's
manifests: manifests:
```bash ```bash
$ kubectl create namespace test kubectl create namespace test
$ kubectl config set-context $(kubectl config current-context) --namespace=test kubectl config set-context $(kubectl config current-context) --namespace=test
``` ```
All subsequent `kubectl` commands will work with the `test` namespace. The All subsequent `kubectl` commands will work with the `test` namespace. The
@ -17,18 +22,17 @@ ConfigMap - there. Please note that the namespace for service accounts and
cluster role bindings in [operator RBAC rules](../manifests/operator-service-account-rbac.yaml) cluster role bindings in [operator RBAC rules](../manifests/operator-service-account-rbac.yaml)
needs to be adjusted to the non-default value. needs to be adjusted to the non-default value.
## Specify the namespace to watch ### Specify the namespace to watch
Watching a namespace for an operator means tracking requests to change Watching a namespace for an operator means tracking requests to change Postgres
Postgresql clusters in the namespace such as "increase the number of Postgresql clusters in the namespace such as "increase the number of Postgres replicas to
replicas to 5" and reacting to the requests, in this example by actually 5" and reacting to the requests, in this example by actually scaling up.
scaling up.
By default, the operator watches the namespace it is deployed to. You can By default, the operator watches the namespace it is deployed to. You can
change this by setting the `WATCHED_NAMESPACE` var in the `env` section of the change this by setting the `WATCHED_NAMESPACE` var in the `env` section of the
[operator deployment](../manifests/postgres-operator.yaml) manifest or by [operator deployment](../manifests/postgres-operator.yaml) manifest or by
altering the `watched_namespace` field in the operator altering the `watched_namespace` field in the operator
[ConfigMap](../manifests/configmap.yaml#L6). [ConfigMap](../manifests/configmap.yaml#L79).
In the case both are set, the env var takes the precedence. To make the In the case both are set, the env var takes the precedence. To make the
operator listen to all namespaces, explicitly set the field/env var to "`*`". operator listen to all namespaces, explicitly set the field/env var to "`*`".
@ -36,14 +40,14 @@ Note that for an operator to manage pods in the watched namespace, the
operator's service account (as specified in the operator deployment manifest) operator's service account (as specified in the operator deployment manifest)
has to have appropriate privileges to access the watched namespace. The has to have appropriate privileges to access the watched namespace. The
operator may not be able to function in the case it watches all namespaces but operator may not be able to function in the case it watches all namespaces but
lacks access rights to any of them (except Kubernetes system namespaces like lacks access rights to any of them (except K8s system namespaces like
`kube-system`). The reason is that for multiple namespaces operations such as `kube-system`). The reason is that for multiple namespaces operations such as
'list pods' execute at the cluster scope and fail at the first violation of 'list pods' execute at the cluster scope and fail at the first violation of
access rights. access rights.
The watched namespace also needs to have a (possibly different) service account The watched namespace also needs to have a (possibly different) service account
in the case database pods need to talk to the Kubernetes API (e.g. when using in the case database pods need to talk to the K8s API (e.g. when using
Kubernetes-native configuration of Patroni). The operator checks that the K8s-native configuration of Patroni). The operator checks that the
`pod_service_account_name` exists in the target namespace, and, if not, deploys `pod_service_account_name` exists in the target namespace, and, if not, deploys
there the `pod_service_account_definition` from the operator there the `pod_service_account_definition` from the operator
[`Config`](../pkg/util/config/config.go) with the default value of: [`Config`](../pkg/util/config/config.go) with the default value of:
@ -61,52 +65,65 @@ namespace. The operator performs **no** further syncing of this account.
## Non-default cluster domain ## Non-default cluster domain
If your cluster uses a different dns domain than `cluster.local`, this needs If your cluster uses a DNS domain other than the default `cluster.local`, this
to be set in the operator ConfigMap. This is used by the operator to connect needs to be set in the operator configuration (`cluster_domain` variable). This
to the clusters after creation. is used by the operator to connect to the clusters after creation.
## Role-based access control for the operator ## Role-based access control for the operator
The `manifests/operator-service-account-rbac.yaml` defines cluster roles and ### Service account and cluster roles
bindings needed for the operator to function under access control restrictions.
To deploy the operator with this RBAC policy use: The manifest [`operator-service-account-rbac.yaml`](../manifests/operator-service-account-rbac.yaml)
defines the service account, cluster roles and bindings needed for the operator
to function under access control restrictions. To deploy the operator with this
RBAC policy use:
```bash ```bash
$ kubectl create -f manifests/configmap.yaml kubectl create -f manifests/configmap.yaml
$ kubectl create -f manifests/operator-service-account-rbac.yaml kubectl create -f manifests/operator-service-account-rbac.yaml
$ kubectl create -f manifests/postgres-operator.yaml kubectl create -f manifests/postgres-operator.yaml
$ kubectl create -f manifests/minimal-postgres-manifest.yaml kubectl create -f manifests/minimal-postgres-manifest.yaml
``` ```
Note that the service account in `operator-rbac.yaml` is named Note that the service account is named `zalando-postgres-operator`. You may have
`zalando-postgres-operator`. You may have to change the `service_account_name` to change the `service_account_name` in the operator ConfigMap and
in the operator ConfigMap and `serviceAccountName` in the postgres-operator `serviceAccountName` in the `postgres-operator` deployment appropriately. This
deployment appropriately. is done intentionally to avoid breaking those setups that already work with the
default `operator` account. In the future the operator should ideally be run
under the `zalando-postgres-operator` service account.
This is done intentionally to avoid breaking those setups that already work The service account defined in `operator-service-account-rbac.yaml` acquires
with the default `operator` account. In the future the operator should ideally some privileges not used by the operator (i.e. we only need `list` and `watch`
be run under the `zalando-postgres-operator` service account. on `configmaps` resources). This is also done intentionally to avoid breaking
The service account defined in `operator-rbac.yaml` acquires some privileges
not really used by the operator (i.e. we only need `list` and `watch` on
`configmaps` resources), this is also done intentionally to avoid breaking
things if someone decides to configure the same service account in the things if someone decides to configure the same service account in the
operator's ConfigMap to run postgres clusters. operator's ConfigMap to run Postgres clusters.
### Use taints and tolerations for dedicated PostgreSQL nodes ### Give K8S users access to create/list `postgresqls`
To ensure Postgres pods are running on nodes without any other application By default `postgresql` custom resources can only be listed and changed by
pods, you can use cluster admins. To allow read and/or write access to other human users apply
[taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) the `user-facing-clusterrole` manifest:
```bash
kubectl create -f manifests/user-facing-clusterroles.yaml
```
It creates zalando-postgres-operator:user:view, :edit and :admin clusterroles
that are aggregated into the K8s [default roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings).
## Use taints and tolerations for dedicated PostgreSQL nodes
To ensure Postgres pods are running on nodes without any other application pods,
you can use [taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
and configure the required toleration in the operator ConfigMap. and configure the required toleration in the operator ConfigMap.
As an example you can set following node taint: As an example you can set following node taint:
```bash ```bash
$ kubectl taint nodes <nodeName> postgres=:NoSchedule kubectl taint nodes <nodeName> postgres=:NoSchedule
``` ```
And configure the toleration for the PostgreSQL pods by adding following line And configure the toleration for the Postgres pods by adding following line
to the ConfigMap: to the ConfigMap:
```yaml ```yaml
@ -119,14 +136,18 @@ data:
... ...
``` ```
Note that the Kubernetes version 1.13 brings [taint-based eviction](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions) to the beta stage and enables it by default. Note that the K8s version 1.13 brings [taint-based eviction](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions)
Postgres pods by default receive tolerations for `unreachable` and `noExecute` taints with the timeout of `5m`. to the beta stage and enables it by default. Postgres pods by default receive
Depending on your setup, you may want to adjust these parameters to prevent master pods from being evicted by the Kubernetes runtime. tolerations for `unreachable` and `noExecute` taints with the timeout of `5m`.
To prevent eviction completely, specify the toleration by leaving out the `tolerationSeconds` value (similar to how Kubernetes' own DaemonSets are configured) Depending on your setup, you may want to adjust these parameters to prevent
master pods from being evicted by the K8s runtime. To prevent eviction
completely, specify the toleration by leaving out the `tolerationSeconds` value
(similar to how Kubernetes' own DaemonSets are configured)
### Enable pod anti affinity ## Enable pod anti affinity
To ensure Postgres pods are running on different topologies, you can use [pod anti affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) To ensure Postgres pods are running on different topologies, you can use
[pod anti affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/)
and configure the required topology in the operator ConfigMap. and configure the required topology in the operator ConfigMap.
Enable pod anti affinity by adding following line to the operator ConfigMap: Enable pod anti affinity by adding following line to the operator ConfigMap:
@ -140,9 +161,10 @@ data:
enable_pod_antiaffinity: "true" enable_pod_antiaffinity: "true"
``` ```
By default the topology key for the pod anti affinity is set to `kubernetes.io/hostname`, By default the topology key for the pod anti affinity is set to
you can set another topology key e.g. `failure-domain.beta.kubernetes.io/zone` by adding following line `kubernetes.io/hostname`, you can set another topology key e.g.
to the operator ConfigMap, see [built-in node labels](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#interlude-built-in-node-labels) for available topology keys: `failure-domain.beta.kubernetes.io/zone` by adding following line to the
operator ConfigMap, see [built-in node labels](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#interlude-built-in-node-labels) for available topology keys:
```yaml ```yaml
apiVersion: v1 apiVersion: v1
@ -154,7 +176,7 @@ data:
pod_antiaffinity_topology_key: "failure-domain.beta.kubernetes.io/zone" pod_antiaffinity_topology_key: "failure-domain.beta.kubernetes.io/zone"
``` ```
### Pod Disruption Budget ## Pod Disruption Budget
By default the operator uses a PodDisruptionBudget (PDB) to protect the cluster By default the operator uses a PodDisruptionBudget (PDB) to protect the cluster
from voluntarily disruptions and hence unwanted DB downtime. The `MinAvailable` from voluntarily disruptions and hence unwanted DB downtime. The `MinAvailable`
@ -168,14 +190,15 @@ The PDB is only relaxed in two scenarios:
The PDB is still in place having `MinAvailable` set to `0`. If enabled it will The PDB is still in place having `MinAvailable` set to `0`. If enabled it will
be automatically set to `1` on scale up. Disabling PDBs helps avoiding blocking be automatically set to `1` on scale up. Disabling PDBs helps avoiding blocking
Kubernetes upgrades in managed K8s environments at the cost of prolonged DB Kubernetes upgrades in managed K8s environments at the cost of prolonged DB
downtime. See PR #384 for the use case. downtime. See PR [#384](https://github.com/zalando/postgres-operator/pull/384)
for the use case.
### Add cluster-specific labels ## Add cluster-specific labels
In some cases, you might want to add `labels` that are specific to a given In some cases, you might want to add `labels` that are specific to a given
postgres cluster, in order to identify its child objects. Postgres cluster, in order to identify its child objects. The typical use case
The typical use case is to add labels that identifies the `Pods` created by the is to add labels that identifies the `Pods` created by the operator, in order
operator, in order to implement fine-controlled `NetworkPolicies`. to implement fine-controlled `NetworkPolicies`.
**OperatorConfiguration** **OperatorConfiguration**
@ -259,7 +282,7 @@ data:
This ConfigMap is then added as a source of environment variables to the This ConfigMap is then added as a source of environment variables to the
Postgres StatefulSet/pods. Postgres StatefulSet/pods.
## Limiting the number of instances in clusters with `min_instances` and `max_instances` ## Limiting the number of min and max instances in clusters
As a preventive measure, one can restrict the minimum and the maximum number of As a preventive measure, one can restrict the minimum and the maximum number of
instances permitted by each Postgres cluster managed by the operator. If either instances permitted by each Postgres cluster managed by the operator. If either
@ -269,9 +292,9 @@ either the min or the max boundary. For instance, of a cluster manifest has 1
instance and the `min_instances` is set to 3, the cluster will be created with 3 instance and the `min_instances` is set to 3, the cluster will be created with 3
instances. By default, both parameters are set to `-1`. instances. By default, both parameters are set to `-1`.
## Load balancers ## Load balancers and allowed IP ranges
For any Postgresql/Spilo cluster, the operator creates two separate Kubernetes For any Postgres/Spilo cluster, the operator creates two separate K8s
services: one for the master pod and one for replica pods. To expose these services: one for the master pod and one for replica pods. To expose these
services to an outer network, one can attach load balancers to them by setting services to an outer network, one can attach load balancers to them by setting
`enableMasterLoadBalancer` and/or `enableReplicaLoadBalancer` to `true` in the `enableMasterLoadBalancer` and/or `enableReplicaLoadBalancer` to `true` in the
@ -280,72 +303,109 @@ manifest, the operator configmap's settings `enable_master_load_balancer` and
`enable_replica_load_balancer` apply. Note that the operator settings affect `enable_replica_load_balancer` apply. Note that the operator settings affect
all Postgresql services running in all namespaces watched by the operator. all Postgresql services running in all namespaces watched by the operator.
To limit the range of IP adresses that can reach a load balancer, specify the To limit the range of IP addresses that can reach a load balancer, specify the
desired ranges in the `allowedSourceRanges` field (applies to both master and desired ranges in the `allowedSourceRanges` field (applies to both master and
replica load balancers). To prevent exposing load balancers to the entire replica load balancers). To prevent exposing load balancers to the entire
Internet, this field is set at cluster creation time to `127.0.0.1/32` unless Internet, this field is set at cluster creation time to `127.0.0.1/32` unless
overwritten explicitly. If you want to revoke all IP ranges from an existing overwritten explicitly. If you want to revoke all IP ranges from an existing
cluster, please set the `allowedSourceRanges` field to `127.0.0.1/32` or to an cluster, please set the `allowedSourceRanges` field to `127.0.0.1/32` or to an
empty sequence `[]`. Setting the field to `null` or omitting it entirely may empty sequence `[]`. Setting the field to `null` or omitting it entirely may
lead to Kubernetes removing this field from the manifest due to its lead to K8s removing this field from the manifest due to its
[handling of null fields](https://kubernetes.io/docs/concepts/overview/object-management-kubectl/declarative-config/#how-apply-calculates-differences-and-merges-changes). [handling of null fields](https://kubernetes.io/docs/concepts/overview/object-management-kubectl/declarative-config/#how-apply-calculates-differences-and-merges-changes).
Then the resultant manifest will not contain the necessary change, and the Then the resultant manifest will not contain the necessary change, and the
operator will respectively do noting with the existing source ranges. operator will respectively do noting with the existing source ranges.
## Running periodic 'autorepair' scans of Kubernetes objects ## Running periodic 'autorepair' scans of K8s objects
The Postgres operator periodically scans all Kubernetes objects belonging to The Postgres Operator periodically scans all K8s objects belonging to each
each cluster and repairs all discrepancies between them and the definitions cluster and repairs all discrepancies between them and the definitions generated
generated from the current cluster manifest. There are two types of scans: from the current cluster manifest. There are two types of scans:
* `sync scan`, running every `resync_period` seconds for every cluster * `sync scan`, running every `resync_period` seconds for every cluster
* `repair scan`, coming every `repair_period` only for those clusters that didn't * `repair scan`, coming every `repair_period` only for those clusters that
report success as a result of the last operation applied to them. didn't report success as a result of the last operation applied to them.
## Postgres roles supported by the operator ## Postgres roles supported by the operator
The operator is capable of maintaining roles of multiple kinds within a The operator is capable of maintaining roles of multiple kinds within a
Postgres database cluster: Postgres database cluster:
* **System roles** are roles necessary for the proper work of Postgres itself such as a replication role or the initial superuser role. The operator delegates creating such roles to Patroni and only establishes relevant secrets. * **System roles** are roles necessary for the proper work of Postgres itself
such as a replication role or the initial superuser role. The operator delegates
creating such roles to Patroni and only establishes relevant secrets.
* **Infrastructure roles** are roles for processes originating from external systems, e.g. monitoring robots. The operator creates such roles in all Postgres clusters it manages assuming that Kubernetes secrets with the relevant credentials exist beforehand. * **Infrastructure roles** are roles for processes originating from external
systems, e.g. monitoring robots. The operator creates such roles in all Postgres
clusters it manages, assuming that K8s secrets with the relevant
credentials exist beforehand.
* **Per-cluster robot users** are also roles for processes originating from external systems but defined for an individual Postgres cluster in its manifest. A typical example is a role for connections from an application that uses the database. * **Per-cluster robot users** are also roles for processes originating from
external systems but defined for an individual Postgres cluster in its manifest.
A typical example is a role for connections from an application that uses the
database.
* **Human users** originate from the Teams API that returns a list of the team members given a team id. The operator differentiates between (a) product teams that own a particular Postgres cluster and are granted admin rights to maintain it, and (b) Postgres superuser teams that get the superuser access to all Postgres databases running in a Kubernetes cluster for the purposes of maintaining and troubleshooting. * **Human users** originate from the Teams API that returns a list of the team
members given a team id. The operator differentiates between (a) product teams
that own a particular Postgres cluster and are granted admin rights to maintain
it, and (b) Postgres superuser teams that get the superuser access to all
Postgres databases running in a K8s cluster for the purposes of maintaining and
troubleshooting.
## Understanding rolling update of Spilo pods ## Understanding rolling update of Spilo pods
The operator logs reasons for a rolling update with the `info` level and a diff between the old and new StatefulSet specs with the `debug` level. To benefit from numerous escape characters in the latter log entry, view it in CLI with `echo -e`. Note that the resultant message will contain some noise because the `PodTemplate` used by the operator is yet to be updated with the default values used internally in Kubernetes. The operator logs reasons for a rolling update with the `info` level and a diff
between the old and new StatefulSet specs with the `debug` level. To benefit
from numerous escape characters in the latter log entry, view it in CLI with
`echo -e`. Note that the resultant message will contain some noise because the
`PodTemplate` used by the operator is yet to be updated with the default values
used internally in K8s.
## Logical backups ## Logical backups
The operator can manage k8s cron jobs to run logical backups of Postgres clusters. The cron job periodically spawns a batch job that runs a single pod. The backup script within this pod's container can connect to a DB for a logical backup. The operator updates cron jobs during Sync if the job schedule changes; the job name acts as the job identifier. These jobs are to be enabled for each indvidual Postgres cluster by setting `enableLogicalBackup: true` in its manifest. Notes: The operator can manage k8s cron jobs to run logical backups of Postgres
clusters. The cron job periodically spawns a batch job that runs a single pod.
The backup script within this pod's container can connect to a DB for a logical
backup. The operator updates cron jobs during Sync if the job schedule changes;
the job name acts as the job identifier. These jobs are to be enabled for each
individual Postgres cluster by setting `enableLogicalBackup: true` in its
manifest. Notes:
1. The [example image](../docker/logical-backup/Dockerfile) implements the backup via `pg_dumpall` and upload of compressed and encrypted results to an S3 bucket; the default image ``registry.opensource.zalan.do/acid/logical-backup`` is the same image built with the Zalando-internal CI pipeline. `pg_dumpall` requires a `superuser` access to a DB and runs on the replica when possible. 1. The [example image](../docker/logical-backup/Dockerfile) implements the
backup via `pg_dumpall` and upload of compressed and encrypted results to an S3
bucket; the default image ``registry.opensource.zalan.do/acid/logical-backup``
is the same image built with the Zalando-internal CI pipeline. `pg_dumpall`
requires a `superuser` access to a DB and runs on the replica when possible.
2. Due to the [limitation of Kubernetes cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) it is highly advisable to set up additional monitoring for this feature; such monitoring is outside of the scope of operator responsibilities. 2. Due to the [limitation of K8s cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations)
it is highly advisable to set up additional monitoring for this feature; such
monitoring is outside of the scope of operator responsibilities.
3. The operator does not remove old backups. 3. The operator does not remove old backups.
4. You may use your own image by overwriting the relevant field in the operator configuration. Any such image must ensure the logical backup is able to finish [in presence of pod restarts](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#handling-pod-and-container-failures) and [simultaneous invocations](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) of the backup cron job. 4. You may use your own image by overwriting the relevant field in the operator
configuration. Any such image must ensure the logical backup is able to finish
[in presence of pod restarts](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#handling-pod-and-container-failures)
and [simultaneous invocations](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations)
of the backup cron job.
5. For that feature to work, your RBAC policy must enable operations on the `cronjobs` resource from the `batch` API group for the operator service account. See [example RBAC](../manifests/operator-service-account-rbac.yaml) 5. For that feature to work, your RBAC policy must enable operations on the
`cronjobs` resource from the `batch` API group for the operator service account.
See [example RBAC](../manifests/operator-service-account-rbac.yaml)
## Access to cloud resources from clusters in non cloud environment ## Access to cloud resources from clusters in non-cloud environment
To access cloud resources like S3 from a cluster in a bare metal setup you can use To access cloud resources like S3 from a cluster on bare metal you can use
`additional_secret_mount` and `additional_secret_mount_path` config parameters. `additional_secret_mount` and `additional_secret_mount_path` configuration
With this you can provision cloud credentials to the containers in the pods of the StatefulSet. parameters. The cloud credentials will be provisioned in the Postgres containers
This works this way that it mounts a volume from the given secret in the pod and this can by mounting an additional volume from the given secret to database pods. They
then accessed in the container over the configured mount path. Via [Custum Pod Environment Variables](#custom-pod-environment-variables) can then be accessed over the configured mount path. Via
you can then point the different cloud sdk's (aws, google etc.) to this mounted secret. [Custom Pod Environment Variables](#custom-pod-environment-variables) you can
With this credentials the cloud sdk can then access cloud resources to upload logs etc. point different cloud SDK's (AWS, GCP etc.) to this mounted secret, e.g. to
access cloud resources for uploading logs etc.
A secret can be pre provisioned in different ways: A secret can be pre-provisioned in different ways:
* Generic secret created via `kubectl create secret generic some-cloud-creds --from-file=some-cloud-credentials-file.json` * Generic secret created via `kubectl create secret generic some-cloud-creds --from-file=some-cloud-credentials-file.json`
* Automatically provisioned via a custom K8s controller like
* Automaticly provisioned via a Controller like [kube-aws-iam-controller](https://github.com/mikkeloscar/kube-aws-iam-controller). This controller would then also rotate the credentials. Please visit the documention for more information. [kube-aws-iam-controller](https://github.com/mikkeloscar/kube-aws-iam-controller)

295
docs/developer.md
View File

@ -1,128 +1,28 @@
## Intro # Developer Guide
See [minikube installation guide](https://github.com/kubernetes/minikube/releases) Read this guide if you want to debug the operator, fix bugs or contribute new
features and tests.
Make sure you use the latest version of Minikube.
After the installation, issue
```bash
$ minikube start
```
Note: if you are running on a Mac, you may also use Docker for Mac Kubernetes
instead of a docker-machine.
Once you have it started successfully, use [the quickstart
guide](https://github.com/kubernetes/minikube#quickstart) in order to test your
that your setup is working.
Note: if you use multiple Kubernetes clusters, you can switch to Minikube with
`kubectl config use-context minikube`
## Deploying the operator
### Kubernetes manifest
A ConfigMap is used to store the configuration of the operator. Alternatively,
a CRD-based configuration can be used, as described [here](reference/operator_parameters).
```bash
$ kubectl --context minikube create -f manifests/configmap.yaml
```
First you need to install the service account definition in your Minikube cluster.
```bash
$ kubectl --context minikube create -f manifests/operator-service-account-rbac.yaml
```
Next deploy the postgres-operator from the docker image Zalando is using:
```bash
$ kubectl --context minikube create -f manifests/postgres-operator.yaml
```
If you prefer to build the image yourself follow up down below.
### Helm chart
Alternatively, the operator can be installed by using the provided [Helm](https://helm.sh/)
chart which saves you the manual steps. Therefore, you would need to install
the helm CLI on your machine. After initializing helm (and its server
component Tiller) in your local cluster you can install the operator chart.
You can define a release name that is prepended to the operator resource's
names.
Use `--name zalando` to match with the default service account name as older
operator versions do not support custom names for service accounts. When relying
solely on the CRD-based configuration edit the `serviceAccount` section in the
[values yaml file](../charts/values.yaml) by setting the name to `"operator"`.
```bash
$ helm init
$ helm install --name zalando ./charts/postgres-operator
```
## Check if CustomResourceDefinition has been registered
```bash
$ kubectl --context minikube get crd
NAME KIND
postgresqls.acid.zalan.do CustomResourceDefinition.v1beta1.apiextensions.k8s.io
```
## Create a new Spilo cluster
```bash
$ kubectl --context minikube create -f manifests/minimal-postgres-manifest.yaml
```
## Watch pods being created
```bash
$ kubectl --context minikube get pods -w --show-labels
```
## Connect to PostgreSQL
We can use the generated secret of the `postgres` robot user to connect to our
`acid-minimal-cluster` master running in Minikube:
```bash
$ export HOST_PORT=$(minikube service acid-minimal-cluster --url | sed 's,.*/,,')
$ export PGHOST=$(echo $HOST_PORT | cut -d: -f 1)
$ export PGPORT=$(echo $HOST_PORT | cut -d: -f 2)
$ export PGPASSWORD=$(kubectl --context minikube get secret postgres.acid-minimal-cluster.credentials -o 'jsonpath={.data.password}' | base64 -d)
$ psql -U postgres
```
# Setup development environment
The following steps guide you through the setup to work on the operator itself.
## Setting up Go ## Setting up Go
Postgres operator is written in Go. Use the [installation Postgres Operator is written in Go. Use the [installation instructions](https://golang.org/doc/install#install)
instructions](https://golang.org/doc/install#install) if you don't have Go on if you don't have Go on your system. You won't be able to compile the operator
your system. You won't be able to compile the operator with Go older than 1.7. with Go older than 1.7. We recommend installing [the latest one](https://golang.org/dl/).
We recommend installing [the latest one](https://golang.org/dl/).
Go projects expect their source code and all the dependencies to be located Go projects expect their source code and all the dependencies to be located
under the [GOPATH](https://github.com/golang/go/wiki/GOPATH). Normally, one under the [GOPATH](https://github.com/golang/go/wiki/GOPATH). Normally, one
would create a directory for the GOPATH (i.e. ~/go) and place the source code would create a directory for the GOPATH (i.e. ~/go) and place the source code
under the ~/go/src subdirectories. under the ~/go/src sub directories.
Given the schema above, the postgres operator source code located at Given the schema above, the Postgres Operator source code located at
`github.com/zalando/postgres-operator` should be put at `github.com/zalando/postgres-operator` should be put at
-`~/go/src/github.com/zalando/postgres-operator`. -`~/go/src/github.com/zalando/postgres-operator`.
```bash ```bash
$ export GOPATH=~/go export GOPATH=~/go
$ mkdir -p ${GOPATH}/src/github.com/zalando/ mkdir -p ${GOPATH}/src/github.com/zalando/
$ cd ${GOPATH}/src/github.com/zalando/ cd ${GOPATH}/src/github.com/zalando/
$ git clone https://github.com/zalando/postgres-operator.git git clone https://github.com/zalando/postgres-operator.git
``` ```
## Building the operator ## Building the operator
@ -130,55 +30,74 @@ Given the schema above, the postgres operator source code located at
You need Glide to fetch all dependencies. Install it with: You need Glide to fetch all dependencies. Install it with:
```bash ```bash
$ make tools make tools
``` ```
Next, install dependencies with glide by issuing: Next, install dependencies with glide by issuing:
```bash ```bash
$ make deps make deps
``` ```
This would take a while to complete. You have to redo `make deps` every time This would take a while to complete. You have to redo `make deps` every time
you dependencies list changes, i.e. after adding a new library dependency. you dependencies list changes, i.e. after adding a new library dependency.
Build the operator docker image and pushing it to Pier One: Build the operator with the `make docker` command. You may define the TAG
variable to assign an explicit tag to your docker image and the IMAGE to set
the image name. By default, the tag is computed with
`git describe --tags --always --dirty` and the image is
`registry.opensource.zalan.do/acid/postgres-operator`
```bash ```bash
$ make docker push export TAG=$(git describe --tags --always --dirty)
make docker
``` ```
You may define the TAG variable to assign an explicit tag to your docker image
and the IMAGE to set the image name. By default, the tag is computed with
`git describe --tags --always --dirty` and the image is
`pierone.stups.zalan.do/acid/postgres-operator`
Building the operator binary (for testing the out-of-cluster option): Building the operator binary (for testing the out-of-cluster option):
```bash ```bash
$ make make
``` ```
The binary will be placed into the build directory. The binary will be placed into the build directory.
## Deploying self build image ## Deploying self build image
The fastest way to run your docker image locally is to reuse the docker from The fastest way to run and test your docker image locally is to reuse the docker
minikube. The following steps will get you the docker image built and deployed. from [minikube](https://github.com/kubernetes/minikube/releases) or use the
`load docker-image` from [kind](https://kind.sigs.k8s.io/). The following steps
will get you the docker image built and deployed.
```bash ```bash
$ eval $(minikube docker-env) # minikube
$ export TAG=$(git describe --tags --always --dirty) eval $(minikube docker-env)
$ make docker make docker
$ sed -e "s/\(image\:.*\:\).*$/\1$TAG/" manifests/postgres-operator.yaml|kubectl --context minikube create -f -
# kind
make docker
kind load docker-image <image> --name <kind-cluster-name>
``` ```
# Code generation Then create a new Postgres Operator deployment. You can reuse the provided
manifest but replace the version and tag. Don't forget to also apply
configuration and RBAC manifests first, e.g.:
The operator employs k8s-provided code generation to obtain deep copy methods ```bash
and Kubernetes-like APIs for its custom resource definitons, namely the Postgres kubectl create -f manifests/configmap.yaml
CRD and the operator CRD. The usage of the code generation follows conventions kubectl create -f manifests/operator-service-account-rbac.yaml
from the k8s community. Relevant scripts live in the `hack` directory: sed -e "s/\(image\:.*\:\).*$/\1$TAG/" manifests/postgres-operator.yaml | kubectl create -f -
# check if the operator is coming up
kubectl get pod -l name=postgres-operator
```
## Code generation
The operator employs K8s-provided code generation to obtain deep copy methods
and K8s-like APIs for its custom resource definitions, namely the
Postgres CRD and the operator CRD. The usage of the code generation follows
conventions from the k8s community. Relevant scripts live in the `hack`
directory:
* `update-codegen.sh` triggers code generation for the APIs defined in `pkg/apis/acid.zalan.do/`, * `update-codegen.sh` triggers code generation for the APIs defined in `pkg/apis/acid.zalan.do/`,
* `verify-codegen.sh` checks if the generated code is up-to-date (to be used within CI). * `verify-codegen.sh` checks if the generated code is up-to-date (to be used within CI).
@ -199,18 +118,22 @@ operator itself with `--v=8` to log all REST API requests). To attach a debugger
to the operator, use the `-outofcluster` option to run the operator locally on to the operator, use the `-outofcluster` option to run the operator locally on
the developer's laptop (and not in a docker container). the developer's laptop (and not in a docker container).
# Debugging the operator ## Debugging the operator
There is a web interface in the operator to observe its internal state. The There is a web interface in the operator to observe its internal state. The
operator listens on port 8080. It is possible to expose it to the operator listens on port 8080. It is possible to expose it to the
localhost:8080 by doing: `localhost:8080` by doing:
$ kubectl --context minikube port-forward $(kubectl --context minikube get pod -l name=postgres-operator -o jsonpath={.items..metadata.name}) 8080:8080 ```bash
kubectl --context minikube port-forward $(kubectl --context minikube get pod -l name=postgres-operator -o jsonpath={.items..metadata.name}) 8080:8080
```
The inner 'query' gets the name of the postgres operator pod, and the outer The inner query gets the name of the Postgres Operator pod, and the outer one
enables port forwarding. Afterwards, you can access the operator API with: enables port forwarding. Afterwards, you can access the operator API with:
$ curl --location http://127.0.0.1:8080/$endpoint | jq . ```bash
curl --location http://127.0.0.1:8080/$endpoint | jq .
```
The available endpoints are listed below. Note that the worker ID is an integer The available endpoints are listed below. Note that the worker ID is an integer
from 0 up to 'workers' - 1 (value configured in the operator configuration and from 0 up to 'workers' - 1 (value configured in the operator configuration and
@ -242,14 +165,13 @@ The operator also supports pprof endpoints listed at the
* /debug/pprof/trace * /debug/pprof/trace
It's possible to attach a debugger to troubleshoot postgres-operator inside a It's possible to attach a debugger to troubleshoot postgres-operator inside a
docker container. It's possible with gdb and docker container. It's possible with [gdb](https://www.gnu.org/software/gdb/)
[delve](https://github.com/derekparker/delve). Since the latter one is a and [delve](https://github.com/derekparker/delve). Since the latter one is a
specialized debugger for golang, we will use it as an example. To use it you specialized debugger for Go, we will use it as an example. To use it you need:
need:
* Install delve locally * Install delve locally
``` ```bash
go get -u github.com/derekparker/delve/cmd/dlv go get -u github.com/derekparker/delve/cmd/dlv
``` ```
@ -261,7 +183,8 @@ RUN go get github.com/derekparker/delve/cmd/dlv
``` ```
* Update the `Makefile` to build the project with debugging symbols. For that * Update the `Makefile` to build the project with debugging symbols. For that
you need to add `gcflags` to a build target for corresponding OS (e.g. linux) you need to add `gcflags` to a build target for corresponding OS (e.g.
GNU/Linux)
``` ```
-gcflags "-N -l" -gcflags "-N -l"
@ -276,61 +199,75 @@ CMD ["/root/go/bin/dlv", "--listen=:DLV_PORT", "--headless=true", "--api-version
* Forward the listening port * Forward the listening port
``` ```bash
kubectl port-forward POD_NAME DLV_PORT:DLV_PORT kubectl port-forward POD_NAME DLV_PORT:DLV_PORT
``` ```
* Attach to it * Attach to it
``` ```bash
$ dlv connect 127.0.0.1:DLV_PORT dlv connect 127.0.0.1:DLV_PORT
``` ```
## Unit tests ## Unit tests
To run all unit tests, you can simply do: To run all unit tests, you can simply do:
``` ```bash
$ go test ./... go test ./...
``` ```
For go 1.9 `vendor` directory would be excluded automatically. For previous For go 1.9 `vendor` directory would be excluded automatically. For previous
versions you can exclude it manually: versions you can exclude it manually:
``` ```bash
$ go test $(glide novendor) go test $(glide novendor)
``` ```
In case if you need to debug your unit test, it's possible to use delve: In case if you need to debug your unit test, it's possible to use delve:
``` ```bash
$ dlv test ./pkg/util/retryutil/ dlv test ./pkg/util/retryutil/
Type 'help' for list of commands. Type 'help' for list of commands.
(dlv) c (dlv) c
PASS PASS
``` ```
To test the multinamespace setup, you can use To test the multi-namespace setup, you can use
``` ```bash
./run_operator_locally.sh --rebuild-operator ./run_operator_locally.sh --rebuild-operator
``` ```
It will automatically create an `acid-minimal-cluster` in the namespace `test`. It will automatically create an `acid-minimal-cluster` in the namespace `test`.
Then you can for example check the Patroni logs: Then you can for example check the Patroni logs:
``` ```bash
kubectl logs acid-minimal-cluster-0 kubectl logs acid-minimal-cluster-0
``` ```
## End-to-end tests ## End-to-end tests
The operator provides reference e2e (end-to-end) tests to ensure various infra parts work smoothly together. The operator provides reference e2e (end-to-end) tests to ensure various infra
Each e2e execution tests a Postgres operator image built from the current git branch. The test runner starts a [kind](https://kind.sigs.k8s.io/) (local k8s) cluster and Docker container with tests. The k8s API client from within the container connects to the `kind` cluster using the standard Docker `bridge` network. parts work smoothly together. Each e2e execution tests a Postgres Operator image
The tests utilize examples from `/manifests` (ConfigMap is used for the operator configuration) to avoid maintaining yet another set of configuration files. The kind cluster is deleted if tests complete successfully. built from the current git branch. The test runner starts a [kind](https://kind.sigs.k8s.io/)
(local k8s) cluster and Docker container with tests. The k8s API client from
within the container connects to the `kind` cluster using the standard Docker
`bridge` network. The tests utilize examples from `/manifests` (ConfigMap is
used for the operator configuration) to avoid maintaining yet another set of
configuration files. The kind cluster is deleted if tests complete successfully.
End-to-end tests are executed automatically during builds; to invoke them locally use `make e2e-run` from the project's top directory. Run `make e2e-tools e2e-build` to install `kind` and build the tests' image locally before the first run. End-to-end tests are executed automatically during builds:
End-to-end tests are written in Python and use `flake8` for code quality. Please run flake8 [before submitting a PR](http://flake8.pycqa.org/en/latest/user/using-hooks.html). ```bash
# invoke them from the project's top directory
make e2e-run
# install kind and build test image before first run
make e2e-tools e2e-build
```
End-to-end tests are written in Python and use `flake8` for code quality.
Please run flake8 [before submitting a PR](http://flake8.pycqa.org/en/latest/user/using-hooks.html).
## Introduce additional configuration parameters ## Introduce additional configuration parameters
@ -340,17 +277,17 @@ be updated. As explained [here](reference/operator_parameters.md), it's possible
to configure the operator either with a ConfigMap or CRD, but currently we aim to configure the operator either with a ConfigMap or CRD, but currently we aim
to synchronize parameters everywhere. to synchronize parameters everywhere.
When choosing a parameter name for a new option in a PG manifest, keep in mind When choosing a parameter name for a new option in a Postgres cluster manifest,
the naming conventions there. The `snake_case` variables come from the Patroni/Postgres world, while the `camelCase` from the k8s world. keep in mind the naming conventions there. We use `camelCase` for manifest
parameters (with exceptions for certain Patroni/Postgres options) and
`snake_case` variables in the configuration. Only introduce new manifest
variables if you feel a per-cluster configuration is necessary.
Note: If one option is defined in the operator configuration and in the cluster Note: If one option is defined in the operator configuration and in the cluster
[manifest](../manifests/complete-postgres-manifest.yaml), the latter takes [manifest](../manifests/complete-postgres-manifest.yaml), the latter takes
precedence. precedence.
So, first define the parameters in: ### Go code
* the [ConfigMap](../manifests/configmap.yaml) manifest
* the CR's [default configuration](../manifests/postgresql-operator-default-configuration.yaml)
* the Helm chart [values](../charts/postgres-operator/values.yaml)
Update the following Go files that obtain the configuration parameter from the Update the following Go files that obtain the configuration parameter from the
manifest files: manifest files:
@ -358,12 +295,30 @@ manifest files:
* [operator_config.go](../pkg/controller/operator_config.go) * [operator_config.go](../pkg/controller/operator_config.go)
* [config.go](../pkg/util/config/config.go) * [config.go](../pkg/util/config/config.go)
Postgres manifest parameters are defined in the [api package](../pkg/apis/acid.zalan.do/v1/postgresql_type.go).
The operator behavior has to be implemented at least in [k8sres.go](../pkg/cluster/k8sres.go). The operator behavior has to be implemented at least in [k8sres.go](../pkg/cluster/k8sres.go).
Please, reflect your changes in tests, for example in: Please, reflect your changes in tests, for example in:
* [config_test.go](../pkg/util/config/config_test.go) * [config_test.go](../pkg/util/config/config_test.go)
* [k8sres_test.go](../pkg/cluster/k8sres_test.go) * [k8sres_test.go](../pkg/cluster/k8sres_test.go)
* [util_test.go](../pkg/apis/acid.zalan.do/v1/util_test.go) * [util_test.go](../pkg/apis/acid.zalan.do/v1/util_test.go)
Finally, document the new configuration option(s) for the operator in its ### Updating manifest files
[reference](reference/operator_parameters.md) document and explain the feature
in the [administrator docs](administrator.md). For the CRD-based configuration, please update the following files:
* the default [OperatorConfiguration](../manifests/postgresql-operator-default-configuration.yaml)
* the Helm chart's [values-crd file](../charts/postgres-operator/values.yaml)
Reflect the changes in the ConfigMap configuration as well (note that numeric
and boolean parameters have to use double quotes here):
* [ConfigMap](../manifests/configmap.yaml) manifest
* the Helm chart's default [values file](../charts/postgres-operator/values.yaml)
### Updating documentation
Finally, add a section for each new configuration option and/or cluster manifest
parameter in the reference documents:
* [config reference](reference/operator_parameters.md)
* [manifest reference](reference/cluster_manifest.md)
It also helps users to explain new features with examples in the
[administrator docs](administrator.md).

59
docs/index.md
View File

@ -1,49 +1,68 @@
# Introduction # Concepts
The Postgres [operator](https://coreos.com/blog/introducing-operators.html) The Postgres [operator](https://coreos.com/blog/introducing-operators.html)
manages PostgreSQL clusters on Kubernetes: manages PostgreSQL clusters on Kubernetes (K8s):
1. The operator watches additions, updates, and deletions of PostgreSQL cluster 1. The operator watches additions, updates, and deletions of PostgreSQL cluster
manifests and changes the running clusters accordingly. For example, when a manifests and changes the running clusters accordingly. For example, when a
user submits a new manifest, the operator fetches that manifest and spawns a user submits a new manifest, the operator fetches that manifest and spawns a
new Postgres cluster along with all necessary entities such as Kubernetes new Postgres cluster along with all necessary entities such as K8s
StatefulSets and Postgres roles. See this StatefulSets and Postgres roles. See this
[Postgres cluster manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/complete-postgres-manifest.yaml) [Postgres cluster manifest](../manifests/complete-postgres-manifest.yaml)
for settings that a manifest may contain. for settings that a manifest may contain.
2. The operator also watches updates to [its own configuration](https://github.com/zalando/postgres-operator/blob/master/manifests/configmap.yaml) 2. The operator also watches updates to [its own configuration](../manifests/configmap.yaml)
and alters running Postgres clusters if necessary. For instance, if a pod and alters running Postgres clusters if necessary. For instance, if the
docker image is changed, the operator carries out the rolling update. That docker image in a pod is changed, the operator carries out the rolling
is, the operator re-spawns one-by-one pods of each StatefulSet it manages update, which means it re-spawns pods of each managed StatefulSet one-by-one
with the new Docker image. with the new Docker image.
3. Finally, the operator periodically synchronizes the actual state of each 3. Finally, the operator periodically synchronizes the actual state of each
Postgres cluster with the desired state defined in the cluster's manifest. Postgres cluster with the desired state defined in the cluster's manifest.
## Concepts 4. The operator aims to be hands free as configuration works only via manifests.
This enables easy integration in automated deploy pipelines with no access to
K8s directly.
### Scope ## Scope
The scope of the postgres operator is on provisioning, modifying configuration The scope of the Postgres Operator is on provisioning, modifying configuration
and cleaning up Postgres clusters that use Patroni, basically to make it easy and cleaning up Postgres clusters that use Patroni, basically to make it easy
and convenient to run Patroni based clusters on Kubernetes. The provisioning and convenient to run Patroni based clusters on K8s. The provisioning
and modifying includes Kubernetes resources on one side but also e.g. database and modifying includes K8s resources on one side but also e.g. database
and role provisioning once the cluster is up and running. We try to leave as and role provisioning once the cluster is up and running. We try to leave as
much work as possible to Kubernetes and to Patroni where it fits, especially much work as possible to K8s and to Patroni where it fits, especially
the cluster bootstrap and high availability. The operator is however involved the cluster bootstrap and high availability. The operator is however involved
in some overarching orchestration, like rolling updates to improve the user in some overarching orchestration, like rolling updates to improve the user
experience. experience.
Monitoring of clusters is not in scope, for this good tools already exist from Monitoring or tuning Postgres is not in scope of the operator in the current
ZMON to Prometheus and more Postgres specific options. state. Other tools like [ZMON](https://opensource.zalando.com/zmon/),
[Prometheus](https://prometheus.io/) or more Postgres specific options can be
used to complement it.
### Status ## Overview of involved entities
Here is a diagram, that summarizes what would be created by the operator, when a
new Postgres cluster CRD is submitted:
![postgresql-operator](diagrams/operator.png "K8S resources, created by operator")
This picture is not complete without an overview of what is inside a single
cluster pod, so let's zoom in:
![pod](diagrams/pod.png "Database pod components")
These two diagrams should help you to understand the basics of what kind of
functionality the operator provides.
## Status
This project is currently in active development. It is however already This project is currently in active development. It is however already
[used internally by Zalando](https://jobs.zalando.com/tech/blog/postgresql-in-a-time-of-kubernetes/) [used internally by Zalando](https://jobs.zalando.com/tech/blog/postgresql-in-a-time-of-kubernetes/)
in order to run Postgres clusters on Kubernetes in larger numbers for staging in order to run Postgres clusters on K8s in larger numbers for staging
environments and a growing number of production clusters. In this environment environments and a growing number of production clusters. In this environment
the operator is deployed to multiple Kubernetes clusters, where users deploy the operator is deployed to multiple K8s clusters, where users deploy
manifests via our CI/CD infrastructure or rely on a slim user interface to manifests via our CI/CD infrastructure or rely on a slim user interface to
create manifests. create manifests.
@ -59,4 +78,4 @@ Please, report any issues discovered to https://github.com/zalando/postgres-oper
4. "Blue elephant on-demand: Postgres + Kubernetes" talk by Oleksii Kliukin and Jan Mussler, FOSDEM 2018: [video](https://fosdem.org/2018/schedule/event/blue_elephant_on_demand_postgres_kubernetes/) | [slides (pdf)](https://www.postgresql.eu/events/fosdem2018/sessions/session/1735/slides/59/FOSDEM%202018_%20Blue_Elephant_On_Demand.pdf) 4. "Blue elephant on-demand: Postgres + Kubernetes" talk by Oleksii Kliukin and Jan Mussler, FOSDEM 2018: [video](https://fosdem.org/2018/schedule/event/blue_elephant_on_demand_postgres_kubernetes/) | [slides (pdf)](https://www.postgresql.eu/events/fosdem2018/sessions/session/1735/slides/59/FOSDEM%202018_%20Blue_Elephant_On_Demand.pdf)
3. "Kube-Native Postgres" talk by Josh Berkus, KubeCon 2017: [video](https://www.youtube.com/watch?v=Zn1vd7sQ_bc) 5. "Kube-Native Postgres" talk by Josh Berkus, KubeCon 2017: [video](https://www.youtube.com/watch?v=Zn1vd7sQ_bc)

150
docs/quickstart.md
View File

@ -1,54 +1,82 @@
## Prerequisites: # Quickstart
In order to run the Postgres Operator locally in minikube you need to install This guide aims to give you a quick look and feel for using the Postgres
the following tools: Operator on a local Kubernetes environment.
* [minikube](https://github.com/kubernetes/minikube/releases) ## Prerequisites
* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-via-curl)
Note that you can also use built-in Kubernetes support in the Docker Desktop Since the Postgres Operator is designed for the Kubernetes (K8s) framework,
hence set it up first. For local tests we recommend to use one of the following
solutions:
* [minikube](https://github.com/kubernetes/minikube/releases), which creates a
single-node K8s cluster inside a VM (requires KVM or VirtualBox),
* [kind](https://kind.sigs.k8s.io/), which allows creating multi-nodes K8s
clusters running on Docker (requires Docker)
To interact with the K8s infrastructure install it's CLI runtime [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-via-curl).
This quickstart assumes that you have started minikube or created a local kind
cluster. Note that you can also use built-in K8s support in the Docker Desktop
for Mac to follow the steps of this tutorial. You would have to replace for Mac to follow the steps of this tutorial. You would have to replace
`minikube start` and `minikube delete` with your launch actions for the Docker `minikube start` and `minikube delete` with your launch actions for the Docker
built-in Kubernetes support. built-in K8s support.
Clone the repository and change to the directory. Then start minikube. ## Configuration Options
Configuring the Postgres Operator is only possible before deploying a new
Postgres cluster. This can work in two ways: via a ConfigMap or a custom
`OperatorConfiguration` object. More details on configuration can be found
[here](reference/operator_parameters.md).
## Deployment options
The Postgres Operator can be deployed in the following ways:
* Manual deployment
* Helm chart
* Operator Lifecycle Manager (OLM)
### Manual deployment setup
The Postgres Operator can be installed simply by applying yaml manifests. Note,
we provide the `/manifests` directory as an example only; you should consider
adjusting the manifests to your K8s environment (e.g. namespaces).
```bash ```bash
# First, clone the repository and change to the directory
git clone https://github.com/zalando/postgres-operator.git git clone https://github.com/zalando/postgres-operator.git
cd postgres-operator cd postgres-operator
minikube start # apply the manifests in the following order
```
If you want to configure the Postgres Operator it must happen before deploying a
Postgres cluster. This can happen in two ways: Via a ConfigMap or a
`OperatorConfiguration` object, which adheres a CustomResourceDefinition (CRD).
More details on configuration can be found [here](reference/operator_parameters.md).
## Manual deployment setup
The Postgres Operator can be installed simply by applying yaml manifests.
```bash
kubectl create -f manifests/configmap.yaml # configuration kubectl create -f manifests/configmap.yaml # configuration
kubectl create -f manifests/operator-service-account-rbac.yaml # identity and permissions kubectl create -f manifests/operator-service-account-rbac.yaml # identity and permissions
kubectl create -f manifests/postgres-operator.yaml # deployment kubectl create -f manifests/postgres-operator.yaml # deployment
``` ```
## Helm chart When using kubectl 1.14 or newer the mentioned manifests could be also be
bundled in one [Kustomization](https://github.com/kubernetes-sigs/kustomize)
manifest.
For convenience, we have automated starting the operator with minikube using the
`run_operator_locally` script. It applies the [`acid-minimal-cluster`](../manifests/minimal-postgres-manifest).
manifest.
```bash
./run_operator_locally.sh
```
### Helm chart
Alternatively, the operator can be installed by using the provided [Helm](https://helm.sh/) Alternatively, the operator can be installed by using the provided [Helm](https://helm.sh/)
chart which saves you the manual steps. Therefore, you would need to install chart which saves you the manual steps. Therefore, install the helm CLI on your
the helm CLI on your machine. After initializing helm (and its server machine. After initializing helm (and its server component Tiller) in your local
component Tiller) in your local cluster you can install the operator chart. cluster you can install the operator chart. You can define a release name that
You can define a release name that is prepended to the operator resource's is prepended to the operator resource's names.
names.
Use `--name zalando` to match with the default service account name as older Use `--name zalando` to match with the default service account name as older
operator versions do not support custom names for service accounts. When relying operator versions do not support custom names for service accounts. To use
solely on the CRD-based configuration edit the `serviceAccount` section in the CRD-based configuration you need to specify the [values-crd yaml file](../charts/values-crd.yaml).
[values yaml file](../charts/values.yaml) by setting the name to `"operator"`.
```bash ```bash
# 1) initialize helm # 1) initialize helm
@ -57,6 +85,20 @@ helm init
helm install --name zalando ./charts/postgres-operator helm install --name zalando ./charts/postgres-operator
``` ```
### Operator Lifecycle Manager (OLM)
The [Operator Lifecycle Manager (OLM)](https://github.com/operator-framework/operator-lifecycle-manager)
has been designed to facilitate management of K8s operators. It has to be
installed in your K8s environment. When OLM is set up simply download and deploy
the Postgres Operator with the following command:
```bash
kubectl create -f https://operatorhub.io/install/postgres-operator.yaml
```
This installs the operator in the `operators` namespace. More information can be
found on [operatorhub.io](https://operatorhub.io/operator/postgres-operator).
## Create a Postgres cluster ## Create a Postgres cluster
Starting the operator may take a few seconds. Check if the operator pod is Starting the operator may take a few seconds. Check if the operator pod is
@ -96,48 +138,38 @@ kubectl get svc -l application=spilo -L spilo-role
## Connect to the Postgres cluster via psql ## Connect to the Postgres cluster via psql
You can retrieve the host and port of the Postgres master from minikube. You can create a port-forward on a database pod to connect to Postgres. See the
Retrieve the password from the Kubernetes Secret that is created in your cluster. [user guide](user.md#connect-to-postgresql) for instructions. With minikube it's
also easy to retrieve the connections string from the K8s service that is
pointing to the master pod:
```bash ```bash
export HOST_PORT=$(minikube service acid-minimal-cluster --url | sed 's,.*/,,') export HOST_PORT=$(minikube service acid-minimal-cluster --url | sed 's,.*/,,')
export PGHOST=$(echo $HOST_PORT | cut -d: -f 1) export PGHOST=$(echo $HOST_PORT | cut -d: -f 1)
export PGPORT=$(echo $HOST_PORT | cut -d: -f 2) export PGPORT=$(echo $HOST_PORT | cut -d: -f 2)
```
Retrieve the password from the K8s Secret that is created in your cluster.
```bash
export PGPASSWORD=$(kubectl get secret postgres.acid-minimal-cluster.credentials -o 'jsonpath={.data.password}' | base64 -d) export PGPASSWORD=$(kubectl get secret postgres.acid-minimal-cluster.credentials -o 'jsonpath={.data.password}' | base64 -d)
psql -U postgres psql -U postgres
``` ```
## Delete a Postgres cluster ## Delete a Postgres cluster
To delete a Postgres cluster simply delete the postgresql custom resource. To delete a Postgres cluster simply delete the `postgresql` custom resource.
```bash ```bash
kubectl delete postgresql acid-minimal-cluster kubectl delete postgresql acid-minimal-cluster
# tear down cleanly
minikube delete
``` ```
This should remove the associated StatefulSet, database Pods, Services and
Endpoints. The PersistentVolumes are released and the PodDisruptionBudget is
deleted. Secrets however are not deleted and backups will remain in place.
## Running and testing the operator When deleting a cluster while it is still starting up or got stuck during that
phase it can [happen](https://github.com/zalando/postgres-operator/issues/551)
The best way to test the operator is to run it in [minikube](https://kubernetes.io/docs/getting-started-guides/minikube/). that the `postgresql` resource is deleted leaving orphaned components behind.
Minikube is a tool to run Kubernetes cluster locally. This can cause troubles when creating a new Postgres cluster. For a fresh setup
you can delete your local minikube or kind cluster and start again.
For convenience, we have automated starting the operator and submitting the
`acid-minimal-cluster`. From inside the cloned repository execute the
`run_operator_locally` shell script.
```bash
./run_operator_locally.sh
```
Note we provide the `/manifests` directory as an example only; you should
consider adjusting the manifests to your particular setting.
### Configuration Options
The operator can be configured with the provided ConfigMap
(`manifests/configmap.yaml`) or the operator's own CRD. See
[developer docs](developer.md) for details.

141
docs/reference/cluster_manifest.md
View File

@ -1,10 +1,12 @@
Individual postgres clusters are described by the Kubernetes *cluster manifest* # Cluster manifest reference
that has the structure defined by the `postgres CRD` (custom resource
Individual Postgres clusters are described by the Kubernetes *cluster manifest*
that has the structure defined by the `postgresql` CRD (custom resource
definition). The following section describes the structure of the manifest and definition). The following section describes the structure of the manifest and
the purpose of individual keys. You can take a look at the examples of the the purpose of individual keys. You can take a look at the examples of the
[minimal](https://github.com/zalando/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml) [minimal](../manifests/minimal-postgres-manifest.yaml)
and the and the
[complete](https://github.com/zalando/postgres-operator/blob/master/manifests/complete-postgres-manifest.yaml) [complete](../manifests/complete-postgres-manifest.yaml)
cluster manifests. cluster manifests.
When Kubernetes resources, such as memory, CPU or volumes, are configured, When Kubernetes resources, such as memory, CPU or volumes, are configured,
@ -19,7 +21,7 @@ applied.
## Manifest structure ## Manifest structure
A postgres manifest is a `YAML` document. On the top level both individual A Postgres manifest is a `YAML` document. On the top level both individual
parameters and parameter groups can be defined. Parameter names are written parameters and parameter groups can be defined. Parameter names are written
in camelCase. in camelCase.
@ -54,26 +56,31 @@ These parameters are grouped directly under the `spec` key in the manifest.
name of the team the cluster belongs to. Changing it after the cluster name of the team the cluster belongs to. Changing it after the cluster
creation is not supported. Required field. creation is not supported. Required field.
* **numberOfInstances**
total number of instances for a given cluster. The operator parameters
`max_instances` and `min_instances` may also adjust this number. Required
field.
* **dockerImage** * **dockerImage**
custom docker image that overrides the **docker_image** operator parameter. custom docker image that overrides the **docker_image** operator parameter.
It should be a [Spilo](https://github.com/zalando/spilo) image. Optional. It should be a [Spilo](https://github.com/zalando/spilo) image. Optional.
* **spiloFSGroup** * **spiloFSGroup**
the Persistent Volumes for the spilo pods in the StatefulSet will be owned the Persistent Volumes for the Spilo pods in the StatefulSet will be owned and
and writable by the group ID specified. This will override the **spilo_fsgroup** writable by the group ID specified. This will override the **spilo_fsgroup**
operator parameter. This is required to run Spilo as a non-root process, but operator parameter. This is required to run Spilo as a non-root process, but
requires a custom spilo image. Note the FSGroup of a Pod cannot be changed requires a custom Spilo image. Note the FSGroup of a Pod cannot be changed
without recreating a new Pod. without recreating a new Pod. Optional.
* **enableMasterLoadBalancer** * **enableMasterLoadBalancer**
boolean flag to override the operator defaults (set by the boolean flag to override the operator defaults (set by the
`enable_master_load_balancer` parameter) to define whether to enable the load `enable_master_load_balancer` parameter) to define whether to enable the load
balancer pointing to the postgres primary. Optional. balancer pointing to the Postgres primary. Optional.
* **enableReplicaLoadBalancer** * **enableReplicaLoadBalancer**
boolean flag to override the operator defaults (set by the boolean flag to override the operator defaults (set by the
`enable_replica_load_balancer` parameter) to define whether to enable the `enable_replica_load_balancer` parameter) to define whether to enable the
load balancer pointing to the postgres standby instances. Optional. load balancer pointing to the Postgres standby instances. Optional.
* **allowedSourceRanges** * **allowedSourceRanges**
when one or more load balancers are enabled for the cluster, this parameter when one or more load balancers are enabled for the cluster, this parameter
@ -82,11 +89,6 @@ These parameters are grouped directly under the `spec` key in the manifest.
this parameter. Optional, when empty the load balancer service becomes this parameter. Optional, when empty the load balancer service becomes
inaccessible from outside of the Kubernetes cluster. inaccessible from outside of the Kubernetes cluster.
* **numberOfInstances**
total number of instances for a given cluster. The operator parameters
`max_instances` and `min_instances` may also adjust this number. Required
field.
* **users** * **users**
a map of usernames to user flags for the users that should be created in the a map of usernames to user flags for the users that should be created in the
cluster by the operator. User flags are a list, allowed elements are cluster by the operator. User flags are a list, allowed elements are
@ -114,7 +116,7 @@ These parameters are grouped directly under the `spec` key in the manifest.
that should be assigned to the cluster pods. When not specified, the value that should be assigned to the cluster pods. When not specified, the value
is taken from the `pod_priority_class_name` operator parameter, if not set is taken from the `pod_priority_class_name` operator parameter, if not set
then the default priority class is taken. The priority class itself must be then the default priority class is taken. The priority class itself must be
defined in advance. defined in advance. Optional.
* **enableShmVolume** * **enableShmVolume**
Start a database pod without limitations on shm memory. By default docker Start a database pod without limitations on shm memory. By default docker
@ -126,35 +128,36 @@ These parameters are grouped directly under the `spec` key in the manifest.
about mounting a volume will be made based on operator configuration about mounting a volume will be made based on operator configuration
(`enable_shm_volume`, which is `true` by default). It it's present and value (`enable_shm_volume`, which is `true` by default). It it's present and value
is `false`, then no volume will be mounted no matter how operator was is `false`, then no volume will be mounted no matter how operator was
configured (so you can override the operator configuration). configured (so you can override the operator configuration). Optional.
* **enableLogicalBackup** * **enableLogicalBackup**
Determines if the logical backup of this cluster should be taken and uploaded Determines if the logical backup of this cluster should be taken and uploaded
to S3. Default: false. to S3. Default: false. Optional.
* **logicalBackupSchedule** * **logicalBackupSchedule**
Schedule for the logical backup k8s cron job. Please take [the reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) Schedule for the logical backup k8s cron job. Please take
into account. Default: "30 00 \* \* \*" [the reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule)
into account. Optional. Default is: "30 00 \* \* \*"
## Postgres parameters ## Postgres parameters
Those parameters are grouped under the `postgresql` top-level key. Those parameters are grouped under the `postgresql` top-level key, which is
required in the manifest.
* **version** * **version**
the postgres major version of the cluster. Looks at the [Spilo the Postgres major version of the cluster. Looks at the [Spilo
project](https://github.com/zalando/spilo/releases) for the list of supported project](https://github.com/zalando/spilo/releases) for the list of supported
versions. Changing the cluster version once the cluster has been bootstrapped versions. Changing the cluster version once the cluster has been bootstrapped
is not supported. Required field. is not supported. Required field.
* **parameters** * **parameters**
a dictionary of postgres parameter names and values to apply to the resulting a dictionary of Postgres parameter names and values to apply to the resulting
cluster. Optional (Spilo automatically sets reasonable defaults for cluster. Optional (Spilo automatically sets reasonable defaults for parameters
parameters like work_mem or max_connections). like `work_mem` or `max_connections`).
## Patroni parameters ## Patroni parameters
Those parameters are grouped under the `patroni` top-level key. See the [patroni Those parameters are grouped under the `patroni` top-level key. See the [Patroni
documentation](https://patroni.readthedocs.io/en/latest/SETTINGS.html) for the documentation](https://patroni.readthedocs.io/en/latest/SETTINGS.html) for the
explanation of `ttl` and `loop_wait` parameters. explanation of `ttl` and `loop_wait` parameters.
@ -176,19 +179,19 @@ explanation of `ttl` and `loop_wait` parameters.
authentication. Optional. authentication. Optional.
* **ttl** * **ttl**
patroni `ttl` parameter value, optional. The default is set by the Spilo Patroni `ttl` parameter value, optional. The default is set by the Spilo
docker image. Optional. docker image. Optional.
* **loop_wait** * **loop_wait**
patroni `loop_wait` parameter value, optional. The default is set by the Patroni `loop_wait` parameter value, optional. The default is set by the
Spilo docker image. Optional. Spilo docker image. Optional.
* **retry_timeout** * **retry_timeout**
patroni `retry_timeout` parameter value, optional. The default is set by the Patroni `retry_timeout` parameter value, optional. The default is set by the
Spilo docker image. Optional. Spilo docker image. Optional.
* **maximum_lag_on_failover** * **maximum_lag_on_failover**
patroni `maximum_lag_on_failover` parameter value, optional. The default is Patroni `maximum_lag_on_failover` parameter value, optional. The default is
set by the Spilo docker image. Optional. set by the Spilo docker image. Optional.
* **slots** * **slots**
@ -199,38 +202,34 @@ explanation of `ttl` and `loop_wait` parameters.
automatically created by Patroni for cluster members and permanent replication automatically created by Patroni for cluster members and permanent replication
slots. Optional. slots. Optional.
* **standby**
initializes cluster as a standby creating a cascading replication, where standby leader is streaming from specified remote location
## Postgres container resources ## Postgres container resources
Those parameters define [CPU and memory requests and Those parameters define [CPU and memory requests and limits](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
limits](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for the Postgres container. They are grouped under the `resources` top-level
for the postgres container. They are grouped under the `resources` top-level key with subgroups `requests` and `limits`.
key. There are two subgroups, `requests` and `limits`.
### Requests ### Requests
CPU and memory requests for the postgres container. CPU and memory requests for the Postgres container.
* **cpu** * **cpu**
CPU requests for the postgres container. Optional, overrides the CPU requests for the Postgres container. Optional, overrides the
`default_cpu_requests` operator configuration parameter. Optional. `default_cpu_requests` operator configuration parameter. Optional.
* **memory** * **memory**
memory requests for the postgres container. Optional, overrides the memory requests for the Postgres container. Optional, overrides the
`default_memory_request` operator configuration parameter. Optional. `default_memory_request` operator configuration parameter. Optional.
#### Limits ### Limits
CPU and memory limits for the postgres container. CPU and memory limits for the Postgres container.
* **cpu** * **cpu**
CPU limits for the postgres container. Optional, overrides the CPU limits for the Postgres container. Optional, overrides the
`default_cpu_limits` operator configuration parameter. Optional. `default_cpu_limits` operator configuration parameter. Optional.
* **memory** * **memory**
memory limits for the postgres container. Optional, overrides the memory limits for the Postgres container. Optional, overrides the
`default_memory_limits` operator configuration parameter. Optional. `default_memory_limits` operator configuration parameter. Optional.
## Parameters defining how to clone the cluster from another one ## Parameters defining how to clone the cluster from another one
@ -257,8 +256,13 @@ under the `clone` top-level key and do not affect the already running cluster.
timestamp. When this parameter is set the operator will not consider cloning timestamp. When this parameter is set the operator will not consider cloning
from the live cluster, even if it is running, and instead goes to S3. Optional. from the live cluster, even if it is running, and instead goes to S3. Optional.
* **s3_wal_path**
the url to S3 bucket containing the WAL archive of the cluster to be cloned.
Optional.
* **s3_endpoint** * **s3_endpoint**
the url of the S3-compatible service should be set when cloning from non AWS S3. Optional. the url of the S3-compatible service should be set when cloning from non AWS
S3. Optional.
* **s3_access_key_id** * **s3_access_key_id**
the access key id, used for authentication on S3 service. Optional. the access key id, used for authentication on S3 service. Optional.
@ -267,13 +271,25 @@ under the `clone` top-level key and do not affect the already running cluster.
the secret access key, used for authentication on S3 service. Optional. the secret access key, used for authentication on S3 service. Optional.
* **s3_force_path_style** * **s3_force_path_style**
to enable path-style addressing(i.e., http://s3.amazonaws.com/BUCKET/KEY) when connecting to an S3-compatible service to enable path-style addressing(i.e., http://s3.amazonaws.com/BUCKET/KEY)
that lack of support for sub-domain style bucket URLs (i.e., http://BUCKET.s3.amazonaws.com/KEY). Optional. when connecting to an S3-compatible service that lack of support for
sub-domain style bucket URLs (i.e., http://BUCKET.s3.amazonaws.com/KEY).
Optional.
### EBS volume resizing ## Standby cluster
On startup, an existing `standby` top-level key creates a standby Postgres
cluster streaming from a remote location. So far only streaming from a S3 WAL
archive is supported.
* **s3_wal_path**
the url to S3 bucket containing the WAL archive of the remote primary.
Required when the `standby` section is present.
## EBS volume resizing
Those parameters are grouped under the `volume` top-level key and define the Those parameters are grouped under the `volume` top-level key and define the
properties of the persistent storage that stores postgres data. properties of the persistent storage that stores Postgres data.
* **size** * **size**
the size of the target EBS volume. Usual Kubernetes size modifiers, i.e. `Gi` the size of the target EBS volume. Usual Kubernetes size modifiers, i.e. `Gi`
@ -286,13 +302,13 @@ properties of the persistent storage that stores postgres data.
for the details on storage classes. Optional. for the details on storage classes. Optional.
* **subPath** * **subPath**
Subpath to use when mounting volume into Spilo container Subpath to use when mounting volume into Spilo container. Optional.
### Sidecar definitions ## Sidecar definitions
Those parameters are defined under the `sidecars` key. They consist of a list Those parameters are defined under the `sidecars` key. They consist of a list
of dictionaries, each defining one sidecar (an extra container running of dictionaries, each defining one sidecar (an extra container running
along the main postgres container on the same pod). The following keys can be along the main Postgres container on the same pod). The following keys can be
defined in the sidecar dictionary: defined in the sidecar dictionary:
* **name** * **name**
@ -306,16 +322,11 @@ defined in the sidecar dictionary:
(https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/) (https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/)
for environment variables. Optional. for environment variables. Optional.
* **resources** see below. Optional. * **resources**
[CPU and memory requests and limits](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container)
for each sidecar container. Optional.
#### Sidecar container resources ### Requests
Those parameters define [CPU and memory requests and
limits](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
for the sidecar container. They are grouped under the `resources` key for each sidecar.
There are two subgroups, `requests` and `limits`.
##### Requests
CPU and memory requests for the sidecar container. CPU and memory requests for the sidecar container.
@ -327,7 +338,7 @@ CPU and memory requests for the sidecar container.
memory requests for the sidecar container. Optional, overrides the memory requests for the sidecar container. Optional, overrides the
`default_memory_request` operator configuration parameter. Optional. `default_memory_request` operator configuration parameter. Optional.
##### Limits ### Limits
CPU and memory limits for the sidecar container. CPU and memory limits for the sidecar container.

6
docs/reference/command_line_and_environment.md
View File

@ -50,9 +50,9 @@ The following environment variables are accepted by the operator:
`scalyr_api_key` operator parameter. `scalyr_api_key` operator parameter.
* **CRD_READY_WAIT_TIMEOUT** * **CRD_READY_WAIT_TIMEOUT**
defines the timeout for the complete postgres CRD creation. When not set defines the timeout for the complete `postgresql` CRD creation. When not set
default is 30s. default is 30s.
* **CRD_READY_WAIT_INTERVAL** * **CRD_READY_WAIT_INTERVAL**
defines the interval between consecutive attempts waiting for the postgres defines the interval between consecutive attempts waiting for the
CRD to be created. The default is 5s. `postgresql` CRD to be created. The default is 5s.

117
docs/reference/operator_parameters.md
View File

@ -1,3 +1,5 @@
# Configuration parameters
There are two mutually-exclusive methods to set the Postgres Operator There are two mutually-exclusive methods to set the Postgres Operator
configuration. configuration.
@ -8,12 +10,12 @@ configuration.
maps. String values containing ':' should be enclosed in quotes. The maps. String values containing ':' should be enclosed in quotes. The
configuration is flat, parameter group names below are not reflected in the configuration is flat, parameter group names below are not reflected in the
configuration structure. There is an configuration structure. There is an
[example](https://github.com/zalando/postgres-operator/blob/master/manifests/configmap.yaml) [example](../manifests/configmap.yaml)
* CRD-based configuration. The configuration is stored in a custom YAML * CRD-based configuration. The configuration is stored in a custom YAML
manifest. The manifest is an instance of the custom resource definition (CRD) manifest. The manifest is an instance of the custom resource definition (CRD)
called `OperatorConfiguration`. The operator registers this CRD during the called `OperatorConfiguration`. The operator registers this CRD during the
start and uses it for configuration if the [operator deployment manifest ](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L21) start and uses it for configuration if the [operator deployment manifest](../manifests/postgres-operator.yaml#L36)
sets the `POSTGRES_OPERATOR_CONFIGURATION_OBJECT` env variable to a non-empty sets the `POSTGRES_OPERATOR_CONFIGURATION_OBJECT` env variable to a non-empty
value. The variable should point to the `postgresql-operator-configuration` value. The variable should point to the `postgresql-operator-configuration`
object in the operator's namespace. object in the operator's namespace.
@ -22,7 +24,7 @@ configuration.
simply represented in the usual YAML way. There are no default values built-in simply represented in the usual YAML way. There are no default values built-in
in the operator, each parameter that is not supplied in the configuration in the operator, each parameter that is not supplied in the configuration
receives an empty value. In order to create your own configuration just copy receives an empty value. In order to create your own configuration just copy
the [default one](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml) the [default one](../manifests/postgresql-operator-default-configuration.yaml)
and change it. and change it.
To test the CRD-based configuration locally, use the following To test the CRD-based configuration locally, use the following
@ -32,10 +34,10 @@ configuration.
kubectl create -f manifests/postgresql-operator-default-configuration.yaml kubectl create -f manifests/postgresql-operator-default-configuration.yaml
kubectl get operatorconfigurations postgresql-operator-default-configuration -o yaml kubectl get operatorconfigurations postgresql-operator-default-configuration -o yaml
``` ```
Note that the operator first attempts to register the CRD of the `OperatorConfiguration` Note that the operator first attempts to register the CRD of the
and then waits for an instance to be created. In between these two event the `OperatorConfiguration` and then waits for an instance to be created. In
operator pod may be failing since it cannot fetch the not-yet-existing between these two event the operator pod may be failing since it cannot fetch
`OperatorConfiguration` instance. the not-yet-existing `OperatorConfiguration` instance.
The CRD-based configuration is more powerful than the one based on ConfigMaps The CRD-based configuration is more powerful than the one based on ConfigMaps
and should be used unless there is a compatibility requirement to use an already and should be used unless there is a compatibility requirement to use an already
@ -56,11 +58,11 @@ parameters, those parameters have no effect and are replaced by the
`CRD_READY_WAIT_INTERVAL` and `CRD_READY_WAIT_TIMEOUT` environment variables. `CRD_READY_WAIT_INTERVAL` and `CRD_READY_WAIT_TIMEOUT` environment variables.
They will be deprecated and removed in the future. They will be deprecated and removed in the future.
For the configmap operator configuration, the [default parameter values](https://github.com/zalando-incubator/postgres-operator/blob/master/pkg/util/config/config.go#L14) For the configmap configuration, the [default parameter values](../pkg/util/config/config.go#L14)
mentioned here are likely to be overwritten in your local operator installation mentioned here are likely to be overwritten in your local operator installation
via your local version of the operator configmap. In the case you use the via your local version of the operator configmap. In the case you use the
operator CRD, all the CRD defaults are provided in the operator CRD, all the CRD defaults are provided in the
[operator's default configuration manifest](https://github.com/zalando-incubator/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml) [operator's default configuration manifest](../manifests/postgresql-operator-default-configuration.yaml)
Variable names are underscore-separated words. Variable names are underscore-separated words.
@ -75,7 +77,7 @@ Those are top-level keys, containing both leaf keys and groups.
Kubernetes-native DCS). Kubernetes-native DCS).
* **docker_image** * **docker_image**
Spilo docker image for postgres instances. For production, don't rely on the Spilo docker image for Postgres instances. For production, don't rely on the
default image, as it might be not the most up-to-date one. Instead, build default image, as it might be not the most up-to-date one. Instead, build
your own Spilo image from the [github your own Spilo image from the [github
repository](https://github.com/zalando/spilo). repository](https://github.com/zalando/spilo).
@ -98,12 +100,12 @@ Those are top-level keys, containing both leaf keys and groups.
create/update/delete/sync clusters concurrently. The default is `4`. create/update/delete/sync clusters concurrently. The default is `4`.
* **max_instances** * **max_instances**
operator will cap the number of instances in any managed postgres cluster up operator will cap the number of instances in any managed Postgres cluster up
to the value of this parameter. When `-1` is specified, no limits are applied. to the value of this parameter. When `-1` is specified, no limits are applied.
The default is `-1`. The default is `-1`.
* **min_instances** * **min_instances**
operator will run at least the number of instances for any given postgres operator will run at least the number of instances for any given Postgres
cluster equal to the value of this parameter. When `-1` is specified, no cluster equal to the value of this parameter. When `-1` is specified, no
limits are applied. The default is `-1`. limits are applied. The default is `-1`.
@ -120,7 +122,7 @@ Those are top-level keys, containing both leaf keys and groups.
containers with high memory limits due to the lack of memory on Kubernetes containers with high memory limits due to the lack of memory on Kubernetes
cluster nodes. This affects all containers created by the operator (Postgres, cluster nodes. This affects all containers created by the operator (Postgres,
Scalyr sidecar, and other sidecars); to set resources for the operator's own Scalyr sidecar, and other sidecars); to set resources for the operator's own
container, change the [operator deployment manually](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L13). container, change the [operator deployment manually](../manifests/postgres-operator.yaml#L20).
The default is `false`. The default is `false`.
## Postgres users ## Postgres users
@ -129,11 +131,11 @@ Parameters describing Postgres users. In a CRD-configuration, they are grouped
under the `users` key. under the `users` key.
* **super_username** * **super_username**
postgres `superuser` name to be created by `initdb`. The default is Postgres `superuser` name to be created by `initdb`. The default is
`postgres`. `postgres`.
* **replication_username** * **replication_username**
postgres username used for replication between instances. The default is Postgres username used for replication between instances. The default is
`standby`. `standby`.
## Kubernetes resources ## Kubernetes resources
@ -158,17 +160,16 @@ configuration they are grouped under the `kubernetes` key.
This definition must bind pod service account to a role with permission This definition must bind pod service account to a role with permission
sufficient for the pods to start and for Patroni to access k8s endpoints; sufficient for the pods to start and for Patroni to access k8s endpoints;
service account on its own lacks any such rights starting with k8s v1.8. If service account on its own lacks any such rights starting with k8s v1.8. If
not excplicitly defined by the user, a simple definition that binds the not explicitly defined by the user, a simple definition that binds the
account to the operator's own 'zalando-postgres-operator' cluster role will account to the operator's own 'zalando-postgres-operator' cluster role will
be used. The default is empty. be used. The default is empty.
* **pod_terminate_grace_period** * **pod_terminate_grace_period**
Postgres pods are [terminated Postgres pods are [terminated forcefully](https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods)
forcefully](https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods)
after this timeout. The default is `5m`. after this timeout. The default is `5m`.
* **watched_namespace** * **watched_namespace**
The operator watches for postgres objects in the given namespace. If not The operator watches for Postgres objects in the given namespace. If not
specified, the value is taken from the operator namespace. A special `*` specified, the value is taken from the operator namespace. A special `*`
value makes it watch all namespaces. The default is empty (watch the operator value makes it watch all namespaces. The default is empty (watch the operator
pod namespace). pod namespace).
@ -195,9 +196,9 @@ configuration they are grouped under the `kubernetes` key.
`{username}.{cluster}.credentials.{tprkind}.{tprgroup}`. `{username}.{cluster}.credentials.{tprkind}.{tprgroup}`.
* **cluster_domain** * **cluster_domain**
defines the default dns domain for the kubernetes cluster the operator is defines the default DNS domain for the kubernetes cluster the operator is
running in. The default is `cluster.local`. Used by the operator to connect running in. The default is `cluster.local`. Used by the operator to connect
to the postgres clusters after creation. to the Postgres clusters after creation.
* **oauth_token_secret_name** * **oauth_token_secret_name**
a name of the secret containing the `OAuth2` token to pass to the teams API. a name of the secret containing the `OAuth2` token to pass to the teams API.
@ -217,9 +218,8 @@ configuration they are grouped under the `kubernetes` key.
* **inherited_labels** * **inherited_labels**
list of labels that can be inherited from the cluster manifest, and added to list of labels that can be inherited from the cluster manifest, and added to
each child objects (`StatefulSet`, `Pod`, `Service` and `Endpoints`) created each child objects (`StatefulSet`, `Pod`, `Service` and `Endpoints`) created
by the opertor. by the operator. Typical use case is to dynamically pass labels that are
Typical use case is to dynamically pass labels that are specific to a given specific to a given Postgres cluster, in order to implement `NetworkPolicy`.
postgres cluster, in order to implement `NetworkPolicy`.
The default is empty. The default is empty.
* **cluster_name_label** * **cluster_name_label**
@ -243,7 +243,7 @@ configuration they are grouped under the `kubernetes` key.
* **pod_environment_configmap** * **pod_environment_configmap**
a name of the ConfigMap with environment variables to populate on every pod. a name of the ConfigMap with environment variables to populate on every pod.
Right now this ConfigMap is searched in the namespace of the postgres cluster. Right now this ConfigMap is searched in the namespace of the Postgres cluster.
All variables from that ConfigMap are injected to the pod's environment, on All variables from that ConfigMap are injected to the pod's environment, on
conflicts they are overridden by the environment variables generated by the conflicts they are overridden by the environment variables generated by the
operator. The default is empty. operator. The default is empty.
@ -254,8 +254,9 @@ configuration they are grouped under the `kubernetes` key.
be defined in advance. Default is empty (use the default priority class). be defined in advance. Default is empty (use the default priority class).
* **spilo_fsgroup** * **spilo_fsgroup**
the Persistent Volumes for the spilo pods in the StatefulSet will be owned and writable by the group ID specified. the Persistent Volumes for the Spilo pods in the StatefulSet will be owned and
This is required to run Spilo as a non-root process, but requires a custom spilo image. Note the FSGroup of a Pod writable by the group ID specified. This is required to run Spilo as a
non-root process, but requires a custom Spilo image. Note the FSGroup of a Pod
cannot be changed without recreating a new Pod. cannot be changed without recreating a new Pod.
* **spilo_privileged** * **spilo_privileged**
@ -291,19 +292,19 @@ Those parameters are grouped under the `postgres_pod_resources` key in a
CRD-based configuration. CRD-based configuration.
* **default_cpu_request** * **default_cpu_request**
CPU request value for the postgres containers, unless overridden by CPU request value for the Postgres containers, unless overridden by
cluster-specific settings. The default is `100m`. cluster-specific settings. The default is `100m`.
* **default_memory_request** * **default_memory_request**
memory request value for the postgres containers, unless overridden by memory request value for the Postgres containers, unless overridden by
cluster-specific settings. The default is `100Mi`. cluster-specific settings. The default is `100Mi`.
* **default_cpu_limit** * **default_cpu_limit**
CPU limits for the postgres containers, unless overridden by cluster-specific CPU limits for the Postgres containers, unless overridden by cluster-specific
settings. The default is `3`. settings. The default is `3`.
* **default_memory_limit** * **default_memory_limit**
memory limits for the postgres containers, unless overridden by cluster-specific memory limits for the Postgres containers, unless overridden by cluster-specific
settings. The default is `1Gi`. settings. The default is `1Gi`.
## Operator timeouts ## Operator timeouts
@ -334,11 +335,11 @@ CRD-based configuration.
cluster or recreating pods. The default is `10m`. cluster or recreating pods. The default is `10m`.
* **ready_wait_interval** * **ready_wait_interval**
the interval between consecutive attempts waiting for the postgres CRD to be the interval between consecutive attempts waiting for the `postgresql` CRD to
created. The default is `5s`. be created. The default is `5s`.
* **ready_wait_timeout** * **ready_wait_timeout**
the timeout for the complete postgres CRD creation. The default is `30s`. the timeout for the complete `postgresql` CRD creation. The default is `30s`.
## Load balancer related options ## Load balancer related options
@ -394,7 +395,7 @@ yet officially supported.
Spilo are S3 and GCS. The default is empty. Spilo are S3 and GCS. The default is empty.
* **log_s3_bucket** * **log_s3_bucket**
S3 bucket to use for shipping postgres daily logs. Works only with S3 on AWS. S3 bucket to use for shipping Postgres daily logs. Works only with S3 on AWS.
The bucket has to be present and accessible by Postgres pods. The default is The bucket has to be present and accessible by Postgres pods. The default is
empty. empty.
@ -413,6 +414,28 @@ yet officially supported.
* **additional_secret_mount_path** * **additional_secret_mount_path**
Path to mount the above Secret in the filesystem of the container(s). The default is empty. Path to mount the above Secret in the filesystem of the container(s). The default is empty.
## Logical backup
These parameters configure a k8s cron job managed by the operator to produce
Postgres logical backups. In the CRD-based configuration those parameters are
grouped under the `logical_backup` key.
* **logical_backup_schedule**
Backup schedule in the cron format. Please take the
[reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule)
into account. Default: "30 00 \* \* \*"
* **logical_backup_docker_image**
An image for pods of the logical backup job. The [example image](../../docker/logical-backup/Dockerfile)
runs `pg_dumpall` on a replica if possible and uploads compressed results to
an S3 bucket under the key `/spilo/pg_cluster_name/cluster_k8s_uuid/logical_backups`.
The default image is the same image built with the Zalando-internal CI
pipeline. Default: "registry.opensource.zalan.do/acid/logical-backup"
* **logical_backup_s3_bucket**
S3 bucket to store backup results. The bucket has to be present and
accessible by Postgres pods. Default: empty.
## Debugging the operator ## Debugging the operator
Options to aid debugging of the operator itself. Grouped under the `debug` key. Options to aid debugging of the operator itself. Grouped under the `debug` key.
@ -423,7 +446,7 @@ Options to aid debugging of the operator itself. Grouped under the `debug` key.
* **enable_database_access** * **enable_database_access**
boolean parameter that toggles the functionality of the operator that require boolean parameter that toggles the functionality of the operator that require
access to the postgres database, i.e. creating databases and users. The access to the Postgres database, i.e. creating databases and users. The
default is `true`. default is `true`.
## Automatic creation of human users in the database ## Automatic creation of human users in the database
@ -442,7 +465,7 @@ key.
`https://teams.example.com/api/`. `https://teams.example.com/api/`.
* **team_api_role_configuration** * **team_api_role_configuration**
postgres parameters to apply to each team member role. The default is Postgres parameters to apply to each team member role. The default is
'*log_statement:all*'. It is possible to supply multiple options, separating '*log_statement:all*'. It is possible to supply multiple options, separating
them by commas. Options containing commas within the value are not supported, them by commas. Options containing commas within the value are not supported,
with the exception of the `search_path`. For instance: with the exception of the `search_path`. For instance:
@ -527,23 +550,3 @@ scalyr sidecar. In the CRD-based configuration they are grouped under the
* **scalyr_memory_limit** * **scalyr_memory_limit**
Memory limit value for the Scalyr sidecar. The default is `1Gi`. Memory limit value for the Scalyr sidecar. The default is `1Gi`.
## Logical backup
These parameters configure a k8s cron job managed by the operator to produce
Postgres logical backups. In the CRD-based configuration those parameters are
grouped under the `logical_backup` key.
* **logical_backup_schedule**
Backup schedule in the cron format. Please take [the reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) into account. Default: "30 00 \* \* \*"
* **logical_backup_docker_image**
An image for pods of the logical backup job. The [example image](../../docker/logical-backup/Dockerfile)
runs `pg_dumpall` on a replica if possible and uploads compressed results to
an S3 bucket under the key `/spilo/pg_cluster_name/cluster_k8s_uuid/logical_backups`.
The default image is the same image built with the Zalando-internal CI
pipeline. Default: "registry.opensource.zalan.do/acid/logical-backup"
* **logical_backup_s3_bucket**
S3 bucket to store backup results. The bucket has to be present and
accessible by Postgres pods. Default: empty.

165
docs/user.md
View File

@ -1,7 +1,11 @@
# User Guide
Learn how to work with the Postgres Operator in a Kubernetes (K8s) environment.
## Create a manifest for a new PostgreSQL cluster ## Create a manifest for a new PostgreSQL cluster
As an example you can take this Make sure you have [set up](quickstart.md) the operator. Then you can create a
[minimal example](https://github.com/zalando/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml): new Postgres cluster by applying manifest like this [minimal example](../manifests/minimal-postgres-manifest.yaml):
```yaml ```yaml
apiVersion: "acid.zalan.do/v1" apiVersion: "acid.zalan.do/v1"
@ -29,27 +33,19 @@ spec:
version: "10" version: "10"
``` ```
## Create a new Spilo cluster Once you cloned the Postgres Operator [repository](https://github.com/zalando/postgres-operator)
you can find this example also in the manifests folder:
```bash ```bash
$ kubectl create -f manifests/minimal-postgres-manifest.yaml kubectl create -f manifests/minimal-postgres-manifest.yaml
``` ```
## Watch pods being created ## Watch pods being created
```bash ```bash
$ kubectl get pods -w --show-labels kubectl get pods -w --show-labels
``` ```
## Give K8S users access to create/list postgresqls
```bash
$ kubectl create -f manifests/user-facing-clusterroles.yaml
```
Creates zalando-postgres-operator:users:view, :edit and :admin clusterroles that are
aggregated into the default roles.
## Connect to PostgreSQL ## Connect to PostgreSQL
With a `port-forward` on one of the database pods (e.g. the master) you can With a `port-forward` on one of the database pods (e.g. the master) you can
@ -69,33 +65,37 @@ Open another CLI and connect to the database. Use the generated secret of the
in Minikube: in Minikube:
```bash ```bash
$ export PGPASSWORD=$(kubectl get secret postgres.acid-minimal-cluster.credentials -o 'jsonpath={.data.password}' | base64 -d) export PGPASSWORD=$(kubectl get secret postgres.acid-minimal-cluster.credentials -o 'jsonpath={.data.password}' | base64 -d)
$ psql -U postgres -p 6432 psql -U postgres -p 6432
``` ```
# Defining database roles in the operator ## Defining database roles in the operator
Postgres operator allows defining roles to be created in the resulting database Postgres Operator allows defining roles to be created in the resulting database
cluster. It covers three use-cases: cluster. It covers three use-cases:
* `manifest roles`: create application roles specific to the cluster described in the manifest. * `manifest roles`: create application roles specific to the cluster described
* `infrastructure roles`: create application roles that should be automatically created on every in the manifest.
* `infrastructure roles`: create application roles that should be automatically
created on every
cluster managed by the operator. cluster managed by the operator.
* `teams API roles`: automatically create users for every member of the team owning the database * `teams API roles`: automatically create users for every member of the team
cluster. owning the database cluster.
In the next sections, we will cover those use cases in more details. In the next sections, we will cover those use cases in more details.
## Manifest roles ### Manifest roles
Manifest roles are defined directly in the cluster manifest. See Manifest roles are defined directly in the cluster manifest. See
[minimal postgres manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml) [minimal postgres manifest](../manifests/minimal-postgres-manifest.yaml)
for an example of `zalando` role, defined with `superuser` and `createdb` for an example of `zalando` role, defined with `superuser` and `createdb` flags.
flags.
Manifest roles are defined as a dictionary, with a role name as a key and a Manifest roles are defined as a dictionary, with a role name as a key and a
list of role options as a value. For a role without any options it is best to supply the empty list of role options as a value. For a role without any options it is best to
list `[]`. It is also possible to leave this field empty as in our example manifests, but in certain cases such empty field may removed by Kubernetes [due to the `null` value it gets](https://kubernetes.io/docs/concepts/overview/object-management-kubectl/declarative-config/#how-apply-calculates-differences-and-merges-changes) (`foobar_user:` is equivalent to `foobar_user: null`). supply the empty list `[]`. It is also possible to leave this field empty as in
our example manifests. In certain cases such empty field may be missing later
removed by K8s [due to the `null` value it gets](https://kubernetes.io/docs/concepts/overview/object-management-kubectl/declarative-config/#how-apply-calculates-differences-and-merges-changes)
(`foobar_user:` is equivalent to `foobar_user: null`).
The operator accepts the following options: `superuser`, `inherit`, `login`, The operator accepts the following options: `superuser`, `inherit`, `login`,
`nologin`, `createrole`, `createdb`, `replication`, `bypassrls`. `nologin`, `createrole`, `createdb`, `replication`, `bypassrls`.
@ -107,13 +107,13 @@ The operator automatically generates a password for each manifest role and
places it in the secret named places it in the secret named
`{username}.{team}-{clustername}.credentials.postgresql.acid.zalan.do` in the `{username}.{team}-{clustername}.credentials.postgresql.acid.zalan.do` in the
same namespace as the cluster. This way, the application running in the same namespace as the cluster. This way, the application running in the
Kubernetes cluster and working with the database can obtain the password right K8s cluster and connecting to Postgres can obtain the password right from the
from the secret, without ever sharing it outside of the cluster. secret, without ever sharing it outside of the cluster.
At the moment it is not possible to define membership of the manifest role in At the moment it is not possible to define membership of the manifest role in
other roles. other roles.
## Infrastructure roles ### Infrastructure roles
An infrastructure role is a role that should be present on every PostgreSQL An infrastructure role is a role that should be present on every PostgreSQL
cluster managed by the operator. An example of such a role is a monitoring cluster managed by the operator. An example of such a role is a monitoring
@ -122,7 +122,7 @@ user. There are two ways to define them:
* With the infrastructure roles secret only * With the infrastructure roles secret only
* With both the the secret and the infrastructure role ConfigMap. * With both the the secret and the infrastructure role ConfigMap.
### Infrastructure roles secret #### Infrastructure roles secret
The infrastructure roles secret is specified by the `infrastructure_roles_secret_name` The infrastructure roles secret is specified by the `infrastructure_roles_secret_name`
parameter. The role definition looks like this (values are base64 encoded): parameter. The role definition looks like this (values are base64 encoded):
@ -134,22 +134,21 @@ parameter. The role definition looks like this (values are base64 encoded):
``` ```
The block above describes the infrastructure role 'dbuser' with password The block above describes the infrastructure role 'dbuser' with password
'secret' that is a member of the 'operator' role. For the following 'secret' that is a member of the 'operator' role. For the following definitions
definitions one must increase the index, i.e. the next role will be defined as one must increase the index, i.e. the next role will be defined as 'user2' and
'user2' and so on. The resulting role will automatically be a login role. so on. The resulting role will automatically be a login role.
Note that with definitions that solely use the infrastructure roles secret Note that with definitions that solely use the infrastructure roles secret
there is no way to specify role options (like superuser or nologin) or role there is no way to specify role options (like superuser or nologin) or role
memberships. This is where the ConfigMap comes into play. memberships. This is where the ConfigMap comes into play.
### Secret plus ConfigMap #### Secret plus ConfigMap
A [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) A [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/)
allows for defining more details regarding the infrastructure roles. allows for defining more details regarding the infrastructure roles. Therefore,
Therefore, one should use the new style that specifies infrastructure roles one should use the new style that specifies infrastructure roles using both the
using both the secret and a ConfigMap. The ConfigMap must have the same name as secret and a ConfigMap. The ConfigMap must have the same name as the secret.
the secret. The secret should contain an entry with 'rolename:rolepassword' for The secret should contain an entry with 'rolename:rolepassword' for each role.
each role.
```yaml ```yaml
dbuser: c2VjcmV0 dbuser: c2VjcmV0
@ -179,14 +178,14 @@ Since an infrastructure role is created uniformly on all clusters managed by
the operator, it makes no sense to define it without the password. Such the operator, it makes no sense to define it without the password. Such
definitions will be ignored with a prior warning. definitions will be ignored with a prior warning.
See [infrastructure roles secret](https://github.com/zalando/postgres-operator/blob/master/manifests/infrastructure-roles.yaml) See [infrastructure roles secret](../manifests/infrastructure-roles.yaml)
and [infrastructure roles configmap](https://github.com/zalando/postgres-operator/blob/master/manifests/infrastructure-roles-configmap.yaml) for the examples. and [infrastructure roles configmap](../manifests/infrastructure-roles-configmap.yaml)
for the examples.
## Use taints and tolerations for dedicated PostgreSQL nodes ## Use taints and tolerations for dedicated PostgreSQL nodes
To ensure Postgres pods are running on nodes without any other application To ensure Postgres pods are running on nodes without any other application pods,
pods, you can use you can use [taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
[taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
and configure the required toleration in the manifest. and configure the required toleration in the manifest.
```yaml ```yaml
@ -208,7 +207,6 @@ You can spin up a new cluster as a clone of the existing one, using a clone
section in the spec. There are two options here: section in the spec. There are two options here:
* Clone directly from a source cluster using `pg_basebackup` * Clone directly from a source cluster using `pg_basebackup`
* Clone from an S3 bucket * Clone from an S3 bucket
### Clone directly ### Clone directly
@ -247,10 +245,10 @@ spec:
``` ```
Here `cluster` is a name of a source cluster that is going to be cloned. A new Here `cluster` is a name of a source cluster that is going to be cloned. A new
cluster will be cloned from S3, using the latest backup before the cluster will be cloned from S3, using the latest backup before the `timestamp`.
`timestamp`. In this case, `uid` field is also mandatory - operator will use it In this case, `uid` field is also mandatory - operator will use it to find a
to find a correct key inside an S3 bucket. You can find this field from correct key inside an S3 bucket. You can find this field in the metadata of the
metadata of a source cluster: source cluster:
```yaml ```yaml
apiVersion: acid.zalan.do/v1 apiVersion: acid.zalan.do/v1
@ -263,7 +261,8 @@ metadata:
Note that timezone is required for `timestamp`. Otherwise, offset is relative Note that timezone is required for `timestamp`. Otherwise, offset is relative
to UTC, see [RFC 3339 section 5.6) 3339 section 5.6](https://www.ietf.org/rfc/rfc3339.txt). to UTC, see [RFC 3339 section 5.6) 3339 section 5.6](https://www.ietf.org/rfc/rfc3339.txt).
For non AWS S3 following settings can be set to support cloning from other S3 implementations: For non AWS S3 following settings can be set to support cloning from other S3
implementations:
```yaml ```yaml
apiVersion: "acid.zalan.do/v1" apiVersion: "acid.zalan.do/v1"
@ -283,7 +282,12 @@ spec:
## Setting up a standby cluster ## Setting up a standby cluster
Standby clusters are like normal cluster but they are streaming from a remote cluster. As the first version of this feature, the only scenario covered by operator is to stream from a wal archive of the master. Following the more popular infrastructure of using Amazon's S3 buckets, it is mentioned as s3_wal_path here. To make a cluster as standby add a section standby in the YAML file as follows. Standby clusters are like normal cluster but they are streaming from a remote
cluster. As the first version of this feature, the only scenario covered by
operator is to stream from a WAL archive of the master. Following the more
popular infrastructure of using Amazon's S3 buckets, it is mentioned as
`s3_wal_path` here. To start a cluster as standby add the following `standby`
section in the YAML file:
```yaml ```yaml
spec: spec:
@ -293,15 +297,24 @@ spec:
Things to note: Things to note:
- An empty string is provided in s3_wal_path of the standby cluster will result in error and no statefulset will be created. - An empty string in the `s3_wal_path` field of the standby cluster will result
in an error and no statefulset will be created.
- Only one pod can be deployed for stand-by cluster. - Only one pod can be deployed for stand-by cluster.
- To manually promote the standby_cluster, use patronictl and remove config entry. - To manually promote the standby_cluster, use `patronictl` and remove config
- There is no way to transform a non-standby cluster to standby cluster through operator. Hence, if a cluster is created without standby section in YAML and later modified by adding that section, there will be no effect on the cluster. However, it can be done through Patroni by adding the [standby_cluster] (https://github.com/zalando/patroni/blob/bd2c54581abb42a7d3a3da551edf0b8732eefd27/docs/replica_bootstrap.rst#standby-cluster) section using patronictl edit-config. Note that the transformed standby cluster will not be doing any streaming, rather will just be in standby mode and allow read-only transactions only. entry.
- There is no way to transform a non-standby cluster to a standby cluster
through the operator. Adding the standby section to the manifest of a running
Postgres cluster will have no effect. However, it can be done through Patroni
by adding the [standby_cluster] (https://github.com/zalando/patroni/blob/bd2c54581abb42a7d3a3da551edf0b8732eefd27/docs/replica_bootstrap.rst#standby-cluster)
section using `patronictl edit-config`. Note that the transformed standby
cluster will not be doing any streaming. It will be in standby mode and allow
read-only transactions only.
## Sidecar Support ## Sidecar Support
Each cluster can specify arbitrary sidecars to run. These containers could be used for Each cluster can specify arbitrary sidecars to run. These containers could be
log aggregation, monitoring, backups or other tasks. A sidecar can be specified like this: used for log aggregation, monitoring, backups or other tasks. A sidecar can be
specified like this:
```yaml ```yaml
apiVersion: "acid.zalan.do/v1" apiVersion: "acid.zalan.do/v1"
@ -326,21 +339,21 @@ spec:
value: "any-k8s-env-things" value: "any-k8s-env-things"
``` ```
In addition to any environment variables you specify, the following environment variables In addition to any environment variables you specify, the following environment
are always passed to sidecars: variables are always passed to sidecars:
- `POD_NAME` - field reference to `metadata.name` - `POD_NAME` - field reference to `metadata.name`
- `POD_NAMESPACE` - field reference to `metadata.namespace` - `POD_NAMESPACE` - field reference to `metadata.namespace`
- `POSTGRES_USER` - the superuser that can be used to connect to the database - `POSTGRES_USER` - the superuser that can be used to connect to the database
- `POSTGRES_PASSWORD` - the password for the superuser - `POSTGRES_PASSWORD` - the password for the superuser
The PostgreSQL volume is shared with sidecars and is mounted at `/home/postgres/pgdata`. The PostgreSQL volume is shared with sidecars and is mounted at
`/home/postgres/pgdata`.
## InitContainers Support ## InitContainers Support
Each cluster can specify arbitrary init containers to run. These containers can be Each cluster can specify arbitrary init containers to run. These containers can
used to run custom actions before any normal and sidecar containers start. be used to run custom actions before any normal and sidecar containers start.
An init container can be specified like this: An init container can be specified like this:
```yaml ```yaml
@ -361,7 +374,6 @@ spec:
`initContainers` accepts full `v1.Container` definition. `initContainers` accepts full `v1.Container` definition.
## Increase volume size ## Increase volume size
PostgreSQL operator supports statefulset volume resize if you're using the PostgreSQL operator supports statefulset volume resize if you're using the
@ -390,19 +402,26 @@ actions:
* call AWS API to change the volume size * call AWS API to change the volume size
* connect to the pod using `kubectl exec` and resize the filesystem with * connect to pod using `kubectl exec` and resize filesystem with `resize2fs`.
`resize2fs`.
Fist step has a limitation, AWS rate-limits this operation to no more than once Fist step has a limitation, AWS rate-limits this operation to no more than once
every 6 hours. every 6 hours. Note, that if the statefulset is scaled down before resizing the
Note that if the statefulset is scaled down before resizing the size changes new size is only applied to the volumes attached to the running pods. The
are only applied to the volumes attached to the running pods. The size of the size of volumes that correspond to the previously running pods is not changed.
volumes that correspond to the previously running pods is not changed.
## Logical backups ## Logical backups
If you add You can enable logical backups from the cluster manifest by adding the following
parameter in the spec section:
``` ```
enableLogicalBackup: true enableLogicalBackup: true
``` ```
to the cluster manifest, the operator will create and sync a k8s cron job to do periodic logical backups of this particular Postgres cluster. Due to the [limitation of Kubernetes cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) it is highly advisable to set up additional monitoring for this feature; such monitoring is outside of the scope of operator responsibilities. See [configuration reference](reference/cluster_manifest.md) and [administrator documentation](administrator.md) for details on how backups are executed.
The operator will create and sync a K8s cron job to do periodic logical backups
of this particular Postgres cluster. Due to the [limitation of K8s cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations)
it is highly advisable to set up additional monitoring for this feature; such
monitoring is outside the scope of operator responsibilities. See
[configuration reference](reference/cluster_manifest.md) and
[administrator documentation](administrator.md) for details on how backups are
executed.

26
manifests/complete-postgres-manifest.yaml
View File

@ -50,12 +50,12 @@ spec:
- hostssl all all 0.0.0.0/0 md5 - hostssl all all 0.0.0.0/0 md5
- host all all 0.0.0.0/0 md5 - host all all 0.0.0.0/0 md5
# slots: # slots:
# permanent_physical_1: # - permanent_physical_1:
# type: physical # type: physical
# permanent_logical_1: # - permanent_logical_1:
# type: logical # type: logical
# database: foo # database: foo
# plugin: pgoutput # plugin: pgoutput
ttl: 30 ttl: 30
loop_wait: &loop_wait 10 loop_wait: &loop_wait 10
retry_timeout: 10 retry_timeout: 10
@ -63,15 +63,15 @@ spec:
# restore a Postgres DB with point-in-time-recovery # restore a Postgres DB with point-in-time-recovery
# with a non-empty timestamp, clone from an S3 bucket using the latest backup before the timestamp # with a non-empty timestamp, clone from an S3 bucket using the latest backup before the timestamp
# with an empty/absent timestamp, clone from an existing alive cluster using pg_basebackup # with an empty/absent timestamp, clone from an existing alive cluster using pg_basebackup
# clone: # clone:
# uid: "efd12e58-5786-11e8-b5a7-06148230260c" # uid: "efd12e58-5786-11e8-b5a7-06148230260c"
# cluster: "acid-batman" # cluster: "acid-batman"
# timestamp: "2017-12-19T12:40:33+01:00" # timezone required (offset relative to UTC, see RFC 3339 section 5.6) # timestamp: "2017-12-19T12:40:33+01:00" # timezone required (offset relative to UTC, see RFC 3339 section 5.6)
# s3_wal_path: "s3://custom/path/to/bucket" # s3_wal_path: "s3://custom/path/to/bucket"
# run periodic backups with k8s cron jobs # run periodic backups with k8s cron jobs
# enableLogicalBackup: true # enableLogicalBackup: true
# logicalBackupSchedule: "30 00 * * *" # logicalBackupSchedule: "30 00 * * *"
maintenanceWindows: maintenanceWindows:
- 01:00-06:00 #UTC - 01:00-06:00 #UTC
- Sat:00:00-04:00 - Sat:00:00-04:00

4
run_operator_locally.sh
View File

@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# #
# Deploy a Postgres operator to a minikube aka local Kubernetes cluster # Deploy a Postgres Operator to a minikube aka local Kubernetes cluster
# Optionally re-build the operator binary beforehand to test local changes # Optionally re-build the operator binary beforehand to test local changes
# Known limitations: # Known limitations:
@ -213,7 +213,7 @@ function submit_postgresql_manifest(){
function main(){ function main(){
if ! [[ $(basename "$PWD") == "postgres-operator" ]]; then if ! [[ $(basename "$PWD") == "postgres-operator" ]]; then
echo "Please execute the script only from the root directory of the Postgres operator repo." echo "Please execute the script only from the root directory of the Postgres Operator repo."
exit 1 exit 1
fi fi