public_git
/
postgres-operator
mirror of https://github.com/zalando/postgres-operator.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

update docs and README

This commit is contained in:
Felix Kunde 2019-07-01 14:54:32 +02:00
parent 44aef63766
commit d1d2341477
6 changed files with 188 additions and 140 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -15,7 +15,7 @@ pipelines with no access to Kubernetes directly.
By default, the operator is building up on two other Open Source projects
developed at Zalando. [Spilo](https://github.com/zalando/spilo) provides the
Docker image that contains PostgreSQL incl. some pre-compiled extensions. Spilo
also includes [Patroni]((https://github.com/zalando/spilo), to manage highly
also includes [Patroni]((https://github.com/zalando/spilo) to manage highly
available Postgres cluster powered by streaming replication.
# Getting started

123
docs/administrator.md
View File

@ -1,9 +1,11 @@
# Administrator Guide
Learn how to configure and manage the Postgres Operator in your Kubernetes
Learn how to configure and manage the Postgres Operator in your Kubernetes (K8s)
environment.
## Select the namespace to deploy to
## Namespaces
### Select the namespace to deploy to
The operator can run in a namespace other than `default`. For example, to use
the `test` namespace, run the following before deploying the operator's
@ -20,7 +22,7 @@ ConfigMap - there. Please note that the namespace for service accounts and
cluster role bindings in [operator RBAC rules](../manifests/operator-service-account-rbac.yaml)
needs to be adjusted to the non-default value.
## Specify the namespace to watch
### Specify the namespace to watch
Watching a namespace for an operator means tracking requests to change
Postgresql clusters in the namespace such as "increase the number of Postgresql
@ -39,14 +41,14 @@ Note that for an operator to manage pods in the watched namespace, the
operator's service account (as specified in the operator deployment manifest)
has to have appropriate privileges to access the watched namespace. The
operator may not be able to function in the case it watches all namespaces but
lacks access rights to any of them (except Kubernetes system namespaces like
lacks access rights to any of them (except K8s system namespaces like
`kube-system`). The reason is that for multiple namespaces operations such as
'list pods' execute at the cluster scope and fail at the first violation of
access rights.
The watched namespace also needs to have a (possibly different) service account
in the case database pods need to talk to the Kubernetes API (e.g. when using
Kubernetes-native configuration of Patroni). The operator checks that the
in the case database pods need to talk to the K8s API (e.g. when using
K8s-native configuration of Patroni). The operator checks that the
`pod_service_account_name` exists in the target namespace, and, if not, deploys
there the `pod_service_account_definition` from the operator
[`Config`](../pkg/util/config/config.go) with the default value of:
@ -64,15 +66,18 @@ namespace. The operator performs **no** further syncing of this account.
## Non-default cluster domain
If your cluster uses a different dns domain than `cluster.local`, this needs
to be set in the operator ConfigMap. This is used by the operator to connect
to the clusters after creation.
If your cluster uses a different DNS domain than `cluster.local`, this needs
to be set in the operator configuration (`cluster_domain` variable). This is
used by the operator to connect to the clusters after creation.
## Role-based access control for the operator
The `manifests/operator-service-account-rbac.yaml` defines cluster roles and
bindings needed for the operator to function under access control restrictions.
To deploy the operator with this RBAC policy use:
### Service account and cluster roles
The manifest [`operator-service-account-rbac.yaml`](../manifests/operator-service-account-rbac.yaml)
defines the service account, cluster roles and bindings needed for the operator
to function under access control restrictions. To deploy the operator with this
RBAC policy use:
```bash
$ kubectl create -f manifests/configmap.yaml
@ -81,26 +86,36 @@ To deploy the operator with this RBAC policy use:
$ kubectl create -f manifests/minimal-postgres-manifest.yaml
```
Note that the service account in `operator-rbac.yaml` is named
`zalando-postgres-operator`. You may have to change the `service_account_name`
in the operator ConfigMap and `serviceAccountName` in the postgres-operator
deployment appropriately.
Note that the service account is named `zalando-postgres-operator`. You may have
to change the `service_account_name` in the operator ConfigMap and
`serviceAccountName` in the `postgres-operator` deployment appropriately. This
is done intentionally to avoid breaking those setups that already work with the
default `operator` account. In the future the operator should ideally be run
under the `zalando-postgres-operator` service account.
This is done intentionally to avoid breaking those setups that already work
with the default `operator` account. In the future the operator should ideally
be run under the `zalando-postgres-operator` service account.
The service account defined in `operator-service-account-rbac.yaml` acquires
some privileges not really used by the operator (i.e. we only need `list` and
`watch` on `configmaps` resources), this is also done intentionally to avoid
breaking things if someone decides to configure the same service account in the
operator's ConfigMap to run Postgres clusters.
The service account defined in `operator-rbac.yaml` acquires some privileges
not really used by the operator (i.e. we only need `list` and `watch` on
`configmaps` resources), this is also done intentionally to avoid breaking
things if someone decides to configure the same service account in the
operator's ConfigMap to run postgres clusters.
### Give K8S users access to create/list postgresqls
### Use taints and tolerations for dedicated PostgreSQL nodes
By default `postgresql` custom resources can only by listed and changed by
cluster admins. To allow read and/or write access to other human users apply
the `ùser-facing-clusterrole` manifest:
To ensure Postgres pods are running on nodes without any other application
pods, you can use
[taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
```bash
$ kubectl create -f manifests/user-facing-clusterroles.yaml
```
It creates zalando-postgres-operator:user:view, :edit and :admin clusterroles
that are aggregated into the K8s [default roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings).
## Use taints and tolerations for dedicated PostgreSQL nodes
To ensure Postgres pods are running on nodes without any other application pods,
you can use [taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
and configure the required toleration in the operator ConfigMap.
As an example you can set following node taint:
@ -109,7 +124,7 @@ As an example you can set following node taint:
$ kubectl taint nodes <nodeName> postgres=:NoSchedule
```
And configure the toleration for the PostgreSQL pods by adding following line
And configure the toleration for the Postgres pods by adding following line
to the ConfigMap:
```yaml
@ -122,15 +137,15 @@ data:
...
```
Note that the Kubernetes version 1.13 brings [taint-based eviction](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions) to the beta stage and enables it by default.
Postgres pods by default receive tolerations for `unreachable` and `noExecute`
taints with the timeout of `5m`. Depending on your setup, you may want to adjust
these parameters to prevent master pods from being evicted by the Kubernetes
runtime. To prevent eviction completely, specify the toleration by leaving out
the `tolerationSeconds` value (similar to how Kubernetes' own DaemonSets are
configured)
Note that the K8s version 1.13 brings [taint-based eviction](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions)
to the beta stage and enables it by default. Postgres pods by default receive
tolerations for `unreachable` and `noExecute` taints with the timeout of `5m`.
Depending on your setup, you may want to adjust these parameters to prevent
master pods from being evicted by the K8s runtime. To prevent eviction
completely, specify the toleration by leaving out the `tolerationSeconds` value
(similar to how Kubernetes' own DaemonSets are configured)
### Enable pod anti affinity
## Enable pod anti affinity
To ensure Postgres pods are running on different topologies, you can use
[pod anti affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/)
@ -162,10 +177,10 @@ data:
pod_antiaffinity_topology_key: "failure-domain.beta.kubernetes.io/zone"
```
### Add cluster-specific labels
## Add cluster-specific labels
In some cases, you might want to add `labels` that are specific to a given
postgres cluster, in order to identify its child objects. The typical use case
Postgres cluster, in order to identify its child objects. The typical use case
is to add labels that identifies the `Pods` created by the operator, in order
to implement fine-controlled `NetworkPolicies`.
@ -251,7 +266,7 @@ data:
This ConfigMap is then added as a source of environment variables to the
Postgres StatefulSet/pods.
## Limiting the number of instances in clusters with `min_instances` and `max_instances`
## Limiting the number of min and max instances in clusters
As a preventive measure, one can restrict the minimum and the maximum number of
instances permitted by each Postgres cluster managed by the operator. If either
@ -261,9 +276,9 @@ either the min or the max boundary. For instance, of a cluster manifest has 1
instance and the `min_instances` is set to 3, the cluster will be created with 3
instances. By default, both parameters are set to `-1`.
## Load balancers
## Load balancers and allowed IP ranges
For any Postgresql/Spilo cluster, the operator creates two separate Kubernetes
For any Postgresql/Spilo cluster, the operator creates two separate K8s
services: one for the master pod and one for replica pods. To expose these
services to an outer network, one can attach load balancers to them by setting
`enableMasterLoadBalancer` and/or `enableReplicaLoadBalancer` to `true` in the
@ -279,21 +294,21 @@ Internet, this field is set at cluster creation time to `127.0.0.1/32` unless
overwritten explicitly. If you want to revoke all IP ranges from an existing
cluster, please set the `allowedSourceRanges` field to `127.0.0.1/32` or to an
empty sequence `[]`. Setting the field to `null` or omitting it entirely may
lead to Kubernetes removing this field from the manifest due to its
lead to K8s removing this field from the manifest due to its
[handling of null fields](https://kubernetes.io/docs/concepts/overview/object-management-kubectl/declarative-config/#how-apply-calculates-differences-and-merges-changes).
Then the resultant manifest will not contain the necessary change, and the
operator will respectively do noting with the existing source ranges.
## Running periodic 'autorepair' scans of Kubernetes objects
## Running periodic 'autorepair' scans of K8s objects
The Postgres Operator periodically scans all Kubernetes objects belonging to
each cluster and repairs all discrepancies between them and the definitions
generated from the current cluster manifest. There are two types of scans:
The Postgres Operator periodically scans all K8s objects belonging to each
cluster and repairs all discrepancies between them and the definitions generated
from the current cluster manifest. There are two types of scans:
* `sync scan`, running every `resync_period` seconds for every cluster
* `repair scan`, coming every `repair_period` only for those clusters that didn't
report success as a result of the last operation applied to them.
* `repair scan`, coming every `repair_period` only for those clusters that
didn't report success as a result of the last operation applied to them.
## Postgres roles supported by the operator
@ -306,7 +321,7 @@ creating such roles to Patroni and only establishes relevant secrets.
* **Infrastructure roles** are roles for processes originating from external
systems, e.g. monitoring robots. The operator creates such roles in all Postgres
clusters it manages assuming that Kubernetes secrets with the relevant
clusters it manages assuming that K8s secrets with the relevant
credentials exist beforehand.
* **Per-cluster robot users** are also roles for processes originating from
@ -318,8 +333,8 @@ database.
members given a team id. The operator differentiates between (a) product teams
that own a particular Postgres cluster and are granted admin rights to maintain
it, and (b) Postgres superuser teams that get the superuser access to all
Postgres databases running in a Kubernetes cluster for the purposes of
maintaining and troubleshooting.
Postgres databases running in a K8s cluster for the purposes of maintaining and
troubleshooting.
## Understanding rolling update of Spilo pods
@ -328,7 +343,7 @@ between the old and new StatefulSet specs with the `debug` level. To benefit
from numerous escape characters in the latter log entry, view it in CLI with
`echo -e`. Note that the resultant message will contain some noise because the
`PodTemplate` used by the operator is yet to be updated with the default values
used internally in Kubernetes.
used internally in K8s.
## Logical backups
@ -346,7 +361,7 @@ bucket; the default image ``registry.opensource.zalan.do/acid/logical-backup``
is the same image built with the Zalando-internal CI pipeline. `pg_dumpall`
requires a `superuser` access to a DB and runs on the replica when possible.
2. Due to the [limitation of Kubernetes cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations)
2. Due to the [limitation of K8s cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations)
it is highly advisable to set up additional monitoring for this feature; such
monitoring is outside of the scope of operator responsibilities.

92
docs/developer.md
View File

@ -1,6 +1,6 @@
# Developer Guide
Reed this guide if you want to debug the operator, fix bugs or contribute new
Read this guide if you want to debug the operator, fix bugs or contribute new
features and tests.
## Setting up Go
@ -63,22 +63,43 @@ The binary will be placed into the build directory.
## Deploying self build image
The fastest way to run your docker image locally is to reuse the docker from
minikube. The following steps will get you the docker image built and deployed.
The fastest way to run and test your docker image locally is to reuse the docker
from [minikube]((https://github.com/kubernetes/minikube/releases)) or use the
`load docker-image` from [kind](https://kind.sigs.k8s.io/). The following steps
will get you the docker image built and deployed.
```bash
# minikube
$ eval $(minikube docker-env)
$ export TAG=$(git describe --tags --always --dirty)
$ make docker
$ sed -e "s/\(image\:.*\:\).*$/\1$TAG/" manifests/postgres-operator.yaml|kubectl --context minikube create -f -
# kind
$ export TAG=$(git describe --tags --always --dirty)
$ make docker
$ kind load docker-image <image> --name <kind-cluster-name>
```
Then create a new Postgres Operator deployment. You can reuse the provided
manifest but replace the version and tag. Don't forget to also apply
configuration and RBAC manifests first, e.g.:
```bash
$ kubectl create -f manifests/configmap.yaml
$ kubectl create -f manifests/operator-service-account-rbac.yaml
$ sed -e "s/\(image\:.*\:\).*$/\1$TAG/" manifests/postgres-operator.yaml | kubectl create -f -
# check if the operator is coming up
$ kubectl get pod -l name=postgres-operator
```
## Code generation
The operator employs k8s-provided code generation to obtain deep copy methods
and Kubernetes-like APIs for its custom resource definitons, namely the Postgres
CRD and the operator CRD. The usage of the code generation follows conventions
from the k8s community. Relevant scripts live in the `hack` directory:
The operator employs K8s-provided code generation to obtain deep copy methods
and K8s-like APIs for its custom resource definitions, namely the
Postgres CRD and the operator CRD. The usage of the code generation follows
conventions from the k8s community. Relevant scripts live in the `hack`
directory:
* `update-codegen.sh` triggers code generation for the APIs defined in `pkg/apis/acid.zalan.do/`,
* `verify-codegen.sh` checks if the generated code is up-to-date (to be used within CI).
@ -110,7 +131,9 @@ localhost:8080 by doing:
The inner 'query' gets the name of the postgres operator pod, and the outer
enables port forwarding. Afterwards, you can access the operator API with:
```
$ curl --location http://127.0.0.1:8080/$endpoint | jq .
```
The available endpoints are listed below. Note that the worker ID is an integer
from 0 up to 'workers' - 1 (value configured in the operator configuration and
@ -150,7 +173,7 @@ need:
* Install delve locally
```
go get -u github.com/derekparker/delve/cmd/dlv
$ go get -u github.com/derekparker/delve/cmd/dlv
```
* Add following dependencies to the `Dockerfile`
@ -177,13 +200,13 @@ CMD ["/root/go/bin/dlv", "--listen=:DLV_PORT", "--headless=true", "--api-version
* Forward the listening port
```
kubectl port-forward POD_NAME DLV_PORT:DLV_PORT
$ kubectl port-forward POD_NAME DLV_PORT:DLV_PORT
```
* Attach to it
```
$ dlv connect 127.0.0.1:DLV_PORT
$ dlv connect 127.0.0.1:DLV_PORT
```
## Unit tests
@ -191,46 +214,60 @@ $ dlv connect 127.0.0.1:DLV_PORT
To run all unit tests, you can simply do:
```
$ go test ./...
$ go test ./...
```
For go 1.9 `vendor` directory would be excluded automatically. For previous
versions you can exclude it manually:
```
$ go test $(glide novendor)
$ go test $(glide novendor)
```
In case if you need to debug your unit test, it's possible to use delve:
```
$ dlv test ./pkg/util/retryutil/
Type 'help' for list of commands.
(dlv) c
PASS
$ dlv test ./pkg/util/retryutil/
Type 'help' for list of commands.
(dlv) c
PASS
```
To test the multinamespace setup, you can use
```
./run_operator_locally.sh --rebuild-operator
$ ./run_operator_locally.sh --rebuild-operator
```
It will automatically create an `acid-minimal-cluster` in the namespace `test`.
Then you can for example check the Patroni logs:
```
kubectl logs acid-minimal-cluster-0
$ kubectl logs acid-minimal-cluster-0
```
## End-to-end tests
The operator provides reference e2e (end-to-end) tests to ensure various infra parts work smoothly together.
Each e2e execution tests a Postgres Operator image built from the current git branch. The test runner starts a [kind](https://kind.sigs.k8s.io/) (local k8s) cluster and Docker container with tests. The k8s API client from within the container connects to the `kind` cluster using the standard Docker `bridge` network.
The tests utilize examples from `/manifests` (ConfigMap is used for the operator configuration) to avoid maintaining yet another set of configuration files. The kind cluster is deleted if tests complete successfully.
The operator provides reference e2e (end-to-end) tests to ensure various infra
parts work smoothly together. Each e2e execution tests a Postgres Operator image
built from the current git branch. The test runner starts a [kind](https://kind.sigs.k8s.io/)
(local k8s) cluster and Docker container with tests. The k8s API client from
within the container connects to the `kind` cluster using the standard Docker
`bridge` network. The tests utilize examples from `/manifests` (ConfigMap is
used for the operator configuration) to avoid maintaining yet another set of
configuration files. The kind cluster is deleted if tests complete successfully.
End-to-end tests are executed automatically during builds; to invoke them locally use `make e2e-run` from the project's top directory. Run `make e2e-tools e2e-build` to install `kind` and build the tests' image locally before the first run.
End-to-end tests are executed automatically during builds:
End-to-end tests are written in Python and use `flake8` for code quality. Please run flake8 [before submitting a PR](http://flake8.pycqa.org/en/latest/user/using-hooks.html).
```bash
# invoke them from the project's top directory
$ make e2e-run
# install kind and build test image before first run
$ make e2e-tools e2e-build
```
End-to-end tests are written in Python and use `flake8` for code quality.
Please run flake8 [before submitting a PR](http://flake8.pycqa.org/en/latest/user/using-hooks.html).
## Introduce additional configuration parameters
@ -240,8 +277,11 @@ be updated. As explained [here](reference/operator_parameters.md), it's possible
to configure the operator either with a ConfigMap or CRD, but currently we aim
to synchronize parameters everywhere.
When choosing a parameter name for a new option in a PG manifest, keep in mind
the naming conventions there. The `snake_case` variables come from the Patroni/Postgres world, while the `camelCase` from the k8s world.
When choosing a parameter name for a new option in a Postgres cluster manifest,
keep in mind the naming conventions there. We use `camelCase` for manifest
parameters (with exceptions for certain Patroni/Postgres options) and
`snake_case` variables in the configuration. Only introduce new manifest
variables if you feel a per-cluster configuration is necessary.
Note: If one option is defined in the operator configuration and in the cluster
[manifest](../manifests/complete-postgres-manifest.yaml), the latter takes

18
docs/index.md
View File

@ -1,12 +1,12 @@
# Introduction
The Postgres [operator](https://coreos.com/blog/introducing-operators.html)
manages PostgreSQL clusters on Kubernetes:
manages PostgreSQL clusters on Kubernetes (K8s):
1. The operator watches additions, updates, and deletions of PostgreSQL cluster
manifests and changes the running clusters accordingly. For example, when a
user submits a new manifest, the operator fetches that manifest and spawns a
new Postgres cluster along with all necessary entities such as Kubernetes
new Postgres cluster along with all necessary entities such as K8s
StatefulSets and Postgres roles. See this
[Postgres cluster manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/complete-postgres-manifest.yaml)
for settings that a manifest may contain.
@ -22,7 +22,7 @@ manages PostgreSQL clusters on Kubernetes:
4. The operator aims to be hands free and configuration happens only via
manifests and its own config. This enables easy integration in automated
deploy pipelines with no access to Kubernetes directly.
deploy pipelines with no access to K8s directly.
## Concepts
@ -30,10 +30,10 @@ manages PostgreSQL clusters on Kubernetes:
The scope of the postgres operator is on provisioning, modifying configuration
and cleaning up Postgres clusters that use Patroni, basically to make it easy
and convenient to run Patroni based clusters on Kubernetes. The provisioning
and modifying includes Kubernetes resources on one side but also e.g. database
and convenient to run Patroni based clusters on K8s. The provisioning
and modifying includes K8s resources on one side but also e.g. database
and role provisioning once the cluster is up and running. We try to leave as
much work as possible to Kubernetes and to Patroni where it fits, especially
much work as possible to K8s and to Patroni where it fits, especially
the cluster bootstrap and high availability. The operator is however involved
in some overarching orchestration, like rolling updates to improve the user
experience.
@ -61,9 +61,9 @@ functionality the operator provides.
This project is currently in active development. It is however already
[used internally by Zalando](https://jobs.zalando.com/tech/blog/postgresql-in-a-time-of-kubernetes/)
in order to run Postgres clusters on Kubernetes in larger numbers for staging
in order to run Postgres clusters on K8s in larger numbers for staging
environments and a growing number of production clusters. In this environment
the operator is deployed to multiple Kubernetes clusters, where users deploy
the operator is deployed to multiple K8s clusters, where users deploy
manifests via our CI/CD infrastructure or rely on a slim user interface to
create manifests.
@ -71,7 +71,7 @@ Please, report any issues discovered to https://github.com/zalando/postgres-oper
## Talks
1. "PostgreSQL and Kubernetes: DBaaS without a vendor-lock" talk by Oleksii Kliukin, PostgreSQL Sessions 2018: [video](https://www.youtube.com/watch?v=q26U2rQcqMw) | [slides](https://speakerdeck.com/alexeyklyukin/postgresql-and-kubernetes-dbaas-without-a-vendor-lock)
1. "PostgreSQL and K8s: DBaaS without a vendor-lock" talk by Oleksii Kliukin, PostgreSQL Sessions 2018: [video](https://www.youtube.com/watch?v=q26U2rQcqMw) | [slides](https://speakerdeck.com/alexeyklyukin/postgresql-and-kubernetes-dbaas-without-a-vendor-lock)
2. "PostgreSQL High Availability on Kubernetes with Patroni" talk by Oleksii Kliukin, Atmosphere 2018: [video](https://www.youtube.com/watch?v=cFlwQOPPkeg) | [slides](https://speakerdeck.com/alexeyklyukin/postgresql-high-availability-on-kubernetes-with-patroni)

33
docs/quickstart.md
View File

@ -16,10 +16,10 @@ For local tests we recommend to use one of the following solutions:
To interact with the K8s infrastructure install it's CLI runtime [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-via-curl).
This quickstart assumes that you haved started minikube or created a local kind
cluster. Note that you can also use built-in Kubernetes support in the Docker
cluster. Note that you can also use built-in K8s support in the Docker
Desktop for Mac to follow the steps of this tutorial. You would have to replace
`minikube start` and `minikube delete` with your launch actions for the Docker
built-in Kubernetes support.
built-in K8s support.
## Configuration Options
@ -32,7 +32,7 @@ Postgres cluster. This can happen in two ways: Via a ConfigMap or a custom
## Deployment options
The Postgres Operator can be deployed in different way:
The Postgres Operator can be deployed in different ways:
* Manual deployment
* Helm chart
@ -42,7 +42,7 @@ The Postgres Operator can be deployed in different way:
The Postgres Operator can be installed simply by applying yaml manifests. Note,
we provide the `/manifests` directory as an example only; you should consider
adjusting the manifests to your particular setting (e.g. namespaces).
adjusting the manifests to your K8s environment (e.g. namespaces).
```bash
# First, clone the repository and change to the directory
@ -67,15 +67,15 @@ For convenience, we have automated starting the operator and submitting the
Alternatively, the operator can be installed by using the provided [Helm](https://helm.sh/)
chart which saves you the manual steps. Therefore, you would need to install
the helm CLI on your machine. After initializing helm (and its server
component Tiller) in your local cluster you can install the operator chart.
You can define a release name that is prepended to the operator resource's
names.
the helm CLI on your machine. After initializing helm (and its server component
Tiller) in your local cluster you can install the operator chart. You can define
a release name that is prepended to the operator resource's names.
Use `--name zalando` to match with the default service account name as older
operator versions do not support custom names for service accounts. When relying
solely on the CRD-based configuration edit the `serviceAccount` section in the
[values yaml file](../charts/values.yaml) by setting the name to `"operator"`.
solely on the CRD-based configuration use the [values-crd yaml file](../charts/values-crd.yaml)
and comment the ConfigMap template in the [helmignore](../charts/.helmignore)
file.
```bash
# 1) initialize helm
@ -139,13 +139,20 @@ kubectl get svc -l application=spilo -L spilo-role
## Connect to the Postgres cluster via psql
You can retrieve the host and port of the Postgres master from minikube.
Retrieve the password from the Kubernetes Secret that is created in your cluster.
You can create a port-forward on a database pod to connect to Postgres. See the
[user guide](user.md#connect-to-postgresql) for instructions. With minikube it's
also easy to retrieve the connections string from the K8s service that is
pointing to the master pod:
```bash
export HOST_PORT=$(minikube service acid-minimal-cluster --url | sed 's,.*/,,')
export PGHOST=$(echo $HOST_PORT | cut -d: -f 1)
export PGPORT=$(echo $HOST_PORT | cut -d: -f 2)
```
Retrieve the password from the K8s Secret that is created in your cluster.
```bash
export PGPASSWORD=$(kubectl get secret postgres.acid-minimal-cluster.credentials -o 'jsonpath={.data.password}' | base64 -d)
psql -U postgres
```
@ -165,4 +172,4 @@ deleted. Secrets however are not deleted. When deleting a cluster while it is
still starting up it can [happen](https://github.com/zalando/postgres-operator/issues/551)
the `postgresql` resource is deleted leaving orphaned components behind. This
can cause troubles when creating a new Postgres cluster. For a fresh setup you
can delete your local minikube and kind cluster and start again.
can delete your local minikube or kind cluster and start again.

60
docs/user.md
View File

@ -1,6 +1,6 @@
# User Guide
Learn how to work with the Postgres Operator in a Kubernetes environment.
Learn how to work with the Postgres Operator in a Kubernetes (K8s) environment.
## Create a manifest for a new PostgreSQL cluster
@ -46,15 +46,6 @@ $ kubectl create -f manifests/minimal-postgres-manifest.yaml
$ kubectl get pods -w --show-labels
```
## Give K8S users access to create/list postgresqls
```bash
$ kubectl create -f manifests/user-facing-clusterroles.yaml
```
Creates zalando-postgres-operator:users:view, :edit and :admin clusterroles that
are aggregated into the default roles.
## Connect to PostgreSQL
With a `port-forward` on one of the database pods (e.g. the master) you can
@ -97,14 +88,13 @@ In the next sections, we will cover those use cases in more details.
Manifest roles are defined directly in the cluster manifest. See
[minimal postgres manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml)
for an example of `zalando` role, defined with `superuser` and `createdb`
flags.
for an example of `zalando` role, defined with `superuser` and `createdb` flags.
Manifest roles are defined as a dictionary, with a role name as a key and a
list of role options as a value. For a role without any options it is best to
supply the empty list `[]`. It is also possible to leave this field empty as in
our example manifests, but in certain cases such empty field may removed by
Kubernetes [due to the `null` value it gets](https://kubernetes.io/docs/concepts/overview/object-management-kubectl/declarative-config/#how-apply-calculates-differences-and-merges-changes)
our example manifests, but in certain cases such empty field may removed by K8s
[due to the `null` value it gets](https://kubernetes.io/docs/concepts/overview/object-management-kubectl/declarative-config/#how-apply-calculates-differences-and-merges-changes)
(`foobar_user:` is equivalent to `foobar_user: null`).
The operator accepts the following options: `superuser`, `inherit`, `login`,
@ -117,7 +107,7 @@ The operator automatically generates a password for each manifest role and
places it in the secret named
`{username}.{team}-{clustername}.credentials.postgresql.acid.zalan.do` in the
same namespace as the cluster. This way, the application running in the
Kubernetes cluster and working with the database can obtain the password right
K8s cluster and working with the database can obtain the password right
from the secret, without ever sharing it outside of the cluster.
At the moment it is not possible to define membership of the manifest role in
@ -144,9 +134,9 @@ parameter. The role definition looks like this (values are base64 encoded):
```
The block above describes the infrastructure role 'dbuser' with password
'secret' that is a member of the 'operator' role. For the following
definitions one must increase the index, i.e. the next role will be defined as
'user2' and so on. The resulting role will automatically be a login role.
'secret' that is a member of the 'operator' role. For the following definitions
one must increase the index, i.e. the next role will be defined as 'user2' and
so on. The resulting role will automatically be a login role.
Note that with definitions that solely use the infrastructure roles secret
there is no way to specify role options (like superuser or nologin) or role
@ -155,11 +145,10 @@ memberships. This is where the ConfigMap comes into play.
### Secret plus ConfigMap
A [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/)
allows for defining more details regarding the infrastructure roles.
Therefore, one should use the new style that specifies infrastructure roles
using both the secret and a ConfigMap. The ConfigMap must have the same name as
the secret. The secret should contain an entry with 'rolename:rolepassword' for
each role.
allows for defining more details regarding the infrastructure roles. Therefore,
one should use the new style that specifies infrastructure roles using both the
secret and a ConfigMap. The ConfigMap must have the same name as the secret.
The secret should contain an entry with 'rolename:rolepassword' for each role.
```yaml
dbuser: c2VjcmV0
@ -194,9 +183,8 @@ and [infrastructure roles configmap](https://github.com/zalando/postgres-operato
## Use taints and tolerations for dedicated PostgreSQL nodes
To ensure Postgres pods are running on nodes without any other application
pods, you can use
[taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
To ensure Postgres pods are running on nodes without any other application pods,
you can use [taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
and configure the required toleration in the manifest.
```yaml
@ -257,10 +245,10 @@ spec:
```
Here `cluster` is a name of a source cluster that is going to be cloned. A new
cluster will be cloned from S3, using the latest backup before the
`timestamp`. In this case, `uid` field is also mandatory - operator will use it
to find a correct key inside an S3 bucket. You can find this field from
metadata of a source cluster:
cluster will be cloned from S3, using the latest backup before the `timestamp`.
In this case, `uid` field is also mandatory - operator will use it to find a
correct key inside an S3 bucket. You can find this field from metadata of a
source cluster:
```yaml
apiVersion: acid.zalan.do/v1
@ -386,14 +374,12 @@ actions:
* call AWS API to change the volume size
* connect to the pod using `kubectl exec` and resize the filesystem with
`resize2fs`.
* connect to pod using `kubectl exec` and resize filesystem with `resize2fs`.
Fist step has a limitation, AWS rate-limits this operation to no more than once
every 6 hours.
Note that if the statefulset is scaled down before resizing the size changes
are only applied to the volumes attached to the running pods. The size of the
volumes that correspond to the previously running pods is not changed.
every 6 hours. Note, that if the statefulset is scaled down before resizing the
size changes are only applied to the volumes attached to the running pods. The
size of volumes that correspond to the previously running pods is not changed.
## Logical backups
@ -403,7 +389,7 @@ If you add
```
to the cluster manifest, the operator will create and sync a k8s cron job to do
periodic logical backups of this particular Postgres cluster. Due to the
[limitation of Kubernetes cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations)
[limitation of K8s cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations)
it is highly advisable to set up additional monitoring for this feature; such
monitoring is outside of the scope of operator responsibilities. See
[configuration reference](reference/cluster_manifest.md) and