mirror of
https://github.com/zalando/postgres-operator.git
synced 2026-10-06 07:31:35 +02:00
merge with master and resolve conflicts
This commit is contained in:
+85
-57
@@ -11,11 +11,11 @@ switchover (planned failover) of the master to the Pod with new minor version.
|
||||
The switch should usually take less than 5 seconds, still clients have to
|
||||
reconnect.
|
||||
|
||||
Major version upgrades are supported via [cloning](user.md#clone-directly). The
|
||||
new cluster manifest must have a higher `version` string than the source cluster
|
||||
and will be created from a basebackup. Depending of the cluster size, downtime
|
||||
in this case can be significant as writes to the database should be stopped and
|
||||
all WAL files should be archived first before cloning is started.
|
||||
Major version upgrades are supported via [cloning](user.md#how-to-clone-an-existing-postgresql-cluster).
|
||||
The new cluster manifest must have a higher `version` string than the source
|
||||
cluster and will be created from a basebackup. Depending of the cluster size,
|
||||
downtime in this case can be significant as writes to the database should be
|
||||
stopped and all WAL files should be archived first before cloning is started.
|
||||
|
||||
Note, that simply changing the version string in the `postgresql` manifest does
|
||||
not work at present and leads to errors. Neither Patroni nor Postgres Operator
|
||||
@@ -47,6 +47,12 @@ patching the CRD manifest:
|
||||
zk8 patch crd postgresqls.acid.zalan.do -p '{"spec":{"validation": null}}'
|
||||
```
|
||||
|
||||
## Non-default cluster domain
|
||||
|
||||
If your cluster uses a DNS domain other than the default `cluster.local`, this
|
||||
needs to be set in the operator configuration (`cluster_domain` variable). This
|
||||
is used by the operator to connect to the clusters after creation.
|
||||
|
||||
## Namespaces
|
||||
|
||||
### Select the namespace to deploy to
|
||||
@@ -89,36 +95,13 @@ lacks access rights to any of them (except K8s system namespaces like
|
||||
'list pods' execute at the cluster scope and fail at the first violation of
|
||||
access rights.
|
||||
|
||||
The watched namespace also needs to have a (possibly different) service account
|
||||
in the case database pods need to talk to the K8s API (e.g. when using
|
||||
K8s-native configuration of Patroni). The operator checks that the
|
||||
`pod_service_account_name` exists in the target namespace, and, if not, deploys
|
||||
there the `pod_service_account_definition` from the operator
|
||||
[`Config`](../pkg/util/config/config.go) with the default value of:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: operator
|
||||
```
|
||||
|
||||
In this definition, the operator overwrites the account's name to match
|
||||
`pod_service_account_name` and the `default` namespace to match the target
|
||||
namespace. The operator performs **no** further syncing of this account.
|
||||
|
||||
## Non-default cluster domain
|
||||
|
||||
If your cluster uses a DNS domain other than the default `cluster.local`, this
|
||||
needs to be set in the operator configuration (`cluster_domain` variable). This
|
||||
is used by the operator to connect to the clusters after creation.
|
||||
|
||||
## Role-based access control for the operator
|
||||
|
||||
The manifest [`operator-service-account-rbac.yaml`](../manifests/operator-service-account-rbac.yaml)
|
||||
defines the service account, cluster roles and bindings needed for the operator
|
||||
to function under access control restrictions. To deploy the operator with this
|
||||
RBAC policy use:
|
||||
to function under access control restrictions. The file also includes a cluster
|
||||
role `postgres-pod` with privileges for Patroni to watch and manage pods and
|
||||
endpoints. To deploy the operator with this RBAC policies use:
|
||||
|
||||
```bash
|
||||
kubectl create -f manifests/operatorconfiguration.crd.yaml
|
||||
@@ -128,14 +111,14 @@ kubectl create -f manifests/postgres-operator.yaml
|
||||
kubectl create -f manifests/minimal-postgres-manifest.yaml
|
||||
```
|
||||
|
||||
### Service account and cluster roles
|
||||
### Namespaced service account and role binding
|
||||
|
||||
Note that the service account is named `zalando-postgres-operator`. You may have
|
||||
to change the `service_account_name` in the operator ConfigMap and
|
||||
`serviceAccountName` in the `postgres-operator` deployment appropriately. This
|
||||
is done intentionally to avoid breaking those setups that already work with the
|
||||
default `operator` account. In the future the operator should ideally be run
|
||||
under the `zalando-postgres-operator` service account.
|
||||
For each namespace the operator watches it creates (or reads) a service account
|
||||
and role binding to be used by the Postgres Pods. The service account is bound
|
||||
to the `postgres-pod` cluster role. The name and definitions of these resources
|
||||
can be [configured](reference/operator_parameters.md#kubernetes-resources).
|
||||
Note, that the operator performs **no** further syncing of namespaced service
|
||||
accounts and role bindings.
|
||||
|
||||
### Give K8s users access to create/list `postgresqls`
|
||||
|
||||
@@ -377,6 +360,17 @@ cluster manifest. In the case any of these variables are omitted from the
|
||||
manifest, the operator configuration settings `enable_master_load_balancer` and
|
||||
`enable_replica_load_balancer` apply. Note that the operator settings affect
|
||||
all Postgresql services running in all namespaces watched by the operator.
|
||||
If load balancing is enabled two default annotations will be applied to its
|
||||
services:
|
||||
|
||||
- `external-dns.alpha.kubernetes.io/hostname` with the value defined by the
|
||||
operator configs `master_dns_name_format` and `replica_dns_name_format`.
|
||||
This value can't be overwritten. If any changing in its value is needed, it
|
||||
MUST be done changing the DNS format operator config parameters; and
|
||||
- `service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout` with
|
||||
a default value of "3600". This value can be overwritten with the operator
|
||||
config parameter `custom_service_annotations` or the cluster parameter
|
||||
`serviceAnnotations`.
|
||||
|
||||
To limit the range of IP addresses that can reach a load balancer, specify the
|
||||
desired ranges in the `allowedSourceRanges` field (applies to both master and
|
||||
@@ -487,37 +481,71 @@ A secret can be pre-provisioned in different ways:
|
||||
|
||||
## Setting up the Postgres Operator UI
|
||||
|
||||
With the v1.2 release the Postgres Operator is shipped with a browser-based
|
||||
Since the v1.2 release the Postgres Operator is shipped with a browser-based
|
||||
configuration user interface (UI) that simplifies managing Postgres clusters
|
||||
with the operator. The UI runs with Node.js and comes with it's own Docker
|
||||
image.
|
||||
with the operator.
|
||||
|
||||
Run NPM to continuously compile `tags/js` code. Basically, it creates an
|
||||
`app.js` file in: `static/build/app.js`
|
||||
### Building the UI image
|
||||
|
||||
```
|
||||
(cd ui/app && npm start)
|
||||
```
|
||||
|
||||
To build the Docker image open a shell and change to the `ui` folder. Then run:
|
||||
The UI runs with Node.js and comes with it's own Docker
|
||||
image. However, installing Node.js to build the operator UI is not required. It
|
||||
is handled via Docker containers when running:
|
||||
|
||||
```bash
|
||||
docker build -t registry.opensource.zalan.do/acid/postgres-operator-ui:v1.2.0 .
|
||||
make docker
|
||||
```
|
||||
|
||||
Apply all manifests for the `ui/manifests` folder to deploy the Postgres
|
||||
Operator UI on K8s. For local tests you don't need the Ingress resource.
|
||||
### Configure endpoints and options
|
||||
|
||||
The UI talks to the K8s API server as well as the Postgres Operator [REST API](developer.md#debugging-the-operator).
|
||||
K8s API server URLs are loaded from the machine's kubeconfig environment by
|
||||
default. Alternatively, a list can also be passed when starting the Python
|
||||
application with the `--cluster` option.
|
||||
|
||||
The Operator API endpoint can be configured via the `OPERATOR_API_URL`
|
||||
environment variables in the [deployment manifest](../ui/manifests/deployment.yaml#L40).
|
||||
You can also expose the operator API through a [service](../manifests/api-service.yaml).
|
||||
Some displayed options can be disabled from UI using simple flags under the
|
||||
`OPERATOR_UI_CONFIG` field in the deployment.
|
||||
|
||||
### Deploy the UI on K8s
|
||||
|
||||
Now, apply all manifests from the `ui/manifests` folder to deploy the Postgres
|
||||
Operator UI on K8s. Replace the image tag in the deployment manifest if you
|
||||
want to test the image you've built with `make docker`. Make sure the pods for
|
||||
the operator and the UI are both running.
|
||||
|
||||
```bash
|
||||
kubectl apply -f ui/manifests
|
||||
sed -e "s/\(image\:.*\:\).*$/\1$TAG/" manifests/deployment.yaml | kubectl apply -f manifests/
|
||||
kubectl get all -l application=postgres-operator-ui
|
||||
```
|
||||
|
||||
Make sure the pods for the operator and the UI are both running. For local
|
||||
testing you need to apply proxying and port forwarding so that the UI can talk
|
||||
to the K8s and Postgres Operator REST API. You can use the provided
|
||||
`run_local.sh` script for this. Make sure it uses the correct URL to your K8s
|
||||
API server, e.g. for minikube it would be `https://192.168.99.100:8443`.
|
||||
### Local testing
|
||||
|
||||
For local testing you need to apply K8s proxying and operator pod port
|
||||
forwarding so that the UI can talk to the K8s and Postgres Operator REST API.
|
||||
The Ingress resource is not needed. You can use the provided `run_local.sh`
|
||||
script for this. Make sure that:
|
||||
|
||||
* Python dependencies are installed on your machine
|
||||
* the K8s API server URL is set for kubectl commands, e.g. for minikube it would usually be `https://192.168.99.100:8443`.
|
||||
* the pod label selectors for port forwarding are correct
|
||||
|
||||
When testing with minikube you have to build the image in its docker environment
|
||||
(running `make docker` doesn't do it for you). From the `ui` directory execute:
|
||||
|
||||
```bash
|
||||
# compile and build operator UI
|
||||
make docker
|
||||
|
||||
# build in image in minikube docker env
|
||||
eval $(minikube docker-env)
|
||||
docker build -t registry.opensource.zalan.do/acid/postgres-operator-ui:v1.3.0 .
|
||||
|
||||
# apply UI manifests next to a running Postgres Operator
|
||||
kubectl apply -f manifests/
|
||||
|
||||
# install python dependencies to run UI locally
|
||||
pip3 install -r requirements
|
||||
./run_local.sh
|
||||
```
|
||||
|
||||
+7
-3
@@ -31,9 +31,13 @@ status page.
|
||||

|
||||
|
||||
Usually, the startup should only take up to 1 minute. If you feel the process
|
||||
got stuck click on the "Logs" button to inspect the operator logs. From the
|
||||
"Status" field in the top menu you can also retrieve the logs and queue of each
|
||||
worker the operator is using. The number of concurrent workers can be
|
||||
got stuck click on the "Logs" button to inspect the operator logs. If the logs
|
||||
look fine, but the UI seems to got stuck, check if you are have configured the
|
||||
same [cluster name label](../ui/manifests/deployment.yaml#L45) like for the
|
||||
[operator](../manifests/configmap.yaml#L13).
|
||||
|
||||
From the "Status" field in the top menu you can also retrieve the logs and queue
|
||||
of each worker the operator is using. The number of concurrent workers can be
|
||||
[configured](reference/operator_parameters.md#general).
|
||||
|
||||

|
||||
|
||||
+56
-1
@@ -53,6 +53,7 @@ kubectl create -f manifests/operatorconfiguration.crd.yaml # registers the CRD
|
||||
kubectl create -f manifests/postgresql-operator-default-configuration.yaml # configuration
|
||||
kubectl create -f manifests/operator-service-account-rbac.yaml # identity and permissions
|
||||
kubectl create -f manifests/postgres-operator.yaml # deployment
|
||||
kubectl create -f manifests/api-service.yaml # operator API to be used by UI
|
||||
```
|
||||
|
||||
There is a [Kustomization](https://github.com/kubernetes-sigs/kustomize)
|
||||
@@ -106,7 +107,7 @@ kubectl create -f https://operatorhub.io/install/postgres-operator.yaml
|
||||
This installs the operator in the `operators` namespace. More information can be
|
||||
found on [operatorhub.io](https://operatorhub.io/operator/postgres-operator).
|
||||
|
||||
## Create a Postgres cluster
|
||||
## Check if Postgres Operator is running
|
||||
|
||||
Starting the operator may take a few seconds. Check if the operator pod is
|
||||
running before applying a Postgres cluster manifest.
|
||||
@@ -117,7 +118,61 @@ kubectl get pod -l name=postgres-operator
|
||||
|
||||
# if you've created the operator using helm chart
|
||||
kubectl get pod -l app.kubernetes.io/name=postgres-operator
|
||||
```
|
||||
|
||||
If the operator doesn't get into `Running` state, either check the latest K8s
|
||||
events of the deployment or pod with `kubectl describe` or inspect the operator
|
||||
logs:
|
||||
|
||||
```bash
|
||||
kubectl logs "$(kubectl get pod -l name=postgres-operator --output='name')"
|
||||
```
|
||||
|
||||
## Deploy the operator UI
|
||||
|
||||
In the following paragraphs we describe how to access and manage PostgreSQL
|
||||
clusters from the command line with kubectl. But it can also be done from the
|
||||
browser-based [Postgres Operator UI](operator-ui.md). Before deploying the UI
|
||||
make sure the operator is running and its REST API is reachable through a
|
||||
[K8s service](../manifests/api-service.yaml). The URL to this API must be
|
||||
configured in the [deployment manifest](../ui/manifests/deployment.yaml#L43)
|
||||
of the UI.
|
||||
|
||||
To deploy the UI simply apply all its manifests files or use the UI helm chart:
|
||||
|
||||
```bash
|
||||
# manual deployment
|
||||
kubectl apply -f ui/manifests/
|
||||
|
||||
# or helm chart
|
||||
helm install postgres-operator-ui ./charts/postgres-operator-ui
|
||||
```
|
||||
|
||||
Like with the operator, check if the UI pod gets into `Running` state:
|
||||
|
||||
```bash
|
||||
# if you've created the operator using yaml manifests
|
||||
kubectl get pod -l name=postgres-operator-ui
|
||||
|
||||
# if you've created the operator using helm chart
|
||||
kubectl get pod -l app.kubernetes.io/name=postgres-operator-ui
|
||||
```
|
||||
|
||||
You can now access the web interface by port forwarding the UI pod (mind the
|
||||
label selector) and enter `localhost:8081` in your browser:
|
||||
|
||||
```bash
|
||||
kubectl port-forward "$(kubectl get pod -l name=postgres-operator-ui --output='name')" 8081
|
||||
```
|
||||
|
||||
Available option are explained in detail in the [UI docs](operator-ui.md).
|
||||
|
||||
## Create a Postgres cluster
|
||||
|
||||
If the operator pod is running it listens to new events regarding `postgresql`
|
||||
resources. Now, it's time to submit your first Postgres cluster manifest.
|
||||
|
||||
```bash
|
||||
# create a Postgres cluster
|
||||
kubectl create -f manifests/minimal-postgres-manifest.yaml
|
||||
```
|
||||
|
||||
@@ -122,6 +122,11 @@ These parameters are grouped directly under the `spec` key in the manifest.
|
||||
A map of key value pairs that gets attached as [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/)
|
||||
to each pod created for the database.
|
||||
|
||||
* **serviceAnnotations**
|
||||
A map of key value pairs that gets attached as [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/)
|
||||
to the services created for the database cluster. Check the
|
||||
[administrator docs](https://github.com/zalando/postgres-operator/blob/master/docs/administrator.md#load-balancers-and-allowed-ip-ranges)
|
||||
for more information regarding default values and overwrite rules.
|
||||
|
||||
* **enableShmVolume**
|
||||
Start a database pod without limitations on shm memory. By default Docker
|
||||
|
||||
@@ -110,8 +110,10 @@ Those are top-level keys, containing both leaf keys and groups.
|
||||
|
||||
* **min_instances**
|
||||
operator will run at least the number of instances for any given Postgres
|
||||
cluster equal to the value of this parameter. When `-1` is specified, no
|
||||
limits are applied. The default is `-1`.
|
||||
cluster equal to the value of this parameter. Standby clusters can still run
|
||||
with `numberOfInstances: 1` as this is the [recommended setup](../user.md#setting-up-a-standby-cluster).
|
||||
When `-1` is specified for `min_instances`, no limits are applied. The default
|
||||
is `-1`.
|
||||
|
||||
* **resync_period**
|
||||
period between consecutive sync requests. The default is `30m`.
|
||||
@@ -152,21 +154,22 @@ configuration they are grouped under the `kubernetes` key.
|
||||
service account used by Patroni running on individual Pods to communicate
|
||||
with the operator. Required even if native Kubernetes support in Patroni is
|
||||
not used, because Patroni keeps pod labels in sync with the instance role.
|
||||
The default is `operator`.
|
||||
The default is `postgres-pod`.
|
||||
|
||||
* **pod_service_account_definition**
|
||||
The operator tries to create the pod Service Account in the namespace that
|
||||
doesn't define such an account using the YAML definition provided by this
|
||||
option. If not defined, a simple definition that contains only the name will
|
||||
be used. The default is empty.
|
||||
On Postgres cluster creation the operator tries to create the service account
|
||||
for the Postgres pods if it does not exist in the namespace. The internal
|
||||
default service account definition (defines only the name) can be overwritten
|
||||
with this parameter. Make sure to provide a valid YAML or JSON string. The
|
||||
default is empty.
|
||||
|
||||
* **pod_service_account_role_binding_definition**
|
||||
This definition must bind pod service account to a role with permission
|
||||
This definition must bind the pod service account to a role with permission
|
||||
sufficient for the pods to start and for Patroni to access K8s endpoints;
|
||||
service account on its own lacks any such rights starting with K8s v1.8. If
|
||||
not explicitly defined by the user, a simple definition that binds the
|
||||
account to the operator's own 'zalando-postgres-operator' cluster role will
|
||||
be used. The default is empty.
|
||||
account to the 'postgres-pod' [cluster role](../../manifests/operator-service-account-rbac.yaml#L198)
|
||||
will be used. The default is empty.
|
||||
|
||||
* **pod_terminate_grace_period**
|
||||
Postgres pods are [terminated forcefully](https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods)
|
||||
@@ -318,11 +321,19 @@ CRD-based configuration.
|
||||
|
||||
* **default_cpu_limit**
|
||||
CPU limits for the Postgres containers, unless overridden by cluster-specific
|
||||
settings. The default is `3`.
|
||||
settings. The default is `1`.
|
||||
|
||||
* **default_memory_limit**
|
||||
memory limits for the Postgres containers, unless overridden by cluster-specific
|
||||
settings. The default is `1Gi`.
|
||||
settings. The default is `500Mi`.
|
||||
|
||||
* **min_cpu_limit**
|
||||
hard CPU minimum what we consider to be required to properly run Postgres
|
||||
clusters with Patroni on Kubernetes. The default is `250m`.
|
||||
|
||||
* **min_memory_limit**
|
||||
hard memory minimum what we consider to be required to properly run Postgres
|
||||
clusters with Patroni on Kubernetes. The default is `250Mi`.
|
||||
|
||||
## Operator timeouts
|
||||
|
||||
@@ -380,8 +391,9 @@ In the CRD-based configuration they are grouped under the `load_balancer` key.
|
||||
`false`.
|
||||
|
||||
* **custom_service_annotations**
|
||||
when load balancing is enabled, LoadBalancer service is created and
|
||||
this parameter takes service annotations that are applied to service.
|
||||
This key/value map provides a list of annotations that get attached to each
|
||||
service of a cluster created by the operator. If the annotation key is also
|
||||
provided by the cluster definition, the manifest value is used.
|
||||
Optional.
|
||||
|
||||
* **master_dns_name_format** defines the DNS name string template for the
|
||||
@@ -453,8 +465,11 @@ grouped under the `logical_backup` key.
|
||||
S3 bucket to store backup results. The bucket has to be present and
|
||||
accessible by Postgres pods. Default: empty.
|
||||
|
||||
* **logical_backup_s3_region**
|
||||
Specifies the region of the bucket which is required with some non-AWS S3 storage services. The default is empty.
|
||||
|
||||
* **logical_backup_s3_endpoint**
|
||||
When using non-AWS S3 storage, endpoint can be set as a ENV variable.
|
||||
When using non-AWS S3 storage, endpoint can be set as a ENV variable. The default is empty.
|
||||
|
||||
* **logical_backup_s3_sse**
|
||||
Specify server side encription that S3 storage is using. If empty string
|
||||
@@ -579,4 +594,4 @@ scalyr sidecar. In the CRD-based configuration they are grouped under the
|
||||
CPU limit value for the Scalyr sidecar. The default is `1`.
|
||||
|
||||
* **scalyr_memory_limit**
|
||||
Memory limit value for the Scalyr sidecar. The default is `1Gi`.
|
||||
Memory limit value for the Scalyr sidecar. The default is `500Mi`.
|
||||
|
||||
+103
-46
@@ -65,7 +65,7 @@ our test cluster.
|
||||
|
||||
```bash
|
||||
# get name of master pod of acid-minimal-cluster
|
||||
export PGMASTER=$(kubectl get pods -o jsonpath={.items..metadata.name} -l application=spilo,version=acid-minimal-cluster,spilo-role=master)
|
||||
export PGMASTER=$(kubectl get pods -o jsonpath={.items..metadata.name} -l application=spilo,cluster-name=acid-minimal-cluster,spilo-role=master)
|
||||
|
||||
# set up port forward
|
||||
kubectl port-forward $PGMASTER 6432:5432
|
||||
@@ -232,11 +232,11 @@ spec:
|
||||
memory: 300Mi
|
||||
```
|
||||
|
||||
The minimum limit to properly run the `postgresql` resource is `256m` for `cpu`
|
||||
and `256Mi` for `memory`. If a lower value is set in the manifest the operator
|
||||
will cancel ADD or UPDATE events on this resource with an error. If no
|
||||
resources are defined in the manifest the operator will obtain the configured
|
||||
[default requests](reference/operator_parameters.md#kubernetes-resource-requests).
|
||||
The minimum limits to properly run the `postgresql` resource are configured to
|
||||
`250m` for `cpu` and `250Mi` for `memory`. If a lower value is set in the
|
||||
manifest the operator will raise the limits to the configured minimum values.
|
||||
If no resources are defined in the manifest they will be obtained from the
|
||||
configured [default requests](reference/operator_parameters.md#kubernetes-resource-requests).
|
||||
|
||||
## Use taints and tolerations for dedicated PostgreSQL nodes
|
||||
|
||||
@@ -254,29 +254,22 @@ spec:
|
||||
|
||||
## How to clone an existing PostgreSQL cluster
|
||||
|
||||
You can spin up a new cluster as a clone of the existing one, using a clone
|
||||
You can spin up a new cluster as a clone of the existing one, using a `clone`
|
||||
section in the spec. There are two options here:
|
||||
|
||||
* Clone directly from a source cluster using `pg_basebackup`
|
||||
* Clone from an S3 bucket
|
||||
* Clone from an S3 bucket (recommended)
|
||||
* Clone directly from a source cluster
|
||||
|
||||
### Clone directly
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
clone:
|
||||
cluster: "acid-batman"
|
||||
```
|
||||
|
||||
Here `cluster` is a name of a source cluster that is going to be cloned. The
|
||||
cluster to clone is assumed to be running and the clone procedure invokes
|
||||
`pg_basebackup` from it. The operator will setup the cluster to be cloned to
|
||||
connect to the service of the source cluster by name (if the cluster is called
|
||||
test, then the connection string will look like host=test port=5432), which
|
||||
means that you can clone only from clusters within the same namespace.
|
||||
Note, that cloning can also be used for [major version upgrades](administrator.md#minor-and-major-version-upgrade)
|
||||
of PostgreSQL.
|
||||
|
||||
### Clone from S3
|
||||
|
||||
Cloning from S3 has the advantage that there is no impact on your production
|
||||
database. A new Postgres cluster is created by restoring the data of another
|
||||
source cluster. If you create it in the same Kubernetes environment, use a
|
||||
different name.
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
clone:
|
||||
@@ -287,7 +280,8 @@ spec:
|
||||
|
||||
Here `cluster` is a name of a source cluster that is going to be cloned. A new
|
||||
cluster will be cloned from S3, using the latest backup before the `timestamp`.
|
||||
In this case, `uid` field is also mandatory - operator will use it to find a
|
||||
Note, that a time zone is required for `timestamp` in the format of +00:00 which
|
||||
is UTC. The `uid` field is also mandatory. The operator will use it to find a
|
||||
correct key inside an S3 bucket. You can find this field in the metadata of the
|
||||
source cluster:
|
||||
|
||||
@@ -299,9 +293,6 @@ metadata:
|
||||
uid: efd12e58-5786-11e8-b5a7-06148230260c
|
||||
```
|
||||
|
||||
Note that timezone is required for `timestamp`. Otherwise, offset is relative
|
||||
to UTC, see [RFC 3339 section 5.6) 3339 section 5.6](https://www.ietf.org/rfc/rfc3339.txt).
|
||||
|
||||
For non AWS S3 following settings can be set to support cloning from other S3
|
||||
implementations:
|
||||
|
||||
@@ -317,14 +308,35 @@ spec:
|
||||
s3_force_path_style: true
|
||||
```
|
||||
|
||||
### Clone directly
|
||||
|
||||
Another way to get a fresh copy of your source DB cluster is via basebackup. To
|
||||
use this feature simply leave out the timestamp field from the clone section.
|
||||
The operator will connect to the service of the source cluster by name. If the
|
||||
cluster is called test, then the connection string will look like host=test
|
||||
port=5432), which means that you can clone only from clusters within the same
|
||||
namespace.
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
clone:
|
||||
cluster: "acid-batman"
|
||||
```
|
||||
|
||||
Be aware that on a busy source database this can result in an elevated load!
|
||||
|
||||
## Setting up a standby cluster
|
||||
|
||||
Standby clusters are like normal cluster but they are streaming from a remote
|
||||
cluster. As the first version of this feature, the only scenario covered by
|
||||
operator is to stream from a WAL archive of the master. Following the more
|
||||
popular infrastructure of using Amazon's S3 buckets, it is mentioned as
|
||||
`s3_wal_path` here. To start a cluster as standby add the following `standby`
|
||||
section in the YAML file:
|
||||
Standby cluster is a [Patroni feature](https://github.com/zalando/patroni/blob/master/docs/replica_bootstrap.rst#standby-cluster)
|
||||
that first clones a database, and keeps replicating changes afterwards. As the
|
||||
replication is happening by the means of archived WAL files (stored on S3 or
|
||||
the equivalent of other cloud providers), the standby cluster can exist in a
|
||||
different location than its source database. Unlike cloning, the PostgreSQL
|
||||
version between source and target cluster has to be the same.
|
||||
|
||||
To start a cluster as standby, add the following `standby` section in the YAML
|
||||
file and specify the S3 bucket path. An empty path will result in an error and
|
||||
no statefulset will be created.
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
@@ -332,20 +344,65 @@ spec:
|
||||
s3_wal_path: "s3 bucket path to the master"
|
||||
```
|
||||
|
||||
Things to note:
|
||||
At the moment, the operator only allows to stream from the WAL archive of the
|
||||
master. Thus, it is recommended to deploy standby clusters with only [one pod](../manifests/standby-manifest.yaml#L10).
|
||||
You can raise the instance count when detaching. Note, that the same pod role
|
||||
labels like for normal clusters are used: The standby leader is labeled as
|
||||
`master`.
|
||||
|
||||
- An empty string in the `s3_wal_path` field of the standby cluster will result
|
||||
in an error and no statefulset will be created.
|
||||
- Only one pod can be deployed for stand-by cluster.
|
||||
- To manually promote the standby_cluster, use `patronictl` and remove config
|
||||
entry.
|
||||
- There is no way to transform a non-standby cluster to a standby cluster
|
||||
through the operator. Adding the standby section to the manifest of a running
|
||||
Postgres cluster will have no effect. However, it can be done through Patroni
|
||||
by adding the [standby_cluster](https://github.com/zalando/patroni/blob/bd2c54581abb42a7d3a3da551edf0b8732eefd27/docs/replica_bootstrap.rst#standby-cluster)
|
||||
section using `patronictl edit-config`. Note that the transformed standby
|
||||
cluster will not be doing any streaming. It will be in standby mode and allow
|
||||
read-only transactions only.
|
||||
### Providing credentials of source cluster
|
||||
|
||||
A standby cluster is replicating the data (including users and passwords) from
|
||||
the source database and is read-only. The system and application users (like
|
||||
standby, postgres etc.) all have a password that does not match the credentials
|
||||
stored in secrets which are created by the operator. One solution is to create
|
||||
secrets beforehand and paste in the credentials of the source cluster.
|
||||
Otherwise, you will see errors in the Postgres logs saying users cannot log in
|
||||
and the operator logs will complain about not being able to sync resources.
|
||||
|
||||
When you only run a standby leader, you can safely ignore this, as it will be
|
||||
sorted out once the cluster is detached from the source. It is also harmless if
|
||||
you don’t plan it. But, when you created a standby replica, too, fix the
|
||||
credentials right away. WAL files will pile up on the standby leader if no
|
||||
connection can be established between standby replica(s). You can also edit the
|
||||
secrets after their creation. Find them by:
|
||||
|
||||
```bash
|
||||
kubectl get secrets --all-namespaces | grep <standby-cluster-name>
|
||||
```
|
||||
|
||||
### Promote the standby
|
||||
|
||||
One big advantage of standby clusters is that they can be promoted to a proper
|
||||
database cluster. This means it will stop replicating changes from the source,
|
||||
and start accept writes itself. This mechanism makes it possible to move
|
||||
databases from one place to another with minimal downtime. Currently, the
|
||||
operator does not support promoting a standby cluster. It has to be done
|
||||
manually using `patronictl edit-config` inside the postgres container of the
|
||||
standby leader pod. Remove the following lines from the YAML structure and the
|
||||
leader promotion happens immediately. Before doing so, make sure that the
|
||||
standby is not behind the source database.
|
||||
|
||||
```yaml
|
||||
standby_cluster:
|
||||
create_replica_methods:
|
||||
- bootstrap_standby_with_wale
|
||||
- basebackup_fast_xlog
|
||||
restore_command: envdir "/home/postgres/etc/wal-e.d/env-standby" /scripts/restore_command.sh
|
||||
"%f" "%p"
|
||||
```
|
||||
|
||||
Finally, remove the `standby` section from the postgres cluster manifest.
|
||||
|
||||
### Turn a normal cluster into a standby
|
||||
|
||||
There is no way to transform a non-standby cluster to a standby cluster through
|
||||
the operator. Adding the `standby` section to the manifest of a running
|
||||
Postgres cluster will have no effect. But, as explained in the previous
|
||||
paragraph it can be done manually through `patronictl edit-config`. This time,
|
||||
by adding the `standby_cluster` section to the Patroni configuration. However,
|
||||
the transformed standby cluster will not be doing any streaming. It will be in
|
||||
standby mode and allow read-only transactions only.
|
||||
|
||||
## Sidecar Support
|
||||
|
||||
|
||||
Reference in New Issue
Block a user