public_git/postgres-operator
1
0
Fork 0
mirror of https://github.com/zalando/postgres-operator.git synced 2026-09-30 14:01:46 +02:00
Code Issues Projects Releases Wiki Activity

Add CRD validation (#599)

Browse Source 
* add CRD manifests with validation
* update documentation
* patroni slots is not an array but a nested hash map
* make deps call tools
* cover validation in docs and export it in crds.go
* add toggle to disable creation of CRD validation and document it
* use templated service account also for CRD-configured helm deployment
This commit is contained in:
Felix Kunde
2019-11-28 12:02:05 +01:00
committed by GitHub
parent 052940862a
commit a3b34f146f
24 changed files with 2270 additions and 108 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/administrator.md
+68 -14
View File
@@ -3,6 +3,30 @@
Learn how to configure and manage the Postgres Operator in your Kubernetes (K8s)
environment.
## CRD Validation
[CustomResourceDefinitions](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions)
will be registered with schema validation by default when the operator is
deployed. The `OperatorConfiguration` CRD will only get created if the
`POSTGRES_OPERATOR_CONFIGURATION_OBJECT` [environment variable](../manifests/postgres-operator.yaml#L36)
in the deployment yaml is set and not empty.
When submitting manifests of [`postgresql`](../manifests/postgresql.crd.yaml) or
[`OperatorConfiguration`](../manifests/operatorconfiguration.crd.yaml) custom
resources with kubectl, validation can be bypassed with `--validate=false`. The
operator can also be configured to not register CRDs with validation on `ADD` or
`UPDATE` events. Running instances are not affected when enabling the validation
afterwards unless the manifests is not changed then. Note, that the provided CRD
manifests contain the validation for users to understand what schema is
enforced.
Once the validation is enabled it can only be disabled manually by editing or
patching the CRD manifest:
```bash
zk8 patch crd postgresqls.acid.zalan.do -p '{"spec":{"validation": null}}'
```
## Namespaces
### Select the namespace to deploy to
@@ -32,7 +56,7 @@ By default, the operator watches the namespace it is deployed to. You can
change this by setting the `WATCHED_NAMESPACE` var in the `env` section of the
[operator deployment](../manifests/postgres-operator.yaml) manifest or by
altering the `watched_namespace` field in the operator
[ConfigMap](../manifests/configmap.yaml#L79).
[configuration](../manifests/postgresql-operator-default-configuration.yaml#L49).
In the case both are set, the env var takes the precedence. To make the
operator listen to all namespaces, explicitly set the field/env var to "`*`".
@@ -115,7 +139,7 @@ that are aggregated into the K8s [default roles](https://kubernetes.io/docs/refe
To ensure Postgres pods are running on nodes without any other application pods,
you can use [taints and tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
and configure the required toleration in the operator ConfigMap.
and configure the required toleration in the operator configuration.
As an example you can set following node taint:
@@ -136,6 +160,21 @@ data:
...
```
For an OperatorConfiguration resource the toleration should be defined like
this:
```yaml
apiVersion: "acid.zalan.do/v1"
kind: OperatorConfiguration
metadata:
name: postgresql-configuration
configuration:
kubernetes:
toleration:
postgres: "key:postgres,operator:Exists,effect:NoSchedule"
...
```
Note that the K8s version 1.13 brings [taint-based eviction](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions)
to the beta stage and enables it by default. Postgres pods by default receive
tolerations for `unreachable` and `noExecute` taints with the timeout of `5m`.
@@ -148,7 +187,7 @@ completely, specify the toleration by leaving out the `tolerationSeconds` value
To ensure Postgres pods are running on different topologies, you can use
[pod anti affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/)
and configure the required topology in the operator ConfigMap.
and configure the required topology in the operator configuration.
Enable pod anti affinity by adding following line to the operator ConfigMap:
@@ -161,21 +200,22 @@ data:
enable_pod_antiaffinity: "true"
```
By default the topology key for the pod anti affinity is set to
`kubernetes.io/hostname`, you can set another topology key e.g.
`failure-domain.beta.kubernetes.io/zone` by adding following line to the
operator ConfigMap, see [built-in node labels](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#interlude-built-in-node-labels) for available topology keys:
Likewise, when using an OperatorConfiguration resource add:
```yaml
apiVersion: v1
kind: ConfigMap
apiVersion: "acid.zalan.do/v1"
kind: OperatorConfiguration
metadata:
name: postgres-operator
data:
enable_pod_antiaffinity: "true"
pod_antiaffinity_topology_key: "failure-domain.beta.kubernetes.io/zone"
name: postgresql-configuration
configuration:
kubernetes:
enable_pod_antiaffinity: true
```
By default the topology key for the pod anti affinity is set to
`kubernetes.io/hostname`, you can set another topology key e.g.
`failure-domain.beta.kubernetes.io/zone`. See [built-in node labels](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#interlude-built-in-node-labels) for available topology keys.
## Pod Disruption Budget
By default the operator uses a PodDisruptionBudget (PDB) to protect the cluster
@@ -280,6 +320,20 @@ data:
...
```
**OperatorConfiguration**
```yaml
apiVersion: "acid.zalan.do/v1"
kind: OperatorConfiguration
metadata:
name: postgresql-operator-configuration
configuration:
kubernetes:
# referencing config map with custom settings
pod_environment_configmap: postgres-pod-config
...
```
**referenced ConfigMap `postgres-pod-config`**
```yaml
@@ -312,7 +366,7 @@ services: one for the master pod and one for replica pods. To expose these
services to an outer network, one can attach load balancers to them by setting
`enableMasterLoadBalancer` and/or `enableReplicaLoadBalancer` to `true` in the
cluster manifest. In the case any of these variables are omitted from the
manifest, the operator configmap's settings `enable_master_load_balancer` and
manifest, the operator configuration settings `enable_master_load_balancer` and
`enable_replica_load_balancer` apply. Note that the operator settings affect
all Postgresql services running in all namespaces watched by the operator.
docs/developer.md
+3 -1
View File
@@ -33,7 +33,7 @@ by setting the `GO111MODULE` environment variable to `on`. The make targets do
this for you, so simply run
```bash
make tools deps
make deps
```
This would take a while to complete. You have to redo `make deps` every time
@@ -284,6 +284,7 @@ manifest files:
Postgres manifest parameters are defined in the [api package](../pkg/apis/acid.zalan.do/v1/postgresql_type.go).
The operator behavior has to be implemented at least in [k8sres.go](../pkg/cluster/k8sres.go).
Validation of CRD parameters is controlled in [crd.go](../pkg/apis/acid.zalan.do/v1/crds.go).
Please, reflect your changes in tests, for example in:
* [config_test.go](../pkg/util/config/config_test.go)
* [k8sres_test.go](../pkg/cluster/k8sres_test.go)
@@ -294,6 +295,7 @@ Please, reflect your changes in tests, for example in:
For the CRD-based configuration, please update the following files:
* the default [OperatorConfiguration](../manifests/postgresql-operator-default-configuration.yaml)
* the Helm chart's [values-crd file](../charts/postgres-operator/values.yaml)
* the CRD's [validation](../manifests/operatorconfiguration.crd.yaml)
Reflect the changes in the ConfigMap configuration as well (note that numeric
and boolean parameters have to use double quotes here):
docs/quickstart.md
+14 -13
View File
@@ -55,8 +55,8 @@ kubectl create -f manifests/postgres-operator.yaml # deployment
```
There is a [Kustomization](https://github.com/kubernetes-sigs/kustomize)
manifest that [combines the mentioned resources](../manifests/kustomization.yaml) -
it can be used with kubectl 1.14 or newer as easy as:
manifest that [combines the mentioned resources](../manifests/kustomization.yaml)
(except for the CRD) - it can be used with kubectl 1.14 or newer as easy as:
```bash
kubectl apply -k github.com/zalando/postgres-operator/manifests
@@ -86,8 +86,9 @@ To use CRD-based configuration you need to specify the [values-crd yaml file](..
helm install postgres-operator ./charts/postgres-operator -f ./charts/postgres-operator/values-crd.yaml
```
The chart works with both Helm 2 and Helm 3. Documentation for installing
applications with helm2 can be found in the [helm2 docs](https://v2.helm.sh/docs/).
The chart works with both Helm 2 and Helm 3. The `crd-install` hook from v2 will
be skipped with warning when using v3. Documentation for installing applications
with Helm 2 can be found in the [v2 docs](https://v2.helm.sh/docs/).
### Operator Lifecycle Manager (OLM)
@@ -119,15 +120,15 @@ kubectl get pod -l app.kubernetes.io/name=postgres-operator
kubectl create -f manifests/minimal-postgres-manifest.yaml
```
After the cluster manifest is submitted the operator will create Service and
Endpoint resources and a StatefulSet which spins up new Pod(s) given the number
of instances specified in the manifest. All resources are named like the
cluster. The database pods can be identified by their number suffix, starting
from `-0`. They run the [Spilo](https://github.com/zalando/spilo) container
image by Zalando. As for the services and endpoints, there will be one for the
master pod and another one for all the replicas (`-repl` suffix). Check if all
components are coming up. Use the label `application=spilo` to filter and list
the label `spilo-role` to see who is currently the master.
After the cluster manifest is submitted and passed the validation the operator
will create Service and Endpoint resources and a StatefulSet which spins up new
Pod(s) given the number of instances specified in the manifest. All resources
are named like the cluster. The database pods can be identified by their number
suffix, starting from `-0`. They run the [Spilo](https://github.com/zalando/spilo)
container image by Zalando. As for the services and endpoints, there will be one
for the master pod and another one for all the replicas (`-repl` suffix). Check
if all components are coming up. Use the label `application=spilo` to filter and
list the label `spilo-role` to see who is currently the master.
```bash
# check the deployed cluster
docs/reference/operator_parameters.md
+11 -7
View File
@@ -29,21 +29,20 @@ configuration.
To test the CRD-based configuration locally, use the following
```bash
kubectl create -f manifests/operatorconfiguration.crd.yaml # registers the CRD
kubectl create -f manifests/postgresql-operator-default-configuration.yaml
kubectl create -f manifests/operator-service-account-rbac.yaml
kubectl create -f manifests/postgres-operator.yaml # set the env var as mentioned above
kubectl create -f manifests/postgresql-operator-default-configuration.yaml
kubectl get operatorconfigurations postgresql-operator-default-configuration -o yaml
```
Note that the operator first attempts to register the CRD of the
`OperatorConfiguration` and then waits for an instance to be created. In
between these two event the operator pod may be failing since it cannot fetch
the not-yet-existing `OperatorConfiguration` instance.
The CRD-based configuration is more powerful than the one based on ConfigMaps
and should be used unless there is a compatibility requirement to use an already
existing configuration. Even in that case, it should be rather straightforward
to convert the configmap based configuration into the CRD-based one and restart
the operator. The ConfigMaps-based configuration will be deprecated and
to convert the ConfigMap-based configuration into the CRD-based one and restart
the operator. The ConfigMap-based configuration will be deprecated and
subsequently removed in future releases.
Note that for the CRD-based configuration groups of configuration options below
@@ -71,6 +70,11 @@ Variable names are underscore-separated words.
Those are top-level keys, containing both leaf keys and groups.
* **enable_crd_validation**
toggles if the operator will create or update CRDs with
[OpenAPI v3 schema validation](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#validation)
The default is `true`.
* **etcd_host**
Etcd connection string for Patroni defined as `host:port`. Not required when
Patroni native Kubernetes support is used. The default is empty (use