From 4d20a3810697042f34db3e2ad671f8296f23c646 Mon Sep 17 00:00:00 2001 From: erthalion <9erthalion6@gmail.com> Date: Tue, 5 Jun 2018 11:42:01 +0200 Subject: [PATCH 1/7] Add section about volume increase --- docs/user.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/docs/user.md b/docs/user.md index d0ebcc1b0..7647ef71f 100644 --- a/docs/user.md +++ b/docs/user.md @@ -242,3 +242,34 @@ metadata: Note that timezone required for `timestamp` (offset relative to UTC, see RFC 3339 section 5.6) + +## Increase volume size + +PostgreSQL operator supports statefulset volume resize if you're using the +operator on top of AWS. For that you need to apply manifest with a new size, +and the operator will find out that a volume size needs to be changed: + +``` +apiVersion: "acid.zalan.do/v1" +kind: postgresql + +metadata: + name: acid-test-cluster +spec: + volume: + size: 5Gi # new volume size +``` + +You can only increase a volume size in this way, if a new requested size is +smaller than the previous one nothing will be done. After this update all the +new pods in a statefulset will be created with a new volume size. To increase a +volume size on already existing pods in a statefulset, the operator will +perform the following steps: + +* modify EBS volume size + +* resize an actuall filesystem use `resize2fs` + +Note that if before a volume size was increased a statefulset was scaled down +and (after the change was applied) scaled back, those pods that were down will +have an old volume size, since a statefulset doesn't delete volumes. From dab6c01cc7dedd1248652d9ddd1e074cf2b82607 Mon Sep 17 00:00:00 2001 From: erthalion <9erthalion6@gmail.com> Date: Wed, 6 Jun 2018 17:36:21 +0200 Subject: [PATCH 2/7] Change and clarify wording --- docs/user.md | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/docs/user.md b/docs/user.md index 7647ef71f..360b58e97 100644 --- a/docs/user.md +++ b/docs/user.md @@ -246,8 +246,8 @@ Note that timezone required for `timestamp` (offset relative to UTC, see RFC ## Increase volume size PostgreSQL operator supports statefulset volume resize if you're using the -operator on top of AWS. For that you need to apply manifest with a new size, -and the operator will find out that a volume size needs to be changed: +operator on top of AWS. For that you need to change the size field of the +volume description in the cluster manifest and apply it: ``` apiVersion: "acid.zalan.do/v1" @@ -260,16 +260,22 @@ spec: size: 5Gi # new volume size ``` -You can only increase a volume size in this way, if a new requested size is -smaller than the previous one nothing will be done. After this update all the -new pods in a statefulset will be created with a new volume size. To increase a -volume size on already existing pods in a statefulset, the operator will -perform the following steps: +The operator compares the new value of the size field with the previous one and +acts on differences. -* modify EBS volume size +You can only enlarge the volume with the process described above, shrinking is +not supported and will emit a warning. After this update all the new volumes in +a statefulset are allocated according to the new size. To enlarge persistent +volumes attached to the running pods, the operator performs the following +actions: -* resize an actuall filesystem use `resize2fs` +* call AWS API to change the volume size -Note that if before a volume size was increased a statefulset was scaled down -and (after the change was applied) scaled back, those pods that were down will -have an old volume size, since a statefulset doesn't delete volumes. +* connect to the pod using `kubectl exec` and resize the filesystem with + `resize2fs`. + +Fist step has a limitation, AWS rate-limits this operation to no more than once +every 6 hours. +Note that if the statefulset is scaled down before resizing the size changes +are only applied to the volumes attached to the running pods. The size of the +volumes that correspond to the previously running pods is not changed. From df40cd831d4e0b63be48e0f0136bab19a62e55e9 Mon Sep 17 00:00:00 2001 From: erthalion <9erthalion6@gmail.com> Date: Thu, 7 Jun 2018 10:25:50 +0200 Subject: [PATCH 3/7] Adjust the wording --- docs/user.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user.md b/docs/user.md index 360b58e97..2420a2681 100644 --- a/docs/user.md +++ b/docs/user.md @@ -247,7 +247,7 @@ Note that timezone required for `timestamp` (offset relative to UTC, see RFC PostgreSQL operator supports statefulset volume resize if you're using the operator on top of AWS. For that you need to change the size field of the -volume description in the cluster manifest and apply it: +volume description in the cluster manifest and apply the change: ``` apiVersion: "acid.zalan.do/v1" @@ -265,7 +265,7 @@ acts on differences. You can only enlarge the volume with the process described above, shrinking is not supported and will emit a warning. After this update all the new volumes in -a statefulset are allocated according to the new size. To enlarge persistent +the statefulset are allocated according to the new size. To enlarge persistent volumes attached to the running pods, the operator performs the following actions: From 9cb48e08891c3fc57e32bdf32cbd9bbdee2bedd5 Mon Sep 17 00:00:00 2001 From: Oleksii Kliukin Date: Fri, 8 Jun 2018 13:21:57 +0200 Subject: [PATCH 4/7] Document operator configuration parameters. (#313) --- docs/operator_parameters.md | 306 ++++++++++++++++++++++++++++++++++++ pkg/util/config/config.go | 1 - 2 files changed, 306 insertions(+), 1 deletion(-) create mode 100644 docs/operator_parameters.md diff --git a/docs/operator_parameters.md b/docs/operator_parameters.md new file mode 100644 index 000000000..6d277c589 --- /dev/null +++ b/docs/operator_parameters.md @@ -0,0 +1,306 @@ +Postgres Operator Configuration +=============================== + +Postgres operator is configured via a ConfigMap defined by the +`CONFIG_MAP_NAME` environment variable. Variable names are underscore-separated +words. + +Available Configuration Parameters +---------------------------------- + +### General +* **etcd_host** + Etcd connection string for Patroni defined as `host:port`. Not required when + Patroni native Kubernetes support is used. The default is empty (use + Kubernetes-native DCS). + +* **docker_image** + Spilo docker image for postgres instances. For production, don't rely on the + default image, as it might be not the most up-to-date one. Instead, build + your own Spilo image from the [github + repository](https://github.com/zalando/spilo). + +* **workers** + number of working routines the operator spawns to process requests to + create/update/delete/sync clusters concurrently. The default is `4`. + +* **max_instances** + operator will cap the number of instances in any managed postgres cluster up + to the value of this parameter. When `-1` is specified, no limits are applied. + The default is `-1`. + +* **min_instances** + operator will run at least the number of instances for any given postgres + cluster equal to the value of this parameter. When `-1` is specified, no limits + are applied. The default is `-1`. + +* **resync_period** + period between consecutive sync requests. The default is `5m`. + +### Postgres users +* **super_username** + postgres `superuser` name to be created by `initdb`. The default is + `postgres`. + +* **replication_username** + postgres username used for replication between instances. The default is + `standby`. + +### Kubernetes resources +* **pod_service_account_name** + service account used by Patroni running on individual Pods to communicate + with the operator. Required even if native Kubernetes support in Patroni is + not used, because Patroni keeps pod labels in sync with the instance role. + The default is `operator`. + +* **pod_service_account_definition** + The operator tries to create the pod Service Account in the namespace that + doesn't define such an account using the YAML definition provided by this + option. If not defined, a simple definition that contains only the name will + be used. The default is empty. + +* **pod_terminate_grace_period** + Patroni pods are [terminated + forcefully](https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods) + after this timeout. The default is `5m`. + +* **watched_namespace** + The operator watches for postgres objects in the given namespace. If not + specified, the value is taken from the operator namespace. A special `*` + value makes it watch all namespaces. The default is empty (watch the operator pod + namespace). + +* **pdb_name_format** + defines the template for PDB (Pod Disruption Budget) names created by the + operator. The default is `postgres-{cluster}-pdb`, where `{cluster}` is + replaced by the cluster name. Only the `{cluster}` placeholders is allowed in + the template. + +* **secret_name_template** + a template for the name of the database user secrets generated by the + operator. `{username}` is replaced with name of the secret, `{cluster}` with + the name of the cluster, `{tprkind}` with the kind of CRD (formerly known as + TPR) and `{tprgroup}` with the group of the CRD. No other placeholders are + allowed. The default is + `{username}.{cluster}.credentials.{tprkind}.{tprgroup}`. + +* **oauth_token_secret_name** + a name of the secret containing the `OAuth2` token to pass to the teams API. + The default is `postgresql-operator`. + +* **infrastructure_roles_secret_name** + name of the secret containing infrastructure roles names and passwords. + +* **pod_role_label** + name of the label assigned to the postgres pods (and services/endpoints) by + the operator. The default is `spilo-role`. + +* **cluster_labels** + list of `name:value` pairs for additional labels assigned to the cluster + objects. The default is `application:spilo`. + +* **cluster_name_label** + name of the label assigned to Kubernetes objects created by the operator that + indicates which cluster a given object belongs to. The default is + `cluster-name`. + +* **node_readiness_label** + a set of labels that a running and active node should possess to be + considered `ready`. The operator uses values of those labels to detect the + start of the Kubernetes cluster upgrade procedure and move master pods off + the nodes to be decommissioned. When the set is not empty, the operator also + assigns the `Affinity` clause to the postgres pods to be scheduled only on + `ready` nodes. The default is empty. + +* **toleration** + a dictionary that should contain `key`, `operator`, `value` and + `effect` keys. In that case, the operator defines a pod toleration + according to the values of those keys. See [kubernetes + documentation](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) + for details on taints and tolerations. The default is empty. + +* **pod_environment_configmap** + a name of the ConfigMap with environment variables to populate on every pod. + Right now this ConfigMap is searched in the namespace of the postgres cluster. + All variables from that ConfigMap are injected to the pod's environment, on + conflicts they are overridden by the environment variables generated by the + operator. The default is empty. + +### Kubernetes resource requests +* **default_cpu_request** + CPU request value for the postgres containers, unless overridden by + cluster-specific settings. The default is `100m`. + +* **default_memory_request** + memory request value for the postgres containers, unless overridden by + cluster-specific settings. The default is `100Mi`. + +* **default_cpu_limit** + CPU limits for the postgres containers, unless overridden by cluster-specific + settings. The default is `3`. + +* **default_memory_limit** + memory limits for the postgres containers, unless overridden by cluster-specific + settings. The default is `1Gi`. + +### Operator timeouts +* **resource_check_interval** + interval to wait between consecutive attempts to check for the presence of + some Kubernetes resource (i.e. `StatefulSet` or `PodDisruptionBudget`). The + default is `3s`. + +* **resource_check_timeout** + timeout when waiting for the presence of a certain Kubernetes resource (i.e. + `StatefulSet` or `PodDisruptionBudget`) before declaring the operation + unsuccessful. The default is `10m`. + +* **pod_label_wait_timeout** + timeout when waiting for the pod role and cluster labels. Bigger value gives + Patroni more time to start the instance; smaller makes the operator detect + possible issues faster. The default is `10m`. + +* **pod_deletion_wait_timeout** + timeout when waiting for the pods to be deleted when removing the cluster or + recreating pods. The default is `10m`. + +* **ready_wait_interval** + the interval between consecutive attempts waiting for the postgres CRD to be + created. The default is `5s`. + +* **ready_wait_timeout** + the timeout for the complete postgres CRD creation. The default is `30s`. + +### Load balancer related options +* **db_hosted_zone** + DNS zone for the cluster DNS name when the load balancer is configured for + the cluster. Only used when combined with + [external-dns](https://github.com/kubernetes-incubator/external-dns) and with + the cluster that has the load balancer enabled. The default is + `db.example.com`. + +* **enable_master_load_balancer** + toggles service type load balancer pointing to the master pod of the cluster. + Can be overridden by individual cluster settings. The default is `true`. + +* **enable_replica_load_balancer** + toggles service type load balancer pointing to the replica pod of the + cluster. Can be overridden by individual cluster settings. The default is + `false`. + +* **master_dns_name_format** defines the DNS name string template for the + master load balancer cluster. The default is + `{cluster}.{team}.{hostedzone}`, where `{cluster}` is replaced by the cluster + name, `{team}` is replaced with the team name and `{hostedzone}` is replaced + with the hosted zone (the value of the `db_hosted_zone` parameter). No other + placeholders are allowed. + +** **replica_dns_name_format** defines the DNS name string template for the + replica load balancer cluster. The default is + `{cluster}-repl.{team}.{hostedzone}`, where `{cluster}` is replaced by the + cluster name, `{team}` is replaced with the team name and `{hostedzone}` is + replaced with the hosted zone (the value of the `db_hosted_zone` parameter). + No other placeholders are allowed. + +### AWS or GSC interaction +* **wal_s3_bucket** + S3 bucket to use for shipping WAL segments with WAL-E. A bucket has to be + present and accessible by Patroni managed pods. At the moment, supported + services by Spilo are S3 and GCS. The default is empty. + +* **log_s3_bucket** + S3 bucket to use for shipping postgres daily logs. Works only with S3 on AWS. + The bucket has to be present and accessible by Patroni managed pods. At the + moment Spilo does not yet support this. The default is empty. + +* **kube_iam_role** + AWS IAM role to supply in the `iam.amazonaws.com/role` annotation of Patroni + pods. Only used when combined with + [kube2iam](https://github.com/jtblin/kube2iam) project on AWS. The default is empty. + +### Debugging the operator +* **debug_logging** + boolean parameter that toggles verbose debug logs from the operator. The + default is `true`. + +* **enable_db_access** + boolean parameter that toggles the functionality of the operator that require + access to the postgres database, i.e. creating databases and users. The default + is `true`. + +### Automatic creation of human users in the database +* **enable_teams_api** + boolean parameter that toggles usage of the Teams API by the operator. + The default is `true`. + +* **teams_api_url** + contains the URL of the Teams API service. There is a [demo + implementation](https://github.com/ikitiki/fake-teams-api). The default is + `https://teams.example.com/api/`. + +* **team_api_role_configuration** + postgres parameters to apply to each team member role. The default is + '*log_statement:all*'. It is possible to supply multiple options, separating + them by commas. Options containing commas within the value are not supported, + with the exception of the `search_path`. For instance: + + ```yaml + teams_api_role_configuration: "log_statement:all,search_path:'data,public'" + ``` + The default is `"log_statement:all"` + +* **enable_team_superuser** + whether to grant superuser to team members created from the Teams API. + The default is `false`. + +* **team_admin_role** + role name to grant to team members created from the Teams API. The default is + `admin`, that role is created by Spilo as a `NOLOGIN` role. + +* **pam_role_name** + when set, the operator will add all team member roles to this group and add a + `pg_hba` line to authenticate members of that role via `pam`. The default is + `zalandos`. + +* **pam_configuration** + when set, should contain a URL to use for authentication against the username + and the token supplied as the password. Used in conjunction with + [pam_oauth2](https://github.com/CyberDem0n/pam-oauth2) module. The default is + `https://info.example.com/oauth2/tokeninfo?access_token= uid + realm=/employees`. + +* **protected_roles** + List of roles that cannot be overwritten by an application, team or + infrastructure role. The default is `admin`. + +### Logging and REST API +* **api_port** + REST API listener listens to this port. The default is `8080`. + +* **ring_log_lines** + number of lines in the ring buffer used to store cluster logs. The default is `100`. + +* **cluster_history_entries** + number of entries in the cluster history ring buffer. The default is `1000`. + +## Scalyr options +* **scalyr_api_key** + API key for the Scalyr sidecar. The default is empty. + +* **scalyr_image** + Docker image for the Scalyr sidecar. The default is empty. + +* **scalyr_server_url** + server URL for the Scalyr sidecar. The default is `https://upload.eu.scalyr.com`. + +* **scalyr_cpu_request** + CPU request value for the Scalyr sidecar. The default is `100m`. + +* **scalyr_memory_request** + Memory request value for the Scalyr sidecar. The default is `50Mi`. + +* **scalyr_cpu_limit** + CPU limit value for the Scalyr sidecar. The default is `1`. + +* **scalyr_memory_limit** + Memory limit value for the Scalyr sidecar. The default is `1Gi`. + diff --git a/pkg/util/config/config.go b/pkg/util/config/config.go index 2bce980b1..182bb077c 100644 --- a/pkg/util/config/config.go +++ b/pkg/util/config/config.go @@ -76,7 +76,6 @@ type Config struct { // value of this string must be valid JSON or YAML; see initPodServiceAccount PodServiceAccountDefinition string `name:"pod_service_account_definition" default:""` DbHostedZone string `name:"db_hosted_zone" default:"db.example.com"` - EtcdScope string `name:"etcd_scope" default:"service"` WALES3Bucket string `name:"wal_s3_bucket"` LogS3Bucket string `name:"log_s3_bucket"` KubeIAMRole string `name:"kube_iam_role"` From b518a31d0c19f6563281dba5c16854bf2462bc58 Mon Sep 17 00:00:00 2001 From: Oleksii Kliukin Date: Tue, 12 Jun 2018 11:57:00 +0200 Subject: [PATCH 5/7] Document cluster manifests. (#320) Document cluster manifests options. Review by @erthalion and @zerg-junior. --- docs/cluster_manifest.md | 218 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 218 insertions(+) create mode 100644 docs/cluster_manifest.md diff --git a/docs/cluster_manifest.md b/docs/cluster_manifest.md new file mode 100644 index 000000000..db7ce17c1 --- /dev/null +++ b/docs/cluster_manifest.md @@ -0,0 +1,218 @@ +Postgres Cluster Manifest +========================= + +Individual postgres clusters are described by the Kubernetes *cluster manifest* +that has the structure defined by the `postgres CRD` (custom resource +definition). The following section describes the structure of the manifest and +the purpose of individual keys. You can take a look at the examples of the +[minimal](https://github.com/zalando-incubator/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml) +and the +[complete](https://github.com/zalando-incubator/postgres-operator/blob/master/manifests/complete-postgres-manifest.yaml) +cluster manifests. + +When Kubernetes resources, such as memory, CPU or volumes, are configured, +their amount is usually described as a string together with the units of +measurements. Please, refer to the [Kubernetes +documentation](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) +for the possible values of those. + +Manifest structure +------------------ + +A postgres manifest is a `YAML` document. On the top level both individual +parameters and parameter groups can be defined. Parameter names are written +in camelCase. + +### Cluster metadata + +Those parameters are grouped under the `metadata` top-level key. + +* **name** + the name of the cluster. Must start with the `teamId` followed by a dash. + Changing it after the cluster creation is not supported. Required field. + +* **namespace** + the namespace where the operator creates Kubernetes objects (i.e. pods, + services, secrets) for the cluster. Changing it after the cluster creation + results in deploying or updating a completely separate cluster in the target + namespace. Optional (if present, should match the namespace where the + manifest is applied). + +### Top-level parameters + +Those are parameters grouped directly under the `spec` key in the manifest. + +* **teamId** + name of the team the cluster belongs to. Changing it after the cluster + creation is not supported. Required field. + +* **dockerImage** + custom docker image that overrides the **docker_image** operator parameter. + It should be a [Spilo](https://github.com/zalando/spilo) image. Optional. + +* **enableMasterLoadBalancer** + boolean flag to override the operator defaults (set by the + `enable_master_load_balancer` parameter) to define whether to enable the load + balancer pointing to the postgres primary. Optional. + +* **enableReplicaLoadBalancer** + boolean flag to override the operator defaults (set by the + `enable_replica_load_balancer` parameter) to define whether to enable the + load balancer pointing to the postgres standby instances. Optional. + +* **allowedSourceRanges** + when one or more load balancers are enabled for the cluster, this parameter + defines the comma-separated range of IP networks (in CIDR-notation). The + corresponding load balancer is accessible only to the networks defined by + this parameter. Optional, when empty the load balancer service becomes + inaccessible from outside of the Kubernetes cluster. + +* **numberOfInstances** + total number of instances for a given cluster. The operator parameters + `max_instances` and `min_instances` may also adjust this number. Required + field. + +* **users** + a map of usernames to user flags for the users that should be created in the + cluster by the operator. User flags are a list, allowed elements are + `SUPERUSER` `REPLICATION` `INHERIT`, `LOGIN`, `NOLOGIN`, `CREATEROLE`, + `CREATEDB`, `BYPASSURL`. A login user is created by default unless NOLOGIN is + specified, in which case the operator creates a role. One can specify empty + flags by providing a JSON empty array '*[]*'. Optional. + +* **databases** + a map of database names to database owners for the databases that should be + created by the operator. The owner users should already exist on the cluster + (i.e. mentioned in the `user` parameter). Optional. + +* **tolerations** + a list of tolerations that apply to the cluster pods. Each element of that + list is a dictionary with the following fields: `key`, `operator`, `value`, + `effect` and `tolerationSeconds`. Each field is optional. See [Kubernetes + examples](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) + for details on tolerations and possible values of those keys. When set, this + value overrides the `pod_toleration` setting from the operator. Optional. + +### Postgres parameters + +Those parameters are grouped under the `postgresql` top-level key. + +* **version** + the postgres major version of the cluster. Looks at the [Spilo + project](https://github.com/zalando/spilo/releases) for the list of supported + versions. Changing the cluster version once the cluster has been bootstrapped + is not supported. Required field. + +* **parameters** + a dictionary of postgres parameter names and values to apply to the resulting + cluster. Optional (Spilo automatically sets reasonable defaults for + parameters like work_mem or max_connections). + +### Patroni parameters + +Those parameters are grouped under the `patroni` top-level key. See the [patroni +documentation](https://patroni.readthedocs.io/en/latest/SETTINGS.html) for the +explanation of `ttl` and `loop_wait` parameters. + +* **initdb** + a map of key-value pairs describing initdb parameters. For `data-checksum`, + `debug`, `no-locale`, `noclean`, `nosync` and `sync-only` parameters use + `true` as the value if you want to set them. Changes to this option do not + affect the already initialized clusters. Optional. + +* **pg_hba** + list of custom `pg_hba` lines to replace default ones. Note that the default + ones include + + ``` + hostsll all +pamrole all pam + ``` + , where pamrole is the name of the role for the pam authentication; any + custom `pg_hba` should include the pam line to avoid breaking pam + authentication. Optional. + +* **ttl** + patroni `ttl` parameter value, optional. The default is set by the Spilo + docker image. Optional. + +* **loop_wait** + patroni `loop_wait` parameter value, optional. The default is set by the + Spilo docker image. Optional. + +* **retry_timeout** + patroni `retry_timeout` parameter value, optional. The default is set by the + Spilo docker image. Optional. + +* **maximum_lag_on_failover** + patroni `maximum_lag_on_failover` parameter value, optional. The default is + set by the Spilo docker image. Optional. + +### Postgres container resources + +Those parameters define [CPU and memory requests and +limits](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) +for the postgres container. They are grouped under the `resources` top-level +key. There are two subgroups, `requests` and `limits`. + +#### Requests + +CPU and memory requests for the postgres container. + +* **cpu** + CPU requests for the postgres container. Optional, overrides the + `default_cpu_requests` operator configuration parameter. Optional. + +* **memory** + memory requests for the postgres container. Optional, overrides the + `default_memory_request` operator configuration parameter. Optional. + +#### Limits + +CPU and memory limits for the postgres container. + +* **cpu** + CPU limits for the postgres container. Optional, overrides the + `default_cpu_limits` operator configuration parameter. Optional. + +* **memory** + memory limits for the postgres container. Optional, overrides the + `default_memory_limits` operator configuration parameter. Optional. + +### Parameters defining how to clone the cluster from another one + +Those parameters are applied when the cluster should be a clone of another one +that is either already running or has a basebackup on S3. They are grouped +under the `clone` top-level key and do not affect the already running cluster. + +* **cluster** + name of the cluster to clone from. Translated to either the service name or + the key inside the S3 bucket containing base backups. Required when the + `clone` section is present. + +* **uid** + Kubernetes UID of the cluster to clone from. Since cluster name is not a + unique identifier of the cluster (as identically named clusters may exist in + different namespaces) , the operator uses UID in the S3 bucket name in order + to guarantee uniqueness. Has no effect when cloning from the running + clusters. Optional. + +* **timestamp** + the timestamp up to which the recovery should proceed. The operator always + configures non-inclusive recovery target, stopping right before the given + timestamp. When this parameter is set the operator will not consider cloning + from the live cluster, even if it is running, and instead goes to S3. Optional. + +### EBS volume resizing + +Those parameters are grouped under the `volume` top-level key and define the +properties of the persistent storage that stores postgres data. + +* **size** + the size of the target EBS volume. Usual Kubernetes size modifiers, i.e. `Gi` + or `Mi`, apply. Required. + +* **storageClass** + the name of the Kubernetes storage class to draw the persistent volume from. + See [Kubernetes + documentation](https://kubernetes.io/docs/concepts/storage/storage-classes/) + for the details on storage classes. Optional. From 5d02c57e042ec75b5f8b849e1c3f8396e863250b Mon Sep 17 00:00:00 2001 From: Oleksii Kliukin Date: Tue, 12 Jun 2018 19:12:11 +0200 Subject: [PATCH 6/7] Docs/reference (#323) Document operator command-line options and environment variables. --- README.md | 25 ++++++--- docs/administrator.md | 2 - docs/developer.md | 6 +-- docs/{concepts.md => index.md} | 28 ++++++++-- docs/quickstart.md | 54 +++++++++++++++++++ docs/reference.md | 23 -------- docs/{ => reference}/cluster_manifest.md | 19 +++---- .../reference/command_line_and_environment.md | 50 +++++++++++++++++ docs/{ => reference}/operator_parameters.md | 23 ++++---- docs/user.md | 12 ++--- mkdocs.yml | 16 ++++++ 11 files changed, 187 insertions(+), 71 deletions(-) rename docs/{concepts.md => index.md} (58%) create mode 100644 docs/quickstart.md delete mode 100644 docs/reference.md rename docs/{ => reference}/cluster_manifest.md (96%) create mode 100644 docs/reference/command_line_and_environment.md rename docs/{ => reference}/operator_parameters.md (97%) create mode 100644 mkdocs.yml diff --git a/README.md b/README.md index 48f269f8b..e7b486087 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,20 @@ manages PostgreSQL clusters on Kubernetes: 3. Finally, the operator periodically synchronizes the actual state of each Postgres cluster with the desired state defined in the cluster's manifest. +There is a browser-friendly version of this documentation at +[postgres-operator.readthedocs.io](https://postgres-operator.readthedocs.io) + +## Table of contents + +* [concepts](docs/index.md) +* [user documentation](docs/user.md) +* [administrator documentation](docs/administrator.md) +* [developer documentation](docs/developer.md) +* [operator configuration reference](docs/reference/operator_parameters.md) +* [cluster manifest reference](docs/reference/cluster_manifest.md) +* [command-line options and environment variables](docs/reference/command_line_and_environment.md) + +the rest of the document is a tutorial to get you up and running with the operator on Minikube. ## Quickstart @@ -34,6 +48,11 @@ Prerequisites: * [minikube](https://github.com/kubernetes/minikube/releases) * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-via-curl) +Note that you can also use built-in Kubernetes support in the Docker Desktop +for Mac to follow the steps of this tutorial. You would have to replace +`minikube start` and `minikube delete` with your launch actionsfor the Docker +built-in Kubernetes support. + ### Local execution ```bash @@ -77,10 +96,4 @@ Minikube is a tool to run Kubernetes cluster locally. The operator can be configured with the provided ConfigMap (`manifests/configmap.yaml`). -## Table of contents -* [concepts](docs/concepts.md) -* [user documentation](docs/user.md) -* [administrator documentation](docs/administrator.md) -* [developer documentation](docs/developer.md) -* [reference](docs/reference.md) diff --git a/docs/administrator.md b/docs/administrator.md index ef64dd224..bb775ed02 100644 --- a/docs/administrator.md +++ b/docs/administrator.md @@ -1,5 +1,3 @@ -# How to deploy PostgreSQL operator - ## Create ConfigMap ConfigMap is used to store the configuration of the operator diff --git a/docs/developer.md b/docs/developer.md index 5235c57dd..407af6439 100644 --- a/docs/developer.md +++ b/docs/developer.md @@ -1,5 +1,3 @@ -# Installing and starting minikube - ## Intro See [minikube installation guide](https://github.com/kubernetes/minikube/releases) @@ -12,9 +10,7 @@ After the installation, issue $ minikube start ``` -Note: if you are running on a Mac, make sure to use the [xhyve -driver](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#xhyve-driver) -instead of the default docker-machine one for performance reasons. +Note: if you are running on a Mac, you may also use Docker for Mac Kubernetes instead of a docker-machine. Once you have it started successfully, use [the quickstart guide](https://github.com/kubernetes/minikube#quickstart) in order to test your diff --git a/docs/concepts.md b/docs/index.md similarity index 58% rename from docs/concepts.md rename to docs/index.md index e93d5fc9c..c3327eae7 100644 --- a/docs/concepts.md +++ b/docs/index.md @@ -1,6 +1,28 @@ -# Concepts +# Introduction -## Scope +The Postgres [operator](https://coreos.com/blog/introducing-operators.html) +manages PostgreSQL clusters on Kubernetes: + +1. The operator watches additions, updates, and deletions of PostgreSQL cluster + manifests and changes the running clusters accordingly. For example, when a + user submits a new manifest, the operator fetches that manifest and spawns a + new Postgres cluster along with all necessary entities such as Kubernetes + StatefulSets and Postgres roles. See this + [Postgres cluster manifest](https://github.com/zalando-incubator/postgres-operator/blob/master/manifests/complete-postgres-manifest.yaml) + for settings that a manifest may contain. + +2. The operator also watches updates to [its own configuration](https://github.com/zalando-incubator/postgres-operator/blob/master/manifests/configmap.yaml) + and alters running Postgres clusters if necessary. For instance, if a pod + docker image is changed, the operator carries out the rolling update. That + is, the operator re-spawns one-by-one pods of each StatefulSet it manages + with the new Docker image. + +3. Finally, the operator periodically synchronizes the actual state of each + Postgres cluster with the desired state defined in the cluster's manifest. + +## Concepts + +### Scope The scope of the postgres operator is on provisioning, modifying configuration and cleaning up Postgres clusters that use Patroni, basically to make it easy @@ -15,7 +37,7 @@ experience. Monitoring of clusters is not in scope, for this good tools already exist from ZMON to Prometheus and more Postgres specific options. -## Status +### Status This project is currently in active development. It is however already [used internally by Zalando](https://jobs.zalando.com/tech/blog/postgresql-in-a-time-of-kubernetes/) diff --git a/docs/quickstart.md b/docs/quickstart.md new file mode 100644 index 000000000..7b9c667e1 --- /dev/null +++ b/docs/quickstart.md @@ -0,0 +1,54 @@ +## Prerequisites: + +In order to run the Postgres operator locally in minikube you need to install the following tools: + +* [minikube](https://github.com/kubernetes/minikube/releases) +* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-via-curl) + +Note that you can also use built-in Kubernetes support in the Docker Desktop +for Mac to follow the steps of this tutorial. You would have to replace +`minikube start` and `minikube delete` with your launch actionsfor the Docker +built-in Kubernetes support. + +## Local execution + +```bash +git clone https://github.com/zalando-incubator/postgres-operator.git +cd postgres-operator + +minikube start + +# start the operator; may take a few seconds +kubectl create -f manifests/configmap.yaml # configuration +kubectl create -f manifests/operator-service-account-rbac.yaml # identity and permissions +kubectl create -f manifests/postgres-operator.yaml # deployment + +# create a Postgres cluster +kubectl create -f manifests/minimal-postgres-manifest.yaml + +# connect to the Postgres master via psql +# operator creates the relevant k8s secret +export HOST_PORT=$(minikube service acid-minimal-cluster --url | sed 's,.*/,,') +export PGHOST=$(echo $HOST_PORT | cut -d: -f 1) +export PGPORT=$(echo $HOST_PORT | cut -d: -f 2) +export PGPASSWORD=$(kubectl get secret postgres.acid-minimal-cluster.credentials -o 'jsonpath={.data.password}' | base64 -d) +psql -U postgres + +# tear down cleanly +minikube delete +``` + +We have automated starting the operator and submitting the `acid-minimal-cluster` for you: +```bash +cd postgres-operator +./run_operator_locally.sh +``` + +## Running and testing the operator + +The best way to test the operator is to run it in [minikube](https://kubernetes.io/docs/getting-started-guides/minikube/). +Minikube is a tool to run Kubernetes cluster locally. + +### Configuration Options + +The operator can be configured with the provided ConfigMap (`manifests/configmap.yaml`). diff --git a/docs/reference.md b/docs/reference.md deleted file mode 100644 index d27ef3f0c..000000000 --- a/docs/reference.md +++ /dev/null @@ -1,23 +0,0 @@ -# Reference - -## Deprecated parameters - -Parameters `useLoadBalancer` and `replicaLoadBalancer` in the PostgreSQL -manifest are deprecated. To retain compatibility with the old manifests they -take effect in the absense of new `enableMasterLoadBalancer` and -`enableReplicaLoadBalancer` parameters (that is, if either of the new ones is -present - all deprecated parameters are ignored). The operator configuration -parameter `enable_load_balancer` is ignored in all cases. - -## Operator Configuration Parameters - -* team_api_role_configuration - a map represented as - *"key1:value1,key2:value2"* of configuration parameters applied to the roles - fetched from the API. For instance, `team_api_role_configuration: - log_statement:all,search_path:'public,"$user"'`. By default is set to - *"log_statement:all"*. See - [PostgreSQL documentation on ALTER ROLE .. SET](https://www.postgresql.org/docs/current/static/sql-alterrole.html) - for to learn about the available options. -* protected_role_names - a list of role names that should be forbidden as the - manifest, infrastructure and teams API roles. The default value is `admin`. - Operator will also disallow superuser and replication roles to be redefined. diff --git a/docs/cluster_manifest.md b/docs/reference/cluster_manifest.md similarity index 96% rename from docs/cluster_manifest.md rename to docs/reference/cluster_manifest.md index db7ce17c1..44ab99c83 100644 --- a/docs/cluster_manifest.md +++ b/docs/reference/cluster_manifest.md @@ -1,5 +1,3 @@ -Postgres Cluster Manifest -========================= Individual postgres clusters are described by the Kubernetes *cluster manifest* that has the structure defined by the `postgres CRD` (custom resource @@ -16,14 +14,13 @@ measurements. Please, refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for the possible values of those. -Manifest structure ------------------- +## Manifest structure A postgres manifest is a `YAML` document. On the top level both individual parameters and parameter groups can be defined. Parameter names are written in camelCase. -### Cluster metadata +## Cluster metadata Those parameters are grouped under the `metadata` top-level key. @@ -38,7 +35,7 @@ Those parameters are grouped under the `metadata` top-level key. namespace. Optional (if present, should match the namespace where the manifest is applied). -### Top-level parameters +## Top-level parameters Those are parameters grouped directly under the `spec` key in the manifest. @@ -93,7 +90,7 @@ Those are parameters grouped directly under the `spec` key in the manifest. for details on tolerations and possible values of those keys. When set, this value overrides the `pod_toleration` setting from the operator. Optional. -### Postgres parameters +## Postgres parameters Those parameters are grouped under the `postgresql` top-level key. @@ -108,7 +105,7 @@ Those parameters are grouped under the `postgresql` top-level key. cluster. Optional (Spilo automatically sets reasonable defaults for parameters like work_mem or max_connections). -### Patroni parameters +## Patroni parameters Those parameters are grouped under the `patroni` top-level key. See the [patroni documentation](https://patroni.readthedocs.io/en/latest/SETTINGS.html) for the @@ -147,14 +144,14 @@ explanation of `ttl` and `loop_wait` parameters. patroni `maximum_lag_on_failover` parameter value, optional. The default is set by the Spilo docker image. Optional. -### Postgres container resources +## Postgres container resources Those parameters define [CPU and memory requests and limits](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for the postgres container. They are grouped under the `resources` top-level key. There are two subgroups, `requests` and `limits`. -#### Requests +### Requests CPU and memory requests for the postgres container. @@ -178,7 +175,7 @@ CPU and memory limits for the postgres container. memory limits for the postgres container. Optional, overrides the `default_memory_limits` operator configuration parameter. Optional. -### Parameters defining how to clone the cluster from another one +## Parameters defining how to clone the cluster from another one Those parameters are applied when the cluster should be a clone of another one that is either already running or has a basebackup on S3. They are grouped diff --git a/docs/reference/command_line_and_environment.md b/docs/reference/command_line_and_environment.md new file mode 100644 index 000000000..1c4a6e8d7 --- /dev/null +++ b/docs/reference/command_line_and_environment.md @@ -0,0 +1,50 @@ +## Command-line options + +The following command-line options are supported for the operator: + +* **-kubeconfig** + the path to the kubeconfig file. Usually named config, it contains + authorization information as well as the URL of the Kubernetes master. + +* **-outofcluster** + run the operator on a client machine, as opposed to a within the cluster. + When running in this mode, the operator cannot connect to databases inside + the cluster, as well as call URLs of in-cluster objects (i.e. teams api + server). Mostly useful for debugging, it also requires setting the + `OPERATOR_NAMESPACE` environment variable for the operator own namespace. + +* **-nodatabaseaccess** + disable database access from the operator. Equivalent to the + `enable_database_access` set to off and can be overridden by the + aforementioned operator configuration option. + +* **-noteamsapi** + disable access to the teams API. Equivalent to the `enable_teams_api` set to + off can can be overridden by the aforementioned operator configuration + option. + +In addition to that, standard [glog +flags](https://godoc.org/github.com/golang/glog) are also supported. For +instance, one may want to add `-alsologtostderr` and `-v=8` to debug the +operator REST calls. + +## Environment variables + +The following environment variables are accepted by the operator: + +* **CONFIG_MAP_NAME** + name of the config map where the operator should look for its configuration. + Must be present. + +* **OPERATOR_NAMESPACE** + name of the namespace the operator runs it. Overrides autodetection by the + operator itself. + +* **WATCHED_NAMESPACE** + the name of the namespace the operator watches. Special '*' character denotes + all namespaces. Empty value defaults to the operator namespace. Overrides the + `watched_namespace` operator parameter. + +* **SCALYR_API_KEY** + the value of the Scalyr API key to supply to the pods. Overrides the + `scalyr_api_key` operator parameter. diff --git a/docs/operator_parameters.md b/docs/reference/operator_parameters.md similarity index 97% rename from docs/operator_parameters.md rename to docs/reference/operator_parameters.md index 6d277c589..115cb055e 100644 --- a/docs/operator_parameters.md +++ b/docs/reference/operator_parameters.md @@ -1,14 +1,9 @@ -Postgres Operator Configuration -=============================== Postgres operator is configured via a ConfigMap defined by the `CONFIG_MAP_NAME` environment variable. Variable names are underscore-separated words. -Available Configuration Parameters ----------------------------------- - -### General +## General * **etcd_host** Etcd connection string for Patroni defined as `host:port`. Not required when Patroni native Kubernetes support is used. The default is empty (use @@ -37,7 +32,7 @@ Available Configuration Parameters * **resync_period** period between consecutive sync requests. The default is `5m`. -### Postgres users +## Postgres users * **super_username** postgres `superuser` name to be created by `initdb`. The default is `postgres`. @@ -46,7 +41,7 @@ Available Configuration Parameters postgres username used for replication between instances. The default is `standby`. -### Kubernetes resources +## Kubernetes resources * **pod_service_account_name** service account used by Patroni running on individual Pods to communicate with the operator. Required even if native Kubernetes support in Patroni is @@ -126,7 +121,7 @@ Available Configuration Parameters conflicts they are overridden by the environment variables generated by the operator. The default is empty. -### Kubernetes resource requests +## Kubernetes resource requests * **default_cpu_request** CPU request value for the postgres containers, unless overridden by cluster-specific settings. The default is `100m`. @@ -143,7 +138,7 @@ Available Configuration Parameters memory limits for the postgres containers, unless overridden by cluster-specific settings. The default is `1Gi`. -### Operator timeouts +## Operator timeouts * **resource_check_interval** interval to wait between consecutive attempts to check for the presence of some Kubernetes resource (i.e. `StatefulSet` or `PodDisruptionBudget`). The @@ -170,7 +165,7 @@ Available Configuration Parameters * **ready_wait_timeout** the timeout for the complete postgres CRD creation. The default is `30s`. -### Load balancer related options +## Load balancer related options * **db_hosted_zone** DNS zone for the cluster DNS name when the load balancer is configured for the cluster. Only used when combined with @@ -201,7 +196,7 @@ Available Configuration Parameters replaced with the hosted zone (the value of the `db_hosted_zone` parameter). No other placeholders are allowed. -### AWS or GSC interaction +## AWS or GSC interaction * **wal_s3_bucket** S3 bucket to use for shipping WAL segments with WAL-E. A bucket has to be present and accessible by Patroni managed pods. At the moment, supported @@ -217,7 +212,7 @@ Available Configuration Parameters pods. Only used when combined with [kube2iam](https://github.com/jtblin/kube2iam) project on AWS. The default is empty. -### Debugging the operator +## Debugging the operator * **debug_logging** boolean parameter that toggles verbose debug logs from the operator. The default is `true`. @@ -272,7 +267,7 @@ Available Configuration Parameters List of roles that cannot be overwritten by an application, team or infrastructure role. The default is `admin`. -### Logging and REST API +## Logging and REST API * **api_port** REST API listener listens to this port. The default is `8080`. diff --git a/docs/user.md b/docs/user.md index 2420a2681..b17cfc784 100644 --- a/docs/user.md +++ b/docs/user.md @@ -1,9 +1,7 @@ -# How to create a new PostgreSQL cluster - ## Create a manifest for a new PostgreSQL cluster As an example you can take this -[minimal example](manifests/minimal-postgres-manifest.yaml): +[minimal example](https://github.com/zalando-incubator/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml): ```yaml apiVersion: "acid.zalan.do/v1" @@ -206,10 +204,10 @@ spec: Here `cluster` is a name of a source cluster that is going to be cloned. The cluster to clone is assumed to be running and the clone procedure invokes -`pg_basebackup` from it. The operator will connect to the service by name (if -the cluster is called test, then the connection string will look like host=test -port=5432), which means that you can clone only from clusters running in the -default namespace. +`pg_basebackup` from it. The operator will setup the cluster to be cloned to +connect to the service of the source cluster by name (if the cluster is called +test, then the connection string will look like host=test port=5432), which +means that you can clone only from clusters within the same namespace. ### Clone from S3 diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 000000000..e50107eed --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,16 @@ +site_name: Postgres Operator +repo_url: https://github.com/zalando-incubator/postgres-operator +theme: readthedocs + +pages: + - Introduction: index.md + - Quickstart: quickstart.md + - Administrator Guide: administrator.md + - User Guide: user.md + - Developer Guide: developer.md + - Reference: + - Operator Configuration: reference/operator_parameters.md + - Cluster Manifest description: reference/cluster_manifest.md + - Command-line options and environment: reference/command_line_and_environment.md + + From d9d2c5cbe562a3bf74e6bdbc7df7a52e232aaacd Mon Sep 17 00:00:00 2001 From: Oleksii Kliukin Date: Wed, 13 Jun 2018 12:32:56 +0200 Subject: [PATCH 7/7] Minor formatting fix --- docs/reference/cluster_manifest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/cluster_manifest.md b/docs/reference/cluster_manifest.md index 44ab99c83..c349e1631 100644 --- a/docs/reference/cluster_manifest.md +++ b/docs/reference/cluster_manifest.md @@ -72,7 +72,7 @@ Those are parameters grouped directly under the `spec` key in the manifest. * **users** a map of usernames to user flags for the users that should be created in the cluster by the operator. User flags are a list, allowed elements are - `SUPERUSER` `REPLICATION` `INHERIT`, `LOGIN`, `NOLOGIN`, `CREATEROLE`, + `SUPERUSER`, `REPLICATION`, `INHERIT`, `LOGIN`, `NOLOGIN`, `CREATEROLE`, `CREATEDB`, `BYPASSURL`. A login user is created by default unless NOLOGIN is specified, in which case the operator creates a role. One can specify empty flags by providing a JSON empty array '*[]*'. Optional.