public_git
/
postgres-operator
mirror of https://github.com/zalando/postgres-operator.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Sketch operator conf/docs

This commit is contained in:
Sergey Dudoladov 2019-01-08 14:40:43 +01:00
parent ad397045f7
commit 4088e0090f
8 changed files with 76 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
docs/administrator.md
View File

@ -226,3 +226,13 @@ The operator is capable of maintaining roles of multiple kinds within a Postgres
## Understanding rolling update of Spilo pods
The operator logs reasons for a rolling update with the `info` level and a diff between the old and new StatefulSet specs with the `debug` level. To benefit from numerous escape characters in the latter log entry, view it in CLI with `echo -e`. Note that the resultant message will contain some noise because the `PodTemplate` used by the operator is yet to be updated with the default values used internally in Kubernetes.
## Logical backups
The operator can launch k8s cron jobs to do periodic logical backups of all PG clusters under its control. The cron job spawns a separate pod with a single container that connects to one of the Postgres replicas for a backup. The operator updates cron jobs during Sync if a schedule or a docker image of a job changes. Notes:
1. The provided `registry.opensource.zalan.do/acid/logical-backup` image implements the backup via `pg_dumpall` and upload of (compressed) results to an S3 bucket; `pg_dumpall` requires a `superuser` access to a DB.
2. Due to the [limitation of Kubernetes cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) it is highly advisable to set up additional monitoring for this feature; such monitoring is outside of the scope of operator responsibilities.
3. The operator does not remove old backups: it is the responsibility of a k8s cluster administrator to define and implement a retention policy for backups.

2
docs/reference/cluster_manifest.md
View File

@ -261,6 +261,6 @@ defined in the sidecar dictionary:
Those parameters are defined under the `logical_backup` key:
* **enable_logical_backup**
determines if the logical backup of this cluster should be uploaded to S3. Default: false.
determines if the logical backup of this cluster should be taken and uploaded to S3. Default: false.
* **logical_backup_schedule**
backup schedule in Cron format. Default: "30 00 * * *"

18
docs/reference/operator_parameters.md
View File

@ -443,4 +443,22 @@ scalyr sidecar. In the CRD-based configuration they are grouped under the
Memory limit value for the Scalyr sidecar. The default is `1Gi`.
## Logical backup
These parameters configure a k8s cron job spawned by the operator to produce Postgres logical backups.
In the CRD-based configuration those parameters are grouped under the `logical_backup` key.
* **enable_logical_backup**
Determines if the operator should initiate the backup of all Postgres clusters it controls. Default: false.
* **logical_backup_schedule**
Backup schedule in the cron format. Default: "30 00 * * *"
* **logical_backup_docker_image**
Docker image for the pods of the cron job. Must implement backup/uploading logic. Default: empty.
* **logical_backup_s3_bucket**
S3 bucket to store backup results. Default: same as the bucket as for WAL.
For the configmap operator configuration, the [default parameter values](https://github.com/zalando-incubator/postgres-operator/blob/master/pkg/util/config/config.go#L14) mentioned here are likely to be overwritten in your local operator installation via your local version of the operator configmap. In the case you use the operator CRD, all the CRD defaults are provided in the [operator's default configuration manifest](https://github.com/zalando-incubator/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml)

4
manifests/configmap.yaml
View File

@ -48,3 +48,7 @@ data:
resource_check_interval: 3s
resource_check_timeout: 10m
resync_period: 5m
# enable_logical_backup: "true"
# logical_backup_schedule: "30 00 * * *"
# logical_backup_docker_image: registry.opensource.zalan.do/acid/logical-backup:master-42

46
pkg/apis/acid.zalan.do/v1/operator_configuration_type.go
View File

@ -120,25 +120,26 @@ type ScalyrConfiguration struct {
}
type OperatorConfigurationData struct {
EtcdHost string `json:"etcd_host,omitempty"`
DockerImage string `json:"docker_image,omitempty"`
Workers uint32 `json:"workers,omitempty"`
MinInstances int32 `json:"min_instances,omitempty"`
MaxInstances int32 `json:"max_instances,omitempty"`
ResyncPeriod Duration `json:"resync_period,omitempty"`
RepairPeriod Duration `json:"repair_period,omitempty"`
Sidecars map[string]string `json:"sidecar_docker_images,omitempty"`
PostgresUsersConfiguration PostgresUsersConfiguration `json:"users"`
Kubernetes KubernetesMetaConfiguration `json:"kubernetes"`
PostgresPodResources PostgresPodResourcesDefaults `json:"postgres_pod_resources"`
SetMemoryRequestToLimit bool `json:"set_memory_request_to_limit,omitempty"`
Timeouts OperatorTimeouts `json:"timeouts"`
LoadBalancer LoadBalancerConfiguration `json:"load_balancer"`
AWSGCP AWSGCPConfiguration `json:"aws_or_gcp"`
OperatorDebug OperatorDebugConfiguration `json:"debug"`
TeamsAPI TeamsAPIConfiguration `json:"teams_api"`
LoggingRESTAPI LoggingRESTAPIConfiguration `json:"logging_rest_api"`
Scalyr ScalyrConfiguration `json:"scalyr"`
EtcdHost string `json:"etcd_host,omitempty"`
DockerImage string `json:"docker_image,omitempty"`
Workers uint32 `json:"workers,omitempty"`
MinInstances int32 `json:"min_instances,omitempty"`
MaxInstances int32 `json:"max_instances,omitempty"`
ResyncPeriod Duration `json:"resync_period,omitempty"`
RepairPeriod Duration `json:"repair_period,omitempty"`
Sidecars map[string]string `json:"sidecar_docker_images,omitempty"`
PostgresUsersConfiguration PostgresUsersConfiguration `json:"users"`
Kubernetes KubernetesMetaConfiguration `json:"kubernetes"`
PostgresPodResources PostgresPodResourcesDefaults `json:"postgres_pod_resources"`
SetMemoryRequestToLimit bool `json:"set_memory_request_to_limit,omitempty"`
Timeouts OperatorTimeouts `json:"timeouts"`
LoadBalancer LoadBalancerConfiguration `json:"load_balancer"`
AWSGCP AWSGCPConfiguration `json:"aws_or_gcp"`
OperatorDebug OperatorDebugConfiguration `json:"debug"`
TeamsAPI TeamsAPIConfiguration `json:"teams_api"`
LoggingRESTAPI LoggingRESTAPIConfiguration `json:"logging_rest_api"`
Scalyr ScalyrConfiguration `json:"scalyr"`
LogicalBackup OperatorLogicalBackupConfiguration `json:"logical_backup"`
}
type OperatorConfigurationUsers struct {
@ -149,3 +150,10 @@ type OperatorConfigurationUsers struct {
}
type Duration time.Duration
type OperatorLogicalBackupConfiguration struct {
EnableLogicalBackup bool `json:"enable_logical_backup,omitempty"`
LogicalBackupSchedule string `json:"logical_backup_schedule,omitempty"`
LogicalBackupDockerImage string `json:"logical_backup_docker_image,omitempty"`
LogicalBackupS3Bucket string `json:"logical_backup_s3_bucket,omitempty"`
}

7
pkg/apis/acid.zalan.do/v1/postgresql_type.go
View File

@ -129,11 +129,8 @@ type UserFlags []string
// PostgresStatus contains status of the PostgreSQL cluster (running, creation failed etc.)
type PostgresStatus string
// CronSchedule is a convenience alias for string
type CronSchedule string
// LogicalBackup contains config of a k8s cron job responsible for running pg_dumpall
type LogicalBackup struct {
EnableLogicalBackup bool `json:"enable_logical_backup,omitempty"`
LogicalBackupSchedule CronSchedule `json:"logical_backup_schedule,omitempty"`
EnableLogicalBackup bool `json:"enable_logical_backup,omitempty"`
LogicalBackupSchedule string `json:"logical_backup_schedule,omitempty"`
}

5
pkg/controller/operator_config.go
View File

@ -97,5 +97,10 @@ func (c *Controller) importConfigurationFromCRD(fromCRD *acidv1.OperatorConfigur
result.ScalyrCPULimit = fromCRD.Scalyr.ScalyrCPULimit
result.ScalyrMemoryLimit = fromCRD.Scalyr.ScalyrMemoryLimit
result.EnableLogicalBackup = fromCRD.LogicalBackup.EnableLogicalBackup
result.LogicalBackupSchedule = fromCRD.LogicalBackup.LogicalBackupSchedule
result.LogicalBackupDockerImage = fromCRD.LogicalBackup.LogicalBackupDockerImage
result.LogicalBackupS3Bucket = fromCRD.LogicalBackup.LogicalBackupS3Bucket
return result
}

9
pkg/util/config/config.go
View File

@ -64,12 +64,21 @@ type Scalyr struct {
ScalyrMemoryLimit string `name:"scalyr_memory_limit" default:"1Gi"`
}
// LogicalBackup
type LogicalBackup struct {
EnableLogicalBackup bool `json:"enable_logical_backup,omitempty"`
LogicalBackupSchedule string `json:"logical_backup_schedule,omitempty"`
LogicalBackupDockerImage string `json:"logical_backup_docker_image,omitempty"`
LogicalBackupS3Bucket string `json:"logical_backup_s3_bucket,omitempty"`
}
// Config describes operator config
type Config struct {
CRD
Resources
Auth
Scalyr
LogicalBackup
WatchedNamespace string `name:"watched_namespace"` // special values: "*" means 'watch all namespaces', the empty string "" means 'watch a namespace where operator is deployed to'
EtcdHost string `name:"etcd_host" default:""` // special values: the empty string "" means Patroni will use k8s as a DCS