Allow configuring the operator via the YAML manifest. (#326)
* Up until now, the operator read its own configuration from the configmap. That has a number of limitations, i.e. when the configuration value is not a scalar, but a map or a list. We use a custom code based on github.com/kelseyhightower/envconfig to decode non-scalar values out of plain text keys, but that breaks when the data inside the keys contains both YAML-special elememtns (i.e. commas) and complex quotes, one good example for that is search_path inside `team_api_role_configuration`. In addition, reliance on the configmap forced a flag structure on the configuration, making it hard to write and to read (see https://github.com/zalando-incubator/postgres-operator/pull/308#issuecomment-395131778). The changes allow to supply the operator configuration in a proper YAML file. That required registering a custom CRD to support the operator configuration and provide an example at manifests/postgresql-operator-default-configuration.yaml. At the moment, both old configmap and the new CRD configuration is supported, so no compatibility issues, however, in the future I'd like to deprecate the configmap-based configuration altogether. Contrary to the configmap-based configuration, the CRD one doesn't embed defaults into the operator code, however, one can use the manifests/postgresql-operator-default-configuration.yaml as a starting point in order to build a custom configuration. Since previously `ReadyWaitInterval` and `ReadyWaitTimeout` parameters used to create the CRD were taken from the operator configuration, which is not possible if the configuration itself is stored in the CRD object, I've added the ability to specify them as environment variables `CRD_READY_WAIT_INTERVAL` and `CRD_READY_WAIT_TIMEOUT` respectively. Per review by @zerg-junior and @Jan-M.
This commit is contained in:
Oleksii Kliukin
GitHub
parent
e90a01050c
commit
3a9378d3b8
20
cmd/main.go
20
cmd/main.go
|
|
@ -7,6 +7,7 @@ import (
|
|||
"os/signal"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/zalando-incubator/postgres-operator/pkg/controller"
|
||||
"github.com/zalando-incubator/postgres-operator/pkg/spec"
|
||||
|
|
@ -20,6 +21,14 @@ var (
|
|||
config spec.ControllerConfig
|
||||
)
|
||||
|
||||
func mustParseDuration(d string) time.Duration {
|
||||
duration, err := time.ParseDuration(d)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return duration
|
||||
}
|
||||
|
||||
func init() {
|
||||
flag.StringVar(&kubeConfigFile, "kubeconfig", "", "Path to kubeconfig file with authorization and master location information.")
|
||||
flag.BoolVar(&outOfCluster, "outofcluster", false, "Whether the operator runs in- our outside of the Kubernetes cluster.")
|
||||
|
|
@ -38,6 +47,17 @@ func init() {
|
|||
log.Printf("Fully qualified configmap name: %v", config.ConfigMapName)
|
||||
|
||||
}
|
||||
if crd_interval := os.Getenv("CRD_READY_WAIT_INTERVAL"); crd_interval != "" {
|
||||
config.CRDReadyWaitInterval = mustParseDuration(crd_interval)
|
||||
} else {
|
||||
config.CRDReadyWaitInterval = 4 * time.Second
|
||||
}
|
||||
|
||||
if crd_timeout := os.Getenv("CRD_READY_WAIT_TIMEOUT"); crd_timeout != "" {
|
||||
config.CRDReadyWaitTimeout = mustParseDuration(crd_timeout)
|
||||
} else {
|
||||
config.CRDReadyWaitTimeout = 30 * time.Second
|
||||
}
|
||||
}
|
||||
|
||||
func main() {
|
||||
|
|
|
|||
|
|
@ -48,3 +48,11 @@ The following environment variables are accepted by the operator:
|
|||
* **SCALYR_API_KEY**
|
||||
the value of the Scalyr API key to supply to the pods. Overrides the
|
||||
`scalyr_api_key` operator parameter.
|
||||
|
||||
* **CRD_READY_WAIT_TIMEOUT**
|
||||
defines the timeout for the complete postgres CRD creation. When not set
|
||||
default is 30s.
|
||||
|
||||
* **CRD_READY_WAIT_INTERVAL**
|
||||
defines the interval between consecutive attempts waiting for the postgres
|
||||
CRD to be created. The default is 5s.
|
||||
|
|
|
|||
|
|
@ -1,9 +1,54 @@
|
|||
There are two mutually-exclusive methods to set the Postgres Operator
|
||||
configuration.
|
||||
|
||||
Postgres operator is configured via a ConfigMap defined by the
|
||||
`CONFIG_MAP_NAME` environment variable. Variable names are underscore-separated
|
||||
words.
|
||||
* ConfigMaps-based, the legacy one. The configuration is supplied in a
|
||||
key-value configmap, defined by the `CONFIG_MAP_NAME` environment variable.
|
||||
Non-scalar values, i.e. lists or maps, are encoded in the value strings using
|
||||
the comma-based syntax for lists and coma-separated `key:value` syntax for
|
||||
maps. String values containing ':' should be enclosed in quotes. The
|
||||
configuration is flat, parameter group names below are not reflected in the
|
||||
configuration structure. There is an
|
||||
[example](https://github.com/zalando-incubator/postgres-operator/blob/master/manifests/configmap.yaml)
|
||||
|
||||
* CRD-based configuration. The configuration is stored in the custom YAML
|
||||
manifest, an instance of the custom resource definition (CRD) called
|
||||
`postgresql-operator-configuration`. This CRD is registered by the operator
|
||||
during the start when `POSTGRES_OPERATOR_CONFIGURATION_OBJECT` variable is
|
||||
set to a non-empty value. The CRD-based configuration is a regular YAML
|
||||
document; non-scalar keys are simply represented in the usual YAML way. The
|
||||
usage of the CRD-based configuration is triggered by setting the
|
||||
`POSTGRES_OPERATOR_CONFIGURATION_OBJECT` variable, which should point to the
|
||||
`postgresql-operator-configuration` object name in the operators namespace.
|
||||
There are no default values built-in in the operator, each parameter that is
|
||||
not supplied in the configuration receives an empty value. In order to
|
||||
create your own configuration just copy the [default
|
||||
one](https://github.com/zalando-incubator/postgres-operator/blob/wip/operator_configuration_via_crd/manifests/postgresql-operator-default-configuration.yaml)
|
||||
and change it.
|
||||
|
||||
CRD-based configuration is more natural and powerful then the one based on
|
||||
ConfigMaps and should be used unless there is a compatibility requirement to
|
||||
use an already existing configuration. Even in that case, it should be rather
|
||||
straightforward to convert the configmap based configuration into the CRD-based
|
||||
one and restart the operator. The ConfigMaps-based configuration will be
|
||||
deprecated and subsequently removed in future releases.
|
||||
|
||||
Note that for the CRD-based configuration configuration groups below correspond
|
||||
to the non-leaf keys in the target YAML (i.e. for the Kubernetes resources the
|
||||
key is `kubernetes`). The key is mentioned alongside the group description. The
|
||||
ConfigMap-based configuration is flat and does not allow non-leaf keys.
|
||||
|
||||
Since in the CRD-based case the operator needs to create a CRD first, which is
|
||||
controlled by the `resource_check_interval` and `resource_check_timeout`
|
||||
parameters, those parameters have no effect and are replaced by the
|
||||
`CRD_READY_WAIT_INTERVAL` and `CRD_READY_WAIT_TIMEOUT` environment variables.
|
||||
They will be deprecated and removed in the future.
|
||||
|
||||
Variable names are underscore-separated words.
|
||||
|
||||
## General
|
||||
|
||||
Those are top-level keys, containing both leaf keys and groups.
|
||||
|
||||
* **etcd_host**
|
||||
Etcd connection string for Patroni defined as `host:port`. Not required when
|
||||
Patroni native Kubernetes support is used. The default is empty (use
|
||||
|
|
@ -38,6 +83,10 @@ words.
|
|||
period between consecutive sync requests. The default is `5m`.
|
||||
|
||||
## Postgres users
|
||||
|
||||
Parameters describing Postgres users. In a CRD-configuration, they are grouped
|
||||
under the `users` key.
|
||||
|
||||
* **super_username**
|
||||
postgres `superuser` name to be created by `initdb`. The default is
|
||||
`postgres`.
|
||||
|
|
@ -47,6 +96,11 @@ words.
|
|||
`standby`.
|
||||
|
||||
## Kubernetes resources
|
||||
|
||||
Parameters to configure cluster-related Kubernetes objects created by the
|
||||
operator, as well as some timeouts associated with them. In a CRD-based
|
||||
configuration they are grouped under the `kubernetes` key.
|
||||
|
||||
* **pod_service_account_name**
|
||||
service account used by Patroni running on individual Pods to communicate
|
||||
with the operator. Required even if native Kubernetes support in Patroni is
|
||||
|
|
@ -127,6 +181,11 @@ words.
|
|||
operator. The default is empty.
|
||||
|
||||
## Kubernetes resource requests
|
||||
|
||||
This group allows you to configure resource requests for the Postgres pods.
|
||||
Those parameters are grouped under the `postgres_pod_resources` key in a
|
||||
CRD-based configuration.
|
||||
|
||||
* **default_cpu_request**
|
||||
CPU request value for the postgres containers, unless overridden by
|
||||
cluster-specific settings. The default is `100m`.
|
||||
|
|
@ -144,6 +203,13 @@ words.
|
|||
settings. The default is `1Gi`.
|
||||
|
||||
## Operator timeouts
|
||||
|
||||
This set of parameters define various timeouts related to some operator
|
||||
actions, affecting pod operations and CRD creation. In the CRD-based
|
||||
configuration `resource_check_interval` and `resource_check_timeout` have no
|
||||
effect, and the parameters are grouped under the `timeouts` key in the
|
||||
CRD-based configuration.
|
||||
|
||||
* **resource_check_interval**
|
||||
interval to wait between consecutive attempts to check for the presence of
|
||||
some Kubernetes resource (i.e. `StatefulSet` or `PodDisruptionBudget`). The
|
||||
|
|
@ -171,6 +237,10 @@ words.
|
|||
the timeout for the complete postgres CRD creation. The default is `30s`.
|
||||
|
||||
## Load balancer related options
|
||||
|
||||
Those options affect the behavior of load balancers created by the operator.
|
||||
In the CRD-based configuration they are grouped under the `load_balancer` key.
|
||||
|
||||
* **db_hosted_zone**
|
||||
DNS zone for the cluster DNS name when the load balancer is configured for
|
||||
the cluster. Only used when combined with
|
||||
|
|
@ -202,6 +272,12 @@ words.
|
|||
No other placeholders are allowed.
|
||||
|
||||
## AWS or GSC interaction
|
||||
|
||||
The options in this group configure operator interactions with non-Kubernetes
|
||||
objects from AWS or Google cloud. They have no effect unless you are using
|
||||
either. In the CRD-based configuration those options are grouped under the
|
||||
`aws_or_gcp` key.
|
||||
|
||||
* **wal_s3_bucket**
|
||||
S3 bucket to use for shipping WAL segments with WAL-E. A bucket has to be
|
||||
present and accessible by Patroni managed pods. At the moment, supported
|
||||
|
|
@ -218,9 +294,12 @@ words.
|
|||
[kube2iam](https://github.com/jtblin/kube2iam) project on AWS. The default is empty.
|
||||
|
||||
* **aws_region**
|
||||
AWS region used to store ESB volumes.
|
||||
AWS region used to store ESB volumes. The default is `eu-central-1`.
|
||||
|
||||
## Debugging the operator
|
||||
|
||||
Options to aid debugging of the operator itself. Grouped under the `debug` key.
|
||||
|
||||
* **debug_logging**
|
||||
boolean parameter that toggles verbose debug logs from the operator. The
|
||||
default is `true`.
|
||||
|
|
@ -230,7 +309,12 @@ words.
|
|||
access to the postgres database, i.e. creating databases and users. The default
|
||||
is `true`.
|
||||
|
||||
### Automatic creation of human users in the database
|
||||
## Automatic creation of human users in the database
|
||||
|
||||
Options to automate creation of human users with the aid of the teams API
|
||||
service. In the CRD-based configuration those are grouped under the `teams_api`
|
||||
key.
|
||||
|
||||
* **enable_teams_api**
|
||||
boolean parameter that toggles usage of the Teams API by the operator.
|
||||
The default is `true`.
|
||||
|
|
@ -276,6 +360,9 @@ words.
|
|||
infrastructure role. The default is `admin`.
|
||||
|
||||
## Logging and REST API
|
||||
|
||||
Parameters affecting logging and REST API listener. In the CRD-based configuration they are grouped under the `logging_rest_api` key.
|
||||
|
||||
* **api_port**
|
||||
REST API listener listens to this port. The default is `8080`.
|
||||
|
||||
|
|
@ -286,6 +373,11 @@ words.
|
|||
number of entries in the cluster history ring buffer. The default is `1000`.
|
||||
|
||||
## Scalyr options
|
||||
|
||||
Those parameters define the resource requests/limits and properties of the
|
||||
scalyr sidecar. In the CRD-based configuration they are grouped under the
|
||||
`scalyr` key.
|
||||
|
||||
* **scalyr_api_key**
|
||||
API key for the Scalyr sidecar. The default is empty.
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,81 @@
|
|||
apiVersion: "acid.zalan.do/v1"
|
||||
kind: postgresql-operator-configuration
|
||||
metadata:
|
||||
name: postgresql-operator-default-configuration
|
||||
configuration:
|
||||
etcd_host: ""
|
||||
docker_image: registry.opensource.zalan.do/acid/spilo-cdp-10:1.4-p8
|
||||
workers: 4
|
||||
min_instances: -1
|
||||
max_instances: -1
|
||||
resync_period: 5m
|
||||
#sidecar_docker_images:
|
||||
# example: "exampleimage:exampletag"
|
||||
users:
|
||||
super_username: postgres
|
||||
replication_username: standby
|
||||
kubernetes:
|
||||
pod_service_account_name: operator
|
||||
pod_terminate_grace_period: 5m
|
||||
pdb_name_format: "postgres-{cluster}-pdb"
|
||||
secret_name_template: "{username}.{cluster}.credentials.{tprkind}.{tprgroup}"
|
||||
oauth_token_secret_name: postgresql-operator
|
||||
pod_role_label: spilo-role
|
||||
cluster_labels:
|
||||
application: spilo
|
||||
cluster_name_label: cluster-name
|
||||
# watched_namespace:""
|
||||
# node_readiness_label: ""
|
||||
# toleration: {}
|
||||
# infrastructure_roles_secret_name: ""
|
||||
# pod_environment_configmap: ""
|
||||
postgres_pod_resources:
|
||||
default_cpu_request: 100m
|
||||
default_memory_request: 100Mi
|
||||
default_cpu_limit: "3"
|
||||
default_memory_limit: 1Gi
|
||||
timeouts:
|
||||
resource_check_interval: 3s
|
||||
resource_check_timeout: 10m
|
||||
pod_label_wait_timeout: 10m
|
||||
pod_deletion_wait_timeout: 10m
|
||||
ready_wait_interval: 4s
|
||||
ready_wait_timeout: 30s
|
||||
load_balancer:
|
||||
enable_master_load_balancer: false
|
||||
enable_replica_load_balancer: false
|
||||
master_dns_name_format: "{cluster}.{team}.{hostedzone}"
|
||||
replica_dns_name_format: "{cluster}-repl.{team}.{hostedzone}"
|
||||
aws_or_gcp:
|
||||
# db_hosted_zone: ""
|
||||
# wal_s3_bucket: ""
|
||||
# log_s3_bucket: ""
|
||||
# kube_iam_role: ""
|
||||
aws_region: eu-central-1
|
||||
debug:
|
||||
debug_logging: true
|
||||
enable_database_access: true
|
||||
teams_api:
|
||||
enable_teams_api: false
|
||||
team_api_role_configuration:
|
||||
log_statement: all
|
||||
enable_team_superuser: false
|
||||
team_admin_role: admin
|
||||
pam_role_name: zalandos
|
||||
# pam_configuration: ""
|
||||
protected_role_names:
|
||||
- admin
|
||||
# teams_api_url: ""
|
||||
logging_rest_api:
|
||||
api_port: 8008
|
||||
ring_log_lines: 100
|
||||
cluster_history_entries: 1000
|
||||
scalyr:
|
||||
scalyr_cpu_request: 100m
|
||||
scalyr_memory_request: 50Mi
|
||||
scalyr_cpu_limit: "1"
|
||||
scalyr_memory_limit: 1Gi
|
||||
# scalyr_api_key: ""
|
||||
# scalyr_image: ""
|
||||
# scalyr_server_url: ""
|
||||
|
||||
|
|
@ -155,7 +155,7 @@ func (c *Cluster) setStatus(status spec.PostgresStatus) {
|
|||
|
||||
_, err = c.KubeClient.CRDREST.Patch(types.MergePatchType).
|
||||
Namespace(c.Namespace).
|
||||
Resource(constants.CRDResource).
|
||||
Resource(constants.PostgresCRDResource).
|
||||
Name(c.Name).
|
||||
Body(request).
|
||||
DoRaw()
|
||||
|
|
|
|||
|
|
@ -424,7 +424,7 @@ func (c *Cluster) credentialSecretNameForCluster(username string, clusterName st
|
|||
return c.OpConfig.SecretNameTemplate.Format(
|
||||
"username", strings.Replace(username, "_", "-", -1),
|
||||
"cluster", clusterName,
|
||||
"tprkind", constants.CRDKind,
|
||||
"tprkind", constants.PostgresCRDKind,
|
||||
"tprgroup", constants.CRDGroup)
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -101,23 +101,24 @@ func (c *Controller) initOperatorConfig() {
|
|||
c.logger.Infoln("no ConfigMap specified. Loading default values")
|
||||
}
|
||||
|
||||
configMapData["watched_namespace"] = c.getEffectiveNamespace(os.Getenv("WATCHED_NAMESPACE"), configMapData["watched_namespace"])
|
||||
|
||||
if c.config.NoDatabaseAccess {
|
||||
configMapData["enable_database_access"] = "false"
|
||||
}
|
||||
if c.config.NoTeamsAPI {
|
||||
configMapData["enable_teams_api"] = "false"
|
||||
}
|
||||
|
||||
c.opConfig = config.NewFromMap(configMapData)
|
||||
c.warnOnDeprecatedOperatorParameters()
|
||||
|
||||
}
|
||||
|
||||
func (c *Controller) modifyConfigFromEnvironment() {
|
||||
c.opConfig.WatchedNamespace = c.getEffectiveNamespace(os.Getenv("WATCHED_NAMESPACE"), c.opConfig.WatchedNamespace)
|
||||
|
||||
if c.config.NoDatabaseAccess {
|
||||
c.opConfig.EnableDBAccess = c.config.NoDatabaseAccess
|
||||
}
|
||||
if c.config.NoTeamsAPI {
|
||||
c.opConfig.EnableTeamsAPI = c.config.NoTeamsAPI
|
||||
}
|
||||
scalyrAPIKey := os.Getenv("SCALYR_API_KEY")
|
||||
if scalyrAPIKey != "" {
|
||||
c.opConfig.ScalyrAPIKey = scalyrAPIKey
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
// warningOnDeprecatedParameters emits warnings upon finding deprecated parmaters
|
||||
|
|
@ -163,20 +164,34 @@ func (c *Controller) initPodServiceAccount() {
|
|||
|
||||
func (c *Controller) initController() {
|
||||
c.initClients()
|
||||
c.initOperatorConfig()
|
||||
|
||||
if configObjectName := os.Getenv("POSTGRES_OPERATOR_CONFIGURATION_OBJECT"); configObjectName != "" {
|
||||
if err := c.createConfigurationCRD(); err != nil {
|
||||
c.logger.Fatalf("could not register Operator Configuration CustomResourceDefinition: %v", err)
|
||||
}
|
||||
if cfg, err := c.readOperatorConfigurationFromCRD(spec.GetOperatorNamespace(), configObjectName); err != nil {
|
||||
c.logger.Fatalf("unable to read operator configuration: %v", err)
|
||||
} else {
|
||||
c.opConfig = c.importConfigurationFromCRD(&cfg.Configuration)
|
||||
}
|
||||
} else {
|
||||
c.initOperatorConfig()
|
||||
}
|
||||
|
||||
c.modifyConfigFromEnvironment()
|
||||
|
||||
if err := c.createPostgresCRD(); err != nil {
|
||||
c.logger.Fatalf("could not register Postgres CustomResourceDefinition: %v", err)
|
||||
}
|
||||
|
||||
c.initPodServiceAccount()
|
||||
|
||||
c.initSharedInformers()
|
||||
|
||||
c.logger.Infof("config: %s", c.opConfig.MustMarshal())
|
||||
|
||||
if c.opConfig.DebugLogging {
|
||||
c.logger.Logger.Level = logrus.DebugLevel
|
||||
}
|
||||
|
||||
if err := c.createCRD(); err != nil {
|
||||
c.logger.Fatalf("could not register CustomResourceDefinition: %v", err)
|
||||
}
|
||||
c.logger.Infof("config: %s", c.opConfig.MustMarshal())
|
||||
|
||||
if infraRoles, err := c.getInfrastructureRoles(&c.opConfig.InfrastructureRolesSecretName); err != nil {
|
||||
c.logger.Warningf("could not get infrastructure roles: %v", err)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,108 @@
|
|||
package controller
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"github.com/zalando-incubator/postgres-operator/pkg/util/config"
|
||||
"github.com/zalando-incubator/postgres-operator/pkg/util/constants"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (c *Controller) readOperatorConfigurationFromCRD(configObjectNamespace, configObjectName string) (*config.OperatorConfiguration, error) {
|
||||
var (
|
||||
opConfig config.OperatorConfiguration
|
||||
)
|
||||
|
||||
req := c.KubeClient.CRDREST.Get().
|
||||
Name(configObjectName).
|
||||
Namespace(configObjectNamespace).
|
||||
Resource(constants.OperatorConfigCRDResource).
|
||||
VersionedParams(&metav1.ListOptions{ResourceVersion: "0"}, metav1.ParameterCodec)
|
||||
|
||||
data, err := req.DoRaw()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("could not get operator configuration object %s: %v", configObjectName, err)
|
||||
}
|
||||
if err = json.Unmarshal(data, &opConfig); err != nil {
|
||||
return nil, fmt.Errorf("could not unmarshal operator configuration object %s, %v", configObjectName, err)
|
||||
}
|
||||
|
||||
return &opConfig, nil
|
||||
}
|
||||
|
||||
// importConfigurationFromCRD is a transitional function that converts CRD configuration to the one based on the configmap
|
||||
func (c *Controller) importConfigurationFromCRD(fromCRD *config.OperatorConfigurationData) *config.Config {
|
||||
result := &config.Config{}
|
||||
|
||||
result.EtcdHost = fromCRD.EtcdHost
|
||||
result.DockerImage = fromCRD.DockerImage
|
||||
result.Workers = fromCRD.Workers
|
||||
result.MinInstances = fromCRD.MinInstances
|
||||
result.MaxInstances = fromCRD.MaxInstances
|
||||
result.ResyncPeriod = time.Duration(fromCRD.ResyncPeriod)
|
||||
result.Sidecars = fromCRD.Sidecars
|
||||
|
||||
result.SuperUsername = fromCRD.PostgresUsersConfiguration.SuperUsername
|
||||
result.ReplicationUsername = fromCRD.PostgresUsersConfiguration.ReplicationUsername
|
||||
|
||||
result.PodServiceAccountName = fromCRD.Kubernetes.PodServiceAccountName
|
||||
result.PodServiceAccountDefinition = fromCRD.Kubernetes.PodServiceAccountDefinition
|
||||
result.PodTerminateGracePeriod = time.Duration(fromCRD.Kubernetes.PodTerminateGracePeriod)
|
||||
result.WatchedNamespace = fromCRD.Kubernetes.WatchedNamespace
|
||||
result.PDBNameFormat = fromCRD.Kubernetes.PDBNameFormat
|
||||
result.SecretNameTemplate = fromCRD.Kubernetes.SecretNameTemplate
|
||||
result.OAuthTokenSecretName = fromCRD.Kubernetes.OAuthTokenSecretName
|
||||
result.InfrastructureRolesSecretName = fromCRD.Kubernetes.InfrastructureRolesSecretName
|
||||
result.PodRoleLabel = fromCRD.Kubernetes.PodRoleLabel
|
||||
result.ClusterLabels = fromCRD.Kubernetes.ClusterLabels
|
||||
result.ClusterNameLabel = fromCRD.Kubernetes.ClusterNameLabel
|
||||
result.NodeReadinessLabel = fromCRD.Kubernetes.NodeReadinessLabel
|
||||
|
||||
result.DefaultCPURequest = fromCRD.PostgresPodResources.DefaultCPURequest
|
||||
result.DefaultMemoryRequest = fromCRD.PostgresPodResources.DefaultMemoryRequest
|
||||
result.DefaultCPULimit = fromCRD.PostgresPodResources.DefaultCPULimit
|
||||
result.DefaultMemoryLimit = fromCRD.PostgresPodResources.DefaultMemoryLimit
|
||||
|
||||
result.ResourceCheckInterval = time.Duration(fromCRD.Timeouts.ResourceCheckInterval)
|
||||
result.ResourceCheckTimeout = time.Duration(fromCRD.Timeouts.ResourceCheckTimeout)
|
||||
result.PodLabelWaitTimeout = time.Duration(fromCRD.Timeouts.PodLabelWaitTimeout)
|
||||
result.PodDeletionWaitTimeout = time.Duration(fromCRD.Timeouts.PodDeletionWaitTimeout)
|
||||
result.ReadyWaitInterval = time.Duration(fromCRD.Timeouts.ReadyWaitInterval)
|
||||
result.ReadyWaitTimeout = time.Duration(fromCRD.Timeouts.ReadyWaitTimeout)
|
||||
|
||||
result.DbHostedZone = fromCRD.LoadBalancer.DbHostedZone
|
||||
result.EnableMasterLoadBalancer = fromCRD.LoadBalancer.EnableMasterLoadBalancer
|
||||
result.EnableReplicaLoadBalancer = fromCRD.LoadBalancer.EnableReplicaLoadBalancer
|
||||
result.MasterDNSNameFormat = fromCRD.LoadBalancer.MasterDNSNameFormat
|
||||
result.ReplicaDNSNameFormat = fromCRD.LoadBalancer.ReplicaDNSNameFormat
|
||||
|
||||
result.WALES3Bucket = fromCRD.AWSGCP.WALES3Bucket
|
||||
result.AWSRegion = fromCRD.AWSGCP.AWSRegion
|
||||
result.LogS3Bucket = fromCRD.AWSGCP.LogS3Bucket
|
||||
result.KubeIAMRole = fromCRD.AWSGCP.KubeIAMRole
|
||||
|
||||
result.DebugLogging = fromCRD.OperatorDebug.DebugLogging
|
||||
result.EnableDBAccess = fromCRD.OperatorDebug.EnableDBAccess
|
||||
result.EnableTeamsAPI = fromCRD.TeamsAPI.EnableTeamsAPI
|
||||
result.TeamsAPIUrl = fromCRD.TeamsAPI.TeamsAPIUrl
|
||||
result.TeamAPIRoleConfiguration = fromCRD.TeamsAPI.TeamAPIRoleConfiguration
|
||||
result.EnableTeamSuperuser = fromCRD.TeamsAPI.EnableTeamSuperuser
|
||||
result.TeamAdminRole = fromCRD.TeamsAPI.TeamAdminRole
|
||||
result.PamRoleName = fromCRD.TeamsAPI.PamRoleName
|
||||
|
||||
result.APIPort = fromCRD.LoggingRESTAPI.APIPort
|
||||
result.RingLogLines = fromCRD.LoggingRESTAPI.RingLogLines
|
||||
result.ClusterHistoryEntries = fromCRD.LoggingRESTAPI.ClusterHistoryEntries
|
||||
|
||||
result.ScalyrAPIKey = fromCRD.Scalyr.ScalyrAPIKey
|
||||
result.ScalyrImage = fromCRD.Scalyr.ScalyrImage
|
||||
result.ScalyrServerURL = fromCRD.Scalyr.ScalyrServerURL
|
||||
result.ScalyrCPURequest = fromCRD.Scalyr.ScalyrCPURequest
|
||||
result.ScalyrMemoryRequest = fromCRD.Scalyr.ScalyrMemoryRequest
|
||||
result.ScalyrCPULimit = fromCRD.Scalyr.ScalyrCPULimit
|
||||
result.ScalyrMemoryLimit = fromCRD.Scalyr.ScalyrMemoryLimit
|
||||
|
||||
return result
|
||||
}
|
||||
|
|
@ -48,7 +48,7 @@ func (c *Controller) clusterListFunc(options metav1.ListOptions) (runtime.Object
|
|||
req := c.KubeClient.CRDREST.
|
||||
Get().
|
||||
Namespace(c.opConfig.WatchedNamespace).
|
||||
Resource(constants.CRDResource).
|
||||
Resource(constants.PostgresCRDResource).
|
||||
VersionedParams(&options, metav1.ParameterCodec)
|
||||
|
||||
b, err := req.DoRaw()
|
||||
|
|
@ -117,7 +117,7 @@ func (c *Controller) clusterWatchFunc(options metav1.ListOptions) (watch.Interfa
|
|||
r, err := c.KubeClient.CRDREST.
|
||||
Get().
|
||||
Namespace(c.opConfig.WatchedNamespace).
|
||||
Resource(constants.CRDResource).
|
||||
Resource(constants.PostgresCRDResource).
|
||||
VersionedParams(&options, metav1.ParameterCodec).
|
||||
FieldsSelectorParam(nil).
|
||||
Stream()
|
||||
|
|
|
|||
|
|
@ -47,20 +47,20 @@ func (c *Controller) clusterWorkerID(clusterName spec.NamespacedName) uint32 {
|
|||
return c.clusterWorkers[clusterName]
|
||||
}
|
||||
|
||||
func (c *Controller) createCRD() error {
|
||||
func (c *Controller) createOperatorCRD(plural, singular, short string) error {
|
||||
crd := &apiextv1beta1.CustomResourceDefinition{
|
||||
ObjectMeta: metav1.ObjectMeta{
|
||||
Name: constants.CRDResource + "." + constants.CRDGroup,
|
||||
Name: plural + "." + constants.CRDGroup,
|
||||
},
|
||||
Spec: apiextv1beta1.CustomResourceDefinitionSpec{
|
||||
Group: constants.CRDGroup,
|
||||
Version: constants.CRDApiVersion,
|
||||
Names: apiextv1beta1.CustomResourceDefinitionNames{
|
||||
Plural: constants.CRDResource,
|
||||
Singular: constants.CRDKind,
|
||||
ShortNames: []string{constants.CRDShort},
|
||||
Kind: constants.CRDKind,
|
||||
ListKind: constants.CRDKind + "List",
|
||||
Plural: plural,
|
||||
Singular: singular,
|
||||
ShortNames: []string{short},
|
||||
Kind: singular,
|
||||
ListKind: singular + "List",
|
||||
},
|
||||
Scope: apiextv1beta1.NamespaceScoped,
|
||||
},
|
||||
|
|
@ -75,7 +75,7 @@ func (c *Controller) createCRD() error {
|
|||
c.logger.Infof("customResourceDefinition %q has been registered", crd.Name)
|
||||
}
|
||||
|
||||
return wait.Poll(c.opConfig.CRD.ReadyWaitInterval, c.opConfig.CRD.ReadyWaitTimeout, func() (bool, error) {
|
||||
return wait.Poll(c.config.CRDReadyWaitInterval, c.config.CRDReadyWaitTimeout, func() (bool, error) {
|
||||
c, err := c.KubeClient.CustomResourceDefinitions().Get(crd.Name, metav1.GetOptions{})
|
||||
if err != nil {
|
||||
return false, err
|
||||
|
|
@ -98,6 +98,14 @@ func (c *Controller) createCRD() error {
|
|||
})
|
||||
}
|
||||
|
||||
func (c *Controller) createPostgresCRD() error {
|
||||
return c.createOperatorCRD(constants.PostgresCRDResource, constants.PostgresCRDKind, constants.PostgresCRDShort)
|
||||
}
|
||||
|
||||
func (c *Controller) createConfigurationCRD() error {
|
||||
return c.createOperatorCRD(constants.OperatorConfigCRDResource, constants.OperatorConfigCRDKind, constants.OperatorConfigCRDShort)
|
||||
}
|
||||
|
||||
func readDecodedRole(s string) (*spec.PgUser, error) {
|
||||
var result spec.PgUser
|
||||
if err := yaml.Unmarshal([]byte(s), &result); err != nil {
|
||||
|
|
|
|||
|
|
@ -150,7 +150,7 @@ var (
|
|||
)
|
||||
|
||||
// Clone makes a deepcopy of the Postgresql structure. The Error field is nulled-out,
|
||||
// as there is no guaratee that the actual implementation of the error interface
|
||||
// as there is no guarantee that the actual implementation of the error interface
|
||||
// will not contain any private fields not-reachable to deepcopy. This should be ok,
|
||||
// since Error is never read from a Kubernetes object.
|
||||
func (p *Postgresql) Clone() *Postgresql {
|
||||
|
|
@ -200,7 +200,7 @@ func (m *MaintenanceWindow) MarshalJSON() ([]byte, error) {
|
|||
m.EndTime.Format("15:04"))), nil
|
||||
}
|
||||
|
||||
// UnmarshalJSON convets a JSON to the maintenance window definition.
|
||||
// UnmarshalJSON converts a JSON to the maintenance window definition.
|
||||
func (m *MaintenanceWindow) UnmarshalJSON(data []byte) error {
|
||||
var (
|
||||
got MaintenanceWindow
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ package spec
|
|||
|
||||
import (
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
|
|
@ -162,10 +163,12 @@ type ControllerConfig struct {
|
|||
RestConfig *rest.Config `json:"-"`
|
||||
InfrastructureRoles map[string]PgUser
|
||||
|
||||
NoDatabaseAccess bool
|
||||
NoTeamsAPI bool
|
||||
ConfigMapName NamespacedName
|
||||
Namespace string
|
||||
NoDatabaseAccess bool
|
||||
NoTeamsAPI bool
|
||||
CRDReadyWaitInterval time.Duration
|
||||
CRDReadyWaitTimeout time.Duration
|
||||
ConfigMapName NamespacedName
|
||||
Namespace string
|
||||
}
|
||||
|
||||
// cached value for the GetOperatorNamespace
|
||||
|
|
@ -185,6 +188,19 @@ func (n *NamespacedName) Decode(value string) error {
|
|||
return n.DecodeWorker(value, GetOperatorNamespace())
|
||||
}
|
||||
|
||||
func (n *NamespacedName) UnmarshalJSON(data []byte) error {
|
||||
result := NamespacedName{}
|
||||
var tmp string
|
||||
if err := json.Unmarshal(data, &tmp); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := result.Decode(tmp); err != nil {
|
||||
return err
|
||||
}
|
||||
*n = result
|
||||
return nil
|
||||
}
|
||||
|
||||
// DecodeWorker separates the decode logic to (unit) test
|
||||
// from obtaining the operator namespace that depends on k8s mounting files at runtime
|
||||
func (n *NamespacedName) DecodeWorker(value, operatorNamespace string) error {
|
||||
|
|
@ -235,3 +251,31 @@ func GetOperatorNamespace() string {
|
|||
}
|
||||
return operatorNamespace
|
||||
}
|
||||
|
||||
type Duration time.Duration
|
||||
|
||||
func (d *Duration) UnmarshalJSON(b []byte) error {
|
||||
var (
|
||||
v interface{}
|
||||
err error
|
||||
)
|
||||
if err = json.Unmarshal(b, &v); err != nil {
|
||||
return err
|
||||
}
|
||||
switch val := v.(type) {
|
||||
case string:
|
||||
t, err := time.ParseDuration(val)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*d = Duration(t)
|
||||
return nil
|
||||
case float64:
|
||||
t := time.Duration(val)
|
||||
*d = Duration(t)
|
||||
return nil
|
||||
default:
|
||||
return fmt.Errorf("could not recognize type %T as a valid type to unmarshal to Duration", val)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,160 @@
|
|||
package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
|
||||
"github.com/zalando-incubator/postgres-operator/pkg/spec"
|
||||
)
|
||||
|
||||
type OperatorConfiguration struct {
|
||||
metav1.TypeMeta `json:",inline"`
|
||||
metav1.ObjectMeta `json:"metadata"`
|
||||
|
||||
Configuration OperatorConfigurationData `json:"configuration"`
|
||||
Error error `json:"-"`
|
||||
}
|
||||
|
||||
type OperatorConfigurationList struct {
|
||||
metav1.TypeMeta `json:",inline"`
|
||||
metav1.ListMeta `json:"metadata"`
|
||||
|
||||
Items []OperatorConfiguration `json:"items"`
|
||||
}
|
||||
|
||||
type PostgresUsersConfiguration struct {
|
||||
SuperUsername string `json:"super_username,omitempty"`
|
||||
ReplicationUsername string `json:"replication_username,omitempty"`
|
||||
}
|
||||
|
||||
type KubernetesMetaConfiguration struct {
|
||||
PodServiceAccountName string `json:"pod_service_account_name,omitempty"`
|
||||
// TODO: change it to the proper json
|
||||
PodServiceAccountDefinition string `json:"pod_service_account_definition,omitempty"`
|
||||
PodTerminateGracePeriod spec.Duration `json:"pod_terminate_grace_period,omitempty"`
|
||||
WatchedNamespace string `json:"watched_namespace,omitempty"`
|
||||
PDBNameFormat stringTemplate `json:"pdb_name_format,omitempty"`
|
||||
SecretNameTemplate stringTemplate `json:"secret_name_template,omitempty"`
|
||||
OAuthTokenSecretName spec.NamespacedName `json:"oauth_token_secret_name,omitempty"`
|
||||
InfrastructureRolesSecretName spec.NamespacedName `json:"infrastructure_roles_secret_name,omitempty"`
|
||||
PodRoleLabel string `json:"pod_role_label,omitempty"`
|
||||
ClusterLabels map[string]string `json:"cluster_labels,omitempty"`
|
||||
ClusterNameLabel string `json:"cluster_name_label,omitempty"`
|
||||
NodeReadinessLabel map[string]string `json:"node_readiness_label,omitempty"`
|
||||
// TODO: use a proper toleration structure?
|
||||
PodToleration map[string]string `json:"toleration,omitempty"`
|
||||
// TODO: use namespacedname
|
||||
PodEnvironmentConfigMap string `json:"pod_environment_configmap,omitempty"`
|
||||
}
|
||||
|
||||
type PostgresPodResourcesDefaults struct {
|
||||
DefaultCPURequest string `json:"default_cpu_request,omitempty"`
|
||||
DefaultMemoryRequest string `json:"default_memory_request,omitempty"`
|
||||
DefaultCPULimit string `json:"default_cpu_limit,omitempty"`
|
||||
DefaultMemoryLimit string `json:"default_memory_limit,omitempty"`
|
||||
}
|
||||
|
||||
type OperatorTimeouts struct {
|
||||
ResourceCheckInterval spec.Duration `json:"resource_check_interval,omitempty"`
|
||||
ResourceCheckTimeout spec.Duration `json:"resource_check_timeout,omitempty"`
|
||||
PodLabelWaitTimeout spec.Duration `json:"pod_label_wait_timeout,omitempty"`
|
||||
PodDeletionWaitTimeout spec.Duration `json:"pod_deletion_wait_timeout,omitempty"`
|
||||
ReadyWaitInterval spec.Duration `json:"ready_wait_interval,omitempty"`
|
||||
ReadyWaitTimeout spec.Duration `json:"ready_wait_timeout,omitempty"`
|
||||
}
|
||||
|
||||
type LoadBalancerConfiguration struct {
|
||||
DbHostedZone string `json:"db_hosted_zone,omitempty"`
|
||||
EnableMasterLoadBalancer bool `json:"enable_master_load_balancer,omitempty"`
|
||||
EnableReplicaLoadBalancer bool `json:"enable_replica_load_balancer,omitempty"`
|
||||
MasterDNSNameFormat stringTemplate `json:"master_dns_name_format,omitempty"`
|
||||
ReplicaDNSNameFormat stringTemplate `json:"replica_dns_name_format,omitempty"`
|
||||
}
|
||||
|
||||
type AWSGCPConfiguration struct {
|
||||
WALES3Bucket string `json:"wal_s3_bucket,omitempty"`
|
||||
AWSRegion string `json:"aws_region,omitempty"`
|
||||
LogS3Bucket string `json:"log_s3_bucket,omitempty"`
|
||||
KubeIAMRole string `json:"kube_iam_role,omitempty"`
|
||||
}
|
||||
|
||||
type OperatorDebugConfiguration struct {
|
||||
DebugLogging bool `json:"debug_logging,omitempty"`
|
||||
EnableDBAccess bool `json:"enable_database_access,omitempty"`
|
||||
}
|
||||
|
||||
type TeamsAPIConfiguration struct {
|
||||
EnableTeamsAPI bool `json:"enable_teams_api,omitempty"`
|
||||
TeamsAPIUrl string `json:"teams_api_url,omitempty"`
|
||||
TeamAPIRoleConfiguration map[string]string `json:"team_api_role_configuration,omitempty"`
|
||||
EnableTeamSuperuser bool `json:"enable_team_superuser,omitempty"`
|
||||
TeamAdminRole string `json:"team_admin_role,omitempty"`
|
||||
PamRoleName string `json:"pam_role_name,omitempty"`
|
||||
PamConfiguration string `json:"pam_configuration,omitempty"`
|
||||
ProtectedRoles []string `json:"protected_role_names,omitempty"`
|
||||
}
|
||||
|
||||
type LoggingRESTAPIConfiguration struct {
|
||||
APIPort int `json:"api_port,omitempty"`
|
||||
RingLogLines int `json:"ring_log_lines,omitempty"`
|
||||
ClusterHistoryEntries int `json:"cluster_history_entries,omitempty"`
|
||||
}
|
||||
|
||||
type ScalyrConfiguration struct {
|
||||
ScalyrAPIKey string `json:"scalyr_api_key,omitempty"`
|
||||
ScalyrImage string `json:"scalyr_image,omitempty"`
|
||||
ScalyrServerURL string `json:"scalyr_server_url,omitempty"`
|
||||
ScalyrCPURequest string `json:"scalyr_cpu_request,omitempty"`
|
||||
ScalyrMemoryRequest string `json:"scalyr_memory_request,omitempty"`
|
||||
ScalyrCPULimit string `json:"scalyr_cpu_limit,omitempty"`
|
||||
ScalyrMemoryLimit string `json:"scalyr_memory_limit,omitempty"`
|
||||
}
|
||||
|
||||
type OperatorConfigurationData struct {
|
||||
EtcdHost string `json:"etcd_host,omitempty"`
|
||||
DockerImage string `json:"docker_image,omitempty"`
|
||||
Workers uint32 `json:"workers,omitempty"`
|
||||
MinInstances int32 `json:"min_instances,omitempty"`
|
||||
MaxInstances int32 `json:"max_instances,omitempty"`
|
||||
ResyncPeriod spec.Duration `json:"resync_period,omitempty"`
|
||||
Sidecars map[string]string `json:"sidecar_docker_images,omitempty"`
|
||||
PostgresUsersConfiguration PostgresUsersConfiguration `json:"users"`
|
||||
Kubernetes KubernetesMetaConfiguration `json:"kubernetes"`
|
||||
PostgresPodResources PostgresPodResourcesDefaults `json:"postgres_pod_resources"`
|
||||
Timeouts OperatorTimeouts `json:"timeouts"`
|
||||
LoadBalancer LoadBalancerConfiguration `json:"load_balancer"`
|
||||
AWSGCP AWSGCPConfiguration `json:"aws_or_gcp"`
|
||||
OperatorDebug OperatorDebugConfiguration `json:"debug"`
|
||||
TeamsAPI TeamsAPIConfiguration `json:"teams_api"`
|
||||
LoggingRESTAPI LoggingRESTAPIConfiguration `json:"logging_rest_api"`
|
||||
Scalyr ScalyrConfiguration `json:"scalyr"`
|
||||
}
|
||||
|
||||
type OperatorConfigurationUsers struct {
|
||||
SuperUserName string `json:"superuser_name,omitempty"`
|
||||
Replication string `json:"replication_user_name,omitempty"`
|
||||
ProtectedRoles []string `json:"protected_roles,omitempty"`
|
||||
TeamAPIRoleConfiguration map[string]string `json:"team_api_role_configuration,omitempty"`
|
||||
}
|
||||
|
||||
type OperatorConfigurationCopy OperatorConfiguration
|
||||
type OperatorConfigurationListCopy OperatorConfigurationList
|
||||
|
||||
func (opc *OperatorConfiguration) UnmarshalJSON(data []byte) error {
|
||||
var ref OperatorConfigurationCopy
|
||||
if err := json.Unmarshal(data, &ref); err != nil {
|
||||
return err
|
||||
}
|
||||
*opc = OperatorConfiguration(ref)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (opcl *OperatorConfigurationList) UnmarshalJSON(data []byte) error {
|
||||
var ref OperatorConfigurationListCopy
|
||||
if err := json.Unmarshal(data, &ref); err != nil {
|
||||
return nil
|
||||
}
|
||||
*opcl = OperatorConfigurationList(ref)
|
||||
return nil
|
||||
}
|
||||
|
|
@ -2,9 +2,12 @@ package constants
|
|||
|
||||
// Different properties of the PostgreSQL Custom Resource Definition
|
||||
const (
|
||||
CRDKind = "postgresql"
|
||||
CRDResource = "postgresqls"
|
||||
CRDShort = "pg"
|
||||
CRDGroup = "acid.zalan.do"
|
||||
CRDApiVersion = "v1"
|
||||
PostgresCRDKind = "postgresql"
|
||||
PostgresCRDResource = "postgresqls"
|
||||
PostgresCRDShort = "pg"
|
||||
CRDGroup = "acid.zalan.do"
|
||||
CRDApiVersion = "v1"
|
||||
OperatorConfigCRDKind = "postgresql-operator-configuration"
|
||||
OperatorConfigCRDResource = "postgresql-operator-configurations"
|
||||
OperatorConfigCRDShort = "pgopconfig"
|
||||
)
|
||||
|
|
|
|||
Loading…
Reference in New Issue