bump tp v1.6.0 (#1265)

* bump tp v1.6.0
* update logical-backup image
* Using smaller image for e2e test.
* fix env var name in docs
* add postgresql-client-13 to logical backup image

Co-authored-by: Jan Mussler <janm81@gmail.com>

.github/ISSUE_TEMPLATE/postgres-operator-issue-template.md

@@ -9,7 +9,7 @@ assignees: ''
 
 Please, answer some short questions which should help us to understand your problem / question better?
 
-- **Which image of the operator are you using?** e.g. registry.opensource.zalan.do/acid/postgres-operator:v1.5.0
+- **Which image of the operator are you using?** e.g. registry.opensource.zalan.do/acid/postgres-operator:v1.6.0
 - **Where do you run it - cloud or metal? Kubernetes or OpenShift?** [AWS K8s | GCP ... | Bare Metal K8s]
 - **Are you running Postgres Operator in production?** [yes | no]
 - **Type of issue?** [Bug report, question, feature request, etc.]

README.md

@@ -14,21 +14,21 @@ pipelines with no access to Kubernetes API directly, promoting infrastructure as
 ### Operator features
 
 * Rolling updates on Postgres cluster changes, incl. quick minor version updates
-* Live volume resize without pod restarts (AWS EBS, PvC)
+* Live volume resize without pod restarts (AWS EBS, PVC)
 * Database connection pooler with PGBouncer
 * Restore and cloning Postgres clusters (incl. major version upgrade)
 * Additionally logical backups to S3 bucket can be configured
 * Standby cluster from S3 WAL archive
 * Configurable for non-cloud environments
 * Basic credential and user management on K8s, eases application deployments
+* Support for custom TLS certificates
 * UI to create and edit Postgres cluster manifests
 * Works well on Amazon AWS, Google Cloud, OpenShift and locally on Kind
-* Support for custom TLS certificates
 * Base support for AWS EBS gp3 migration (iops, throughput pending)
 
 ### PostgreSQL features
 
-* Supports PostgreSQL 13, starting from 9.6+
+* Supports PostgreSQL 13, starting from 9.5+
 * Streaming replication cluster via Patroni
 * Point-In-Time-Recovery with
 [pg_basebackup](https://www.postgresql.org/docs/11/app-pgbasebackup.html) /
@@ -59,13 +59,13 @@ If you are new to the operator, you can skip this and just start using the Postg
 The Postgres operator supports Postgres 13 with the new Spilo Image that includes also the recent Patroni version to support PG13 settings.
 More work on optimizing restarts and rolling upgrades is pending.
 
-If you are already using the Postgres operator in older version with a Spilo 12 Docker Image you need to be aware of the changes for the backup path.
+If you are already using the Postgres operator in older version with a Spilo 12 Docker image you need to be aware of the changes for the backup path.
-We introduce the major version into the backup path to smooth the major version upgrade that is now supported manually.
+We introduce the major version into the backup path to smoothen the [major version upgrade](docs/administrator.md#minor-and-major-version-upgrade) that is now supported manually.
 
-The new operator configuration, sets a compatilibty flag *enable_spilo_wal_path_compat* to make Spilo look in current path but also old format paths for wal segments.
+The new operator configuration can set a compatibility flag *enable_spilo_wal_path_compat* to make Spilo look for wal segments in the current path but also old format paths.
-This comes at potential perf. costs, and should be disabled after a few days.
+This comes at potential performance costs and should be disabled after a few days.
 
-The new Spilo 13 image is: `registry.opensource.zalan.do/acid/spilo-13:2.0-p1`
+The new Spilo 13 image is: `registry.opensource.zalan.do/acid/spilo-13:2.0-p2`
 
 The last Spilo 12 image is: `registry.opensource.zalan.do/acid/spilo-12:1.6-p5`
 

charts/postgres-operator-ui/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v1
 name: postgres-operator-ui
-version: 1.5.0
+version: 1.6.0
-appVersion: 1.5.0
+appVersion: 1.6.0
 home: https://github.com/zalando/postgres-operator
 description: Postgres Operator UI provides a graphical interface for a convenient database-as-a-service user experience
 keywords:

charts/postgres-operator-ui/index.yaml

@@ -1,9 +1,32 @@
 apiVersion: v1
 entries:
   postgres-operator-ui:
+  - apiVersion: v1
+    appVersion: 1.6.0
+    created: "2020-12-17T15:49:56.570324588+01:00"
+    description: Postgres Operator UI provides a graphical interface for a convenient
+      database-as-a-service user experience
+    digest: 9ce86d53b4e79dc405aea5fe2feadd163dfbbde43205782c20206ac0ba9d5e4d
+    home: https://github.com/zalando/postgres-operator
+    keywords:
+    - postgres
+    - operator
+    - ui
+    - cloud-native
+    - patroni
+    - spilo
+    maintainers:
+    - email: opensource@zalando.de
+      name: Zalando
+    name: postgres-operator-ui
+    sources:
+    - https://github.com/zalando/postgres-operator
+    urls:
+    - postgres-operator-ui-1.6.0.tgz
+    version: 1.6.0
   - apiVersion: v1
     appVersion: 1.5.0
-    created: "2020-06-04T17:06:37.153522579+02:00"
+    created: "2020-12-17T15:49:56.569780943+01:00"
     description: Postgres Operator UI provides a graphical interface for a convenient
       database-as-a-service user experience
     digest: c91ea39e6d51d57f4048fb1b6ec53b40823f2690eb88e4e4f1a036367b9fdd61
@@ -24,29 +47,4 @@ entries:
     urls:
     - postgres-operator-ui-1.5.0.tgz
     version: 1.5.0
-  - apiVersion: v1
+generated: "2020-12-17T15:49:56.569108956+01:00"
-    appVersion: 1.4.0
-    created: "2020-06-04T17:06:37.15302073+02:00"
-    description: Postgres Operator UI provides a graphical interface for a convenient
-      database-as-a-service user experience
-    digest: 00e0eff7056d56467cd5c975657fbb76c8d01accd25a4b7aca81bc42aeac961d
-    home: https://github.com/zalando/postgres-operator
-    keywords:
-    - postgres
-    - operator
-    - ui
-    - cloud-native
-    - patroni
-    - spilo
-    maintainers:
-    - email: opensource@zalando.de
-      name: Zalando
-    - email: sk@sik-net.de
-      name: siku4
-    name: postgres-operator-ui
-    sources:
-    - https://github.com/zalando/postgres-operator
-    urls:
-    - postgres-operator-ui-1.4.0.tgz
-    version: 1.4.0
-generated: "2020-06-04T17:06:37.152369987+02:00"

charts/postgres-operator-ui/templates/deployment.yaml

@@ -68,10 +68,8 @@ spec:
                   "resources_visible": true,
                   "users_visible": true,
                   "postgresql_versions": [
+                    "13",
                     "12",
-                    "11",
+                    "11"
-                    "10",
-                    "9.6",
-                    "9.5"
                   ]
                 }

charts/postgres-operator-ui/values.yaml

@@ -8,7 +8,7 @@ replicaCount: 1
 image:
   registry: registry.opensource.zalan.do
   repository: acid/postgres-operator-ui
-  tag: v1.5.0-dirty
+  tag: v1.6.0
   pullPolicy: "IfNotPresent"
 
 # Optionally specify an array of imagePullSecrets.

charts/postgres-operator/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v1
 name: postgres-operator
-version: 1.5.0
+version: 1.6.0
-appVersion: 1.5.0
+appVersion: 1.6.0
 home: https://github.com/zalando/postgres-operator
 description: Postgres Operator creates and manages PostgreSQL clusters running in Kubernetes
 keywords:

charts/postgres-operator/index.yaml

@@ -1,9 +1,31 @@
 apiVersion: v1
 entries:
   postgres-operator:
+  - apiVersion: v1
+    appVersion: 1.6.0
+    created: "2020-12-17T16:16:25.639708821+01:00"
+    description: Postgres Operator creates and manages PostgreSQL clusters running
+      in Kubernetes
+    digest: 2f5f527bae0a22b02f2f7b1e2352665cecf489a990e18212444fa34450b97604
+    home: https://github.com/zalando/postgres-operator
+    keywords:
+    - postgres
+    - operator
+    - cloud-native
+    - patroni
+    - spilo
+    maintainers:
+    - email: opensource@zalando.de
+      name: Zalando
+    name: postgres-operator
+    sources:
+    - https://github.com/zalando/postgres-operator
+    urls:
+    - postgres-operator-1.6.0.tgz
+    version: 1.6.0
   - apiVersion: v1
     appVersion: 1.5.0
-    created: "2020-06-04T17:06:49.41741489+02:00"
+    created: "2020-12-17T16:16:25.637262877+01:00"
     description: Postgres Operator creates and manages PostgreSQL clusters running
       in Kubernetes
     digest: 198351d5db52e65cdf383d6f3e1745d91ac1e2a01121f8476f8b1be728b09531
@@ -23,26 +45,4 @@ entries:
     urls:
     - postgres-operator-1.5.0.tgz
     version: 1.5.0
-  - apiVersion: v1
+generated: "2020-12-17T16:16:25.635647131+01:00"
-    appVersion: 1.4.0
-    created: "2020-06-04T17:06:49.416001109+02:00"
-    description: Postgres Operator creates and manages PostgreSQL clusters running
-      in Kubernetes
-    digest: f8b90fecfc3cb825b94ed17edd9d5cefc36ae61801d4568597b4a79bcd73b2e9
-    home: https://github.com/zalando/postgres-operator
-    keywords:
-    - postgres
-    - operator
-    - cloud-native
-    - patroni
-    - spilo
-    maintainers:
-    - email: opensource@zalando.de
-      name: Zalando
-    name: postgres-operator
-    sources:
-    - https://github.com/zalando/postgres-operator
-    urls:
-    - postgres-operator-1.4.0.tgz
-    version: 1.4.0
-generated: "2020-06-04T17:06:49.414521538+02:00"

charts/postgres-operator/values-crd.yaml

@@ -1,7 +1,7 @@
 image:
   registry: registry.opensource.zalan.do
   repository: acid/postgres-operator
-  tag: v1.5.0
+  tag: v1.6.0
   pullPolicy: "IfNotPresent"
 
 # Optionally specify an array of imagePullSecrets.
@@ -32,7 +32,7 @@ configGeneral:
   # Select if setup uses endpoints (default), or configmaps to manage leader (DCS=k8s)
   # kubernetes_use_configmaps: false
   # Spilo docker image
-  docker_image: registry.opensource.zalan.do/acid/spilo-12:1.6-p3
+  docker_image: registry.opensource.zalan.do/acid/spilo-13:2.0-p2
   # max number of instances in Postgres cluster. -1 = no limit
   min_instances: -1
   # min number of instances in Postgres cluster. -1 = no limit
@@ -248,7 +248,7 @@ configAwsOrGcp:
 # configure K8s cron job managed by the operator
 configLogicalBackup:
   # image for pods of the logical backup job (example runs pg_dumpall)
-  logical_backup_docker_image: "registry.opensource.zalan.do/acid/logical-backup:master-58"
+  logical_backup_docker_image: "registry.opensource.zalan.do/acid/logical-backup:v1.6.0"
   # S3 Access Key ID
   logical_backup_s3_access_key_id: ""
   # S3 bucket to store backup results

charts/postgres-operator/values.yaml

@@ -1,7 +1,7 @@
 image:
   registry: registry.opensource.zalan.do
   repository: acid/postgres-operator
-  tag: v1.5.0
+  tag: v1.6.0
   pullPolicy: "IfNotPresent"
 
 # Optionally specify an array of imagePullSecrets.
@@ -35,7 +35,7 @@ configGeneral:
   # Select if setup uses endpoints (default), or configmaps to manage leader (DCS=k8s)
   # kubernetes_use_configmaps: "false"
   # Spilo docker image
-  docker_image: registry.opensource.zalan.do/acid/spilo-12:1.6-p3
+  docker_image: registry.opensource.zalan.do/acid/spilo-13:2.0-p2
   # max number of instances in Postgres cluster. -1 = no limit
   min_instances: "-1"
   # min number of instances in Postgres cluster. -1 = no limit
@@ -239,7 +239,7 @@ configAwsOrGcp:
 # configure K8s cron job managed by the operator
 configLogicalBackup:
   # image for pods of the logical backup job (example runs pg_dumpall)
-  logical_backup_docker_image: "registry.opensource.zalan.do/acid/logical-backup:master-58"
+  logical_backup_docker_image: "registry.opensource.zalan.do/acid/logical-backup:v.1.6.0"
   # S3 Access Key ID
   logical_backup_s3_access_key_id: ""
   # S3 bucket to store backup results

delivery.yaml

@@ -55,7 +55,7 @@ pipeline:
 
     - id: build-operator-ui
       type: script
-      requires_human_approval: true
+
       commands:
         - desc: 'Prepare environment'
           cmd: |
@@ -83,7 +83,7 @@ pipeline:
 
     - id: build-logical-backup
       type: script
-      requires_human_approval: true
+
       commands:
         - desc: Build image
           cmd: |

docker/logical-backup/Dockerfile

@@ -22,6 +22,7 @@ RUN apt-get update     \
     && curl --silent https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
     && apt-get update \
     && apt-get install --no-install-recommends -y  \
+        postgresql-client-13  \
         postgresql-client-12  \
         postgresql-client-11  \
         postgresql-client-10  \

docs/administrator.md

@@ -22,7 +22,7 @@ started.
 
 Starting with Spilo 13, Postgres Operator can do in-place major version upgrade,
 which should be faster than cloning. However, it is not fully automatic yet.
-First, you need to make sure, that setting the PG_VERSION environment variable
+First, you need to make sure, that setting the `PGVERSION` environment variable
 is enabled in the configuration. Since `v1.6.0`, `enable_pgversion_env_var` is
 enabled by default.
 

e2e/run.sh

@@ -8,7 +8,7 @@ IFS=$'\n\t'
 
 readonly cluster_name="postgres-operator-e2e-tests"
 readonly kubeconfig_path="/tmp/kind-config-${cluster_name}"
-readonly spilo_image="registry.opensource.zalan.do/acid/spilo-13:2.0-p1"
+readonly spilo_image="registry.opensource.zalan.do/acid/spilo-13-e2e:0.3"
 readonly e2e_test_runner_image="registry.opensource.zalan.do/acid/postgres-operator-e2e-tests-runner:0.3"
 
 export GOPATH=${GOPATH-~/go}

e2e/tests/test_e2e.py

@@ -11,8 +11,8 @@ from kubernetes import client
 from tests.k8s_api import K8s
 from kubernetes.client.rest import ApiException
 
-SPILO_CURRENT = "registry.opensource.zalan.do/acid/spilo-13:2.0-p1"
+SPILO_CURRENT = "registry.opensource.zalan.do/acid/spilo-13-e2e:0.3"
-SPILO_LAZY = "registry.opensource.zalan.do/acid/spilo-cdp-13:2.0-p145"
+SPILO_LAZY = "registry.opensource.zalan.do/acid/spilo-13-e2e:0.4"
 
 
 def to_selector(labels):
@@ -112,6 +112,7 @@ class EndToEndTestCase(unittest.TestCase):
         with open("manifests/configmap.yaml", 'r+') as f:
             configmap = yaml.safe_load(f)
             configmap["data"]["workers"] = "1"
+            configmap["data"]["docker_image"] = SPILO_CURRENT
 
         with open("manifests/configmap.yaml", 'w') as f:
             yaml.dump(configmap, f, Dumper=yaml.Dumper)
@@ -121,7 +122,6 @@ class EndToEndTestCase(unittest.TestCase):
                          "operatorconfiguration.crd.yaml",
                          "postgresteam.crd.yaml",
                          "configmap.yaml",
-                         "postgresql-operator-default-configuration.yaml",
                          "postgres-operator.yaml",
                          "api-service.yaml",
                          "infrastructure-roles.yaml",

manifests/complete-postgres-manifest.yaml

@@ -9,7 +9,7 @@ metadata:
 #    "delete-date": "2020-08-31"  # can only be deleted on that day if "delete-date "key is configured
 #    "delete-clustername": "acid-test-cluster"  # can only be deleted when name matches if "delete-clustername" key is configured
 spec:
-  dockerImage: registry.opensource.zalan.do/acid/spilo-12:1.6-p3
+  dockerImage: registry.opensource.zalan.do/acid/spilo-13:2.0-p2
   teamId: "acid"
   numberOfInstances: 2
   users:  # Application/Robot users

manifests/configmap.yaml

@@ -31,7 +31,7 @@ data:
   # default_memory_request: 100Mi
   # delete_annotation_date_key: delete-date
   # delete_annotation_name_key: delete-clustername
-  docker_image: registry.opensource.zalan.do/acid/spilo-13:2.0-p1
+  docker_image: registry.opensource.zalan.do/acid/spilo-13:2.0-p2
   # downscaler_annotations: "deployment-time,downscaler/*"
   # enable_admin_role_for_users: "true"
   # enable_crd_validation: "true"
@@ -48,7 +48,6 @@ data:
   # enable_postgres_team_crd_superusers: "false"
   enable_replica_load_balancer: "false"
   # enable_shm_volume: "true"
-  enable_pgversion_env_var: "true"
   # enable_sidecars: "true"
   enable_spilo_wal_path_compat: "true"
   # enable_team_superuser: "false"
@@ -63,7 +62,7 @@ data:
   # inherited_labels: application,environment
   # kube_iam_role: ""
   # log_s3_bucket: ""
-  logical_backup_docker_image: "registry.opensource.zalan.do/acid/logical-backup"
+  logical_backup_docker_image: "registry.opensource.zalan.do/acid/logical-backup:v1.6.0"
   # logical_backup_s3_access_key_id: ""
   logical_backup_s3_bucket: "my-bucket-url"
   # logical_backup_s3_region: ""

manifests/postgres-operator.yaml

@@ -19,7 +19,7 @@ spec:
       serviceAccountName: postgres-operator
       containers:
       - name: postgres-operator
-        image: registry.opensource.zalan.do/acid/postgres-operator:v1.5.0
+        image: registry.opensource.zalan.do/acid/postgres-operator:v1.6.0
         imagePullPolicy: IfNotPresent
         resources:
           requests:

manifests/postgresql-operator-default-configuration.yaml

@@ -3,7 +3,7 @@ kind: OperatorConfiguration
 metadata:
   name: postgresql-operator-default-configuration
 configuration:
-  docker_image: registry.opensource.zalan.do/acid/spilo-12:1.6-p3
+  docker_image: registry.opensource.zalan.do/acid/spilo-13:2.0-p2
   # enable_crd_validation: true
   # enable_lazy_spilo_upgrade: false
   enable_pgversion_env_var: true
@@ -115,7 +115,7 @@ configuration:
     # wal_gs_bucket: ""
     # wal_s3_bucket: ""
   logical_backup:
-    logical_backup_docker_image: "registry.opensource.zalan.do/acid/logical-backup:master-58"
+    logical_backup_docker_image: "registry.opensource.zalan.do/acid/logical-backup:v.1.6.0"
     # logical_backup_s3_access_key_id: ""
     logical_backup_s3_bucket: "my-bucket-url"
     # logical_backup_s3_endpoint: ""

manifests/standby-manifest.yaml

@@ -9,7 +9,7 @@ spec:
     size: 1Gi
   numberOfInstances: 1
   postgresql:
-    version: "12"
+    version: "13"
 # Make this a standby cluster and provide the s3 bucket path of source cluster for continuous streaming.
   standby:
     s3_wal_path: "s3://path/to/bucket/containing/wal/of/source/cluster/"

pkg/controller/operator_config.go

@@ -39,7 +39,7 @@ func (c *Controller) importConfigurationFromCRD(fromCRD *acidv1.OperatorConfigur
 	result.EnableSpiloWalPathCompat = fromCRD.EnableSpiloWalPathCompat
 	result.EtcdHost = fromCRD.EtcdHost
 	result.KubernetesUseConfigMaps = fromCRD.KubernetesUseConfigMaps
-	result.DockerImage = util.Coalesce(fromCRD.DockerImage, "registry.opensource.zalan.do/acid/spilo-12:1.6-p3")
+	result.DockerImage = util.Coalesce(fromCRD.DockerImage, "registry.opensource.zalan.do/acid/spilo-13:2.0-p2")
 	result.Workers = util.CoalesceUInt32(fromCRD.Workers, 8)
 	result.MinInstances = fromCRD.MinInstances
 	result.MaxInstances = fromCRD.MaxInstances

pkg/util/config/config.go

@@ -149,7 +149,7 @@ type Config struct {
 	WatchedNamespace        string            `name:"watched_namespace"` // special values: "*" means 'watch all namespaces', the empty string "" means 'watch a namespace where operator is deployed to'
 	KubernetesUseConfigMaps bool              `name:"kubernetes_use_configmaps" default:"false"`
 	EtcdHost                string            `name:"etcd_host" default:""` // special values: the empty string "" means Patroni will use K8s as a DCS
-	DockerImage             string            `name:"docker_image" default:"registry.opensource.zalan.do/acid/spilo-12:1.6-p3"`
+	DockerImage             string            `name:"docker_image" default:"registry.opensource.zalan.do/acid/spilo-13:2.0-p2"`
 	SidecarImages           map[string]string `name:"sidecar_docker_images"` // deprecated in favour of SidecarContainers
 	SidecarContainers       []v1.Container    `name:"sidecars"`
 	PodServiceAccountName   string            `name:"pod_service_account_name" default:"postgres-pod"`

ui/manifests/deployment.yaml

@@ -20,7 +20,7 @@ spec:
       serviceAccountName: postgres-operator-ui
       containers:
         - name: "service"
-          image: registry.opensource.zalan.do/acid/postgres-operator-ui:v1.5.0-dirty
+          image: registry.opensource.zalan.do/acid/postgres-operator-ui:v1.6.0
           ports:
             - containerPort: 8081
               protocol: "TCP"

ui/run_local.sh

@@ -23,11 +23,9 @@ default_operator_ui_config='{
   "cost_core": 0.0575,
   "cost_memory": 0.014375,
   "postgresql_versions": [
+    "13",
     "12",
-    "11",
+    "11"
-    "10",
-    "9.6",
-    "9.5"
   ],
   "static_network_whitelist": {
     "localhost": ["172.0.0.1/32"]