Add support for pg17 and remove pg12 (#2773)

* Add support for pg17
* use new gcov2lcov-action
* Use ghcr spilo-17
* Update SPILO_CURRENT and SPILO_LAZY
* Update e2e/run.sh

---------

Co-authored-by: Polina Bungina <27892524+hughcapet@users.noreply.github.com>
This commit is contained in:
Ida Novindasari 2024-12-20 11:22:52 +01:00 committed by GitHub
parent 34df486f00
commit 470a1eab89
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
33 changed files with 106 additions and 108 deletions

View File

@ -22,7 +22,7 @@ jobs:
- name: Run unit tests
run: go test -race -covermode atomic -coverprofile=coverage.out ./...
- name: Convert coverage to lcov
uses: jandelgado/gcov2lcov-action@v1.0.9
uses: jandelgado/gcov2lcov-action@v1.1.1
- name: Coveralls
uses: coverallsapp/github-action@master
with:

View File

@ -28,13 +28,13 @@ pipelines with no access to Kubernetes API directly, promoting infrastructure as
### PostgreSQL features
* Supports PostgreSQL 16, starting from 12+
* Supports PostgreSQL 17, starting from 13+
* Streaming replication cluster via Patroni
* Point-In-Time-Recovery with
[pg_basebackup](https://www.postgresql.org/docs/16/app-pgbasebackup.html) /
[pg_basebackup](https://www.postgresql.org/docs/17/app-pgbasebackup.html) /
[WAL-E](https://github.com/wal-e/wal-e) via [Spilo](https://github.com/zalando/spilo)
* Preload libraries: [bg_mon](https://github.com/CyberDem0n/bg_mon),
[pg_stat_statements](https://www.postgresql.org/docs/16/pgstatstatements.html),
[pg_stat_statements](https://www.postgresql.org/docs/17/pgstatstatements.html),
[pgextwlist](https://github.com/dimitri/pgextwlist),
[pg_auth_mon](https://github.com/RafiaSabih/pg_auth_mon)
* Incl. popular Postgres extensions such as

View File

@ -84,11 +84,11 @@ spec:
"limit_iops": 16000,
"limit_throughput": 1000,
"postgresql_versions": [
"17",
"16",
"15",
"14",
"13",
"12"
"13"
]
}
{{- if .Values.extraEnvs }}

View File

@ -68,7 +68,7 @@ spec:
type: string
docker_image:
type: string
default: "ghcr.io/zalando/spilo-16:3.3-p1"
default: "ghcr.io/zalando/spilo-17:4.0-p2"
enable_crd_registration:
type: boolean
default: true
@ -167,10 +167,10 @@ spec:
type: string
minimal_major_version:
type: string
default: "12"
default: "13"
target_major_version:
type: string
default: "16"
default: "17"
kubernetes:
type: object
properties:

View File

@ -375,11 +375,11 @@ spec:
version:
type: string
enum:
- "12"
- "13"
- "14"
- "15"
- "16"
- "17"
parameters:
type: object
additionalProperties:

View File

@ -38,7 +38,7 @@ configGeneral:
# etcd connection string for Patroni. Empty uses K8s-native DCS.
etcd_host: ""
# Spilo docker image
docker_image: ghcr.io/zalando/spilo-16:3.3-p1
docker_image: ghcr.io/zalando/spilo-17:4.0-p2
# key name for annotation to ignore globally configured instance limits
# ignore_instance_limits_annotation_key: ""
@ -89,9 +89,9 @@ configMajorVersionUpgrade:
# - acid
# minimal Postgres major version that will not automatically be upgraded
minimal_major_version: "12"
minimal_major_version: "13"
# target Postgres major version when upgrading clusters automatically
target_major_version: "16"
target_major_version: "17"
configKubernetes:
# list of additional capabilities for postgres container

View File

@ -1297,7 +1297,7 @@ aws_or_gcp:
If cluster members have to be (re)initialized restoring physical backups
happens automatically either from the backup location or by running
[pg_basebackup](https://www.postgresql.org/docs/16/app-pgbasebackup.html)
[pg_basebackup](https://www.postgresql.org/docs/17/app-pgbasebackup.html)
on one of the other running instances (preferably replicas if they do not lag
behind). You can test restoring backups by [cloning](user.md#how-to-clone-an-existing-postgresql-cluster)
clusters.

View File

@ -638,7 +638,7 @@ the global configuration before adding the `tls` section'.
## Change data capture streams
This sections enables change data capture (CDC) streams via Postgres'
[logical decoding](https://www.postgresql.org/docs/16/logicaldecoding.html)
[logical decoding](https://www.postgresql.org/docs/17/logicaldecoding.html)
feature and `pgoutput` plugin. While the Postgres operator takes responsibility
for providing the setup to publish change events, it relies on external tools
to consume them. At Zalando, we are using a workflow based on
@ -671,7 +671,7 @@ can have the following properties:
The CDC operator is following the [outbox pattern](https://debezium.io/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/).
The application is responsible for putting events into a (JSON/B or VARCHAR)
payload column of the outbox table in the structure of the specified target
event type. The operator will create a [PUBLICATION](https://www.postgresql.org/docs/16/logical-replication-publication.html)
event type. The operator will create a [PUBLICATION](https://www.postgresql.org/docs/17/logical-replication-publication.html)
in Postgres for all tables specified for one `database` and `applicationId`.
The CDC operator will consume from it shortly after transactions are
committed to the outbox table. The `idColumn` will be used in telemetry for

View File

@ -94,9 +94,6 @@ Those are top-level keys, containing both leaf keys and groups.
* **enable_pgversion_env_var**
With newer versions of Spilo, it is preferable to use `PGVERSION` pod environment variable instead of the setting `postgresql.bin_dir` in the `SPILO_CONFIGURATION` env variable. When this option is true, the operator sets `PGVERSION` and omits `postgresql.bin_dir` from `SPILO_CONFIGURATION`. When false, the `postgresql.bin_dir` is set. This setting takes precedence over `PGVERSION`; see PR 222 in Spilo. The default is `true`.
* **enable_spilo_wal_path_compat**
enables backwards compatible path between Spilo 12 and Spilo 13+ images. The default is `false`.
* **enable_team_id_clustername_prefix**
To lower the risk of name clashes between clusters of different teams you
can turn on this flag and the operator will sync only clusters where the
@ -250,12 +247,12 @@ CRD-configuration, they are grouped under the `major_version_upgrade` key.
* **minimal_major_version**
The minimal Postgres major version that will not automatically be upgraded
when `major_version_upgrade_mode` is set to `"full"`. The default is `"12"`.
when `major_version_upgrade_mode` is set to `"full"`. The default is `"13"`.
* **target_major_version**
The target Postgres major version when upgrading clusters automatically
which violate the configured allowed `minimal_major_version` when
`major_version_upgrade_mode` is set to `"full"`. The default is `"16"`.
`major_version_upgrade_mode` is set to `"full"`. The default is `"17"`.
## Kubernetes resources

View File

@ -30,7 +30,7 @@ spec:
databases:
foo: zalando
postgresql:
version: "16"
version: "17"
```
Once you cloned the Postgres Operator [repository](https://github.com/zalando/postgres-operator)
@ -109,7 +109,7 @@ metadata:
spec:
[...]
postgresql:
version: "16"
version: "17"
parameters:
password_encryption: scram-sha-256
```
@ -517,7 +517,7 @@ Postgres Operator will create the following NOLOGIN roles:
The `<dbname>_owner` role is the database owner and should be used when creating
new database objects. All members of the `admin` role, e.g. teams API roles, can
become the owner with the `SET ROLE` command. [Default privileges](https://www.postgresql.org/docs/16/sql-alterdefaultprivileges.html)
become the owner with the `SET ROLE` command. [Default privileges](https://www.postgresql.org/docs/17/sql-alterdefaultprivileges.html)
are configured for the owner role so that the `<dbname>_reader` role
automatically gets read-access (SELECT) to new tables and sequences and the
`<dbname>_writer` receives write-access (INSERT, UPDATE, DELETE on tables,
@ -594,7 +594,7 @@ spec:
### Schema `search_path` for default roles
The schema [`search_path`](https://www.postgresql.org/docs/16/ddl-schemas.html#DDL-SCHEMAS-PATH)
The schema [`search_path`](https://www.postgresql.org/docs/17/ddl-schemas.html#DDL-SCHEMAS-PATH)
for each role will include the role name and the schemas, this role should have
access to. So `foo_bar_writer` does not have to schema-qualify tables from
schemas `foo_bar_writer, bar`, while `foo_writer` can look up `foo_writer` and
@ -695,7 +695,7 @@ handle it.
### HugePages support
The operator supports [HugePages](https://www.postgresql.org/docs/16/kernel-resources.html#LINUX-HUGEPAGES).
The operator supports [HugePages](https://www.postgresql.org/docs/17/kernel-resources.html#LINUX-HUGEPAGES).
To enable HugePages, set the matching resource requests and/or limits in the manifest:
```yaml
@ -838,7 +838,7 @@ spec:
### Clone directly
Another way to get a fresh copy of your source DB cluster is via
[pg_basebackup](https://www.postgresql.org/docs/16/app-pgbasebackup.html). To
[pg_basebackup](https://www.postgresql.org/docs/17/app-pgbasebackup.html). To
use this feature simply leave out the timestamp field from the clone section.
The operator will connect to the service of the source cluster by name. If the
cluster is called test, then the connection string will look like host=test

View File

@ -46,7 +46,7 @@ tools:
# install pinned version of 'kind'
# go install must run outside of a dir with a (module-based) Go project !
# otherwise go install updates project's dependencies and/or behaves differently
cd "/tmp" && GO111MODULE=on go install sigs.k8s.io/kind@v0.23.0
cd "/tmp" && GO111MODULE=on go install sigs.k8s.io/kind@v0.24.0
e2etest: tools copy clean
./run.sh main

View File

@ -8,7 +8,7 @@ IFS=$'\n\t'
readonly cluster_name="postgres-operator-e2e-tests"
readonly kubeconfig_path="/tmp/kind-config-${cluster_name}"
readonly spilo_image="registry.opensource.zalan.do/acid/spilo-16-e2e:0.1"
readonly spilo_image="registry.opensource.zalan.do/acid/spilo-17-e2e:0.3"
readonly e2e_test_runner_image="registry.opensource.zalan.do/acid/postgres-operator-e2e-tests-runner:0.4"
export GOPATH=${GOPATH-~/go}

View File

@ -12,10 +12,9 @@ from kubernetes import client
from tests.k8s_api import K8s
from kubernetes.client.rest import ApiException
SPILO_CURRENT = "registry.opensource.zalan.do/acid/spilo-16-e2e:0.1"
SPILO_LAZY = "registry.opensource.zalan.do/acid/spilo-16-e2e:0.2"
SPILO_FULL_IMAGE = "ghcr.io/zalando/spilo-16:3.2-p3"
SPILO_CURRENT = "registry.opensource.zalan.do/acid/spilo-17-e2e:0.3"
SPILO_LAZY = "registry.opensource.zalan.do/acid/spilo-17-e2e:0.4"
SPILO_FULL_IMAGE = "ghcr.io/zalando/spilo-17:4.0-p2"
def to_selector(labels):
return ",".join(["=".join(lbl) for lbl in labels.items()])
@ -1201,35 +1200,35 @@ class EndToEndTestCase(unittest.TestCase):
k8s = self.k8s
cluster_label = 'application=spilo,cluster-name=acid-upgrade-test'
with open("manifests/minimal-postgres-manifest-12.yaml", 'r+') as f:
with open("manifests/minimal-postgres-lowest-version-manifest.yaml", 'r+') as f:
upgrade_manifest = yaml.safe_load(f)
upgrade_manifest["spec"]["dockerImage"] = SPILO_FULL_IMAGE
with open("manifests/minimal-postgres-manifest-12.yaml", 'w') as f:
with open("manifests/minimal-postgres-lowest-version-manifest.yaml", 'w') as f:
yaml.dump(upgrade_manifest, f, Dumper=yaml.Dumper)
k8s.create_with_kubectl("manifests/minimal-postgres-manifest-12.yaml")
k8s.create_with_kubectl("manifests/minimal-postgres-lowest-version-manifest.yaml")
self.eventuallyEqual(lambda: k8s.count_running_pods(labels=cluster_label), 2, "No 2 pods running")
self.eventuallyEqual(lambda: k8s.get_operator_state(), {"0": "idle"}, "Operator does not get in sync")
self.eventuallyEqual(check_version, 12, "Version is not correct")
self.eventuallyEqual(check_version, 13, "Version is not correct")
master_nodes, _ = k8s.get_cluster_nodes(cluster_labels=cluster_label)
# should upgrade immediately
pg_patch_version_13 = {
pg_patch_version_14 = {
"spec": {
"postgresql": {
"version": "13"
"version": "14"
}
}
}
k8s.api.custom_objects_api.patch_namespaced_custom_object(
"acid.zalan.do", "v1", "default", "postgresqls", "acid-upgrade-test", pg_patch_version_13)
"acid.zalan.do", "v1", "default", "postgresqls", "acid-upgrade-test", pg_patch_version_14)
self.eventuallyEqual(lambda: k8s.get_operator_state(), {"0": "idle"}, "Operator does not get in sync")
k8s.wait_for_pod_failover(master_nodes, 'spilo-role=replica,' + cluster_label)
k8s.wait_for_pod_start('spilo-role=master,' + cluster_label)
k8s.wait_for_pod_start('spilo-role=replica,' + cluster_label)
self.eventuallyEqual(check_version, 13, "Version should be upgraded from 12 to 13")
self.eventuallyEqual(check_version, 14, "Version should be upgraded from 13 to 14")
# check if annotation for last upgrade's success is set
annotations = get_annotations()
@ -1238,10 +1237,10 @@ class EndToEndTestCase(unittest.TestCase):
# should not upgrade because current time is not in maintenanceWindow
current_time = datetime.now()
maintenance_window_future = f"{(current_time+timedelta(minutes=60)).strftime('%H:%M')}-{(current_time+timedelta(minutes=120)).strftime('%H:%M')}"
pg_patch_version_14 = {
pg_patch_version_15 = {
"spec": {
"postgresql": {
"version": "14"
"version": "15"
},
"maintenanceWindows": [
maintenance_window_future
@ -1249,23 +1248,23 @@ class EndToEndTestCase(unittest.TestCase):
}
}
k8s.api.custom_objects_api.patch_namespaced_custom_object(
"acid.zalan.do", "v1", "default", "postgresqls", "acid-upgrade-test", pg_patch_version_14)
"acid.zalan.do", "v1", "default", "postgresqls", "acid-upgrade-test", pg_patch_version_15)
self.eventuallyEqual(lambda: k8s.get_operator_state(), {"0": "idle"}, "Operator does not get in sync")
k8s.wait_for_pod_failover(master_nodes, 'spilo-role=master,' + cluster_label)
k8s.wait_for_pod_start('spilo-role=master,' + cluster_label)
k8s.wait_for_pod_start('spilo-role=replica,' + cluster_label)
self.eventuallyEqual(check_version, 13, "Version should not be upgraded")
self.eventuallyEqual(check_version, 14, "Version should not be upgraded")
second_annotations = get_annotations()
self.assertIsNone(second_annotations.get("last-major-upgrade-failure"), "Annotation for last upgrade's failure should not be set")
# change the version again to trigger operator sync
maintenance_window_current = f"{(current_time-timedelta(minutes=30)).strftime('%H:%M')}-{(current_time+timedelta(minutes=30)).strftime('%H:%M')}"
pg_patch_version_15 = {
pg_patch_version_16 = {
"spec": {
"postgresql": {
"version": "15"
"version": "16"
},
"maintenanceWindows": [
maintenance_window_current
@ -1274,13 +1273,13 @@ class EndToEndTestCase(unittest.TestCase):
}
k8s.api.custom_objects_api.patch_namespaced_custom_object(
"acid.zalan.do", "v1", "default", "postgresqls", "acid-upgrade-test", pg_patch_version_15)
"acid.zalan.do", "v1", "default", "postgresqls", "acid-upgrade-test", pg_patch_version_16)
self.eventuallyEqual(lambda: k8s.get_operator_state(), {"0": "idle"}, "Operator does not get in sync")
k8s.wait_for_pod_failover(master_nodes, 'spilo-role=replica,' + cluster_label)
k8s.wait_for_pod_start('spilo-role=master,' + cluster_label)
k8s.wait_for_pod_start('spilo-role=replica,' + cluster_label)
self.eventuallyEqual(check_version, 15, "Version should be upgraded from 13 to 15")
self.eventuallyEqual(check_version, 16, "Version should be upgraded from 14 to 16")
# check if annotation for last upgrade's success is updated after second upgrade
third_annotations = get_annotations()
@ -1288,7 +1287,7 @@ class EndToEndTestCase(unittest.TestCase):
self.assertNotEqual(annotations.get("last-major-upgrade-success"), third_annotations.get("last-major-upgrade-success"), "Annotation for last upgrade's success is not updated")
# test upgrade with failed upgrade annotation
pg_patch_version_16 = {
pg_patch_version_17 = {
"metadata": {
"annotations": {
"last-major-upgrade-failure": "2024-01-02T15:04:05Z"
@ -1296,18 +1295,18 @@ class EndToEndTestCase(unittest.TestCase):
},
"spec": {
"postgresql": {
"version": "16"
"version": "17"
},
},
}
k8s.api.custom_objects_api.patch_namespaced_custom_object(
"acid.zalan.do", "v1", "default", "postgresqls", "acid-upgrade-test", pg_patch_version_16)
"acid.zalan.do", "v1", "default", "postgresqls", "acid-upgrade-test", pg_patch_version_17)
self.eventuallyEqual(lambda: k8s.get_operator_state(), {"0": "idle"}, "Operator does not get in sync")
k8s.wait_for_pod_failover(master_nodes, 'spilo-role=master,' + cluster_label)
k8s.wait_for_pod_start('spilo-role=master,' + cluster_label)
k8s.wait_for_pod_start('spilo-role=replica,' + cluster_label)
self.eventuallyEqual(check_version, 15, "Version should not be upgraded because annotation for last upgrade's failure is set")
self.eventuallyEqual(check_version, 16, "Version should not be upgraded because annotation for last upgrade's failure is set")
# change the version back to 15 and should remove failure annotation
k8s.api.custom_objects_api.patch_namespaced_custom_object(

View File

@ -25,11 +25,11 @@ RUN apt-get update \
&& curl --silent https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
&& apt-get update \
&& apt-get install --no-install-recommends -y \
postgresql-client-17 \
postgresql-client-16 \
postgresql-client-15 \
postgresql-client-14 \
postgresql-client-13 \
postgresql-client-12 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

View File

@ -10,7 +10,7 @@ metadata:
# "delete-date": "2020-08-31" # can only be deleted on that day if "delete-date "key is configured
# "delete-clustername": "acid-test-cluster" # can only be deleted when name matches if "delete-clustername" key is configured
spec:
dockerImage: ghcr.io/zalando/spilo-16:3.3-p1
dockerImage: ghcr.io/zalando/spilo-17:4.0-p2
teamId: "acid"
numberOfInstances: 2
users: # Application/Robot users
@ -48,7 +48,7 @@ spec:
defaultRoles: true
defaultUsers: false
postgresql:
version: "16"
version: "17"
parameters: # Expert section
shared_buffers: "32MB"
max_connections: "10"

View File

@ -34,7 +34,7 @@ data:
default_memory_request: 100Mi
# delete_annotation_date_key: delete-date
# delete_annotation_name_key: delete-clustername
docker_image: ghcr.io/zalando/spilo-16:3.3-p1
docker_image: ghcr.io/zalando/spilo-17:4.0-p2
# downscaler_annotations: "deployment-time,downscaler/*"
enable_admin_role_for_users: "true"
enable_crd_registration: "true"
@ -112,7 +112,7 @@ data:
min_cpu_limit: 250m
min_instances: "-1"
min_memory_limit: 250Mi
minimal_major_version: "12"
minimal_major_version: "13"
# node_readiness_label: "status:ready"
# node_readiness_label_merge: "OR"
oauth_token_secret_name: postgresql-operator
@ -162,7 +162,7 @@ data:
spilo_privileged: "false"
storage_resize_mode: "pvc"
super_username: postgres
target_major_version: "16"
target_major_version: "17"
team_admin_role: "admin"
team_api_role_configuration: "log_statement:all"
teams_api_url: http://fake-teams-api.default.svc.cluster.local

View File

@ -17,4 +17,4 @@ spec:
preparedDatabases:
bar: {}
postgresql:
version: "12"
version: "13"

View File

@ -17,4 +17,4 @@ spec:
preparedDatabases:
bar: {}
postgresql:
version: "16"
version: "17"

View File

@ -66,7 +66,7 @@ spec:
type: string
docker_image:
type: string
default: "ghcr.io/zalando/spilo-16:3.3-p1"
default: "ghcr.io/zalando/spilo-17:4.0-p2"
enable_crd_registration:
type: boolean
default: true
@ -165,10 +165,10 @@ spec:
type: string
minimal_major_version:
type: string
default: "12"
default: "13"
target_major_version:
type: string
default: "16"
default: "17"
kubernetes:
type: object
properties:

View File

@ -3,7 +3,7 @@ kind: OperatorConfiguration
metadata:
name: postgresql-operator-default-configuration
configuration:
docker_image: ghcr.io/zalando/spilo-16:3.3-p1
docker_image: ghcr.io/zalando/spilo-17:4.0-p2
# enable_crd_registration: true
# crd_categories:
# - all
@ -39,8 +39,8 @@ configuration:
major_version_upgrade_mode: "manual"
# major_version_upgrade_team_allow_list:
# - acid
minimal_major_version: "12"
target_major_version: "16"
minimal_major_version: "13"
target_major_version: "17"
kubernetes:
# additional_pod_capabilities:
# - "SYS_NICE"

View File

@ -373,11 +373,11 @@ spec:
version:
type: string
enum:
- "12"
- "13"
- "14"
- "15"
- "16"
- "17"
parameters:
type: object
additionalProperties:

View File

@ -8,7 +8,7 @@ spec:
size: 1Gi
numberOfInstances: 1
postgresql:
version: "16"
version: "17"
# Make this a standby cluster and provide either the s3 bucket path of source cluster or the remote primary host for continuous streaming.
standby:
# s3_wal_path: "s3://mybucket/spilo/acid-minimal-cluster/abcd1234-2a4b-4b2a-8c9c-c1234defg567/wal/14/"

View File

@ -595,9 +595,6 @@ var PostgresCRDResourceValidation = apiextv1.CustomResourceValidation{
"version": {
Type: "string",
Enum: []apiextv1.JSON{
{
Raw: []byte(`"12"`),
},
{
Raw: []byte(`"13"`),
},
@ -610,6 +607,9 @@ var PostgresCRDResourceValidation = apiextv1.CustomResourceValidation{
{
Raw: []byte(`"16"`),
},
{
Raw: []byte(`"17"`),
},
},
},
"parameters": {
@ -1164,7 +1164,8 @@ var OperatorConfigCRDResourceValidation = apiextv1.CustomResourceValidation{
Type: "boolean",
},
"enable_spilo_wal_path_compat": {
Type: "boolean",
Type: "boolean",
Description: "deprecated",
},
"enable_team_id_clustername_prefix": {
Type: "boolean",

View File

@ -49,8 +49,8 @@ type PostgresUsersConfiguration struct {
type MajorVersionUpgradeConfiguration struct {
MajorVersionUpgradeMode string `json:"major_version_upgrade_mode" default:"manual"` // off - no actions, manual - manifest triggers action, full - manifest and minimal version violation trigger upgrade
MajorVersionUpgradeTeamAllowList []string `json:"major_version_upgrade_team_allow_list,omitempty"`
MinimalMajorVersion string `json:"minimal_major_version" default:"12"`
TargetMajorVersion string `json:"target_major_version" default:"16"`
MinimalMajorVersion string `json:"minimal_major_version" default:"13"`
TargetMajorVersion string `json:"target_major_version" default:"17"`
}
// KubernetesMetaConfiguration defines k8s conf required for all Postgres clusters and the operator itself

View File

@ -219,7 +219,7 @@ var unmarshalCluster = []struct {
"127.0.0.1/32"
],
"postgresql": {
"version": "16",
"version": "17",
"parameters": {
"shared_buffers": "32MB",
"max_connections": "10",
@ -279,7 +279,7 @@ var unmarshalCluster = []struct {
},
Spec: PostgresSpec{
PostgresqlParam: PostgresqlParam{
PgVersion: "16",
PgVersion: "17",
Parameters: map[string]string{
"shared_buffers": "32MB",
"max_connections": "10",
@ -339,7 +339,7 @@ var unmarshalCluster = []struct {
},
Error: "",
},
marshal: []byte(`{"kind":"Postgresql","apiVersion":"acid.zalan.do/v1","metadata":{"name":"acid-testcluster1","creationTimestamp":null},"spec":{"postgresql":{"version":"16","parameters":{"log_statement":"all","max_connections":"10","shared_buffers":"32MB"}},"pod_priority_class_name":"spilo-pod-priority","volume":{"size":"5Gi","storageClass":"SSD", "subPath": "subdir"},"enableShmVolume":false,"patroni":{"initdb":{"data-checksums":"true","encoding":"UTF8","locale":"en_US.UTF-8"},"pg_hba":["hostssl all all 0.0.0.0/0 md5","host all all 0.0.0.0/0 md5"],"ttl":30,"loop_wait":10,"retry_timeout":10,"maximum_lag_on_failover":33554432,"slots":{"permanent_logical_1":{"database":"foo","plugin":"pgoutput","type":"logical"}}},"resources":{"requests":{"cpu":"10m","memory":"50Mi"},"limits":{"cpu":"300m","memory":"3000Mi"}},"teamId":"acid","allowedSourceRanges":["127.0.0.1/32"],"numberOfInstances":2,"users":{"zalando":["superuser","createdb"]},"maintenanceWindows":["Mon:01:00-06:00","Sat:00:00-04:00","05:00-05:15"],"clone":{"cluster":"acid-batman"}},"status":{"PostgresClusterStatus":""}}`),
marshal: []byte(`{"kind":"Postgresql","apiVersion":"acid.zalan.do/v1","metadata":{"name":"acid-testcluster1","creationTimestamp":null},"spec":{"postgresql":{"version":"17","parameters":{"log_statement":"all","max_connections":"10","shared_buffers":"32MB"}},"pod_priority_class_name":"spilo-pod-priority","volume":{"size":"5Gi","storageClass":"SSD", "subPath": "subdir"},"enableShmVolume":false,"patroni":{"initdb":{"data-checksums":"true","encoding":"UTF8","locale":"en_US.UTF-8"},"pg_hba":["hostssl all all 0.0.0.0/0 md5","host all all 0.0.0.0/0 md5"],"ttl":30,"loop_wait":10,"retry_timeout":10,"maximum_lag_on_failover":33554432,"slots":{"permanent_logical_1":{"database":"foo","plugin":"pgoutput","type":"logical"}}},"resources":{"requests":{"cpu":"10m","memory":"50Mi"},"limits":{"cpu":"300m","memory":"3000Mi"}},"teamId":"acid","allowedSourceRanges":["127.0.0.1/32"],"numberOfInstances":2,"users":{"zalando":["superuser","createdb"]},"maintenanceWindows":["Mon:01:00-06:00","Sat:00:00-04:00","05:00-05:15"],"clone":{"cluster":"acid-batman"}},"status":{"PostgresClusterStatus":""}}`),
err: nil},
{
about: "example with clone",
@ -404,7 +404,7 @@ var postgresqlList = []struct {
out PostgresqlList
err error
}{
{"expect success", []byte(`{"apiVersion":"v1","items":[{"apiVersion":"acid.zalan.do/v1","kind":"Postgresql","metadata":{"labels":{"team":"acid"},"name":"acid-testcluster42","namespace":"default","resourceVersion":"30446957","selfLink":"/apis/acid.zalan.do/v1/namespaces/default/postgresqls/acid-testcluster42","uid":"857cd208-33dc-11e7-b20a-0699041e4b03"},"spec":{"allowedSourceRanges":["185.85.220.0/22"],"numberOfInstances":1,"postgresql":{"version":"16"},"teamId":"acid","volume":{"size":"10Gi"}},"status":{"PostgresClusterStatus":"Running"}}],"kind":"List","metadata":{},"resourceVersion":"","selfLink":""}`),
{"expect success", []byte(`{"apiVersion":"v1","items":[{"apiVersion":"acid.zalan.do/v1","kind":"Postgresql","metadata":{"labels":{"team":"acid"},"name":"acid-testcluster42","namespace":"default","resourceVersion":"30446957","selfLink":"/apis/acid.zalan.do/v1/namespaces/default/postgresqls/acid-testcluster42","uid":"857cd208-33dc-11e7-b20a-0699041e4b03"},"spec":{"allowedSourceRanges":["185.85.220.0/22"],"numberOfInstances":1,"postgresql":{"version":"17"},"teamId":"acid","volume":{"size":"10Gi"}},"status":{"PostgresClusterStatus":"Running"}}],"kind":"List","metadata":{},"resourceVersion":"","selfLink":""}`),
PostgresqlList{
TypeMeta: metav1.TypeMeta{
Kind: "List",
@ -425,7 +425,7 @@ var postgresqlList = []struct {
},
Spec: PostgresSpec{
ClusterName: "testcluster42",
PostgresqlParam: PostgresqlParam{PgVersion: "16"},
PostgresqlParam: PostgresqlParam{PgVersion: "17"},
Volume: Volume{Size: "10Gi"},
TeamID: "acid",
AllowedSourceRanges: []string{"185.85.220.0/22"},

View File

@ -72,18 +72,18 @@ func TestGenerateSpiloJSONConfiguration(t *testing.T) {
}{
{
subtest: "Patroni default configuration",
pgParam: &acidv1.PostgresqlParam{PgVersion: "16"},
pgParam: &acidv1.PostgresqlParam{PgVersion: "17"},
patroni: &acidv1.Patroni{},
opConfig: &config.Config{
Auth: config.Auth{
PamRoleName: "zalandos",
},
},
result: `{"postgresql":{"bin_dir":"/usr/lib/postgresql/16/bin"},"bootstrap":{"initdb":[{"auth-host":"md5"},{"auth-local":"trust"}],"dcs":{}}}`,
result: `{"postgresql":{"bin_dir":"/usr/lib/postgresql/17/bin"},"bootstrap":{"initdb":[{"auth-host":"md5"},{"auth-local":"trust"}],"dcs":{}}}`,
},
{
subtest: "Patroni configured",
pgParam: &acidv1.PostgresqlParam{PgVersion: "16"},
pgParam: &acidv1.PostgresqlParam{PgVersion: "17"},
patroni: &acidv1.Patroni{
InitDB: map[string]string{
"encoding": "UTF8",
@ -102,38 +102,38 @@ func TestGenerateSpiloJSONConfiguration(t *testing.T) {
FailsafeMode: util.True(),
},
opConfig: &config.Config{},
result: `{"postgresql":{"bin_dir":"/usr/lib/postgresql/16/bin","pg_hba":["hostssl all all 0.0.0.0/0 md5","host all all 0.0.0.0/0 md5"]},"bootstrap":{"initdb":[{"auth-host":"md5"},{"auth-local":"trust"},"data-checksums",{"encoding":"UTF8"},{"locale":"en_US.UTF-8"}],"dcs":{"ttl":30,"loop_wait":10,"retry_timeout":10,"maximum_lag_on_failover":33554432,"synchronous_mode":true,"synchronous_mode_strict":true,"synchronous_node_count":1,"slots":{"permanent_logical_1":{"database":"foo","plugin":"pgoutput","type":"logical"}},"failsafe_mode":true}}}`,
result: `{"postgresql":{"bin_dir":"/usr/lib/postgresql/17/bin","pg_hba":["hostssl all all 0.0.0.0/0 md5","host all all 0.0.0.0/0 md5"]},"bootstrap":{"initdb":[{"auth-host":"md5"},{"auth-local":"trust"},"data-checksums",{"encoding":"UTF8"},{"locale":"en_US.UTF-8"}],"dcs":{"ttl":30,"loop_wait":10,"retry_timeout":10,"maximum_lag_on_failover":33554432,"synchronous_mode":true,"synchronous_mode_strict":true,"synchronous_node_count":1,"slots":{"permanent_logical_1":{"database":"foo","plugin":"pgoutput","type":"logical"}},"failsafe_mode":true}}}`,
},
{
subtest: "Patroni failsafe_mode configured globally",
pgParam: &acidv1.PostgresqlParam{PgVersion: "16"},
pgParam: &acidv1.PostgresqlParam{PgVersion: "17"},
patroni: &acidv1.Patroni{},
opConfig: &config.Config{
EnablePatroniFailsafeMode: util.True(),
},
result: `{"postgresql":{"bin_dir":"/usr/lib/postgresql/16/bin"},"bootstrap":{"initdb":[{"auth-host":"md5"},{"auth-local":"trust"}],"dcs":{"failsafe_mode":true}}}`,
result: `{"postgresql":{"bin_dir":"/usr/lib/postgresql/17/bin"},"bootstrap":{"initdb":[{"auth-host":"md5"},{"auth-local":"trust"}],"dcs":{"failsafe_mode":true}}}`,
},
{
subtest: "Patroni failsafe_mode configured globally, disabled for cluster",
pgParam: &acidv1.PostgresqlParam{PgVersion: "16"},
pgParam: &acidv1.PostgresqlParam{PgVersion: "17"},
patroni: &acidv1.Patroni{
FailsafeMode: util.False(),
},
opConfig: &config.Config{
EnablePatroniFailsafeMode: util.True(),
},
result: `{"postgresql":{"bin_dir":"/usr/lib/postgresql/16/bin"},"bootstrap":{"initdb":[{"auth-host":"md5"},{"auth-local":"trust"}],"dcs":{"failsafe_mode":false}}}`,
result: `{"postgresql":{"bin_dir":"/usr/lib/postgresql/17/bin"},"bootstrap":{"initdb":[{"auth-host":"md5"},{"auth-local":"trust"}],"dcs":{"failsafe_mode":false}}}`,
},
{
subtest: "Patroni failsafe_mode disabled globally, configured for cluster",
pgParam: &acidv1.PostgresqlParam{PgVersion: "16"},
pgParam: &acidv1.PostgresqlParam{PgVersion: "17"},
patroni: &acidv1.Patroni{
FailsafeMode: util.True(),
},
opConfig: &config.Config{
EnablePatroniFailsafeMode: util.False(),
},
result: `{"postgresql":{"bin_dir":"/usr/lib/postgresql/16/bin"},"bootstrap":{"initdb":[{"auth-host":"md5"},{"auth-local":"trust"}],"dcs":{"failsafe_mode":true}}}`,
result: `{"postgresql":{"bin_dir":"/usr/lib/postgresql/17/bin"},"bootstrap":{"initdb":[{"auth-host":"md5"},{"auth-local":"trust"}],"dcs":{"failsafe_mode":true}}}`,
},
}
for _, tt := range tests {
@ -164,15 +164,15 @@ func TestExtractPgVersionFromBinPath(t *testing.T) {
},
{
subTest: "test current bin path against hard coded template",
binPath: "/usr/lib/postgresql/16/bin",
binPath: "/usr/lib/postgresql/17/bin",
template: pgBinariesLocationTemplate,
expected: "16",
expected: "17",
},
{
subTest: "test alternative bin path against a matching template",
binPath: "/usr/pgsql-16/bin",
binPath: "/usr/pgsql-17/bin",
template: "/usr/pgsql-%v/bin",
expected: "16",
expected: "17",
},
}
@ -2148,7 +2148,7 @@ func TestSidecars(t *testing.T) {
spec = acidv1.PostgresSpec{
PostgresqlParam: acidv1.PostgresqlParam{
PgVersion: "16",
PgVersion: "17",
Parameters: map[string]string{
"max_connections": "100",
},

View File

@ -21,6 +21,7 @@ var VersionMap = map[string]int{
"14": 140000,
"15": 150000,
"16": 160000,
"17": 170000,
}
const (
@ -44,7 +45,7 @@ func (c *Cluster) GetDesiredMajorVersionAsInt() int {
func (c *Cluster) GetDesiredMajorVersion() string {
if c.Config.OpConfig.MajorVersionUpgradeMode == "full" {
// e.g. current is 12, minimal is 12 allowing 12 to 16 clusters, everything below is upgraded
// e.g. current is 13, minimal is 13 allowing 13 to 17 clusters, everything below is upgraded
if IsBiggerPostgresVersion(c.Spec.PgVersion, c.Config.OpConfig.MinimalMajorVersion) {
c.logger.Infof("overwriting configured major version %s to %s", c.Spec.PgVersion, c.Config.OpConfig.TargetMajorVersion)
return c.Config.OpConfig.TargetMajorVersion

View File

@ -39,7 +39,7 @@ func (c *Controller) importConfigurationFromCRD(fromCRD *acidv1.OperatorConfigur
result.EnableTeamIdClusternamePrefix = fromCRD.EnableTeamIdClusternamePrefix
result.EtcdHost = fromCRD.EtcdHost
result.KubernetesUseConfigMaps = fromCRD.KubernetesUseConfigMaps
result.DockerImage = util.Coalesce(fromCRD.DockerImage, "ghcr.io/zalando/spilo-16:3.3-p1")
result.DockerImage = util.Coalesce(fromCRD.DockerImage, "ghcr.io/zalando/spilo-17:4.0-p2")
result.Workers = util.CoalesceUInt32(fromCRD.Workers, 8)
result.MinInstances = fromCRD.MinInstances
result.MaxInstances = fromCRD.MaxInstances
@ -62,8 +62,8 @@ func (c *Controller) importConfigurationFromCRD(fromCRD *acidv1.OperatorConfigur
// major version upgrade config
result.MajorVersionUpgradeMode = util.Coalesce(fromCRD.MajorVersionUpgrade.MajorVersionUpgradeMode, "manual")
result.MajorVersionUpgradeTeamAllowList = fromCRD.MajorVersionUpgrade.MajorVersionUpgradeTeamAllowList
result.MinimalMajorVersion = util.Coalesce(fromCRD.MajorVersionUpgrade.MinimalMajorVersion, "12")
result.TargetMajorVersion = util.Coalesce(fromCRD.MajorVersionUpgrade.TargetMajorVersion, "16")
result.MinimalMajorVersion = util.Coalesce(fromCRD.MajorVersionUpgrade.MinimalMajorVersion, "13")
result.TargetMajorVersion = util.Coalesce(fromCRD.MajorVersionUpgrade.TargetMajorVersion, "17")
// kubernetes config
result.EnableOwnerReferences = util.CoalesceBool(fromCRD.Kubernetes.EnableOwnerReferences, util.False())

View File

@ -175,7 +175,7 @@ type Config struct {
WatchedNamespace string `name:"watched_namespace"` // special values: "*" means 'watch all namespaces', the empty string "" means 'watch a namespace where operator is deployed to'
KubernetesUseConfigMaps bool `name:"kubernetes_use_configmaps" default:"false"`
EtcdHost string `name:"etcd_host" default:""` // special values: the empty string "" means Patroni will use K8s as a DCS
DockerImage string `name:"docker_image" default:"ghcr.io/zalando/spilo-16:3.3-p1"`
DockerImage string `name:"docker_image" default:"ghcr.io/zalando/spilo-17:4.0-p2"`
SidecarImages map[string]string `name:"sidecar_docker_images"` // deprecated in favour of SidecarContainers
SidecarContainers []v1.Container `name:"sidecars"`
PodServiceAccountName string `name:"pod_service_account_name" default:"postgres-pod"`
@ -246,8 +246,8 @@ type Config struct {
EnableTeamIdClusternamePrefix bool `name:"enable_team_id_clustername_prefix" default:"false"`
MajorVersionUpgradeMode string `name:"major_version_upgrade_mode" default:"manual"`
MajorVersionUpgradeTeamAllowList []string `name:"major_version_upgrade_team_allow_list" default:""`
MinimalMajorVersion string `name:"minimal_major_version" default:"12"`
TargetMajorVersion string `name:"target_major_version" default:"16"`
MinimalMajorVersion string `name:"minimal_major_version" default:"13"`
TargetMajorVersion string `name:"target_major_version" default:"17"`
PatroniAPICheckInterval time.Duration `name:"patroni_api_check_interval" default:"1s"`
PatroniAPICheckTimeout time.Duration `name:"patroni_api_check_timeout" default:"5s"`
EnablePatroniFailsafeMode *bool `name:"enable_patroni_failsafe_mode" default:"false"`

View File

@ -73,11 +73,11 @@ spec:
"limit_iops": 16000,
"limit_throughput": 1000,
"postgresql_versions": [
"17",
"16",
"15",
"14",
"13",
"12"
"13"
]
}
# Exemple of settings to make snapshot view working in the ui when using AWS

View File

@ -267,7 +267,7 @@ DEFAULT_UI_CONFIG = {
'users_visible': True,
'databases_visible': True,
'resources_visible': RESOURCES_VISIBLE,
'postgresql_versions': ['12', '13', '14', '15', '16'],
'postgresql_versions': ['13', '14', '15', '16', '17'],
'dns_format_string': '{0}.{1}',
'pgui_link': '',
'static_network_whitelist': {},

View File

@ -305,7 +305,7 @@ def read_versions(
if uid == 'wal' or defaulting(lambda: UUID(uid))
]
BACKUP_VERSION_PREFIXES = ['', '10/', '11/', '12/', '13/', '14/', '15/', '16/']
BACKUP_VERSION_PREFIXES = ['', '10/', '11/', '12/', '13/', '14/', '15/', '16/', '17/']
def read_basebackups(
pg_cluster,

View File

@ -31,11 +31,11 @@ default_operator_ui_config='{
"limit_iops": 16000,
"limit_throughput": 1000,
"postgresql_versions": [
"17",
"16",
"15",
"14",
"13",
"12"
"13"
],
"static_network_whitelist": {
"localhost": ["172.0.0.1/32"]