In Kubernetes, there is tight coupling of API group, CRD and Custom Resources managed by operators. CRDs are cluster resource and unique per cluster.
CRD operatorconfigurations.acid.zalan.do defines the operator configurations for postgres operator.
CRD postgresqls.acid.zalan.do defines minimum and maximum supported postgresql versions.
This leads to issues in a multi-tenant k8s cluster, an example provided below.
In Namespace X, user X deploys web tier application which uses zalando postgres-operator with max version PG 15.
In Namespace Y, user Y deploys web tier application which also uses zalando postgres-operator but needs PG 17 runs into an issue as CRD postgresqls.acid.zalan.do supports only PG 15. This happens because CRD is cluster scope and all namespace scoped applications should adhere to it.
With large k8s clusters, postgresql being a very popular database, zalando postgres operator being defacto way to manage databases, there is a need to keep this configurable so applications can manage the API group of CRDs.
Take API group for postgres operator as an env variable, assuming that the relevant CRDs with API groups are installed.
* Add support for pg17
* use new gcov2lcov-action
* Use ghcr spilo-17
* Update SPILO_CURRENT and SPILO_LAZY
* Update e2e/run.sh
---------
Co-authored-by: Polina Bungina <27892524+hughcapet@users.noreply.github.com>
* Add empty string cases to patterns for pod resources
* Added empty strings test case
* Restored k8sres.go and changed test to zeros
* Updated validation pattern in manifests/operatorconfiguration.crd.yaml and pkg/apis/acid.zalan.do/v1/crds.go
* feat(498): Add ownerReferences to managed entities
* empty owner reference for cross namespace secret and more tests
* update ownerReferences of existing resources
* removing ownerReference requires Update API call
* CR ownerReference on PVC blocks pvc retention policy of statefulset
* make ownerreferences optional and disabled by default
* update unit test to check len ownerReferences
* update codegen
* add owner references e2e test
* update unit test
* add block_owner_deletion field to test owner reference
* fix typos and update docs once more
* reflect code feedback
---------
Co-authored-by: Max Begenau <max@begenau.com>
* Secrets deletion config
* Update e2e/tests/test_e2e.py
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
---------
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* make bucket prefix for logical backup configurable
* include container comparison in logical backup diff
* add unit test and update description for compareContainers
* don't rely on users putting / in the config - reflect other comments from review
* add the pg version 16
* add comma after pg16 in crds api
* change minimal_major_version to 12
* add new spilo image for pg16
* edit the registry from current and lazy spilo
* Update e2e/run.sh
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* Update README.md
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* add pg 11 to be compatible for the existing DBs
* update pq, pyyaml,k8s and kind version
* skip test_infrastructure_roles
* skip another test
* remove the skipping
* adjust the verification of new Patroni version states
---------
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* Add support for PostgreSQL 16
* Include pg16 in local script and logical-backup Dockerfile
---------
Co-authored-by: Tom Molesworth <tom@audioboundary.com>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* Add hugepages-2Mi and 1Gi to ResourceDescription type and crd (#1549, #1788)
* Add tests for hugepages resource requests/limits
* Add tests for hugepages resource requests/limits on sidecars, too
* Add docs for hugepages support
* Add link to kubernetes docs on hugepages
* Add tests for hugepages not being set on container if not requested in custom resource
* Add hugepages resources fields to manifest docs
* Add hugepages resources fields to complete manifest example
* Add hugepages resources fields to chart crd
---------
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* Introduce `masterServiceAnnotations` & `replicaServiceAnnotations`
Introduce `masterServiceAnnotations` & `replicaServiceAnnotations` to the `Postgresql` CRD.
`masterServiceAnnotations` overrides `serviceAnnotations` for master role if not empty.
`replicaServiceAnnotations` overrides `serviceAnnotations` for replica role if not empty.
Existing definition of `serviceAnnotations` continue to work for backward compatibitlity when neither `masterServiceAnnotations` nor `replicaServiceAnnotations` is defined.
This closes https://github.com/zalando/postgres-operator/issues/1927
* Accumulate service annotations
First, global config, then ServiceAnnotations overriding, then MasterServiceAnnotations and ReplicaServiceAnnotations.
This addresses
https://github.com/zalando/postgres-operator/pull/2161#discussion_r1063558711.
* Update admin doc with master & replica service annotations overrides
Addressed https://github.com/zalando/postgres-operator/pull/2161#discussion_r1064744086
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* new config options to specify resources for logical backup jobs
* bug in logical backup script for s3 dumps
* define enum for logical_backup_provider
* changed order of logical backup azure options
* fix unit test for stream comparison
This commit adds support of a not-yet-released Patroni feature that allows postgres to run as primary in case of a failed leader lock update.
* Add Patroni 'failsafe_mode' local parameter (enable for a single PG cluster)
* Allow configuring Patroni 'failsafe_mode' parameter globally
* add toggle to turn off readiness probes
* include PodManagementPolicy and ReadinessProbe in stateful set comparison
* add URI scheme to generated readiness probe
* add the possibility to create a standby cluster that streams from a remote primary
* extending unit tests
* add more docs and e2e test
Co-authored-by: machine424 <ayoubmrini424@gmail.com>
* feat: add ignored annotations when comparing during sync
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
Co-authored-by: Moshe Immerman <moshe@flanksource.com>
* Add support for pooler load balancer
Signed-off-by: Sergey Shatunov <me@prok.pw>
* Rename to enable_master_pooler_load_balancer
Signed-off-by: Sergey Shatunov <me@prok.pw>
* target port should be intval
* enhance pooler e2e test
* add new options to crds.go
Co-authored-by: Sergey Shatunov <me@prok.pw>
* Add optional logical backup retention time
* Set defaults for potentially unbound variables, so that the script will work with older operator versions
* Document retention time parameter for logical backups
* Add retention time parameter to resources and charts
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* provide event stream API
* check manifest settings for logical decoding before creating streams
* operator updates Postgres config and creates replication user
* name FES like the Postgres cluster
* add delete case and fix updating streams + update unit test
* check if fes CRD exists before syncing
* existing slot must use the same plugin
* make id and payload columns configurable
* sync streams only when they are defined in manifest
* introduce applicationId for separate stream CRDs
* add FES to RBAC in chart
* disable streams in chart
* switch to pgoutput plugin and let operator create publications
* reflect code review and additional refactoring
Co-authored-by: Paŭlo Ebermann <paul.ebermann@zalando.de>
* synchronous_node_count support
* notification about Patroni image version
* default synchronous_node_count to 1
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* password rotation in K8s secrets
* add db connection to syncSecrets
* add user retention
* add e2e test
* cleanup on username mismatch if rotation was switched off
* add unit test for syncSecrets + new updateSecret func
Nandan
Ida Novindasari
Motte
Felix Kunde
Samuel Mutel
Jociele Padilha
tm604
Silas
Davide Bizzarri
Stef Graces
Owen Ou
Francois Parquet
Christian Rohmann
Polina Bungina
Dmitry Volodin
preved911
A. Stoewer
Menzorg