A 'kubectl delete crd' is destructive: the API server cascades the
delete to every custom resource of that kind in the cluster, so a
single mistyped command can wipe out all postgresql, operator
configuration, postgresteam and fabriceventstream objects at once.
This is especially dangerous for an operator whose whole value
proposition is the data behind those CRs.
The fix is a 'acid.zalan.do/crd-protection' finalizer on each
operator CRD. The apiserver blocks the deletion while the finalizer
is set, so the destructive action is gated behind a manual step
(patch the finalizer away) that anyone running the delete has to
take deliberately.
Removal procedure is documented in docs/administrator.md under
'Protecting CRDs from accidental deletion'.
Assisted-by: OpenCode + MiniMax M3
* fix data to POSIX and sed working on macos
* add ServiceAccountGetter to the newFakeK8sAnnotationsClient for unit tests
* try to update the service account
* use irsa_role_arn since we need the full arn, and remove enable_irsa
* move sa sync code to existing sync.go file to be all together
* change all Irsa to IRSA to follow go idiomatic that capitalize initialisms or acronyms
* using Update instead of Patch for the service account syn
* document the new option and add the key in the values/configs
* add the new option to the administrator docs
* trying to increase the timeout for the flaky test after sync
---------
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* Add cluster_labels and annotations to logical backup CronJob and Jobs
When using the logical backup feature, the CronJob and its created Jobs
were missing the cluster_labels and annotations that are applied to
other cluster resources. This made it difficult to filter or identify
backup jobs using the same labels as other cluster components.
Changes:
- Added ObjectMeta with labels and annotations to JobTemplateSpec
- Updated CronJob ObjectMeta to use the merged labels (including
'application: spilo-logical-backup')
- Updated tests to expect the new labels
* Add topologySpreadConstraints configuration to pod spec.
* Run update-codegen.sh to add deepcopy for new field to the api.
* Reuse configured TopologySpreadConstraints for logical backup.
* Remove x-kubernetes-preserve-unknown-fields and XPreserveUnknownFields.
* Add topologySpreadConstraint example in the complete manifest.
* Add support for helm chart.
* Add documentation for topologySpreadConstraint.
* Update e2e test to patch topologySpreadConstraints into the postgresqls manifest.
* For e2e test, updated the PVC retention policy to remove redundant PVCs.
* Fix e2e test, expected PVC count in end-to-end test after config changes.
* build multi-arch pooler image
* add pooler build step in delivery.yaml and bump pooler version
* pull from docker hub not zalando registry
* add pooler step to ghcr workflow
* pass infra roles to auth file via pooler entrypoint
* introduce extra pooler secret for mounting auth_file
* use pbgouncer as image name and push to ghcr on next merge
* build with latest pgbouncer
* integrate new image in e2e process and update pooler image default
* update pooler build dependencies
* build pooler image for e2e test
* more Makefile and e2e run script tweaking
---------
Co-authored-by: Ida Novindasari <idanovinda@gmail.com>
* Add support for pg18 and remove pg13
* Update general spilo image and use new rebuilt e2e spilo image
---------
Co-authored-by: Polina Bungina <polina.bungina@zalando.de>
* skip db user actions when its secret failed to sync on update
* need to add new pgUser field to e2e test
* lets collect errors of syncSecret so we still get status updateFailed
Create the second PDB to cover Pods with a special "critical operation" label set.
This label is going to be assigned to all pg cluster's Pods by the Operator during a PG major version upgrade, by Patroni during a cluster/replica bootstrap. It can also be set manually or by any other automation tool.
* Add support for pg17
* use new gcov2lcov-action
* Use ghcr spilo-17
* Update SPILO_CURRENT and SPILO_LAZY
* Update e2e/run.sh
---------
Co-authored-by: Polina Bungina <27892524+hughcapet@users.noreply.github.com>
* sync all resources to cluster fields (CronJob, Streams, Patroni resources)
* separated sync and delete logic for Patroni resources
* align delete streams and secrets logic with other resources
* rename gatherApplicationIds to getDistinctApplicationIds
* improve slot check before syncing streams CRD
* add ownerReferences and annotations diff to Patroni objects
* add extra sync code for config service so it does not get too ugly
* some bugfixes when comparing annotations and return err on found
* sync Patroni resources on update event and extended unit tests
* add config service/endpoint owner references check to e2e tes
* feat(498): Add ownerReferences to managed entities
* empty owner reference for cross namespace secret and more tests
* update ownerReferences of existing resources
* removing ownerReference requires Update API call
* CR ownerReference on PVC blocks pvc retention policy of statefulset
* make ownerreferences optional and disabled by default
* update unit test to check len ownerReferences
* update codegen
* add owner references e2e test
* update unit test
* add block_owner_deletion field to test owner reference
* fix typos and update docs once more
* reflect code feedback
---------
Co-authored-by: Max Begenau <max@begenau.com>
* implement maintenance window for major version upgrade
* e2e test: fix major version upgrade test and extend with the time window
* unit test: add iteration to test isInMaintenanceWindow
* UI: show the window and enable edit via UI
* refactor syncing publication section
* update createOrUpdateStream function to allow resource deletion when removed from manifest
* add minimal FES CRD to enable FES resources creation for E2E test
* fix bug of removing manifest slots in syncStream
* e2e test: fixing typo with major upgrade test
* e2e test: should create and delete FES resource
* e2e test: should not delete manual created resources
* e2e test: enable cluster role for FES with patching instead of deploying in manifest
* Annotate PVC on Sync/Update, not only change PVC template
* Don't rotate pods when only annotations changed
* Annotate Logical Backup's and Pooler's pods
* Annotate PDB, Endpoints created by the Operator, Secrets, Logical Backup jobs
Inherited annotations are only added/updated, not removed
* Secrets deletion config
* Update e2e/tests/test_e2e.py
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
---------
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* add the pg version 16
* add comma after pg16 in crds api
* change minimal_major_version to 12
* add new spilo image for pg16
* edit the registry from current and lazy spilo
* Update e2e/run.sh
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* Update README.md
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* add pg 11 to be compatible for the existing DBs
* update pq, pyyaml,k8s and kind version
* skip test_infrastructure_roles
* skip another test
* remove the skipping
* adjust the verification of new Patroni version states
---------
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* bump pooler image
* set pooler pod security context
* use hard coded RunAsUser 100 and RunAsGroup 101 for pooler pod
* unify generation of TLS secret mounts
* extend documentation on tls support
* add unit test for testing TLS support for pooler
* add e2e test for tls support
* Bumped Spilo image tag to the one that supports PostgreSQL 15. Using CDP version temporarily until non-CDP one is released.
* Added support for PostgreSQL 15 and made it default. 9.5 and 9.6 are now no longer supported
* Bumped spilo image tag to 2.1-p9
* Bumped spilo image in test launcher
Co-authored-by: yoshihiko <ariyoshi10@gmail.com>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
* Allow drop slots when it gets deleted from the manifest
* use leader instead replica to query slots
* fix and extend unit tests for config update checks
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
This commit adds support of a not-yet-released Patroni feature that allows postgres to run as primary in case of a failed leader lock update.
* Add Patroni 'failsafe_mode' local parameter (enable for a single PG cluster)
* Allow configuring Patroni 'failsafe_mode' parameter globally
* allow in place pw rotation of system users
* block postgres user from rotation
* mark pooler pods for replacement
* adding podsGetter where pooler is synced in unit tests
* move rotation code in extra function
* reverse membership for additional owner roles
* remove type RoleOriginSpilo
* use e2e images with cron_admin inside
* let operator resolve reversed membership
* make additional owner roles part of the sync user strategy
* add more context in the docs about additional_owner_roles
* add the possibility to create a standby cluster that streams from a remote primary
* extending unit tests
* add more docs and e2e test
Co-authored-by: machine424 <ayoubmrini424@gmail.com>
* feat: add ignored annotations when comparing during sync
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
Co-authored-by: Moshe Immerman <moshe@flanksource.com>
* Add support for pooler load balancer
Signed-off-by: Sergey Shatunov <me@prok.pw>
* Rename to enable_master_pooler_load_balancer
Signed-off-by: Sergey Shatunov <me@prok.pw>
* target port should be intval
* enhance pooler e2e test
* add new options to crds.go
Co-authored-by: Sergey Shatunov <me@prok.pw>
Jairo Llopis
Felix Kunde
Tiago Condeixa
Raphael Torquato
laiminhtrung1997
Polina Bungina
Ida Novindasari
Motte
Jociele Padilha
yoshihikoueno
Polina Bungina
Philipp B