Commit Graph

626 Commits

Author SHA1 Message Date
cosimomeli eef49500a5
Add support for EBS CSI Driver (#2677)
* Add support for EBS CSI Driver
2024-12-19 12:32:09 +01:00
Felix Kunde 8cc6796537
fix comparing stream annotations and improve unit test (#2820) 2024-12-18 11:22:08 +01:00
Felix Kunde d44bfabe78
do not use extra labels to list stream CRDs (#2803)
* do not use extra labels to list stream CRDs
* add diff on labels for streams + unit test coverage
2024-12-17 08:54:37 +01:00
Felix Kunde 80ef38f7f0
add resource annotation and ignore recovery type (#2817)
* add resource annotation and ignore recovery type
* Update docs/reference/cluster_manifest.md

---------

Co-authored-by: Ida Novindasari <idanovinda@gmail.com>
2024-12-16 18:17:19 +01:00
Felix Kunde 301462c415
remove streams delete and extend unit tests (#2737) 2024-12-16 18:13:52 +01:00
Ida Novindasari fc9a26040a
Integrate spilo with Patroni 4 (#2818) 2024-12-16 11:11:22 +01:00
fahed dorgaa acdb957d8e
fix switch over candidate retrieving (#2760)
* fix switch over candidate retrieving

Signed-off-by: fahed dorgaa <fahed.dorgaa.ext@corp.ovh.com>

---------

Signed-off-by: fahed dorgaa <fahed.dorgaa.ext@corp.ovh.com>
Co-authored-by: fahed dorgaa <fahed.dorgaa.ext@corp.ovh.com>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-11-01 17:06:20 +01:00
Felix Kunde 8231797efa
add cluster field for PVCs (#2785)
* add cluster field for PVCs
* sync volumes on cluster creation
* fully spell pvc in log messages
2024-10-31 14:08:50 +01:00
Felix Kunde 002d0f94a1
quote schema names in case they use special characters and remove strings.Builder (#2782) 2024-10-17 16:52:24 +02:00
Motte f5e122e8ef
Fix resource constraints (#2735)
* Add empty string cases to patterns for pod resources
* Added empty strings test case
* Restored k8sres.go and changed test to zeros
* Updated validation pattern in manifests/operatorconfiguration.crd.yaml and pkg/apis/acid.zalan.do/v1/crds.go
2024-10-16 17:19:07 +02:00
Polina Bungina 41f5fe1dc9
More major upgrade prechecks (#2775)
Skip when 
- it is a standby clusters
- there is no master in the cluster
2024-10-15 14:05:39 +02:00
Polina Bungina 3ca86678cc
Add major upgrade prechecks (#2772)
Don't fail major upgrade (don't set annotation) if replica(s) are not
(yet) streaming or replication lag is too high
2024-10-11 17:11:46 +02:00
Ida Novindasari c25dc57b96
only skip upgrade if failed before after recheck version (#2755) 2024-09-10 10:32:56 +02:00
Ida Novindasari 2e398120d2
Implement major upgrade result annotations (#2727)
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
Co-authored-by: Polina Bungina <27892524+hughcapet@users.noreply.github.com>
2024-08-28 15:26:12 +02:00
Felix Kunde a09b7655c9
update K8s version and reflect necessary changes (#2740) 2024-08-27 18:13:39 +02:00
Felix Kunde 2ae51fb9ce
reflect linter feedback, remove unused argumnents and redundant type from arrays (#2739)
* reflect linter feedback, remove unused argumnents and redundant literal definitions
* add logical backup to TestCreate unit test
2024-08-27 17:56:07 +02:00
Felix Kunde a08d1679f2
align sync and update logs (#2738) 2024-08-27 09:58:32 +02:00
Rob Nickmans cb06a1ec89
fix: add secret only when not in secret file (#2732)
* fix: add secret only when not in secret file
* fix indentation

---------

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-08-20 17:35:09 +02:00
Felix Kunde 2f7e3ee847
fix stream duplication on operator restart (#2733)
* fix stream duplication on operator restart
* add try except to streams e2e test
2024-08-20 14:38:07 +02:00
Felix Kunde c7ee34ed12
fix sync streams and add diffs for annotations and owner references (#2728)
* extend and improve hasSlotsInSync unit test
* fix sync streams and add diffs for annotations and owner references
* incl. current annotations as desired where we do not fully control them
* added one more unit test and fixed sub test names
* pass maintenance windows to function and update unit test
2024-08-14 12:56:14 +02:00
fahed dorgaa aad03f71ea
fix golangci-lint issues (#2715)
Signed-off-by: fahed dorgaa <fahed.dorgaa@gmail.com>
Co-authored-by: fahed dorgaa <fahed.dorgaa.ext@corp.ovh.com>
Co-authored-by: Matthias Adler <macedigital@users.noreply.github.com>
2024-08-14 12:54:44 +02:00
Felix Kunde 25ccc87317
sync all resources to cluster fields (#2713)
* sync all resources to cluster fields (CronJob, Streams, Patroni resources)
* separated sync and delete logic for Patroni resources
* align delete streams and secrets logic with other resources
* rename gatherApplicationIds to getDistinctApplicationIds
* improve slot check before syncing streams CRD
* add ownerReferences and annotations diff to Patroni objects
* add extra sync code for config service so it does not get too ugly
* some bugfixes when comparing annotations and return err on found
* sync Patroni resources on update event and extended unit tests
* add config service/endpoint owner references check to e2e tes
2024-08-13 10:06:46 +02:00
Felix Kunde 31f92a1aa0
extend inherited annotations unit test to include logical backup cron job (#2723)
* extend inherited annotations test to logical backup cron job
* sync on updated when enabled, not only on schedule changes
2024-08-12 13:12:51 +02:00
Felix Kunde a87307e56b
Feat: enable owner references (#2688)
* feat(498): Add ownerReferences to managed entities
* empty owner reference for cross namespace secret and more tests
* update ownerReferences of existing resources
* removing ownerReference requires Update API call
* CR ownerReference on PVC blocks pvc retention policy of statefulset
* make ownerreferences optional and disabled by default
* update unit test to check len ownerReferences
* update codegen
* add owner references e2e test
* update unit test
* add block_owner_deletion field to test owner reference
* fix typos and update docs once more
* reflect code feedback

---------

Co-authored-by: Max Begenau <max@begenau.com>
2024-08-09 17:58:25 +02:00
Felix Kunde d5a88f571a
let operator fix publications without tables (#2722) 2024-08-09 17:20:05 +02:00
Felix Kunde 85b8058029
bump spilo to 16-3.3, drop support for pg11 (#2706)
* bump spilo to 16-3.3, drop support for pg11
* update README
2024-08-09 14:47:23 +02:00
Ida Novindasari e6ae9e3772
Implement per-cluster maintenance window for Postgres automatic upgrade (#2710)
* implement maintenance window for major version upgrade 
* e2e test: fix major version upgrade test and extend with the time window
* unit test: add iteration to test isInMaintenanceWindow
* UI: show the window and enable edit via UI
2024-08-09 14:07:35 +02:00
Ida Novindasari 94d36327ba
stream: slot and FES should not be created if the publication creation fails (#2704)
* slot should not be created if the publication creation fails
* not create FES resource when slot doesn't exist
2024-08-02 15:09:37 +02:00
Ida Novindasari 31f474a95c
Enable slot and publication deletion when stream application is removed (#2684)
* refactor syncing publication section
* update createOrUpdateStream function to allow resource deletion when removed from manifest
* add minimal FES CRD to enable FES resources creation for E2E test
* fix bug of removing manifest slots in syncStream
* e2e test: fixing typo with major upgrade test
* e2e test: should create and delete FES resource
* e2e test: should not delete manual created resources
* e2e test: enable cluster role for FES with patching instead of deploying in manifest
2024-07-25 12:00:23 +02:00
Felix Kunde e71891e2bd
improve logical backup comparison unit test and improve container sync (#2686)
* improve logical backup comparison unit test and improve container sync
* add new comparison function for volume mounts + unit test
2024-07-08 14:06:14 +02:00
Felix Kunde 37d6993439
remove stream resources after drop from Postgres manifest (#2563)
* remove stream resources after drop from Postgres manifest
2024-06-27 14:30:52 +02:00
Polina Bungina 47efca33c9
Improve inherited annotations (#2657)
* Annotate PVC on Sync/Update, not only change PVC template
* Don't rotate pods when only annotations changed
* Annotate Logical Backup's and Pooler's pods
* Annotate PDB, Endpoints created by the Operator, Secrets, Logical Backup jobs

Inherited annotations are only added/updated, not removed
2024-06-26 13:10:37 +02:00
Felix Kunde b550f8ae39
fix unit test for new subPathExpr feature (#2638)
* fix unit test for new subPathExpr feature
* add subPathExpr flag to CRD and re-sort
2024-05-24 15:07:17 +02:00
Samuel Mutel 7bcb73a402
feat: Add SubPathExpr option for additionalVolumes (#2463) 2024-05-24 11:55:22 +02:00
Vasily Oleynikov 843d3e1caa
Fix logical backup job toleration (#2018)
* Fix logical backup job toleration (now cluster and operator-wide instructions will be not ignored)

---------

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-05-17 14:45:21 +02:00
Motte 13d6594cdf
Secrets deletion config (#2582)
* Secrets deletion config
* Update e2e/tests/test_e2e.py

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>

---------

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-05-10 16:31:21 +02:00
Felix Kunde 5357062857
add logical backup retention as manifest option (#2621)
* add logical backup retention as manifest option
* added unit test for logical backup envvar generation
2024-04-29 10:58:52 +02:00
Felix Kunde 83878fe447
make bucket prefix for logical backup configurable (#2609)
* make bucket prefix for logical backup configurable
* include container comparison in logical backup diff
* add unit test and update description for compareContainers
* don't rely on users putting / in the config - reflect other comments from review
2024-04-23 14:24:04 +02:00
Felix Kunde 3ebe4ffb99
reflect new replica states in unit tests (#2581)
* reflect new replica states in unit tests
2024-03-15 14:46:36 +01:00
Felix Kunde 08089ed4b4
add option to prevent PVC removal on cluster deletion (#2579)
* add option to prevent PVC removal on cluster deletion
* Update docs/reference/operator_parameters.md

Co-authored-by: Motte <37443982+dmotte@users.noreply.github.com>
2024-03-14 17:01:26 +01:00
yangw 773c0392a7
fix: db row close error be ignored (#2578) 2024-03-14 12:22:56 +01:00
Jociele Padilha a5663da64f
add the pg version 16 (#2557)
* add the pg version 16

* add comma after pg16 in crds api

* change minimal_major_version to 12

* add new spilo image for pg16

* edit the registry from current and lazy spilo

* Update e2e/run.sh

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>

* Update README.md

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>

* add pg 11 to be compatible for the existing DBs

* update pq, pyyaml,k8s and kind version

* skip test_infrastructure_roles

* skip another test

* remove the skipping

* adjust the verification of new Patroni version states

---------

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-03-13 16:43:25 +01:00
Felix Kunde 8bd9080798
return create and sync error, not setStatus error (#2574)
* return create and sync error, not possible status set error
* update documentation and improve deletion logs
2024-03-12 16:31:59 +01:00
cstohr1 9bb5d8add7
Fix updating SynchronousNodeCount (#2552) (#2558)
CRD support for synchronous_node_count was previously added in #1484, however the desired SynchronousNodeCount was not compared to the actual patroni configuration, which meant it was never updated.
2024-03-05 09:37:06 +01:00
Jan Mussler ae95c2db6c
Recheck version from Patroni IP. (#2533) 2024-02-29 12:12:02 +01:00
Felix Kunde 23f4fdb327
update go and dependencies (#2554) 2024-02-23 13:58:11 +01:00
Felix Kunde 3fb3b34094
change username in secret when switching rotation mode (#2549) 2024-02-22 10:26:13 +01:00
Felix Kunde e34f19be01
update spec when updating status (#2546)
* update spec when updating status
* only setSpec of pg resource is not empty
2024-02-20 10:24:24 +01:00
Felix Kunde c1bfc2c2c0
do not block delete because of emtpy child resources (#2538) 2024-02-15 14:52:52 +01:00
Felix Kunde 3d9d0258f0
patch event stream CRD instead of update (#2535) 2024-02-14 18:20:02 +01:00