Commit Graph

1395 Commits

Author SHA1 Message Date
cosimomeli eef49500a5
Add support for EBS CSI Driver (#2677)
* Add support for EBS CSI Driver
2024-12-19 12:32:09 +01:00
dependabot[bot] e7cc4f9120
Bump golang.org/x/crypto from 0.26.0 to 0.31.0 in /kubectl-pg (#2819)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.26.0 to 0.31.0.
- [Commits](https://github.com/golang/crypto/compare/v0.26.0...v0.31.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-18 16:03:20 +01:00
Felix Kunde 8cc6796537
fix comparing stream annotations and improve unit test (#2820) 2024-12-18 11:22:08 +01:00
dependabot[bot] 5450113eb5
Bump golang.org/x/crypto from 0.26.0 to 0.31.0 (#2816)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.26.0 to 0.31.0.
- [Commits](https://github.com/golang/crypto/compare/v0.26.0...v0.31.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-17 14:07:15 +01:00
Felix Kunde d44bfabe78
do not use extra labels to list stream CRDs (#2803)
* do not use extra labels to list stream CRDs
* add diff on labels for streams + unit test coverage
2024-12-17 08:54:37 +01:00
Felix Kunde 80ef38f7f0
add resource annotation and ignore recovery type (#2817)
* add resource annotation and ignore recovery type
* Update docs/reference/cluster_manifest.md

---------

Co-authored-by: Ida Novindasari <idanovinda@gmail.com>
2024-12-16 18:17:19 +01:00
Felix Kunde 301462c415
remove streams delete and extend unit tests (#2737) 2024-12-16 18:13:52 +01:00
Felix Kunde 4929dd204c
Update major version upgrade docs (#2807)
* Update major version upgrade logs
2024-12-16 11:22:40 +01:00
Ida Novindasari fc9a26040a
Integrate spilo with Patroni 4 (#2818) 2024-12-16 11:11:22 +01:00
dependabot[bot] c206eb38a8
Bump werkzeug from 3.0.3 to 3.0.6 in /ui (#2793)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.3 to 3.0.6.
- [Release notes](https://github.com/pallets/werkzeug/releases)
- [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/werkzeug/compare/3.0.3...3.0.6)

---
updated-dependencies:
- dependency-name: werkzeug
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-06 11:46:53 +01:00
fahed dorgaa acdb957d8e
fix switch over candidate retrieving (#2760)
* fix switch over candidate retrieving

Signed-off-by: fahed dorgaa <fahed.dorgaa.ext@corp.ovh.com>

---------

Signed-off-by: fahed dorgaa <fahed.dorgaa.ext@corp.ovh.com>
Co-authored-by: fahed dorgaa <fahed.dorgaa.ext@corp.ovh.com>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-11-01 17:06:20 +01:00
Felix Kunde 8231797efa
add cluster field for PVCs (#2785)
* add cluster field for PVCs
* sync volumes on cluster creation
* fully spell pvc in log messages
2024-10-31 14:08:50 +01:00
Martin Kucin 45e9227f55
fix(postgres-operator/deployment): Set 'nindent' to 8 for 'extraEnvs' (#2783)
Co-authored-by: martin.kucin <martin.kucin@yunextraffic.com>
2024-10-30 11:11:22 +01:00
Felix Kunde 002d0f94a1
quote schema names in case they use special characters and remove strings.Builder (#2782) 2024-10-17 16:52:24 +02:00
Motte f5e122e8ef
Fix resource constraints (#2735)
* Add empty string cases to patterns for pod resources
* Added empty strings test case
* Restored k8sres.go and changed test to zeros
* Updated validation pattern in manifests/operatorconfiguration.crd.yaml and pkg/apis/acid.zalan.do/v1/crds.go
2024-10-16 17:19:07 +02:00
Prasad Krishnan d21466dbc4
update clusterrole.yaml (#2762)
* update clusterrole.yaml
* Update charts/postgres-operator/templates/clusterrole.yaml

---------

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-10-16 17:18:01 +02:00
Felix Kunde bb73334682
quote admin user to allow names with special characters (#2774) 2024-10-16 17:14:44 +02:00
Polina Bungina 41f5fe1dc9
More major upgrade prechecks (#2775)
Skip when 
- it is a standby clusters
- there is no master in the cluster
2024-10-15 14:05:39 +02:00
Polina Bungina 3ca86678cc
Add major upgrade prechecks (#2772)
Don't fail major upgrade (don't set annotation) if replica(s) are not
(yet) streaming or replication lag is too high
2024-10-11 17:11:46 +02:00
Ida Novindasari c25dc57b96
only skip upgrade if failed before after recheck version (#2755) 2024-09-10 10:32:56 +02:00
Ida Novindasari 2e398120d2
Implement major upgrade result annotations (#2727)
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
Co-authored-by: Polina Bungina <27892524+hughcapet@users.noreply.github.com>
2024-08-28 15:26:12 +02:00
Felix Kunde a09b7655c9
update K8s version and reflect necessary changes (#2740) 2024-08-27 18:13:39 +02:00
Felix Kunde 2ae51fb9ce
reflect linter feedback, remove unused argumnents and redundant type from arrays (#2739)
* reflect linter feedback, remove unused argumnents and redundant literal definitions
* add logical backup to TestCreate unit test
2024-08-27 17:56:07 +02:00
Felix Kunde a08d1679f2
align sync and update logs (#2738) 2024-08-27 09:58:32 +02:00
Felix Kunde cc9074c184
Bump operator to v1.13.0 (#2729)
* bump operator to v1.13.0
* align configmap with CRD config
* remove default from CRD config option additional_secret_mount_path
* enable automatic major version upgrades by default
2024-08-22 12:16:27 +02:00
Rob Nickmans cb06a1ec89
fix: add secret only when not in secret file (#2732)
* fix: add secret only when not in secret file
* fix indentation

---------

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-08-20 17:35:09 +02:00
Polina Bungina 2582b934bf
MaintenanceWindow CRD validation reflects the implementation (#2731) 2024-08-20 14:43:12 +02:00
Felix Kunde 2f7e3ee847
fix stream duplication on operator restart (#2733)
* fix stream duplication on operator restart
* add try except to streams e2e test
2024-08-20 14:38:07 +02:00
Felix Kunde c7ee34ed12
fix sync streams and add diffs for annotations and owner references (#2728)
* extend and improve hasSlotsInSync unit test
* fix sync streams and add diffs for annotations and owner references
* incl. current annotations as desired where we do not fully control them
* added one more unit test and fixed sub test names
* pass maintenance windows to function and update unit test
2024-08-14 12:56:14 +02:00
fahed dorgaa aad03f71ea
fix golangci-lint issues (#2715)
Signed-off-by: fahed dorgaa <fahed.dorgaa@gmail.com>
Co-authored-by: fahed dorgaa <fahed.dorgaa.ext@corp.ovh.com>
Co-authored-by: Matthias Adler <macedigital@users.noreply.github.com>
2024-08-14 12:54:44 +02:00
Felix Kunde 25ccc87317
sync all resources to cluster fields (#2713)
* sync all resources to cluster fields (CronJob, Streams, Patroni resources)
* separated sync and delete logic for Patroni resources
* align delete streams and secrets logic with other resources
* rename gatherApplicationIds to getDistinctApplicationIds
* improve slot check before syncing streams CRD
* add ownerReferences and annotations diff to Patroni objects
* add extra sync code for config service so it does not get too ugly
* some bugfixes when comparing annotations and return err on found
* sync Patroni resources on update event and extended unit tests
* add config service/endpoint owner references check to e2e tes
2024-08-13 10:06:46 +02:00
Felix Kunde 31f92a1aa0
extend inherited annotations unit test to include logical backup cron job (#2723)
* extend inherited annotations test to logical backup cron job
* sync on updated when enabled, not only on schedule changes
2024-08-12 13:12:51 +02:00
Felix Kunde a87307e56b
Feat: enable owner references (#2688)
* feat(498): Add ownerReferences to managed entities
* empty owner reference for cross namespace secret and more tests
* update ownerReferences of existing resources
* removing ownerReference requires Update API call
* CR ownerReference on PVC blocks pvc retention policy of statefulset
* make ownerreferences optional and disabled by default
* update unit test to check len ownerReferences
* update codegen
* add owner references e2e test
* update unit test
* add block_owner_deletion field to test owner reference
* fix typos and update docs once more
* reflect code feedback

---------

Co-authored-by: Max Begenau <max@begenau.com>
2024-08-09 17:58:25 +02:00
Felix Kunde d5a88f571a
let operator fix publications without tables (#2722) 2024-08-09 17:20:05 +02:00
Felix Kunde 85b8058029
bump spilo to 16-3.3, drop support for pg11 (#2706)
* bump spilo to 16-3.3, drop support for pg11
* update README
2024-08-09 14:47:23 +02:00
Ida Novindasari e6ae9e3772
Implement per-cluster maintenance window for Postgres automatic upgrade (#2710)
* implement maintenance window for major version upgrade 
* e2e test: fix major version upgrade test and extend with the time window
* unit test: add iteration to test isInMaintenanceWindow
* UI: show the window and enable edit via UI
2024-08-09 14:07:35 +02:00
Cédric de Saint Martin ce15d10aa3
feat: Add extraEnvs to operator helm chart (#2671)
Signed-off-by: Cédric de Saint Martin <cdesaintmartin@wiremind.io>
2024-08-06 12:31:17 +02:00
Ida Novindasari 94d36327ba
stream: slot and FES should not be created if the publication creation fails (#2704)
* slot should not be created if the publication creation fails
* not create FES resource when slot doesn't exist
2024-08-02 15:09:37 +02:00
Ida Novindasari 31f474a95c
Enable slot and publication deletion when stream application is removed (#2684)
* refactor syncing publication section
* update createOrUpdateStream function to allow resource deletion when removed from manifest
* add minimal FES CRD to enable FES resources creation for E2E test
* fix bug of removing manifest slots in syncStream
* e2e test: fixing typo with major upgrade test
* e2e test: should create and delete FES resource
* e2e test: should not delete manual created resources
* e2e test: enable cluster role for FES with patching instead of deploying in manifest
2024-07-25 12:00:23 +02:00
Felix Kunde 73f72414f6
bump go version to 1.22.5 (#2699) 2024-07-23 13:25:29 +02:00
Felix Kunde e71891e2bd
improve logical backup comparison unit test and improve container sync (#2686)
* improve logical backup comparison unit test and improve container sync
* add new comparison function for volume mounts + unit test
2024-07-08 14:06:14 +02:00
Felix Kunde 37d6993439
remove stream resources after drop from Postgres manifest (#2563)
* remove stream resources after drop from Postgres manifest
2024-06-27 14:30:52 +02:00
Matthias Adler 7cdc23fff0
chore: simplify delivery-yaml for building operator (#2673)
Commit switches builder image to `cdp-runtime/go`, removing the need to install `go` manually.

Also, commit splits "build-postgres-operator" pipeline into 2 distinct steps.

1. Run unit tests based on locally checked out code including set up of dependencies and generated code.
2. Build Docker image if tests are successful
2024-06-26 18:39:20 +02:00
Polina Bungina 47efca33c9
Improve inherited annotations (#2657)
* Annotate PVC on Sync/Update, not only change PVC template
* Don't rotate pods when only annotations changed
* Annotate Logical Backup's and Pooler's pods
* Annotate PDB, Endpoints created by the Operator, Secrets, Logical Backup jobs

Inherited annotations are only added/updated, not removed
2024-06-26 13:10:37 +02:00
Matthias Adler 2ef7d58578
chore: update package dependencies when building image (#2665)
* chore: update package dependencies when building image

Install available updates alongside installation of packages to remove known vulnerabilities from images.

Example for issues in plain alpine:3 image (v3.20):

```sh
$ grype alpine:3
 ✔ Vulnerability DB                [updated]
 ✔ Loaded image                                                            alpine:3
 ✔ Parsed image                    sha256:1d34ffeaf190be23d3de5a8de0a436676b758f48f
 ✔ Cataloged contents              dac15f325cac528994a5efe78787cd03bdd796979bda52fd
   ├── ✔ Packages                        [14 packages]
   ├── ✔ File digests                    [77 files]
   ├── ✔ File metadata                   [77 locations]
   └── ✔ Executables                     [17 executables]
 ✔ Scanned for vulnerabilities     [8 vulnerability matches]
   ├── by severity: 0 critical, 0 high, 6 medium, 0 low, 0 negligible (2 unknown)
   └── by status:   8 fixed, 0 not-fixed, 0 ignored
NAME           INSTALLED   FIXED-IN    TYPE  VULNERABILITY   SEVERITY
busybox        1.36.1-r28  1.36.1-r29  apk   CVE-2023-42365  Medium
busybox        1.36.1-r28  1.36.1-r29  apk   CVE-2023-42364  Medium
busybox-binsh  1.36.1-r28  1.36.1-r29  apk   CVE-2023-42365  Medium
busybox-binsh  1.36.1-r28  1.36.1-r29  apk   CVE-2023-42364  Medium
libcrypto3     3.3.0-r2    3.3.0-r3    apk   CVE-2024-4741   Unknown
libssl3        3.3.0-r2    3.3.0-r3    apk   CVE-2024-4741   Unknown
ssl_client     1.36.1-r28  1.36.1-r29  apk   CVE-2023-42365  Medium
ssl_client     1.36.1-r28  1.36.1-r29  apk   CVE-2023-42364  Medium
```

Issue would be solved by also upgrading installed packages:

```sh
$ apk -U upgrade --no-cache
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/community/x86_64/APKINDEX.tar.gz
(1/5) Upgrading busybox (1.36.1-r28 -> 1.36.1-r29)
Executing busybox-1.36.1-r29.post-upgrade
(2/5) Upgrading busybox-binsh (1.36.1-r28 -> 1.36.1-r29)
(3/5) Upgrading libcrypto3 (3.3.0-r2 -> 3.3.1-r0)
(4/5) Upgrading libssl3 (3.3.0-r2 -> 3.3.1-r0)
(5/5) Upgrading ssl_client (1.36.1-r28 -> 1.36.1-r29)
Executing busybox-1.36.1-r29.trigger
OK: 8 MiB in 14 packages
```

Furthermore, this commit reduces accidental complexity from the Docker build process.
Most notably, use pre-made official golang images for building postgres-operator.

* Update docker/DebugDockerfile

---------

Co-authored-by: Ida Novindasari <idanovinda@gmail.com>
2024-06-18 17:21:03 +02:00
Felix Kunde 7c7aa96935
bump to v1.12.2 (#2664) 2024-06-14 10:53:17 +02:00
Matthias Adler eee1ef2e21
Align Docker images in ghcr pipeline with images assumed in Dockerfile (#2663)
* PG-UI switch to official Python image 

Commit changes build argument for Github Actions to use the same [official Python image][1] as the one used for building "postgres-operator-ui" since commit d60b424d79.

Should fix problem with `publish_ghcr_image` workflow. 

[1]: https://hub.docker.com/_/python

* Use latest Alpine version for Postgres-Operator

Similar to commit 601ce0d321, align image version in Github Actions pipeline with assumed default version in Dockerfile, using latest [Alpine 3](https://hub.docker.com/_/alpine).
2024-06-14 10:25:55 +02:00
Felix Kunde 2e1583e9c0
bump to v1.12.1 (#2658)
* bump to v1.12.1
* align Python version in setup.py with base image
2024-06-13 10:40:07 +02:00
Hemakshi Sachdev 032743b8f0
Fix (#2644) - Add name tags to InfrastructureRole struct (#2659) 2024-06-12 11:12:28 +02:00
Matthias Adler 1f47f59267
fix: use nodejs-lts image for building frontend code (#2653)
* fix: use nodejs-lts image for building frontend code

Node v14 is end-of-life and should no longer be used. Commit changes Makefile to pull in latest node-lts instead.

Also, use local temporary folder for storing npm generated files to workaround permission issue with old npm version, e.g. emitting these errors:

```
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /.npm
npm ERR! errno -13
npm ERR!
npm ERR! Your cache folder contains root-owned files, due to a bug in
npm ERR! previous versions of npm which has since been addressed.
```

Both changes should also fix issue https://github.com/zalando/postgres-operator/issues/2651

* fix: add frontend build step

Commit d60b424d79 accidentally removed build steps that are important for building frontend.

This commit restores previous behavior, but switches to nodejs-lts image for building frontend code.

Should restore `app.js` presence in ghcr image, see https://github.com/zalando/postgres-operator/issues/2651.
2024-06-05 15:09:44 +02:00