Commit Graph

1371 Commits

Author SHA1 Message Date
Felix Kunde cc9074c184
Bump operator to v1.13.0 (#2729)
* bump operator to v1.13.0
* align configmap with CRD config
* remove default from CRD config option additional_secret_mount_path
* enable automatic major version upgrades by default
2024-08-22 12:16:27 +02:00
Rob Nickmans cb06a1ec89
fix: add secret only when not in secret file (#2732)
* fix: add secret only when not in secret file
* fix indentation

---------

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-08-20 17:35:09 +02:00
Polina Bungina 2582b934bf
MaintenanceWindow CRD validation reflects the implementation (#2731) 2024-08-20 14:43:12 +02:00
Felix Kunde 2f7e3ee847
fix stream duplication on operator restart (#2733)
* fix stream duplication on operator restart
* add try except to streams e2e test
2024-08-20 14:38:07 +02:00
Felix Kunde c7ee34ed12
fix sync streams and add diffs for annotations and owner references (#2728)
* extend and improve hasSlotsInSync unit test
* fix sync streams and add diffs for annotations and owner references
* incl. current annotations as desired where we do not fully control them
* added one more unit test and fixed sub test names
* pass maintenance windows to function and update unit test
2024-08-14 12:56:14 +02:00
fahed dorgaa aad03f71ea
fix golangci-lint issues (#2715)
Signed-off-by: fahed dorgaa <fahed.dorgaa@gmail.com>
Co-authored-by: fahed dorgaa <fahed.dorgaa.ext@corp.ovh.com>
Co-authored-by: Matthias Adler <macedigital@users.noreply.github.com>
2024-08-14 12:54:44 +02:00
Felix Kunde 25ccc87317
sync all resources to cluster fields (#2713)
* sync all resources to cluster fields (CronJob, Streams, Patroni resources)
* separated sync and delete logic for Patroni resources
* align delete streams and secrets logic with other resources
* rename gatherApplicationIds to getDistinctApplicationIds
* improve slot check before syncing streams CRD
* add ownerReferences and annotations diff to Patroni objects
* add extra sync code for config service so it does not get too ugly
* some bugfixes when comparing annotations and return err on found
* sync Patroni resources on update event and extended unit tests
* add config service/endpoint owner references check to e2e tes
2024-08-13 10:06:46 +02:00
Felix Kunde 31f92a1aa0
extend inherited annotations unit test to include logical backup cron job (#2723)
* extend inherited annotations test to logical backup cron job
* sync on updated when enabled, not only on schedule changes
2024-08-12 13:12:51 +02:00
Felix Kunde a87307e56b
Feat: enable owner references (#2688)
* feat(498): Add ownerReferences to managed entities
* empty owner reference for cross namespace secret and more tests
* update ownerReferences of existing resources
* removing ownerReference requires Update API call
* CR ownerReference on PVC blocks pvc retention policy of statefulset
* make ownerreferences optional and disabled by default
* update unit test to check len ownerReferences
* update codegen
* add owner references e2e test
* update unit test
* add block_owner_deletion field to test owner reference
* fix typos and update docs once more
* reflect code feedback

---------

Co-authored-by: Max Begenau <max@begenau.com>
2024-08-09 17:58:25 +02:00
Felix Kunde d5a88f571a
let operator fix publications without tables (#2722) 2024-08-09 17:20:05 +02:00
Felix Kunde 85b8058029
bump spilo to 16-3.3, drop support for pg11 (#2706)
* bump spilo to 16-3.3, drop support for pg11
* update README
2024-08-09 14:47:23 +02:00
Ida Novindasari e6ae9e3772
Implement per-cluster maintenance window for Postgres automatic upgrade (#2710)
* implement maintenance window for major version upgrade 
* e2e test: fix major version upgrade test and extend with the time window
* unit test: add iteration to test isInMaintenanceWindow
* UI: show the window and enable edit via UI
2024-08-09 14:07:35 +02:00
Cédric de Saint Martin ce15d10aa3
feat: Add extraEnvs to operator helm chart (#2671)
Signed-off-by: Cédric de Saint Martin <cdesaintmartin@wiremind.io>
2024-08-06 12:31:17 +02:00
Ida Novindasari 94d36327ba
stream: slot and FES should not be created if the publication creation fails (#2704)
* slot should not be created if the publication creation fails
* not create FES resource when slot doesn't exist
2024-08-02 15:09:37 +02:00
Ida Novindasari 31f474a95c
Enable slot and publication deletion when stream application is removed (#2684)
* refactor syncing publication section
* update createOrUpdateStream function to allow resource deletion when removed from manifest
* add minimal FES CRD to enable FES resources creation for E2E test
* fix bug of removing manifest slots in syncStream
* e2e test: fixing typo with major upgrade test
* e2e test: should create and delete FES resource
* e2e test: should not delete manual created resources
* e2e test: enable cluster role for FES with patching instead of deploying in manifest
2024-07-25 12:00:23 +02:00
Felix Kunde 73f72414f6
bump go version to 1.22.5 (#2699) 2024-07-23 13:25:29 +02:00
Felix Kunde e71891e2bd
improve logical backup comparison unit test and improve container sync (#2686)
* improve logical backup comparison unit test and improve container sync
* add new comparison function for volume mounts + unit test
2024-07-08 14:06:14 +02:00
Felix Kunde 37d6993439
remove stream resources after drop from Postgres manifest (#2563)
* remove stream resources after drop from Postgres manifest
2024-06-27 14:30:52 +02:00
Matthias Adler 7cdc23fff0
chore: simplify delivery-yaml for building operator (#2673)
Commit switches builder image to `cdp-runtime/go`, removing the need to install `go` manually.

Also, commit splits "build-postgres-operator" pipeline into 2 distinct steps.

1. Run unit tests based on locally checked out code including set up of dependencies and generated code.
2. Build Docker image if tests are successful
2024-06-26 18:39:20 +02:00
Polina Bungina 47efca33c9
Improve inherited annotations (#2657)
* Annotate PVC on Sync/Update, not only change PVC template
* Don't rotate pods when only annotations changed
* Annotate Logical Backup's and Pooler's pods
* Annotate PDB, Endpoints created by the Operator, Secrets, Logical Backup jobs

Inherited annotations are only added/updated, not removed
2024-06-26 13:10:37 +02:00
Matthias Adler 2ef7d58578
chore: update package dependencies when building image (#2665)
* chore: update package dependencies when building image

Install available updates alongside installation of packages to remove known vulnerabilities from images.

Example for issues in plain alpine:3 image (v3.20):

```sh
$ grype alpine:3
 ✔ Vulnerability DB                [updated]
 ✔ Loaded image                                                            alpine:3
 ✔ Parsed image                    sha256:1d34ffeaf190be23d3de5a8de0a436676b758f48f
 ✔ Cataloged contents              dac15f325cac528994a5efe78787cd03bdd796979bda52fd
   ├── ✔ Packages                        [14 packages]
   ├── ✔ File digests                    [77 files]
   ├── ✔ File metadata                   [77 locations]
   └── ✔ Executables                     [17 executables]
 ✔ Scanned for vulnerabilities     [8 vulnerability matches]
   ├── by severity: 0 critical, 0 high, 6 medium, 0 low, 0 negligible (2 unknown)
   └── by status:   8 fixed, 0 not-fixed, 0 ignored
NAME           INSTALLED   FIXED-IN    TYPE  VULNERABILITY   SEVERITY
busybox        1.36.1-r28  1.36.1-r29  apk   CVE-2023-42365  Medium
busybox        1.36.1-r28  1.36.1-r29  apk   CVE-2023-42364  Medium
busybox-binsh  1.36.1-r28  1.36.1-r29  apk   CVE-2023-42365  Medium
busybox-binsh  1.36.1-r28  1.36.1-r29  apk   CVE-2023-42364  Medium
libcrypto3     3.3.0-r2    3.3.0-r3    apk   CVE-2024-4741   Unknown
libssl3        3.3.0-r2    3.3.0-r3    apk   CVE-2024-4741   Unknown
ssl_client     1.36.1-r28  1.36.1-r29  apk   CVE-2023-42365  Medium
ssl_client     1.36.1-r28  1.36.1-r29  apk   CVE-2023-42364  Medium
```

Issue would be solved by also upgrading installed packages:

```sh
$ apk -U upgrade --no-cache
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/community/x86_64/APKINDEX.tar.gz
(1/5) Upgrading busybox (1.36.1-r28 -> 1.36.1-r29)
Executing busybox-1.36.1-r29.post-upgrade
(2/5) Upgrading busybox-binsh (1.36.1-r28 -> 1.36.1-r29)
(3/5) Upgrading libcrypto3 (3.3.0-r2 -> 3.3.1-r0)
(4/5) Upgrading libssl3 (3.3.0-r2 -> 3.3.1-r0)
(5/5) Upgrading ssl_client (1.36.1-r28 -> 1.36.1-r29)
Executing busybox-1.36.1-r29.trigger
OK: 8 MiB in 14 packages
```

Furthermore, this commit reduces accidental complexity from the Docker build process.
Most notably, use pre-made official golang images for building postgres-operator.

* Update docker/DebugDockerfile

---------

Co-authored-by: Ida Novindasari <idanovinda@gmail.com>
2024-06-18 17:21:03 +02:00
Felix Kunde 7c7aa96935
bump to v1.12.2 (#2664) 2024-06-14 10:53:17 +02:00
Matthias Adler eee1ef2e21
Align Docker images in ghcr pipeline with images assumed in Dockerfile (#2663)
* PG-UI switch to official Python image 

Commit changes build argument for Github Actions to use the same [official Python image][1] as the one used for building "postgres-operator-ui" since commit d60b424d79.

Should fix problem with `publish_ghcr_image` workflow. 

[1]: https://hub.docker.com/_/python

* Use latest Alpine version for Postgres-Operator

Similar to commit 601ce0d321, align image version in Github Actions pipeline with assumed default version in Dockerfile, using latest [Alpine 3](https://hub.docker.com/_/alpine).
2024-06-14 10:25:55 +02:00
Felix Kunde 2e1583e9c0
bump to v1.12.1 (#2658)
* bump to v1.12.1
* align Python version in setup.py with base image
2024-06-13 10:40:07 +02:00
Hemakshi Sachdev 032743b8f0
Fix (#2644) - Add name tags to InfrastructureRole struct (#2659) 2024-06-12 11:12:28 +02:00
Matthias Adler 1f47f59267
fix: use nodejs-lts image for building frontend code (#2653)
* fix: use nodejs-lts image for building frontend code

Node v14 is end-of-life and should no longer be used. Commit changes Makefile to pull in latest node-lts instead.

Also, use local temporary folder for storing npm generated files to workaround permission issue with old npm version, e.g. emitting these errors:

```
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /.npm
npm ERR! errno -13
npm ERR!
npm ERR! Your cache folder contains root-owned files, due to a bug in
npm ERR! previous versions of npm which has since been addressed.
```

Both changes should also fix issue https://github.com/zalando/postgres-operator/issues/2651

* fix: add frontend build step

Commit d60b424d79 accidentally removed build steps that are important for building frontend.

This commit restores previous behavior, but switches to nodejs-lts image for building frontend code.

Should restore `app.js` presence in ghcr image, see https://github.com/zalando/postgres-operator/issues/2651.
2024-06-05 15:09:44 +02:00
Felix Kunde 393439fdc3
update K8s version in makefile (#2647) 2024-06-05 14:36:49 +02:00
Felix Kunde 6cde8e8c0b
Bump to v1.12.0 (#2639)
* bump tp v1.12.0
* code-generator and apiextensions-apiserver still on to 0.25.9 to allow code-generation on GH
* bump go in github action and mini fix in UI
* update UI Dockerfile

---------

Co-authored-by: Ida Novindasari <idanovinda@gmail.com>
2024-05-31 15:29:29 +02:00
Hemakshi Sachdev 34f9cfbcda
Fix (#2644) - Add json tags to InfrastructureRole struct (#2645)
* Fix (#2644) - Add json tags to InfrastructureRole struct
* Fix (#2644) - Add name tags to InfrastructureRole struct

---------

Co-authored-by: Hemakshi Sachdev <hsachdev@purestorage.com>
2024-05-31 14:17:26 +02:00
Felix Kunde d60b424d79
[UI] use only one logger adapter and update Dockerfile (#2646)
* [UI] use only one logger adapter and update Dockerfile
* remove setLevel on logger
2024-05-31 11:24:31 +02:00
Pratheek Rebala 1210ceca72
Allow scheduling constraints for operator-ui pods (#2326) 2024-05-24 16:27:00 +02:00
Felix Kunde b550f8ae39
fix unit test for new subPathExpr feature (#2638)
* fix unit test for new subPathExpr feature
* add subPathExpr flag to CRD and re-sort
2024-05-24 15:07:17 +02:00
Samuel Mutel 7bcb73a402
feat: Add SubPathExpr option for additionalVolumes (#2463) 2024-05-24 11:55:22 +02:00
Ida Novindasari 1839baaad3
[UI] Remove manual authentication for login user (#2635)
* Remove manual authentication
* update python libraries
* remove psycopg2 and bring back wal-e
* remove unused vars
2024-05-23 10:51:46 +02:00
Felix Kunde 1b08ee1acf
switch to ghcr image in helm chart and examples (#2634)
* switch to ghcr image in helm chart and examples
* change logical backup config for helm chart
* change internal default for logical backup image config to ghcr, too
2024-05-21 17:43:37 +02:00
Vasily Oleynikov 843d3e1caa
Fix logical backup job toleration (#2018)
* Fix logical backup job toleration (now cluster and operator-wide instructions will be not ignored)

---------

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-05-17 14:45:21 +02:00
Motte 13d6594cdf
Secrets deletion config (#2582)
* Secrets deletion config
* Update e2e/tests/test_e2e.py

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>

---------

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2024-05-10 16:31:21 +02:00
Nick Douma 8ee5231648
Apply template on all keys of operatorconfiguration (#2608) 2024-05-10 16:30:29 +02:00
Felix Kunde 5357062857
add logical backup retention as manifest option (#2621)
* add logical backup retention as manifest option
* added unit test for logical backup envvar generation
2024-04-29 10:58:52 +02:00
dependabot[bot] d70cdf1f10
Bump golang.org/x/net from 0.19.0 to 0.23.0 in /kubectl-pg (#2613)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.19.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.19.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-25 15:55:55 +02:00
dependabot[bot] d4c3e236ed
Bump golang.org/x/net from 0.20.0 to 0.23.0 (#2614)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.20.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-25 15:55:41 +02:00
Felix Kunde 83878fe447
make bucket prefix for logical backup configurable (#2609)
* make bucket prefix for logical backup configurable
* include container comparison in logical backup diff
* add unit test and update description for compareContainers
* don't rely on users putting / in the config - reflect other comments from review
2024-04-23 14:24:04 +02:00
Felix Kunde 6ddafadc09
add pdb_master_label_selector to helm chart and example manifests (#2595)
* add pdb_master_label_selector to helm chart and example manifests
2024-03-28 12:06:35 +01:00
Felix Kunde be28f3a017
update helm chart with #2584 (#2585) 2024-03-18 11:05:40 +01:00
Thore 4cd4bee383
Fix duplicate key issue when using configmap config target (#2584) 2024-03-18 10:55:42 +01:00
Felix Kunde 0f96eb20bf
bump tp v1.11.0 (#2580) 2024-03-15 15:02:39 +01:00
Felix Kunde 3ebe4ffb99
reflect new replica states in unit tests (#2581)
* reflect new replica states in unit tests
2024-03-15 14:46:36 +01:00
Felix Kunde 08089ed4b4
add option to prevent PVC removal on cluster deletion (#2579)
* add option to prevent PVC removal on cluster deletion
* Update docs/reference/operator_parameters.md

Co-authored-by: Motte <37443982+dmotte@users.noreply.github.com>
2024-03-14 17:01:26 +01:00
yangw 773c0392a7
fix: db row close error be ignored (#2578) 2024-03-14 12:22:56 +01:00
dependabot[bot] 6653b8f9c6
Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 (#2576)
Bumps google.golang.org/protobuf from 1.31.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-14 12:22:32 +01:00