Commit Graph

90 Commits

Author SHA1 Message Date
Stef Graces bb2617a53f
Add logical backup for azure (#2052)
* Add logical backup for azure
2023-01-05 12:16:41 +01:00
Felix Kunde 819e410959
refactor podAffinity generation (#2156) 2023-01-03 11:34:02 +01:00
Felix Kunde d7e1fb57f1
polish global config about sharing postgresql-run socket (#2155)
* polish global config about sharing postgresql-run socket
2023-01-02 18:28:48 +01:00
Francois Parquet be7b52db92
add preferred during scheduling pod anti affinity (#2048)
* add preferred during scheduling pod anti affinity

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2023-01-02 18:22:47 +01:00
Christian Rohmann 024aab1f13
Add config switch to share pg_socket in /var/run/postgresql via an emptyDir with the sidecar containers (#962) 2023-01-02 12:57:36 +01:00
Felix Kunde 3e148ea57e
enable operator support for pg15 and drop support for 9.5 and 9.6 (#2140)
* enable operator support for pg15 and drop support for 9.5 and 9.6
* not offer 15 in UI before spilo-15 is available
2022-12-15 12:17:27 +01:00
Polina Bungina 4d585250db
Add Patroni failsafe_mode parameter (#2076)
This commit adds support of a not-yet-released Patroni feature that allows postgres to run as primary in case of a failed leader lock update.
* Add Patroni 'failsafe_mode' local parameter (enable for a single PG cluster)
* Allow configuring Patroni 'failsafe_mode' parameter globally
2022-12-02 13:33:02 +01:00
Felix Kunde a119772efb
add toggle to turn off readiness probes (#2004)
* add toggle to turn off readiness probes
* include PodManagementPolicy and ReadinessProbe in stateful set comparison
* add URI scheme to generated readiness probe
2022-10-05 18:25:24 +02:00
Felix Kunde 3bfd63cbe6
Make teamId in cluster name optional (#2001)
* making teamId in clustername optional
* move teamId check to addCluster function
2022-08-24 10:12:50 +02:00
Jociele Padilha b41daf4f76
Set maximum CPU and Memory requests on K8s (#1959)
* Set maximum CPU and Memory requests on K8s
2022-07-28 14:18:27 +02:00
Felix Kunde 1c80ac0acd
add storage_resize_mode mixed to opConfig CRD (#1947) 2022-07-08 11:49:06 +02:00
Felix Kunde 5e4badd99c
annotation to bypass globally configured instance limits (#1943) 2022-06-30 10:40:03 +02:00
Felix Kunde 9eb7517218
sync CRD with example manifests (#1869) 2022-04-25 18:16:05 +02:00
Dmitry Volodin 9bcb25ac7e
Ability to set pod environment variables on cluster resource (#1794)
* Ability to set pod environment variables on cluster resource

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2022-04-11 10:16:35 +02:00
Felix Kunde 0dc370f15d
standby cluster that streams from a remote primary (#1830)
* add the possibility to create a standby cluster that streams from a remote primary
* extending unit tests
* add more docs and e2e test

Co-authored-by: machine424 <ayoubmrini424@gmail.com>
2022-04-04 15:41:11 +02:00
Felix Kunde 654d22d04a
Configure annotations to be ignored in comparisons during sync (#1823)
* feat: add ignored annotations when comparing during sync

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
Co-authored-by: Moshe Immerman <moshe@flanksource.com>
2022-03-24 18:38:37 +01:00
Felix Kunde 36df1bc87c
refactor GenerateResourceRequirements and provide unit tests (#1822)
* refactor GenerateResourceRequirements and provide unit tests
2022-03-24 17:35:00 +01:00
preved911 38db48c7f0
fixing toleration fields requirements (#1797)
Signed-off-by: Ildar Valiullin <preved.911@gmail.com>
2022-03-21 16:05:45 +01:00
Felix Kunde 2719d411c3
grant db owners to cron_admin (#1805)
* grant db owners to cron_admin
* allow specifiying more extra owner roles
* add unit test for InitAdditionalOwnerRoles
* add e2e test
2022-03-18 12:36:12 +01:00
Jociele Padilha 69254abeba
add new parameter for Patroni API (PatroniAPICheckInterval, PatroniAPICheckTimeout) (#1803)
Co-authored-by: Jociele Padilha <jociele.padilha@zalando.de>
2022-03-15 11:34:09 +01:00
Felix Kunde d032e4783e
LoadBalancer toggles for master and replica pooler pods (#1799)
* Add support for pooler load balancer

Signed-off-by: Sergey Shatunov <me@prok.pw>

* Rename to enable_master_pooler_load_balancer

Signed-off-by: Sergey Shatunov <me@prok.pw>

* target port should be intval
* enhance pooler e2e test
* add new options to crds.go

Co-authored-by: Sergey Shatunov <me@prok.pw>
2022-03-04 13:36:17 +01:00
A. Stoewer 695ad44caf
Logical backup retention time (#1337)
* Add optional logical backup retention time
* Set defaults for potentially unbound variables, so that the script will work with older operator versions
* Document retention time parameter for logical backups
* Add retention time parameter to resources and charts

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2022-03-02 17:39:33 +01:00
Felix Kunde d8a159ef1a
create CDC event stream CRD (#1570)
* provide event stream API
* check manifest settings for logical decoding before creating streams
* operator updates Postgres config and creates replication user
* name FES like the Postgres cluster
* add delete case and fix updating streams + update unit test
* check if fes CRD exists before syncing
* existing slot must use the same plugin
* make id and payload columns configurable
* sync streams only when they are defined in manifest
* introduce applicationId for separate stream CRDs
* add FES to RBAC in chart
* disable streams in chart
* switch to pgoutput plugin and let operator create publications
* reflect code review and additional refactoring

Co-authored-by: Paŭlo Ebermann <paul.ebermann@zalando.de>
2022-02-28 10:09:42 +01:00
Menzorg 06c28da97d
synchronous_node_count support (#1484)
* synchronous_node_count support
* notification about Patroni image version
* default synchronous_node_count to 1

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2022-02-25 17:21:42 +01:00
Felix Kunde 3ce0b1e7fa
deprecate crd validation toggle and sync with manifests (#1781)
* deprecate crd validation toggle and sync with manifests
* fix description in pg crd manifests
* change CRD creation strategy
* affinity matchExpression has values
* lower repair period in e2e tests
2022-02-18 15:04:31 +01:00
Felix Kunde 658923d10d
Password rotation in secrets (#1749)
* password rotation in K8s secrets
* add db connection to syncSecrets
* add user retention
* add e2e test
* cleanup on username mismatch if rotation was switched off
* add unit test for syncSecrets + new updateSecret func
2022-02-18 11:54:47 +01:00
Felix Kunde a78a619e90
toleration diff and nodeReadinessLabel merge with manifest matchExpressions (#1729)
* include tolerations in statefulset comparison
* provide alternative merge behavior of nodeSelectorTerms for node readiness label
* add config option to change affinity merge behavior
* reworked e2e tests around node affinity
2022-01-27 15:57:24 +01:00
Damian Peckett fe340192ca
Add configuration flag for disabling operator CRD creation/update (#1733)
* Make CRD registration configurable and drop RBAC permissions when CRD registration is disabled
* add generated deep copy functions

Co-authored-by: Damian Peckett <d.peckett_admin@mgmt.innovo-cloud.de>
2022-01-13 15:20:04 +01:00
James McDonald def9e1d688
Support standby replication from GS (GCS) (#1446)
* Add support for manual gs_wal_path in standby
* Remove separate standby version configuration
* Remove setting standby path via cluster/uid/version
Picking up the version doesn't work reliably without making changes to
Spilo. It's clearer to just specify the full S3/GS bucket path.

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-12-03 11:24:29 +01:00
Jan Mussler 3e275d122a
Allow individual teams to do auto upgrade via operator. (#1699)
* Allow whitelisting of teams to do auto upgrade upgrade via operator.

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-11-29 12:47:18 +01:00
Felix Kunde ab25fb29b7
make Postgres 14 available (#1636)
* make Postgres 14 available
* don't be too hard to 9.5
* bump Spilo image and more docs updates
* update e2e test upgrading to 14
2021-10-12 12:00:59 +02:00
John Rood 2d2ce6197b
Add volume selector (#1385)
* Add volume selector
* Add slightly better documentation and gofmt changes
* Update generated deepcopy
* Add test for PV selector

Co-authored-by: John Rood <j.rood@picturae.com>
2021-08-26 14:57:54 +02:00
Felix Kunde 282b6d2863
allow secrets of default users in a different namespace (#1581)
* allow secrets of default users in a different namespace
* add warning in case secretNamespace is ignored
2021-08-18 16:00:26 +02:00
Felix Kunde 2057ad8191
reorder e2e tests to follow alphabetical sorting (#1567)
* reorder e2e tests to follow alphabetical sorting
* e2e: finish waiting for pod failover only if all pods were replaced
* wait for sync in rolling update timeout test
2021-07-27 11:41:27 +02:00
Rafia Sabih fa604027cf
Move flag to configmap (#1540)
* Move flag to configmap

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-07-02 08:46:21 +02:00
Rafia Sabih 75a9e2be38
Create cross namespace secrets (#1490)
* Create cross namespace secrets

* add test cases

* fixes

* Fixes
- include namespace in secret name only when namespace is provided
- use username.namespace as key to pgUsers only when namespace is
  provided
- avoid conflict in the role creation in db by checking namespace
  alongwith the username

* Update unit tests

* Fix test case

* Fixes

- update regular expression for usernames
- add test to allow check for valid usernames
- create pg roles with namespace (if any) appended in rolename

* add more test cases for valid usernames

* update docs

* fixes as per review comments

* update e2e

* fixes

* Add toggle to allow namespaced secrets

* update docs

* comment update

* Update e2e/tests/test_e2e.py

* few minor fixes

* fix unit tests

* fix e2e

* fix e2e attempt 2

* fix e2e

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-06-11 10:35:30 +02:00
Felix Kunde eeb59c5bfd
Rename roles that are removed from PostgresTeam CRD (#1457)
* rename db roles that are removed from manifests

* extend PostgresTeam e2e test

* make suffix configurable and add deprecated field to pgUser struct

* deny LOGIN from deprecated roles

* update feature documentation
2021-05-21 15:49:39 +02:00
neelasha-09 9e93c0a4ef
Fix for AllowPrivilegeEscalation : issue-1403 (#1412)
* Fix for AllowPrivilegeEscalation : issue-1403

* fixed syntax error

* Aligned the value for parameter

* Aligned the value for parameter

* Update crds.go

* Aligned the parameter spilo_allow_privilege_escalation

* Parameters sorted in Alphabetical order in manifests yaml

* Parameters sorted in Alphabetical order in manifests yaml

* Update pkg/controller/operator_config.go

* Update docs/reference/operator_parameters.md

Co-authored-by: Neelam Sharma <neelasha@amdocs.com>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-03-29 10:37:59 +02:00
Felix Kunde c9acd52700
Major version upgrade config (#1386)
* reflect new major version upgrade options everywhere

* emit events during major version upgrade
2021-03-09 15:28:15 +01:00
georgebarbarosie cee4bf82c7
small fix for postgresql CRD (#1368) 2021-02-18 08:21:26 +01:00
Felix Kunde 12ad8c91fa
configurable container capabilities (#1336)
* configurable container capabilities

* revert change on TestTLS

* fix e2e test

* minor fix
2021-01-29 14:54:48 +01:00
Jan Mussler 4a88f00a3f
Full AWS gp3 support for iops and througput config. (#1261)
Support new AWS EBS volume type `gp3` with `iops` and `throughput` in the manifest.

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-01-25 10:07:18 +01:00
Felix Kunde f927d6616c
add default values to operatorconfiguration crd (#1283)
* add default values to operatorconfiguration crd

* leave default for enable_master_load_balancer to true

* add missing bits for new logical backup option

* fix wrong lb tag and update chart package
2021-01-11 17:24:24 +01:00
Jan Mussler a63ad49ef8
Initial commit for new 1.6 release with Postgres 13 support. (#1257)
* Initial commit for new 1.6 release with Postgres 13 support.
* Updating maintainers, Go version, Codeowners.
* Use lazy upgrade image that contains pg13.
* fix typo for ownerReference
* fix clusterrole in helm chart
* reflect GCP logical backup in validation
* improve PostgresTeam docs
* change defaults for enable_pgversion_env_var and storage_resize_mode
* explain manual part of in-place upgrade
* remove gsoc docs

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-12-17 15:00:29 +01:00
Pavel Tumik 77252e316c
Add node affinity support (#1166)
* Adding nodeaffinity support alongside node_readiness_label
* add documentation for node affinity
* add node affinity e2e test
* add unit test for node affinity

Co-authored-by: Steffen Pøhner Henriksen <str3sses@gmail.com>
Co-authored-by: Adrian Astley <adrian.astley@activision.com>
2020-12-16 14:56:28 +01:00
Pavel Tumik 5076e669cb
Fix timestamp regex (#1178) 2020-12-16 11:17:08 +01:00
Felix Kunde 83fbccac5a
new env var for backwards compatability between spilo 12 and 13 (#1254) 2020-12-14 18:43:53 +01:00
Felix Kunde 6a97316a69
Support inherited annotations for all major objects (#1236)
* add comments where inherited annotations could be added

* add inheritedAnnotations feature

* return nil if no annotations are set

* minor changes

* first downscaler then inherited annotations

* add unit test for inherited annotations

* add pvc to test + minor changes

* missing comma

* fix nil map assignment

* set annotations in the same order it is done in other places

* replace acidClientSet with acid getters in K8s client

* more fixes on clientSet vs getters

* minor changes

* remove endpoints from annotation test

* refine unit test - but deployment and sts are still empty

* fix checkinng sts and deployment

* make annotations setter one liners

* no need for len check anymore

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
2020-12-11 16:34:01 +01:00
Jan Mussler 549f71bb49
Support EBS gp2 to gp3 migration on sync for below 1tb volumes (#1242)
* initial commit for gp3 migration.

* Default volume migration done.

* Added Gomock and one test case with mock.

* Dep update.

* more changes for code gen.

* push fake package.

* Rename var.

* Changes to Makefile and return value.

* Macke mocks phony due to overlap in foldername.

* Learning as one goes. Initialize map.

* Wrong toggle.

* Expect modify call.

* Fix mapping of ids in test.

* Fix volume id.

* volume ids.

* Fixing test setup. Late night...

* create all pvs.

* Fix test case config.

* store volumes and compare.

* More logs.

* Logging of migration action.

* Ensure to log errors.

* Log warning if modify failed, e.g. due to ebs volume state.

* Add more output.

* Skip local e2e tests.

* Reflect k8s volume id in test data. Extract aws volume id from k8s value.

* Finalizing ebs migration.

* More logs. describe fails.

* Fix non existing fields in gp2 discovery.

* Remove nothing to do flag for migration.

* Final commit for migration.

* add new options to all places

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-12-11 15:52:32 +01:00
Igor Yanchenko b9ef88f842
replace AdditionalProperties with XPreserveUnknownFields FIXES #1206 (#1248) 2020-12-10 16:27:03 +01:00