Commit Graph

85 Commits

Author SHA1 Message Date
yoshihikoueno becf8a4715
Bump spilo and target version for PostgreSQL 15 (#2139)
* Bumped Spilo image tag to the one that supports PostgreSQL 15. Using CDP version temporarily until non-CDP one is released.
* Added support for PostgreSQL 15 and made it default. 9.5 and 9.6 are now no longer supported
* Bumped spilo image tag to 2.1-p9
* Bumped spilo image in test launcher

Co-authored-by: yoshihiko <ariyoshi10@gmail.com>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2023-01-04 12:01:30 +01:00
idanovinda 486d5d66e0
Allow drop slots when it gets deleted from the manifest (#2089)
* Allow drop slots when it gets deleted from the manifest
* use leader instead replica to query slots
* fix and extend unit tests for config update checks

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2023-01-03 15:46:59 +01:00
Felix Kunde e80cccb93b
use random short name for stream CRDs (#2137)
* use random short name for stream CRDs
2022-12-27 16:52:01 +01:00
Polina Bungina 4d585250db
Add Patroni failsafe_mode parameter (#2076)
This commit adds support of a not-yet-released Patroni feature that allows postgres to run as primary in case of a failed leader lock update.
* Add Patroni 'failsafe_mode' local parameter (enable for a single PG cluster)
* Allow configuring Patroni 'failsafe_mode' parameter globally
2022-12-02 13:33:02 +01:00
Felix Kunde 4786f53f03
Fix password rotation (#2043)
* fix password rotation
* test connection with rotation user in e2e test + minor changes
2022-10-13 11:33:26 +02:00
Felix Kunde ce8b009c66
fix team member deprecation (#2072) 2022-10-11 18:02:41 +02:00
Philipp B 84fe38a069
switch to batch API v1 for Jobs (#2066) 2022-10-07 11:27:58 +02:00
Felix Kunde 2aa52094db
switch to policy API v1 for PDBs (#2008)
* switch to policy API v1 for PDBs
* update e2e test dependencies
* use kind 0.14.0
* bump K8s client in e2e docker image
* bump e2e tests-runner
2022-10-06 09:43:17 +02:00
Felix Kunde b2642fa2fc
allow in place pw rotation of system users (#1953)
* allow in place pw rotation of system users
* block postgres user from rotation
* mark pooler pods for replacement
* adding podsGetter where pooler is synced in unit tests
* move rotation code in extra function
2022-08-18 14:14:31 +02:00
Jociele Padilha b41daf4f76
Set maximum CPU and Memory requests on K8s (#1959)
* Set maximum CPU and Memory requests on K8s
2022-07-28 14:18:27 +02:00
Felix Kunde a77d5df158
reverse membership for additional owner roles (#1862)
* reverse membership for additional owner roles
* remove type RoleOriginSpilo
* use e2e images with cron_admin inside
* let operator resolve reversed membership
* make additional owner roles part of the sync user strategy
* add more context in the docs about additional_owner_roles
2022-04-28 11:15:40 +02:00
Felix Kunde 0dc370f15d
standby cluster that streams from a remote primary (#1830)
* add the possibility to create a standby cluster that streams from a remote primary
* extending unit tests
* add more docs and e2e test

Co-authored-by: machine424 <ayoubmrini424@gmail.com>
2022-04-04 15:41:11 +02:00
Felix Kunde 60e0685c32
define readinessProbe on statefulSet (#1825)
* define readinessProbe on statefulSet 
* do not error out on deleting Patroni cluster objects
* change delete order for patroni objects
2022-03-30 18:19:34 +02:00
Felix Kunde 654d22d04a
Configure annotations to be ignored in comparisons during sync (#1823)
* feat: add ignored annotations when comparing during sync

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
Co-authored-by: Moshe Immerman <moshe@flanksource.com>
2022-03-24 18:38:37 +01:00
Felix Kunde a020708ef1
fix unit test and improve stability in e2e test (#1819)
* fix unit test and improve stability in e2e test
* fix resource handling
2022-03-21 10:05:20 +01:00
Felix Kunde 2719d411c3
grant db owners to cron_admin (#1805)
* grant db owners to cron_admin
* allow specifiying more extra owner roles
* add unit test for InitAdditionalOwnerRoles
* add e2e test
2022-03-18 12:36:12 +01:00
Felix Kunde 6ba05fee22
Pooler sync fix (#1811)
* always sync pooler objects
* do not capitalize log messages in Go
* stretch pooler e2e test
2022-03-17 19:22:18 +01:00
Felix Kunde d032e4783e
LoadBalancer toggles for master and replica pooler pods (#1799)
* Add support for pooler load balancer

Signed-off-by: Sergey Shatunov <me@prok.pw>

* Rename to enable_master_pooler_load_balancer

Signed-off-by: Sergey Shatunov <me@prok.pw>

* target port should be intval
* enhance pooler e2e test
* add new options to crds.go

Co-authored-by: Sergey Shatunov <me@prok.pw>
2022-03-04 13:36:17 +01:00
Felix Kunde 8b404fd049
minor fixes to password rotation (#1796)
* minor fixes to password rotation
* rework unit test
2022-02-25 17:46:26 +01:00
Felix Kunde 46547c4088
do not recreate pods if previous Patroni API calls fail (#1767)
* do not recreate pods if previous Patroni API calls fail
* move retry reads against Patroni API to pod.go
* remove final failover check in node affinity test
* make test_min_resource_limits more robust
2022-02-25 09:33:04 +01:00
Felix Kunde 3ce0b1e7fa
deprecate crd validation toggle and sync with manifests (#1781)
* deprecate crd validation toggle and sync with manifests
* fix description in pg crd manifests
* change CRD creation strategy
* affinity matchExpression has values
* lower repair period in e2e tests
2022-02-18 15:04:31 +01:00
Felix Kunde 658923d10d
Password rotation in secrets (#1749)
* password rotation in K8s secrets
* add db connection to syncSecrets
* add user retention
* add e2e test
* cleanup on username mismatch if rotation was switched off
* add unit test for syncSecrets + new updateSecret func
2022-02-18 11:54:47 +01:00
Felix Kunde a78a619e90
toleration diff and nodeReadinessLabel merge with manifest matchExpressions (#1729)
* include tolerations in statefulset comparison
* provide alternative merge behavior of nodeSelectorTerms for node readiness label
* add config option to change affinity merge behavior
* reworked e2e tests around node affinity
2022-01-27 15:57:24 +01:00
Felix Kunde 411abbe31e
handle case when Patroni returns that lag is unknown (#1724)
* handle case when Patroni returns that lag is unknown
* remove some prints from e2e test
2021-12-17 12:36:23 +01:00
Felix Kunde 087c379687
move to go 1.17.4 and update dependencies (#1717)
* move to go 1.17.4 and update dependencies
* use go install for mockgen and kind
2021-12-16 15:13:30 +01:00
Felix Kunde 1eafd688d0
restart master first in some edge cases (#1655)
* restart master first in some edge cases

* edge case is when desired is lower than effective

* wait after config patch and restart on sync whenever we see pending_restart

* convert options to int to check decrease and add unit test

* minor update to e2e tests

* wait only after restart not every sync

* using spilo 14 e2e images
2021-10-26 16:43:19 +02:00
Felix Kunde 6caf3a5359
e2e: wait for replica after failover before fetching leader (#1649) 2021-10-14 17:58:53 +02:00
Felix Kunde 2a33bf3313
improve Patroni config sync (#1635)
* improve Patroni config sync
* collect new and updated slots to patch patroni
* refactor httpGet in Patroni and extend unit tests
* GetMemberData should call the patroni endpoint
* add PATCH test
2021-10-13 17:17:26 +02:00
Felix Kunde e5fcb46ede
fix TestNodeAffinity in e2e tests (#1643)
* fix TestNodeAffinity in e2e tests
* skip node_readiness_label test for now
2021-10-13 10:52:36 +02:00
Felix Kunde ab25fb29b7
make Postgres 14 available (#1636)
* make Postgres 14 available
* don't be too hard to 9.5
* bump Spilo image and more docs updates
* update e2e test upgrading to 14
2021-10-12 12:00:59 +02:00
Felix Kunde f7b221de25
bump kind and use k8s = self.k8s in every test (#1640) 2021-10-08 17:24:17 +02:00
Felix Kunde 66620d5049
refactor restarting instances (#1535)
* refactor restarting instances and reduce listPods calls
* only add parameters to set if it differs from effective config
* update e2e test for updating Postgres config
* patch config only once
2021-08-09 16:23:41 +02:00
Felix Kunde 2057ad8191
reorder e2e tests to follow alphabetical sorting (#1567)
* reorder e2e tests to follow alphabetical sorting
* e2e: finish waiting for pod failover only if all pods were replaced
* wait for sync in rolling update timeout test
2021-07-27 11:41:27 +02:00
Felix Kunde 58bab073da
fix searching for users with namespace in name (#1569)
* fix searching for users with namespace in name and improve e2e test
* remove reformatting username to query
2021-07-27 09:46:55 +02:00
Rafia Sabih fa604027cf
Move flag to configmap (#1540)
* Move flag to configmap

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-07-02 08:46:21 +02:00
Igor Yanchenko ebb3204cdd
restart instances via rest api instead of recreating pods, fixes bug with being unable to decrease some values, like max_connections (#1103)
* restart instances via rest api instead of recreating pods
* Ignore differences in bootstrap.dcs when compare SPILO_CONFIGURATION
* isBootstrapOnlyParameter is rewritten, instead of whitelist it uses blacklist
* added e2e test for max_connections decreasing
* documentation updated
* pending_restart flag added to restart api call, wait fot ttl seconds after restart
* refactoring, /restart returns error if pending_restart is set to true and patroni is not pending restart
* restart postgresql instances within pods only if pod's restart is not required
* patroni might need to restart postgresql after pods were recreated if values like max_connections decreased
* instancesRestart is not critical, try to restart pods if not successful
* cleanup

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-06-14 11:00:58 +02:00
Rafia Sabih 75a9e2be38
Create cross namespace secrets (#1490)
* Create cross namespace secrets

* add test cases

* fixes

* Fixes
- include namespace in secret name only when namespace is provided
- use username.namespace as key to pgUsers only when namespace is
  provided
- avoid conflict in the role creation in db by checking namespace
  alongwith the username

* Update unit tests

* Fix test case

* Fixes

- update regular expression for usernames
- add test to allow check for valid usernames
- create pg roles with namespace (if any) appended in rolename

* add more test cases for valid usernames

* update docs

* fixes as per review comments

* update e2e

* fixes

* Add toggle to allow namespaced secrets

* update docs

* comment update

* Update e2e/tests/test_e2e.py

* few minor fixes

* fix unit tests

* fix e2e

* fix e2e attempt 2

* fix e2e

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-06-11 10:35:30 +02:00
Felix Kunde a37e78bd9a
bump operator to v1.6.3 (#1503) 2021-05-28 11:53:10 +02:00
Felix Kunde eeb59c5bfd
Rename roles that are removed from PostgresTeam CRD (#1457)
* rename db roles that are removed from manifests

* extend PostgresTeam e2e test

* make suffix configurable and add deprecated field to pgUser struct

* deny LOGIN from deprecated roles

* update feature documentation
2021-05-21 15:49:39 +02:00
Felix Kunde f0f7f25d30
Fix go lint errors (#1468)
* fix linter errors
* fix linter errors in kubectl plugin
* update PyYAML dependency in e2e tests
* declare a testVolume in volume_test
2021-05-10 11:48:03 +02:00
Felix Kunde ff8143770c
Improve rolling upgrades and rolling upgrade continue (#1341)
* add TODOs for moving rooling update label on pods
* steer rolling update via pod annotation
* rename patch method and fix reading flag on sync
* pass only pods to recreatePods function
* do not take address of iterator if you use it later
* add e2e test and pass switchover targets to recreatePods
* add wait_for_pod_failover for e2e test
* add one more e2e test case
* helm chart remove 1.6.0 archive from 1.6.0 archive
* reflect code review feedback
2021-02-26 15:38:58 +01:00
Jan Mussler 636a9a8191
Support major version upgrade via manifest and global upgrades via min version (#1372)
Support major version upgrade trigger via manifest. There is `off` `manual` and `full`. Manual is what you expect, and full will auto upgrade clusters below a certain threshold.
2021-02-25 11:42:43 +01:00
Felix Kunde 41858a702c
making pgTeamMap a pointer (#1349)
* making pgTeamMap a pointer
* init empty map
* add e2e test for additional teams and members
* update test_min_resource_limits
* add more waiting in node_affinity_test
* no need for pointers in map of postgresTeamMebership
* another minor update on node affinity test
* refactor and fix fetching additional members
2021-02-16 10:38:20 +01:00
Felix Kunde 12ad8c91fa
configurable container capabilities (#1336)
* configurable container capabilities

* revert change on TestTLS

* fix e2e test

* minor fix
2021-01-29 14:54:48 +01:00
Rafia Sabih e398cf8c7e
Avoid syncing when possible (#1274)
Avoid extra syncing in case there are no changes in pooler requirements.
Add pooler specific labels to pooler secrets.
Add test case to check for pooler secret creation and deletion.

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
2021-01-14 09:53:09 +01:00
Felix Kunde 102178409b
bump tp v1.6.0 (#1265)
* bump tp v1.6.0
* update logical-backup image
* Using smaller image for e2e test.
* fix env var name in docs
* add postgresql-client-13 to logical backup image

Co-authored-by: Jan Mussler <janm81@gmail.com>
2020-12-18 13:10:35 +01:00
Jan Mussler a63ad49ef8
Initial commit for new 1.6 release with Postgres 13 support. (#1257)
* Initial commit for new 1.6 release with Postgres 13 support.
* Updating maintainers, Go version, Codeowners.
* Use lazy upgrade image that contains pg13.
* fix typo for ownerReference
* fix clusterrole in helm chart
* reflect GCP logical backup in validation
* improve PostgresTeam docs
* change defaults for enable_pgversion_env_var and storage_resize_mode
* explain manual part of in-place upgrade
* remove gsoc docs

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-12-17 15:00:29 +01:00
Pavel Tumik 77252e316c
Add node affinity support (#1166)
* Adding nodeaffinity support alongside node_readiness_label
* add documentation for node affinity
* add node affinity e2e test
* add unit test for node affinity

Co-authored-by: Steffen Pøhner Henriksen <str3sses@gmail.com>
Co-authored-by: Adrian Astley <adrian.astley@activision.com>
2020-12-16 14:56:28 +01:00
Felix Kunde 6a97316a69
Support inherited annotations for all major objects (#1236)
* add comments where inherited annotations could be added

* add inheritedAnnotations feature

* return nil if no annotations are set

* minor changes

* first downscaler then inherited annotations

* add unit test for inherited annotations

* add pvc to test + minor changes

* missing comma

* fix nil map assignment

* set annotations in the same order it is done in other places

* replace acidClientSet with acid getters in K8s client

* more fixes on clientSet vs getters

* minor changes

* remove endpoints from annotation test

* refine unit test - but deployment and sts are still empty

* fix checkinng sts and deployment

* make annotations setter one liners

* no need for len check anymore

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
2020-12-11 16:34:01 +01:00
Sergey Dudoladov dc9a5b1e61
Introduce PGVERSION (#1172)
* introduce PGVERSION

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-11-27 18:49:49 +01:00