Commit Graph

152 Commits

Author SHA1 Message Date
Felix Kunde e34f19be01
update spec when updating status (#2546)
* update spec when updating status
* only setSpec of pg resource is not empty
2024-02-20 10:24:24 +01:00
Felix Kunde 886cb86797
allow users to opt out from globally enabled secret rotation (#2528)
* allow users to opt out from globally enabled secret rotation
* cover new option also in e2e test
* change ignore test to existing user
2024-02-09 12:19:06 +01:00
Felix Kunde 4a0c483514
add unit test and documentation for finalizers (#2509)
* add unit test and documentation for finalizers
* error msg with lower case and cover sync case
* try to avoid adding json-patch dependency
* use Update to remove finalizer
* changing status and finalizer during create
* do not call Delete() twice
2024-01-22 12:13:40 +01:00
Felix Kunde 3bad9aaded
fix when syncing standby discription (#2513) 2024-01-12 10:41:17 +01:00
Felix Kunde dad5b132ec
Standby cluster promotion by changing manifest (#2472)
* Standby cluster promotion by changing manifest
* Updated the documentation

---------

Co-authored-by: Senthilnathan M <snathanm@vmware.com>
2024-01-04 12:33:50 +01:00
Felix Kunde 1105228d3a
in sync mode select only syncStandby as switchover candidate (#2278)
* in sync mode select only syncStandby as swicthover candidate
* do not exit retry with err
* unit test: use error from reading byte stream twice
2023-04-06 12:04:55 +02:00
Felix Kunde 80fee5bda4
continue syncing databases and extensions on err (#2262) 2023-03-14 10:58:54 +01:00
Felix Kunde e6fb57a6bd
add c.replicationSlots on sync (#2238) 2023-02-23 13:19:35 +01:00
Felix Kunde 7a90fbcb00
fix sync of stream slots (#2194) 2023-01-27 18:03:37 +01:00
Felix Kunde 7887ebbbce
set wal_level config not on empty parameters map (#2189)
* set wal_level config not on empty parameters map
* UPDATE event must trigger statefulSet sync when streams are added
2023-01-26 09:43:03 +01:00
Felix Kunde b9165190e1
set wal_level for streams in statefulSet sync (#2187)
* set wal_level for streams in statefulSet sync
2023-01-25 17:06:31 +01:00
Felix Kunde 4741b3f734
copy rolconfig during password rotation (#2183)
* copy rolconfig during password rotation

Co-authored-by: idanovinda <idanovinda@gmail.com>
2023-01-25 10:48:23 +01:00
Felix Kunde a4f95e97e0
do not rotate secrets for standby clusters (#2175) 2023-01-17 12:58:14 +01:00
idanovinda 486d5d66e0
Allow drop slots when it gets deleted from the manifest (#2089)
* Allow drop slots when it gets deleted from the manifest
* use leader instead replica to query slots
* fix and extend unit tests for config update checks

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2023-01-03 15:46:59 +01:00
Polina Bungina 4d585250db
Add Patroni failsafe_mode parameter (#2076)
This commit adds support of a not-yet-released Patroni feature that allows postgres to run as primary in case of a failed leader lock update.
* Add Patroni 'failsafe_mode' local parameter (enable for a single PG cluster)
* Allow configuring Patroni 'failsafe_mode' parameter globally
2022-12-02 13:33:02 +01:00
Felix Kunde 70f3ee8e36
skip db sync on failed initUsers during UPDATE (#2083)
* skip db sync on failed initUsers during UPDATE
* provide unit test for teams API being unavailable
* add test for 404 case
2022-10-21 17:50:14 +02:00
Felix Kunde 4786f53f03
Fix password rotation (#2043)
* fix password rotation
* test connection with rotation user in e2e test + minor changes
2022-10-13 11:33:26 +02:00
Felix Kunde ce8b009c66
fix team member deprecation (#2072) 2022-10-11 18:02:41 +02:00
Philipp B 84fe38a069
switch to batch API v1 for Jobs (#2066) 2022-10-07 11:27:58 +02:00
Felix Kunde 2aa52094db
switch to policy API v1 for PDBs (#2008)
* switch to policy API v1 for PDBs
* update e2e test dependencies
* use kind 0.14.0
* bump K8s client in e2e docker image
* bump e2e tests-runner
2022-10-06 09:43:17 +02:00
Felix Kunde e0c4603057
create streams only after postgres instances were restarted (#2034)
* create streams only after postgres instances were restarted
* checkAndSetGlobalPostgreSQLConfiguration returns if config has been patched
* restart can be pending even without a config patch
2022-09-19 15:25:55 +02:00
Felix Kunde d209612b18 use correct keys in updateSecret (#2029) 2022-09-01 10:58:42 +02:00
Felix Kunde 21d00e2ed7
rework map selection in updateSecret (#2010) 2022-08-24 17:33:39 +02:00
Felix Kunde ef324494a0
fetch pooler and fes_user system user only when corresponding features are used (#2009)
* fetch pooler and fes_user system user only when corresponding features are used
* cover error case in unit test
* use string formatting instead of +
2022-08-24 16:28:49 +02:00
Felix Kunde b2642fa2fc
allow in place pw rotation of system users (#1953)
* allow in place pw rotation of system users
* block postgres user from rotation
* mark pooler pods for replacement
* adding podsGetter where pooler is synced in unit tests
* move rotation code in extra function
2022-08-18 14:14:31 +02:00
Felix Kunde 532772c5cd
do not call EBS api when there are no pvs (#1851)
* do not call EBS api when there are no pvs
* no extra aws api call in executeEBSMigration, operate on fetched cluster.EBSVolumes
2022-04-20 12:12:02 +02:00
Felix Kunde 654d22d04a
Configure annotations to be ignored in comparisons during sync (#1823)
* feat: add ignored annotations when comparing during sync

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
Co-authored-by: Moshe Immerman <moshe@flanksource.com>
2022-03-24 18:38:37 +01:00
Felix Kunde 36df1bc87c
refactor GenerateResourceRequirements and provide unit tests (#1822)
* refactor GenerateResourceRequirements and provide unit tests
2022-03-24 17:35:00 +01:00
Felix Kunde d032e4783e
LoadBalancer toggles for master and replica pooler pods (#1799)
* Add support for pooler load balancer

Signed-off-by: Sergey Shatunov <me@prok.pw>

* Rename to enable_master_pooler_load_balancer

Signed-off-by: Sergey Shatunov <me@prok.pw>

* target port should be intval
* enhance pooler e2e test
* add new options to crds.go

Co-authored-by: Sergey Shatunov <me@prok.pw>
2022-03-04 13:36:17 +01:00
Felix Kunde d8a159ef1a
create CDC event stream CRD (#1570)
* provide event stream API
* check manifest settings for logical decoding before creating streams
* operator updates Postgres config and creates replication user
* name FES like the Postgres cluster
* add delete case and fix updating streams + update unit test
* check if fes CRD exists before syncing
* existing slot must use the same plugin
* make id and payload columns configurable
* sync streams only when they are defined in manifest
* introduce applicationId for separate stream CRDs
* add FES to RBAC in chart
* disable streams in chart
* switch to pgoutput plugin and let operator create publications
* reflect code review and additional refactoring

Co-authored-by: Paŭlo Ebermann <paul.ebermann@zalando.de>
2022-02-28 10:09:42 +01:00
Felix Kunde 8b404fd049
minor fixes to password rotation (#1796)
* minor fixes to password rotation
* rework unit test
2022-02-25 17:46:26 +01:00
Felix Kunde 46547c4088
do not recreate pods if previous Patroni API calls fail (#1767)
* do not recreate pods if previous Patroni API calls fail
* move retry reads against Patroni API to pod.go
* remove final failover check in node affinity test
* make test_min_resource_limits more robust
2022-02-25 09:33:04 +01:00
Felix Kunde 658923d10d
Password rotation in secrets (#1749)
* password rotation in K8s secrets
* add db connection to syncSecrets
* add user retention
* add e2e test
* cleanup on username mismatch if rotation was switched off
* add unit test for syncSecrets + new updateSecret func
2022-02-18 11:54:47 +01:00
Felix Kunde 1eafd688d0
restart master first in some edge cases (#1655)
* restart master first in some edge cases

* edge case is when desired is lower than effective

* wait after config patch and restart on sync whenever we see pending_restart

* convert options to int to check decrease and add unit test

* minor update to e2e tests

* wait only after restart not every sync

* using spilo 14 e2e images
2021-10-26 16:43:19 +02:00
Felix Kunde 2a33bf3313
improve Patroni config sync (#1635)
* improve Patroni config sync
* collect new and updated slots to patch patroni
* refactor httpGet in Patroni and extend unit tests
* GetMemberData should call the patroni endpoint
* add PATCH test
2021-10-13 17:17:26 +02:00
Felix Kunde 62ed7e470f
improve pooler sync (#1593)
* remove role from installLookupFunction and run it on database sync, too
* fix condition to decide on syncing pooler
* trigger lookup from database sync only if pooler is set
* use empty spec everywhere and do not sync if one lookupfunction was passed
* do not sync pooler after being disabled
2021-08-27 12:41:37 +02:00
Felix Kunde 66620d5049
refactor restarting instances (#1535)
* refactor restarting instances and reduce listPods calls
* only add parameters to set if it differs from effective config
* update e2e test for updating Postgres config
* patch config only once
2021-08-09 16:23:41 +02:00
Felix Kunde 58bab073da
fix searching for users with namespace in name (#1569)
* fix searching for users with namespace in name and improve e2e test
* remove reformatting username to query
2021-07-27 09:46:55 +02:00
Felix Kunde 54e506c00b
define default access privileges for default users too (#1512)
* define default access privileges for default users too
* extend docs on defaultUsers
2021-06-22 16:45:28 +02:00
Igor Yanchenko ebb3204cdd
restart instances via rest api instead of recreating pods, fixes bug with being unable to decrease some values, like max_connections (#1103)
* restart instances via rest api instead of recreating pods
* Ignore differences in bootstrap.dcs when compare SPILO_CONFIGURATION
* isBootstrapOnlyParameter is rewritten, instead of whitelist it uses blacklist
* added e2e test for max_connections decreasing
* documentation updated
* pending_restart flag added to restart api call, wait fot ttl seconds after restart
* refactoring, /restart returns error if pending_restart is set to true and patroni is not pending restart
* restart postgresql instances within pods only if pod's restart is not required
* patroni might need to restart postgresql after pods were recreated if values like max_connections decreased
* instancesRestart is not critical, try to restart pods if not successful
* cleanup

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-06-14 11:00:58 +02:00
Rafia Sabih 75a9e2be38
Create cross namespace secrets (#1490)
* Create cross namespace secrets

* add test cases

* fixes

* Fixes
- include namespace in secret name only when namespace is provided
- use username.namespace as key to pgUsers only when namespace is
  provided
- avoid conflict in the role creation in db by checking namespace
  alongwith the username

* Update unit tests

* Fix test case

* Fixes

- update regular expression for usernames
- add test to allow check for valid usernames
- create pg roles with namespace (if any) appended in rolename

* add more test cases for valid usernames

* update docs

* fixes as per review comments

* update e2e

* fixes

* Add toggle to allow namespaced secrets

* update docs

* comment update

* Update e2e/tests/test_e2e.py

* few minor fixes

* fix unit tests

* fix e2e

* fix e2e attempt 2

* fix e2e

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-06-11 10:35:30 +02:00
Felix Kunde dd9c3907b7
pick first container if postgres is not found (#1505)
* pick first container if postgres is not found

* minor change
2021-05-28 11:44:10 +02:00
Felix Kunde 7884af2d59
get postgres container by name, not index (#1504) 2021-05-27 18:56:58 +02:00
Felix Kunde eeb59c5bfd
Rename roles that are removed from PostgresTeam CRD (#1457)
* rename db roles that are removed from manifests

* extend PostgresTeam e2e test

* make suffix configurable and add deprecated field to pgUser struct

* deny LOGIN from deprecated roles

* update feature documentation
2021-05-21 15:49:39 +02:00
Felix Kunde f0f7f25d30
Fix go lint errors (#1468)
* fix linter errors
* fix linter errors in kubectl plugin
* update PyYAML dependency in e2e tests
* declare a testVolume in volume_test
2021-05-10 11:48:03 +02:00
Felix Kunde 32e6c135b9
replace statefulset on annotation diff (#1449)
* replace statefulset on annotation diff
* remove update annotation function for statefulset
* add unit test for syncing annotations
* add inherited annotation to unit test
2021-04-22 11:22:52 +02:00
machine424 78bfba85d2
create global default privileges in the appropriate prepared databases (#1421) 2021-03-26 14:19:26 +01:00
Felix Kunde ff8143770c
Improve rolling upgrades and rolling upgrade continue (#1341)
* add TODOs for moving rooling update label on pods
* steer rolling update via pod annotation
* rename patch method and fix reading flag on sync
* pass only pods to recreatePods function
* do not take address of iterator if you use it later
* add e2e test and pass switchover targets to recreatePods
* add wait_for_pod_failover for e2e test
* add one more e2e test case
* helm chart remove 1.6.0 archive from 1.6.0 archive
* reflect code review feedback
2021-02-26 15:38:58 +01:00
Jan Mussler 636a9a8191
Support major version upgrade via manifest and global upgrades via min version (#1372)
Support major version upgrade trigger via manifest. There is `off` `manual` and `full`. Manual is what you expect, and full will auto upgrade clusters below a certain threshold.
2021-02-25 11:42:43 +01:00
Jan Mussler 772f0ca771
Fix volume sync order. (#1340) 2021-02-12 17:36:11 +01:00