Commit Graph
273 Commits
Author SHA1 Message Date
Felix Kunde 2aa52094db switch to policy API v1 for PDBs (#2008)
* switch to policy API v1 for PDBs
* update e2e test dependencies
* use kind 0.14.0
* bump K8s client in e2e docker image
* bump e2e tests-runner
2022-10-06 09:43:17 +02:00
Felix Kunde a119772efb add toggle to turn off readiness probes (#2004)
* add toggle to turn off readiness probes
* include PodManagementPolicy and ReadinessProbe in stateful set comparison
* add URI scheme to generated readiness probe
2022-10-05 18:25:24 +02:00
Felix Kunde 89375186b3 use old LB DNS format when teamId prefix is disabled (#2011)
* use old LB DNS format when teamId prefix is disabled
* support both old and new format in external-dns
* switch dns template from team to namespace
2022-08-25 18:29:54 +02:00
Felix Kunde ef324494a0 fetch pooler and fes_user system user only when corresponding features are used (#2009)
* fetch pooler and fes_user system user only when corresponding features are used
* cover error case in unit test
* use string formatting instead of +
2022-08-24 16:28:49 +02:00
Felix Kunde 88a2931550 bump pooler image to use new alpine base image (#1985)
* bump pooler image to use new alpine base image
* use a safe default for PGHOST pooler env variable
2022-08-08 17:36:43 +02:00
Jociele Padilha b41daf4f76 Set maximum CPU and Memory requests on K8s (#1959)
* Set maximum CPU and Memory requests on K8s
2022-07-28 14:18:27 +02:00
Felix Kunde 5e4badd99c annotation to bypass globally configured instance limits (#1943) 2022-06-30 10:40:03 +02:00
Felix Kunde 7d4da92872 bring back CLONE_WAL_BUCKET_SCOPE_PREFIX (#1902) 2022-05-24 16:27:34 +02:00
Felix Kunde c6f2c68588 ignore case when checking for envVar existence but do not change it (#1889) 2022-05-12 11:59:05 +02:00
Felix Kunde a77d5df158 reverse membership for additional owner roles (#1862)
* reverse membership for additional owner roles
* remove type RoleOriginSpilo
* use e2e images with cron_admin inside
* let operator resolve reversed membership
* make additional owner roles part of the sync user strategy
* add more context in the docs about additional_owner_roles
2022-04-28 11:15:40 +02:00
Felix Kunde 8b6664f1a2 fix container ports (#1864) 2022-04-21 18:52:53 +02:00
Felix Kunde eecd13169c refactor spilo env var generation (#1848)
* refactor spilo env generation
* enhance docs on env vars
* add unit test for appendEnvVar
2022-04-14 11:47:33 +02:00
Dmitry VolodinandFelix Kunde 9bcb25ac7e Ability to set pod environment variables on cluster resource (#1794)
* Ability to set pod environment variables on cluster resource

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2022-04-11 10:16:35 +02:00
Felix Kundeandmachine424 0dc370f15d standby cluster that streams from a remote primary (#1830)
* add the possibility to create a standby cluster that streams from a remote primary
* extending unit tests
* add more docs and e2e test

Co-authored-by: machine424 <ayoubmrini424@gmail.com>
2022-04-04 15:41:11 +02:00
Felix Kunde 60e0685c32 define readinessProbe on statefulSet (#1825)
* define readinessProbe on statefulSet 
* do not error out on deleting Patroni cluster objects
* change delete order for patroni objects
2022-03-30 18:19:34 +02:00
Felix Kunde 36df1bc87c refactor GenerateResourceRequirements and provide unit tests (#1822)
* refactor GenerateResourceRequirements and provide unit tests
2022-03-24 17:35:00 +01:00
Felix Kunde a020708ef1 fix unit test and improve stability in e2e test (#1819)
* fix unit test and improve stability in e2e test
* fix resource handling
2022-03-21 10:05:20 +01:00
Jakob Gillich f3b83c0b05 Fix empty resources spec field failing schema validation (#1589)
In Go, when a struct field is not set, it becomes a struct with
default values for all fields. These default values are included
during serialization. This causes issues with schema validation
where optional fields cannot be omitted because default values
are considered invalid.

This patch addresses this issue for `Resources` fields on several
types by using a pointer value.
2022-03-18 16:16:32 +01:00
Felix Kunde 2719d411c3 grant db owners to cron_admin (#1805)
* grant db owners to cron_admin
* allow specifiying more extra owner roles
* add unit test for InitAdditionalOwnerRoles
* add e2e test
2022-03-18 12:36:12 +01:00
Felix Kunde 6ba05fee22 Pooler sync fix (#1811)
* always sync pooler objects
* do not capitalize log messages in Go
* stretch pooler e2e test
2022-03-17 19:22:18 +01:00
Felix KundeandSergey Shatunov d032e4783e LoadBalancer toggles for master and replica pooler pods (#1799)
* Add support for pooler load balancer

Signed-off-by: Sergey Shatunov <me@prok.pw>

* Rename to enable_master_pooler_load_balancer

Signed-off-by: Sergey Shatunov <me@prok.pw>

* target port should be intval
* enhance pooler e2e test
* add new options to crds.go

Co-authored-by: Sergey Shatunov <me@prok.pw>
2022-03-04 13:36:17 +01:00
A. StoewerandFelix Kunde 695ad44caf Logical backup retention time (#1337)
* Add optional logical backup retention time
* Set defaults for potentially unbound variables, so that the script will work with older operator versions
* Document retention time parameter for logical backups
* Add retention time parameter to resources and charts

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2022-03-02 17:39:33 +01:00
david amick ca0c27a51b Retry when getting the pod_environment_secret (#1777)
* Retry when getting the pod_environment_secret
2022-03-01 17:56:16 +01:00
Dmitry Volodin da83982313 inherited_labels and inherited_annotations not passed to PVC (#1784)
* inherited_labels and inherited_annotations not passed to PVC
* Fix developer.md related to the local operator deployment
2022-03-01 17:07:37 +01:00
MenzorgandFelix Kunde 06c28da97d synchronous_node_count support (#1484)
* synchronous_node_count support
* notification about Patroni image version
* default synchronous_node_count to 1

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2022-02-25 17:21:42 +01:00
Felix Kunde a78a619e90 toleration diff and nodeReadinessLabel merge with manifest matchExpressions (#1729)
* include tolerations in statefulset comparison
* provide alternative merge behavior of nodeSelectorTerms for node readiness label
* add config option to change affinity merge behavior
* reworked e2e tests around node affinity
2022-01-27 15:57:24 +01:00
James McDonaldandFelix Kunde def9e1d688 Support standby replication from GS (GCS) (#1446)
* Add support for manual gs_wal_path in standby
* Remove separate standby version configuration
* Remove setting standby path via cluster/uid/version
Picking up the version doesn't work reliably without making changes to
Spilo. It's clearer to just specify the full S3/GS bucket path.

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-12-03 11:24:29 +01:00
Felix Kunde 2a33bf3313 improve Patroni config sync (#1635)
* improve Patroni config sync
* collect new and updated slots to patch patroni
* refactor httpGet in Patroni and extend unit tests
* GetMemberData should call the patroni endpoint
* add PATCH test
2021-10-13 17:17:26 +02:00
Aaron Peschel 1dd0cd9691 Add Support for Azure WAL-G Backups (#1537)
This commit adds support for using an Azure storage account as a backup
location.

It uses the existing GCS functionality as a reference for what to do,
and follows the example set by GCS as closely as possible.

The decision to name the cloud provider key "aws_or_gcp" is unfortunate
while adding support for Azure, but I have left it alone to allow for
this changeset to be backwards compatible.
2021-08-26 14:59:03 +02:00
John RoodandJohn Rood 2d2ce6197b Add volume selector (#1385)
* Add volume selector
* Add slightly better documentation and gofmt changes
* Update generated deepcopy
* Add test for PV selector

Co-authored-by: John Rood <j.rood@picturae.com>
2021-08-26 14:57:54 +02:00
Felix Kunde f0815fc5bd remove debug log of Spilo env vars (#1591) 2021-08-23 15:44:34 +02:00
Igor YanchenkoandFelix Kunde ebb3204cdd restart instances via rest api instead of recreating pods, fixes bug with being unable to decrease some values, like max_connections (#1103)
* restart instances via rest api instead of recreating pods
* Ignore differences in bootstrap.dcs when compare SPILO_CONFIGURATION
* isBootstrapOnlyParameter is rewritten, instead of whitelist it uses blacklist
* added e2e test for max_connections decreasing
* documentation updated
* pending_restart flag added to restart api call, wait fot ttl seconds after restart
* refactoring, /restart returns error if pending_restart is set to true and patroni is not pending restart
* restart postgresql instances within pods only if pod's restart is not required
* patroni might need to restart postgresql after pods were recreated if values like max_connections decreased
* instancesRestart is not critical, try to restart pods if not successful
* cleanup

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-06-14 11:00:58 +02:00
75a9e2be38 Create cross namespace secrets (#1490)
* Create cross namespace secrets

* add test cases

* fixes

* Fixes
- include namespace in secret name only when namespace is provided
- use username.namespace as key to pgUsers only when namespace is
  provided
- avoid conflict in the role creation in db by checking namespace
  alongwith the username

* Update unit tests

* Fix test case

* Fixes

- update regular expression for usernames
- add test to allow check for valid usernames
- create pg roles with namespace (if any) appended in rolename

* add more test cases for valid usernames

* update docs

* fixes as per review comments

* update e2e

* fixes

* Add toggle to allow namespaced secrets

* update docs

* comment update

* Update e2e/tests/test_e2e.py

* few minor fixes

* fix unit tests

* fix e2e

* fix e2e attempt 2

* fix e2e

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-06-11 10:35:30 +02:00
Felix Kunde dd9c3907b7 pick first container if postgres is not found (#1505)
* pick first container if postgres is not found

* minor change
2021-05-28 11:44:10 +02:00
Felix Kunde 7884af2d59 get postgres container by name, not index (#1504) 2021-05-27 18:56:58 +02:00
Quan HoangandFelix Kunde af5378eea5 Mount additional volumes to 'postgres' container when 'targetContains` is an empty list (#1475)
* Mount additional volumes to 'postgres' container when 'targetContainers' is an empty list

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-05-27 14:56:14 +02:00
Felix Kunde f0f7f25d30 Fix go lint errors (#1468)
* fix linter errors
* fix linter errors in kubectl plugin
* update PyYAML dependency in e2e tests
* declare a testVolume in volume_test
2021-05-10 11:48:03 +02:00
Felix Kunde c18241f187 Bump v1.6.2 (#1433)
* helm chart remove 1.6.0 archive from 1.6.0 archive

* bump operator to v1.6.2

* fix pointer deref

* skip connection pooler sync when empty

* revert pooler change and minor update to version msg

* do not log query on error when creating or altering users
2021-04-01 11:53:07 +02:00
9e93c0a4ef Fix for AllowPrivilegeEscalation : issue-1403 (#1412)
* Fix for AllowPrivilegeEscalation : issue-1403

* fixed syntax error

* Aligned the value for parameter

* Aligned the value for parameter

* Update crds.go

* Aligned the parameter spilo_allow_privilege_escalation

* Parameters sorted in Alphabetical order in manifests yaml

* Parameters sorted in Alphabetical order in manifests yaml

* Update pkg/controller/operator_config.go

* Update docs/reference/operator_parameters.md

Co-authored-by: Neelam Sharma <neelasha@amdocs.com>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2021-03-29 10:37:59 +02:00
Felix Kunde ff8143770c Improve rolling upgrades and rolling upgrade continue (#1341)
* add TODOs for moving rooling update label on pods
* steer rolling update via pod annotation
* rename patch method and fix reading flag on sync
* pass only pods to recreatePods function
* do not take address of iterator if you use it later
* add e2e test and pass switchover targets to recreatePods
* add wait_for_pod_failover for e2e test
* add one more e2e test case
* helm chart remove 1.6.0 archive from 1.6.0 archive
* reflect code review feedback
2021-02-26 15:38:58 +01:00
Jan Mussler 636a9a8191 Support major version upgrade via manifest and global upgrades via min version (#1372)
Support major version upgrade trigger via manifest. There is `off` `manual` and `full`. Manual is what you expect, and full will auto upgrade clusters below a certain threshold.
2021-02-25 11:42:43 +01:00
Felix Kunde ca968ca150 Fix empty capabilities (#1380)
* helm chart remove 1.6.0 archive from 1.6.0 archive
* empty pod capabilities should be nil
2021-02-22 17:27:32 +01:00
Michael Seiwald 17da6bc649 Truncate cronjob name at 52 characters (#1208) 2021-02-15 17:00:21 +01:00
zvier 6aeb92f024 code optimization (#1350)
* pre-allocate cap for slice structure
* if clause is no need because of range, and kubelet also use range
  method to get each capability so there is no side-effect

Signed-off-by: Jeff Zvier <zvier20@gmail.com>
2021-02-09 09:35:24 +01:00
Felix Kunde 0cce565b65 fix when adding only one capability (#1339)
* fix when adding only one capability

* fix error messages in unit test
2021-01-29 16:10:27 +01:00
Felix Kunde 12ad8c91fa configurable container capabilities (#1336)
* configurable container capabilities

* revert change on TestTLS

* fix e2e test

* minor fix
2021-01-29 14:54:48 +01:00
Felix Kunde 4ea0b5f432 set AllowPrivilegeEscalation on container securityContext (#1326) 2021-01-22 14:06:19 +01:00
Rafia SabihandRafia Sabih e398cf8c7e Avoid syncing when possible (#1274)
Avoid extra syncing in case there are no changes in pooler requirements.
Add pooler specific labels to pooler secrets.
Add test case to check for pooler secret creation and deletion.

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
2021-01-14 09:53:09 +01:00
Sergey DudoladovandSergey Dudoladov 168b679506 add a prefix for the name of a logical backup job (#1287)
* add a prefix for the name of a logical backup job

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2021-01-07 10:38:07 +01:00
Felix Kunde 07c4f52ede use pointer type for nodeAffinity (#1263)
* use pointer type for nodeAffinity
2020-12-18 12:07:59 +01:00