Commit Graph

912 Commits

Author SHA1 Message Date
Sergey Dudoladov 2a21cc4393
Compare Postgres pod priority on Sync (#1144)
* compare Postgres pod priority on Sync

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-09-23 17:26:56 +02:00
neelasha-09 ab95eaa6ef
Fixes #1130 (#1139)
* Fixes #1130

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-09-22 17:16:05 +02:00
Rico Berger d09e418b56
Set user and group in security context (#1083)
* Set user and group in security context
2020-09-15 13:27:59 +02:00
Igor Yanchenko d8884a4003
Allow to overwrite default ExternalTrafficPolicy for the service (#1136)
* Allow to overwrite default ExternalTrafficPolicy for the service
2020-09-15 13:19:22 +02:00
Felix Kunde 03437b6374
Update issue templates (#1051)
* Update issue templates

To help us helping them

* update the template

* some updates

* or not on
2020-09-03 08:02:46 +02:00
hlihhovac e03e9f919a
add missing omitempty directive to the attributes of PostgresSpec (#1128)
Co-authored-by: Pavlo Golub <pavlo.golub@gmail.com>
2020-08-31 12:28:52 +02:00
Felix Kunde 5e93aabea6
improve e2e test debugging (#1107)
* print operator log in most tests when they time out
2020-08-28 14:57:19 +02:00
Felix Kunde 30c86758a3
update kind and use with old storage class (#1121)
* update kind and use with old storage class
* specify standard storage class in minimal manifest
* remove existing local storage class in kind
* fix pod distribution test
* exclude k8s master from nodes of interest
2020-08-28 12:16:37 +02:00
Felix Kunde 248ce9fc78
Update to go 1.14.7 (#1122)
* update go version, dependencies, and client-go 1.18.8
2020-08-26 14:00:14 +02:00
Peter Halliday dab704c566
Add kustomize support to Postgres UI. (#1086)
Co-authored-by: Peter Halliday <peter.halliday@walgreens.com>
2020-08-26 12:06:25 +02:00
Felix Kunde 3ddc56e5b9
allow delete only if annotations meet configured criteria (#1069)
* define annotations for delete protection

* change log level and reduce log lines for e2e tests

* reduce wait_for_pod_start even further
2020-08-13 16:36:22 +02:00
Jan Mussler 0d81f972a1
Added build and node directory to gitignore file. (#1102) 2020-08-12 15:45:00 +02:00
Felix Kunde 808030ad17
update go modules (#1097) 2020-08-12 15:37:40 +02:00
Sonay Şevik fc9ee76832
UI Service port forwarding internal port is updated to 80 from 8081. (#1096)
Fix #1093
2020-08-11 15:14:39 +02:00
Felix Kunde dfd0dd90ed
set search_path for default roles (#1065)
* set search_path for default roles

* deployment back to 1.5.0

Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-08-11 10:42:31 +02:00
Felix Kunde 0508266219
Remove all secrets on delete incl. pooler (#1091)
* fix syncSecrets and remove pooler secret

* update log for deleteSecret

* use c.credentialSecretName(username)

* minor fix
2020-08-10 18:26:26 +02:00
Felix Kunde 43163cf83b
allow using both infrastructure_roles_options (#1090)
* allow using both infrastructure_roles_options

* new default values for user and role definition

* use robot_zmon as parent role

* add operator log to debug

* right name for old secret

* only extract if rolesDefs is empty

* set password1 in old infrastructure role

* fix new infra rile secret

* choose different role key for new secret

* set memberof everywhere

* reenable all tests

* reflect feedback

* remove condition for rolesDefs
2020-08-10 15:08:03 +02:00
Dmitry Dolgov 7cf2fae6df
[WIP] Extend infrastructure roles handling (#1064)
Extend infrastructure roles handling

Postgres Operator uses infrastructure roles to provide access to a database for
external users e.g. for monitoring purposes. Such infrastructure roles are
expected to be present in the form of k8s secrets with the following content:

    inrole1: some_encrypted_role
    password1: some_encrypted_password
    user1: some_entrypted_name

    inrole2: some_encrypted_role
    password2: some_encrypted_password
    user2: some_entrypted_name

The format of this content is implied implicitly and not flexible enough. In
case if we do not have possibility to change the format of a secret we want to
use in the Operator, we need to recreate it in this format.

To address this lets make the format of secret content explicitly. The idea is
to introduce a new configuration option for the Operator.

    infrastructure_roles_secrets:
    - secretname: k8s_secret_name
      userkey: some_encrypted_name
      passwordkey: some_encrypted_password
      rolekey: some_encrypted_role

    - secretname: k8s_secret_name
      userkey: some_encrypted_name
      passwordkey: some_encrypted_password
      rolekey: some_encrypted_role

This would allow Operator to use any avalable secrets to prepare infrastructure
roles. To make it backward compatible simulate the old behaviour if the new
option is not present.

The new configuration option is intended be used mainly from CRD, but it's also
available via Operator ConfigMap in a limited fashion. For ConfigMap one can
put there only a string with one secret definition in the following format (as
a string):

    infrastructure_roles_secrets: |
        secretname: k8s_secret_name,
        userkey: some_encrypted_name,
        passwordkey: some_encrypted_password,
        rolekey: some_encrypted_role

Note than only one secret could be specified this way, no multiple secrets are
allowed.

Eventually the resulting list of infrastructure roles would be a total sum of
all supported ways to describe it, namely legacy via
infrastructure_roles_secret_name and infrastructure_roles_secrets from both
ConfigMap and CRD.
2020-08-05 14:18:56 +02:00
Felix Kunde f3ddce81d5
fix random order for pod environment tests (#1085) 2020-07-30 17:48:15 +02:00
hlihhovac 47b11f7f89
change Clone attribute of PostgresSpec to *CloneDescription (#1020)
* change Clone attribute of PostgresSpec to *ConnectionPooler

* update go.mod from master

* fix TestConnectionPoolerSynchronization()

* Update pkg/apis/acid.zalan.do/v1/postgresql_type.go

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>

Co-authored-by: Pavlo Golub <pavlo.golub@gmail.com>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-07-30 16:31:29 +02:00
Felix Kunde 3bee590d43
fix index in TestGenerateSpiloPodEnvVarswq (#1084)
Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-07-30 13:35:37 +02:00
Allison Richardet aab9b0aff9
chart ui: fix target namespace to allow '*' (#1082) 2020-07-30 11:08:33 +02:00
Christian Rohmann ece341d516
Allow pod environment variables to also be sourced from a secret (#946)
* Extend operator configuration to allow for a pod_environment_secret just like pod_environment_configmap

* Add all keys from PodEnvironmentSecrets as ENV vars (using SecretKeyRef to protect the value)

* Apply envVars from pod_environment_configmap and pod_environment_secrets before doing the global settings from the operator config. This allows them to be overriden by the user (via configmap / secret)

* Add ability use a Secret for custom pod envVars (via pod_environment_secret) to admin documentation

* Add pod_environment_secret to Helm chart values.yaml

* Add unit tests for PodEnvironmentConfigMap and PodEnvironmentSecret - highly inspired by @kupson and his very similar PR #481

* Added new parameter pod_environment_secret to operatorconfig CRD and configmap examples

* Add pod_environment_secret to the operationconfiguration CRD

Co-authored-by: Christian Rohmann <christian.rohmann@inovex.de>
2020-07-30 10:48:16 +02:00
Felix Kunde 102a353649
update dependencies (#1080) 2020-07-29 15:57:55 +02:00
Igor Yanchenko 002b47ec32
Use scram-sha-256 hash if postgresql parameter password_encryption set to do so. (#995)
* Use scram-sha-256 hash if postgresql parameter password_encryption set to do so.

* test fixed

* Refactoring

* code style
2020-07-16 14:43:57 +02:00
Toon Sevrin ec932f88d8
Port-forward service instead of pod (#1040) 2020-07-15 13:53:10 +02:00
Felix Kunde 375963424d
delete secrets the right way (#1054)
* delete secrets the right way

* make a one function

* continue deleting secrets even if one delete fails

Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-07-10 15:07:42 +02:00
Igor Yanchenko b80f9767d1
test coverage (#1055) 2020-07-10 09:07:25 +02:00
Felix Kunde c10d30903e
bump pgBouncer image (#1050)
Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-07-08 11:56:58 +02:00
Igor Yanchenko 88735a798a
Resize volume by changing pvc size if enabled in config. (#958)
* Try to resize pvc if resizing pv has failed

* added config option to switch between storage resize strategies

* changes according to requests

* Update pkg/controller/operator_config.go

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>

* enable_storage_resize documented

added examples to the default configuration and helm value files

* enable_storage_resize renamed to volume_resize_mode, off by default

* volume_resize_mode renamed to storage_resize_mode

* Update pkg/apis/acid.zalan.do/v1/crds.go

* pkg/cluster/volumes.go updated

* Update docs/reference/operator_parameters.md

* Update manifests/postgresql-operator-default-configuration.yaml

* Update pkg/controller/operator_config.go

* Update pkg/util/config/config.go

* Update charts/postgres-operator/values-crd.yaml

* Update charts/postgres-operator/values.yaml

* Update docs/reference/operator_parameters.md

* added logging if no changes required

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-07-03 10:53:37 +02:00
Jan Mussler 6869c2cf1b
Added image to readme, added/rewrote features. (#1031)
* Added image to readme, added/rewrote features.
2020-06-23 10:16:40 +02:00
Felix Kunde 0c6655a22d
skip creation later to improve visibility of errors (#1013)
* try to emit error for missing team name in cluster name

* skip creation after new cluster object

* move SetStatus to k8sclient and emit event when skipping creation and rename to SetPostgresCRDStatus

Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-06-17 13:32:16 +02:00
Felix Kunde 0e3fb9ec43
update dependencies (#1019)
Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-06-16 10:51:49 +02:00
Felix Kunde 3d976ebe8b
include volume in list of required fields (#1016)
Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-06-12 15:09:59 +02:00
Felix Kunde fa6929f028
do not block rolling updates with lazy spilo update enabled (#1012)
* do not block rolling updates with lazy spilo update enabled

* treat initContainers like Spilo image

Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-06-11 12:23:39 +02:00
Felix Kunde fe7ffaa112
trigger rolling update when securityContext of PodTemplate changes (#1007)
Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-06-09 10:27:57 +02:00
Felix Kunde 3c352fb460
bump pooler image and more coalescing for CRD config (#1004)
Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-06-05 11:14:17 +02:00
Kamil Solecki 9acdcd8bbf
Make selector match labels defined in the deployment (#1001)
Currently, the deployment manifest specifies two labels: `name` and `team`.
This fixes the service not matching the deployed pods by chosing a correct selector.
2020-06-04 16:49:22 +02:00
alfredw33 2b0def5bc8
Support for GCS WAL-E backups (#620)
* Support for WAL_GS_BUCKET and GOOGLE_APPLICATION_CREDENTIALS environtment variables

* Fixed merge issue but also removed all changes to support macos.

* Updated test to new format

* Missed macos specific changes

* Added documentation and addressed comments

* Update docs/administrator.md

* Update docs/administrator.md

* Update e2e/run.sh

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-06-03 17:33:48 +02:00
Steffen Pøhner Henriksen 0fa61a6ab3
Changed order of sidecar env vars (#980)
* Changed order of sidecar env vars

* Cleaned up test code
2020-05-25 16:32:33 +02:00
Felix Kunde 3a49b485e5
delete secrets of system users too (#974) 2020-05-14 11:34:02 +02:00
Christian Rohmann 8ff7658ed3
Fix pooler delete (#960)
deleteConnectionPooler function incorrectly checks that the delete api response is ResourceNotFound. Looks like the only consequence is a confusing log message, but obviously it's wrong. Remove negation, since having ResourceNotFound as error is the good case.

Co-authored-by: Christian Rohmann <christian.rohmann@inovex.de>
2020-05-13 14:55:54 +02:00
Ask Bjørn Hansen 852f29274a
Fix typo in error message (#969) 2020-05-12 10:05:42 +02:00
Damiano Albani a5bb8d913c
Fix typo (#965) 2020-05-12 09:20:09 +02:00
Felix Kunde 62bde6faa2
fix env var in UI chart (#967)
* fix env var in UI chart

* re-include 1.4.0 in helm chart index

* fix import in UI main.py and updated images
2020-05-08 13:02:27 +02:00
Felix Kunde bb3d2fa678
Bump v1.5.0 (#954)
* bump to v1.5.0

* update helm charts and docs

* update helm charts and packages

* update images for spilo, logical-backup and pooler
2020-05-05 12:52:54 +02:00
Felix Kunde 76d43525f7
define more default values for opConfig CRD (#955) 2020-05-04 16:23:21 +02:00
Rafia Sabih d52296c323
Propagate annotations to the StatefulSet (#932)
* Initial commit

* Corrections

- set the type of the new  configuration parameter to be array of
  strings
- propagate the annotations to statefulset at sync

* Enable regular expression matching

* Improvements

-handle rollingUpdate flag
-modularize code
-rename config parameter name

* fix merge error

* Pass annotations to connection pooler deployment

* update code-gen

* Add documentation and update manifests

* add e2e test and introduce option in configmap

* fix service annotations test

* Add unit test

* fix e2e tests

* better key lookup of annotations tests

* add debug message for annotation tests

* Fix typos

* minor fix for looping

* Handle update path and renaming

- handle the update path to update sts and connection pooler deployment.
  This way no need to wait for sync
- rename the parameter to downscaler_annotations
- handle other review comments

* another try to fix python loops

* Avoid unneccessary update events

* Update manifests

* some final polishing

* fix cluster_test after polishing

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-05-04 14:46:56 +02:00
Petr Barborka be208b61f1
Fix S3 backup list (#880)
* Fix S3 backup list

Co-authored-by: Petr Barborka <petr.barborka@orgis.cz>
2020-04-30 17:10:16 +02:00
Felix Kunde 5af4379118
[UI] add toggle for connection pooler (#953)
* [UI] add toggle for connection pooler

* remove team service logger

* fix new.tag.pug and change port in Makefile
2020-04-30 09:58:07 +02:00