Commit Graph

934 Commits

Author SHA1 Message Date
Rafia Sabih d2c410d72b Add labels 2020-10-05 12:10:02 +02:00
Rafia Sabih a6ffdbae36 Cleanup deleteConnectionPooler 2020-10-05 12:06:58 +02:00
Rafia Sabih 86e6a51fa9 Merge branch 'master' into replica-pooler 2020-10-05 11:47:47 +02:00
Sergey Dudoladov 38e15183a2
update kind (#1156)
Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-10-02 09:31:55 +02:00
Felix Kunde 21475f4547
Cleanup config examples (#1151)
* post polishing for latest PRs

* update travis and go modules

* make deprecation comments in structs less confusing

* have separate pod priority class es for operator and database pods
2020-09-30 17:24:14 +02:00
Sergey Dudoladov 3b6dc4f92d
Improve e2e tests (#1111)
* icnrease vm size

* cache deps

* switch to the absolute cache path as cdp does not support shell expansion

* do not pull non-existing image

* manually install kind

* add alias to kind

* use full kind name

* one more name change

* install kind with other tools

* add bind mounts instead of copying files

* test fetching the runner image

* build image for pierone

* bump up the client-go version to match the master

* bump up go version

* install pinned version of kind before any test run

* do not overwrite local ./manifests during test run

* update the docs

* fix kind name

* update go.* files

* fix deps

* avoid unnecessary image upload

* properly install kind

* Change network to host to make it reachable within e2e runner. May not be the right solution though.

* Small changes. Also use entrypoint vs cmd.

* Bumping spilo. Load before test.

* undo incorrect merge from the master

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
Co-authored-by: Jan Mußler <janm81@gmail.com>
2020-09-25 14:14:19 +02:00
Sergey Dudoladov ffdb47f53a
remove outdated GSOC info (#1148)
Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-09-25 09:46:50 +02:00
Sergey Dudoladov 2a21cc4393
Compare Postgres pod priority on Sync (#1144)
* compare Postgres pod priority on Sync

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-09-23 17:26:56 +02:00
Rafia Sabih 683cb15b0d Add and update tests 2020-09-23 17:09:36 +02:00
Rafia Sabih 1ee79887cf Add sync test 2020-09-23 11:19:01 +02:00
neelasha-09 ab95eaa6ef
Fixes #1130 (#1139)
* Fixes #1130

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-09-22 17:16:05 +02:00
Rafia Sabih 033a7f4e24 Update tests 2020-09-21 22:55:10 +02:00
Rafia Sabih 2936ed0060 Refactor needConnectionPooler
Have one unified function to tell if any connection pooler is required

Add a helper function to list the roles that require connection pooler,
helps in avoiding duplication of code
2020-09-21 16:40:40 +02:00
Rico Berger d09e418b56
Set user and group in security context (#1083)
* Set user and group in security context
2020-09-15 13:27:59 +02:00
Igor Yanchenko d8884a4003
Allow to overwrite default ExternalTrafficPolicy for the service (#1136)
* Allow to overwrite default ExternalTrafficPolicy for the service
2020-09-15 13:19:22 +02:00
Rafia Sabih 7c9b459919 Minor fix 2020-09-11 10:22:29 +02:00
Rafia Sabih 46ff4bb738 Resolve review comments 2020-09-11 10:17:43 +02:00
Rafia Sabih 770fc1e612 Improvements in tests
- Fixed the issue with failing test cases
- Add more test cases for replica connection pooler
- Added docs about the new flag
2020-09-08 17:32:47 +02:00
Rafia Sabih b3dbac5b81 Adding test cases and other changes
- Refactor needConnectionPooler for master and replica separately
- Improve sync function
- Add test cases to create, delete and sync with repplica connection
  pooler

Other changes
2020-09-04 17:32:35 +02:00
Rafia Sabih 374dd00538 Fix sync 2020-09-04 08:02:27 +02:00
Rafia Sabih 1814342dc3 Assorted changes
- Update deleteConnectionPooler to include role
- Rename EnableMasterConnectionPooler back to original name for backward
  compatiility
- other minor chnages and code improvements
2020-09-03 16:01:25 +02:00
Rafia Sabih 503082cf1a fix for labels selector 2020-09-03 09:32:48 +02:00
Felix Kunde 03437b6374
Update issue templates (#1051)
* Update issue templates

To help us helping them

* update the template

* some updates

* or not on
2020-09-03 08:02:46 +02:00
Rafia Sabih a9248b1379 Fix labels for the replica pods 2020-09-02 17:17:56 +02:00
Rafia Sabih fb49376085 Enable connection pooler for replica
- Refactor code for connection pooler deployment and services
- Refactor sync code for connection pooler
- Rename EnableConnectionPooler to EnableMasterConnectionPooler
- Update yamls and tests
2020-09-02 13:46:36 +02:00
hlihhovac e03e9f919a
add missing omitempty directive to the attributes of PostgresSpec (#1128)
Co-authored-by: Pavlo Golub <pavlo.golub@gmail.com>
2020-08-31 12:28:52 +02:00
Felix Kunde 5e93aabea6
improve e2e test debugging (#1107)
* print operator log in most tests when they time out
2020-08-28 14:57:19 +02:00
Rafia Sabih 83ddd5c85b Add new pooler service for replica 2020-08-28 14:55:49 +02:00
Rafia Sabih 3a906aba93 Add pooler for replica 2020-08-28 13:25:46 +02:00
Felix Kunde 30c86758a3
update kind and use with old storage class (#1121)
* update kind and use with old storage class
* specify standard storage class in minimal manifest
* remove existing local storage class in kind
* fix pod distribution test
* exclude k8s master from nodes of interest
2020-08-28 12:16:37 +02:00
Felix Kunde 248ce9fc78
Update to go 1.14.7 (#1122)
* update go version, dependencies, and client-go 1.18.8
2020-08-26 14:00:14 +02:00
Peter Halliday dab704c566
Add kustomize support to Postgres UI. (#1086)
Co-authored-by: Peter Halliday <peter.halliday@walgreens.com>
2020-08-26 12:06:25 +02:00
Felix Kunde 3ddc56e5b9
allow delete only if annotations meet configured criteria (#1069)
* define annotations for delete protection

* change log level and reduce log lines for e2e tests

* reduce wait_for_pod_start even further
2020-08-13 16:36:22 +02:00
Jan Mussler 0d81f972a1
Added build and node directory to gitignore file. (#1102) 2020-08-12 15:45:00 +02:00
Felix Kunde 808030ad17
update go modules (#1097) 2020-08-12 15:37:40 +02:00
Sonay Şevik fc9ee76832
UI Service port forwarding internal port is updated to 80 from 8081. (#1096)
Fix #1093
2020-08-11 15:14:39 +02:00
Felix Kunde dfd0dd90ed
set search_path for default roles (#1065)
* set search_path for default roles

* deployment back to 1.5.0

Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-08-11 10:42:31 +02:00
Felix Kunde 0508266219
Remove all secrets on delete incl. pooler (#1091)
* fix syncSecrets and remove pooler secret

* update log for deleteSecret

* use c.credentialSecretName(username)

* minor fix
2020-08-10 18:26:26 +02:00
Felix Kunde 43163cf83b
allow using both infrastructure_roles_options (#1090)
* allow using both infrastructure_roles_options

* new default values for user and role definition

* use robot_zmon as parent role

* add operator log to debug

* right name for old secret

* only extract if rolesDefs is empty

* set password1 in old infrastructure role

* fix new infra rile secret

* choose different role key for new secret

* set memberof everywhere

* reenable all tests

* reflect feedback

* remove condition for rolesDefs
2020-08-10 15:08:03 +02:00
Dmitry Dolgov 7cf2fae6df
[WIP] Extend infrastructure roles handling (#1064)
Extend infrastructure roles handling

Postgres Operator uses infrastructure roles to provide access to a database for
external users e.g. for monitoring purposes. Such infrastructure roles are
expected to be present in the form of k8s secrets with the following content:

    inrole1: some_encrypted_role
    password1: some_encrypted_password
    user1: some_entrypted_name

    inrole2: some_encrypted_role
    password2: some_encrypted_password
    user2: some_entrypted_name

The format of this content is implied implicitly and not flexible enough. In
case if we do not have possibility to change the format of a secret we want to
use in the Operator, we need to recreate it in this format.

To address this lets make the format of secret content explicitly. The idea is
to introduce a new configuration option for the Operator.

    infrastructure_roles_secrets:
    - secretname: k8s_secret_name
      userkey: some_encrypted_name
      passwordkey: some_encrypted_password
      rolekey: some_encrypted_role

    - secretname: k8s_secret_name
      userkey: some_encrypted_name
      passwordkey: some_encrypted_password
      rolekey: some_encrypted_role

This would allow Operator to use any avalable secrets to prepare infrastructure
roles. To make it backward compatible simulate the old behaviour if the new
option is not present.

The new configuration option is intended be used mainly from CRD, but it's also
available via Operator ConfigMap in a limited fashion. For ConfigMap one can
put there only a string with one secret definition in the following format (as
a string):

    infrastructure_roles_secrets: |
        secretname: k8s_secret_name,
        userkey: some_encrypted_name,
        passwordkey: some_encrypted_password,
        rolekey: some_encrypted_role

Note than only one secret could be specified this way, no multiple secrets are
allowed.

Eventually the resulting list of infrastructure roles would be a total sum of
all supported ways to describe it, namely legacy via
infrastructure_roles_secret_name and infrastructure_roles_secrets from both
ConfigMap and CRD.
2020-08-05 14:18:56 +02:00
Felix Kunde f3ddce81d5
fix random order for pod environment tests (#1085) 2020-07-30 17:48:15 +02:00
hlihhovac 47b11f7f89
change Clone attribute of PostgresSpec to *CloneDescription (#1020)
* change Clone attribute of PostgresSpec to *ConnectionPooler

* update go.mod from master

* fix TestConnectionPoolerSynchronization()

* Update pkg/apis/acid.zalan.do/v1/postgresql_type.go

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>

Co-authored-by: Pavlo Golub <pavlo.golub@gmail.com>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-07-30 16:31:29 +02:00
Felix Kunde 3bee590d43
fix index in TestGenerateSpiloPodEnvVarswq (#1084)
Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-07-30 13:35:37 +02:00
Allison Richardet aab9b0aff9
chart ui: fix target namespace to allow '*' (#1082) 2020-07-30 11:08:33 +02:00
Christian Rohmann ece341d516
Allow pod environment variables to also be sourced from a secret (#946)
* Extend operator configuration to allow for a pod_environment_secret just like pod_environment_configmap

* Add all keys from PodEnvironmentSecrets as ENV vars (using SecretKeyRef to protect the value)

* Apply envVars from pod_environment_configmap and pod_environment_secrets before doing the global settings from the operator config. This allows them to be overriden by the user (via configmap / secret)

* Add ability use a Secret for custom pod envVars (via pod_environment_secret) to admin documentation

* Add pod_environment_secret to Helm chart values.yaml

* Add unit tests for PodEnvironmentConfigMap and PodEnvironmentSecret - highly inspired by @kupson and his very similar PR #481

* Added new parameter pod_environment_secret to operatorconfig CRD and configmap examples

* Add pod_environment_secret to the operationconfiguration CRD

Co-authored-by: Christian Rohmann <christian.rohmann@inovex.de>
2020-07-30 10:48:16 +02:00
Felix Kunde 102a353649
update dependencies (#1080) 2020-07-29 15:57:55 +02:00
Igor Yanchenko 002b47ec32
Use scram-sha-256 hash if postgresql parameter password_encryption set to do so. (#995)
* Use scram-sha-256 hash if postgresql parameter password_encryption set to do so.

* test fixed

* Refactoring

* code style
2020-07-16 14:43:57 +02:00
Toon Sevrin ec932f88d8
Port-forward service instead of pod (#1040) 2020-07-15 13:53:10 +02:00
Felix Kunde 375963424d
delete secrets the right way (#1054)
* delete secrets the right way

* make a one function

* continue deleting secrets even if one delete fails

Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-07-10 15:07:42 +02:00
Igor Yanchenko b80f9767d1
test coverage (#1055) 2020-07-10 09:07:25 +02:00