Commit Graph

939 Commits

Author SHA1 Message Date
Jan Mußler d88e62fc79 Fixing yaml dump. Removing restart pending between tests. 2020-10-23 01:09:02 +02:00
Jan Mußler aa3100ca34 Mount script with tools for easy logaccess and watching objects. 2020-10-22 23:35:43 +02:00
Jan Mußler 0143a470b1 Mostly cosmetic changes to logs. Removed quotes from diff. Move all object diffs to text diff. Enabled padding for log level. 2020-10-22 22:30:34 +02:00
Jan Mußler 2aeaad03f3 Minor changes around running pods and catching error in infrastructure roles. 2020-10-22 16:12:12 +02:00
Jan Mußler 1f3730b2b4 More tests and more nice diff. 2020-10-21 23:30:35 +02:00
Jan Mußler e40abdb249 Move scale function. 2020-10-21 18:12:33 +02:00
Jan Mußler b422cf9d71 Comments updated. 2020-10-21 18:10:20 +02:00
Jan Mußler 6b91bd3282 More e2e changes for scale up and down. 2020-10-21 17:58:16 +02:00
Jan Mußler 39641e81ea Fixing annotations key. 2020-10-21 17:44:21 +02:00
Jan Mußler f03409da06 Fix min resurces end to end test. 2020-10-21 17:27:00 +02:00
Jan Mußler 9b596f1eb7 Extending timeout, allow one sync. 2020-10-21 15:39:09 +02:00
Jan Mußler 2066256a17 Progressing on faster e2e tests. 2020-10-21 15:23:17 +02:00
Jan Mußler 668ef51d9f Printing config as multi log line entity, makes it readable and grepable on startup 2020-10-21 08:33:31 +02:00
Jan Mußler c6c4c4cc8a * Make lazy upgrade test work reliable
* Allow Docker image to take parameters to overwrite unittest
execution
  * Add documentation for running individual tests
  * Fixed String encoding in Patorni state check and error case
2020-10-20 19:20:38 +02:00
Jan Mußler 4fc8ca384d Fix distribution call. 2020-10-19 23:47:40 +02:00
Jan Mußler 966575dd4b * Patroni state function added in k8s
* Lazy upgrade now properly covered with eventual and waiting for pod
start
 * patching config now updates deployment, patching annotation, allowing
to trace change step
 * run.sh no takes NOCLEANUP to stop kind from being deleted
 * if kind config is present, run will not install kind
 * Fast e2e local execution now possible once kind is up
2020-10-19 23:35:08 +02:00
Jan Mußler c1ad71668b WIP 2020-10-19 14:09:22 +02:00
Jan Mußler ccde8c6bf6 More fixes for e2e tests. 2020-10-19 13:53:48 +02:00
Jan Mußler 38e6261d64 Loadbalancer test now uses eventualEqual properly. 2020-10-19 10:26:41 +02:00
Jan Mußler cc4bfb0cfb Build docker image and changed back to os image. 2020-10-18 19:56:54 +02:00
Jan Mussler 21afc07d9f Improving end 2 end tests by implementing proper eventual asserts and timeouts. 2020-10-18 19:23:17 +02:00
Dmitry Dolgov d15f2d3392
Readiness probe (#1169)
Right now there are no readiness probes defined for connection pooler,
which means after a pod restart there is a short time window (between a
container start and connection pooler starting listening to a socket)
when a service can send queries to a new pod, but connection will be
refused. The pooler container is rather lightweight and it start to
listen immediately, so the time window is small, but still.

To fix this add a readiness probe for tcp socket opened by connection
pooler.
2020-10-15 10:16:42 +02:00
Alex Stockinger 692c721854
Introduce ENABLE_JSON_LOGGING env variable (#1158) 2020-10-08 15:32:15 +02:00
Sergey Dudoladov 38e15183a2
update kind (#1156)
Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-10-02 09:31:55 +02:00
Felix Kunde 21475f4547
Cleanup config examples (#1151)
* post polishing for latest PRs

* update travis and go modules

* make deprecation comments in structs less confusing

* have separate pod priority class es for operator and database pods
2020-09-30 17:24:14 +02:00
Sergey Dudoladov 3b6dc4f92d
Improve e2e tests (#1111)
* icnrease vm size

* cache deps

* switch to the absolute cache path as cdp does not support shell expansion

* do not pull non-existing image

* manually install kind

* add alias to kind

* use full kind name

* one more name change

* install kind with other tools

* add bind mounts instead of copying files

* test fetching the runner image

* build image for pierone

* bump up the client-go version to match the master

* bump up go version

* install pinned version of kind before any test run

* do not overwrite local ./manifests during test run

* update the docs

* fix kind name

* update go.* files

* fix deps

* avoid unnecessary image upload

* properly install kind

* Change network to host to make it reachable within e2e runner. May not be the right solution though.

* Small changes. Also use entrypoint vs cmd.

* Bumping spilo. Load before test.

* undo incorrect merge from the master

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
Co-authored-by: Jan Mußler <janm81@gmail.com>
2020-09-25 14:14:19 +02:00
Sergey Dudoladov ffdb47f53a
remove outdated GSOC info (#1148)
Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-09-25 09:46:50 +02:00
Sergey Dudoladov 2a21cc4393
Compare Postgres pod priority on Sync (#1144)
* compare Postgres pod priority on Sync

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-09-23 17:26:56 +02:00
neelasha-09 ab95eaa6ef
Fixes #1130 (#1139)
* Fixes #1130

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-09-22 17:16:05 +02:00
Rico Berger d09e418b56
Set user and group in security context (#1083)
* Set user and group in security context
2020-09-15 13:27:59 +02:00
Igor Yanchenko d8884a4003
Allow to overwrite default ExternalTrafficPolicy for the service (#1136)
* Allow to overwrite default ExternalTrafficPolicy for the service
2020-09-15 13:19:22 +02:00
Felix Kunde 03437b6374
Update issue templates (#1051)
* Update issue templates

To help us helping them

* update the template

* some updates

* or not on
2020-09-03 08:02:46 +02:00
hlihhovac e03e9f919a
add missing omitempty directive to the attributes of PostgresSpec (#1128)
Co-authored-by: Pavlo Golub <pavlo.golub@gmail.com>
2020-08-31 12:28:52 +02:00
Felix Kunde 5e93aabea6
improve e2e test debugging (#1107)
* print operator log in most tests when they time out
2020-08-28 14:57:19 +02:00
Felix Kunde 30c86758a3
update kind and use with old storage class (#1121)
* update kind and use with old storage class
* specify standard storage class in minimal manifest
* remove existing local storage class in kind
* fix pod distribution test
* exclude k8s master from nodes of interest
2020-08-28 12:16:37 +02:00
Felix Kunde 248ce9fc78
Update to go 1.14.7 (#1122)
* update go version, dependencies, and client-go 1.18.8
2020-08-26 14:00:14 +02:00
Peter Halliday dab704c566
Add kustomize support to Postgres UI. (#1086)
Co-authored-by: Peter Halliday <peter.halliday@walgreens.com>
2020-08-26 12:06:25 +02:00
Felix Kunde 3ddc56e5b9
allow delete only if annotations meet configured criteria (#1069)
* define annotations for delete protection

* change log level and reduce log lines for e2e tests

* reduce wait_for_pod_start even further
2020-08-13 16:36:22 +02:00
Jan Mussler 0d81f972a1
Added build and node directory to gitignore file. (#1102) 2020-08-12 15:45:00 +02:00
Felix Kunde 808030ad17
update go modules (#1097) 2020-08-12 15:37:40 +02:00
Sonay Şevik fc9ee76832
UI Service port forwarding internal port is updated to 80 from 8081. (#1096)
Fix #1093
2020-08-11 15:14:39 +02:00
Felix Kunde dfd0dd90ed
set search_path for default roles (#1065)
* set search_path for default roles

* deployment back to 1.5.0

Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-08-11 10:42:31 +02:00
Felix Kunde 0508266219
Remove all secrets on delete incl. pooler (#1091)
* fix syncSecrets and remove pooler secret

* update log for deleteSecret

* use c.credentialSecretName(username)

* minor fix
2020-08-10 18:26:26 +02:00
Felix Kunde 43163cf83b
allow using both infrastructure_roles_options (#1090)
* allow using both infrastructure_roles_options

* new default values for user and role definition

* use robot_zmon as parent role

* add operator log to debug

* right name for old secret

* only extract if rolesDefs is empty

* set password1 in old infrastructure role

* fix new infra rile secret

* choose different role key for new secret

* set memberof everywhere

* reenable all tests

* reflect feedback

* remove condition for rolesDefs
2020-08-10 15:08:03 +02:00
Dmitry Dolgov 7cf2fae6df
[WIP] Extend infrastructure roles handling (#1064)
Extend infrastructure roles handling

Postgres Operator uses infrastructure roles to provide access to a database for
external users e.g. for monitoring purposes. Such infrastructure roles are
expected to be present in the form of k8s secrets with the following content:

    inrole1: some_encrypted_role
    password1: some_encrypted_password
    user1: some_entrypted_name

    inrole2: some_encrypted_role
    password2: some_encrypted_password
    user2: some_entrypted_name

The format of this content is implied implicitly and not flexible enough. In
case if we do not have possibility to change the format of a secret we want to
use in the Operator, we need to recreate it in this format.

To address this lets make the format of secret content explicitly. The idea is
to introduce a new configuration option for the Operator.

    infrastructure_roles_secrets:
    - secretname: k8s_secret_name
      userkey: some_encrypted_name
      passwordkey: some_encrypted_password
      rolekey: some_encrypted_role

    - secretname: k8s_secret_name
      userkey: some_encrypted_name
      passwordkey: some_encrypted_password
      rolekey: some_encrypted_role

This would allow Operator to use any avalable secrets to prepare infrastructure
roles. To make it backward compatible simulate the old behaviour if the new
option is not present.

The new configuration option is intended be used mainly from CRD, but it's also
available via Operator ConfigMap in a limited fashion. For ConfigMap one can
put there only a string with one secret definition in the following format (as
a string):

    infrastructure_roles_secrets: |
        secretname: k8s_secret_name,
        userkey: some_encrypted_name,
        passwordkey: some_encrypted_password,
        rolekey: some_encrypted_role

Note than only one secret could be specified this way, no multiple secrets are
allowed.

Eventually the resulting list of infrastructure roles would be a total sum of
all supported ways to describe it, namely legacy via
infrastructure_roles_secret_name and infrastructure_roles_secrets from both
ConfigMap and CRD.
2020-08-05 14:18:56 +02:00
Felix Kunde f3ddce81d5
fix random order for pod environment tests (#1085) 2020-07-30 17:48:15 +02:00
hlihhovac 47b11f7f89
change Clone attribute of PostgresSpec to *CloneDescription (#1020)
* change Clone attribute of PostgresSpec to *ConnectionPooler

* update go.mod from master

* fix TestConnectionPoolerSynchronization()

* Update pkg/apis/acid.zalan.do/v1/postgresql_type.go

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>

Co-authored-by: Pavlo Golub <pavlo.golub@gmail.com>
Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-07-30 16:31:29 +02:00
Felix Kunde 3bee590d43
fix index in TestGenerateSpiloPodEnvVarswq (#1084)
Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-07-30 13:35:37 +02:00
Allison Richardet aab9b0aff9
chart ui: fix target namespace to allow '*' (#1082) 2020-07-30 11:08:33 +02:00
Christian Rohmann ece341d516
Allow pod environment variables to also be sourced from a secret (#946)
* Extend operator configuration to allow for a pod_environment_secret just like pod_environment_configmap

* Add all keys from PodEnvironmentSecrets as ENV vars (using SecretKeyRef to protect the value)

* Apply envVars from pod_environment_configmap and pod_environment_secrets before doing the global settings from the operator config. This allows them to be overriden by the user (via configmap / secret)

* Add ability use a Secret for custom pod envVars (via pod_environment_secret) to admin documentation

* Add pod_environment_secret to Helm chart values.yaml

* Add unit tests for PodEnvironmentConfigMap and PodEnvironmentSecret - highly inspired by @kupson and his very similar PR #481

* Added new parameter pod_environment_secret to operatorconfig CRD and configmap examples

* Add pod_environment_secret to the operationconfiguration CRD

Co-authored-by: Christian Rohmann <christian.rohmann@inovex.de>
2020-07-30 10:48:16 +02:00