Commit Graph

590 Commits

Author SHA1 Message Date
Jan Mußler 1f6fe9bcad Fix test case config. 2020-12-09 12:03:33 +01:00
Jan Mußler 17966ffcbf create all pvs. 2020-12-08 23:43:57 +01:00
Jan Mußler 3e3f3f8352 Fixing test setup. Late night... 2020-12-08 23:26:20 +01:00
Jan Mußler c9f83f07dc volume ids. 2020-12-08 23:22:12 +01:00
Jan Mußler dc91b60119 Fix volume id. 2020-12-08 23:17:48 +01:00
Jan Mußler 51f4683204 Fix mapping of ids in test. 2020-12-08 23:11:57 +01:00
Jan Mußler 0bab888dc7 Expect modify call. 2020-12-08 23:05:47 +01:00
Jan Mußler 4193dce95e Wrong toggle. 2020-12-08 23:00:49 +01:00
Jan Mußler 523e2d67d8 Learning as one goes. Initialize map. 2020-12-08 22:52:28 +01:00
Jan Mußler 515f2f622f Changes to Makefile and return value. 2020-12-08 19:01:47 +01:00
Jan Mußler e79a6c4ec9 Rename var. 2020-12-08 16:16:04 +01:00
Jan Mußler d4d9f0a62a more changes for code gen. 2020-12-07 23:45:19 +01:00
Jan Mußler b42013455f Added Gomock and one test case with mock. 2020-12-07 22:36:17 +01:00
Jan Mußler 9e095e49d8 Default volume migration done. 2020-12-07 17:52:02 +01:00
Jan Mußler 6bb78c8de2 initial commit for gp3 migration. 2020-12-07 14:57:21 +01:00
Sergey Dudoladov dc9a5b1e61
Introduce PGVERSION (#1172)
* introduce PGVERSION

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-11-27 18:49:49 +01:00
Sergey Dudoladov 6f5751fe55
raise log level for malformed secrets (#1235)
Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-11-27 18:47:50 +01:00
Boyan Bonev 85d1a72cd6
Add scheduler name support - [Update #990] (#1226)
* Add ability to specify alternative schedulers via schedulerName.

Co-authored-by: micah.coletti@gmail.com <micah.coletti@gmail.com>
2020-11-25 10:55:05 +01:00
Felix Kunde cfd83e33c8
preserving fields only when using k8s specs (#1228)
* preserving fields when k8s specs are used with x-kubernetes-preserve-unknown-fields flag
* cleaning up merge errors in postgresql and operatorconfiguration CRD
* add operatorconfiguration CRD and sample manifests in setUpClass of e2e tests
* update generated code and go modules
2020-11-24 16:23:22 +01:00
Jan Mussler c4ae11629b
Fix connection pooler deployment selectors (#1213)
Stick with the existing pooler deployment selector labels to make it compatible with existing deployments.

Make the use of additional labels clear and avoid where not needed.

Deployment Selector and Service Selector now do not use extra labels, pod spec does.
2020-11-23 17:18:18 +01:00
Felix Kunde 3e42e8a896
CRD: preserve unknown fields and add to all category (#1212)
* CRD: preserve unknown fields and add to all category

* allow Pg13

* left over
2020-11-16 10:29:01 +01:00
Rafia Sabih 49158ecb68
Connection pooler for replica (#1127)
* Enable connection pooler for replica
* Refactor code for connection pooler
  - Move all the relevant code to a separate file
  - Move all the related tests to a separate file
  - Avoid using cluster where not required
  - Simplify the logic in sync and other methods
  - Cleanup of duplicated or unused code
* Fix labels for the replica pods
* Update deleteConnectionPooler to include role
* Adding test cases and other changes
   - Fix unit test and delete secret when required only
   - Make sure we use empty fresh cluster for every test case.
* enhance e2e test
* Disable pooler in complete manifest as this is source for e2e too an creates unnecessary pooler setups.

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
Co-authored-by: Jan Mussler <janm81@gmail.com>
2020-11-13 14:52:21 +01:00
Felix Kunde 3fed565328
check resize mode on update events (#1194)
* check resize mode on update events

* add unit test for PVC resizing

* set resize mode to pvc in charts and manifests

* add test for quantityToGigabyte

* just one debug line for syncing volumes

* extend test and update log msg
2020-11-11 13:22:43 +01:00
Sergey Dudoladov e779eab22f
Update e2e pipeline (#1202)
* clean up after test_multi_namespace test

* see the PR description for complete list of changes

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-11-11 10:21:46 +01:00
Felix Kunde b379db20ed
fix redundant appending of infrastructure roles (#1192) 2020-11-05 12:04:51 +01:00
Pavel Tumik db0d089e75
Fix cloning from GCS (#1176)
* Fix clone from gcs

* pass google credentials env var if using GS bucket

* remove requirement for timezone as GCS returns timestamp in local time to the region it is in

* Revert "remove requirement for timezone as GCS returns timestamp in local time to the region it is in"

This reverts commit ac4eb350d9.

* update GCS documentation

* remove sentence about logical backups

* reword pod environment configmap section

* fix documentation
2020-11-03 15:05:44 +01:00
Sergey Dudoladov 4f3bb6aa8c
Remove operator checks that prevent PG major version upgrade (#1160)
* remove checks that prevent major version upgrade

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-11-02 16:49:29 +01:00
Felix Kunde d76419565b
move to apiextensions from v1beta1 to v1 (#746)
* move to apiextensions from v1beta1 to v1

* remove metadata from CRD validation

* some forgotten change
2020-11-02 10:49:29 +01:00
Jan Mussler c694a72352
Make failure in retry a warning not an error. (#1188) 2020-10-29 13:12:25 +01:00
Felix Kunde 9a11e85d57
disable PostgresTeam by default (#1186)
* disable PostgresTeam by default

* fix version in chart
2020-10-28 17:51:37 +01:00
Felix Kunde d658b9672e
PostgresTeam CRD for advanced team management (#1165)
* PostgresTeamCRD for advanced team management

* rework internal structure to be closer to CRD

* superusers instead of admin

* add more util functions and unit tests

* fix initHumanUsers

* check for superusers when creating normal teams

* polishing and fixes

* adding the essential missing pieces

* add documentation and update rbac

* reflect some feedback

* reflect more feedback

* fixing debug logs and raise QueueResyncPeriodTPR

* add two more flags to disable CRD and its superuser support

* fix chart

* update go modules

* move to client 1.19.3 and update codegen
2020-10-28 10:40:10 +01:00
Jan Mussler 3a86dfc8bb
End 2 End tests speedup (#1180)
* Improving end 2 end tests, especially speed of execution and error, by implementing proper eventual asserts and timeouts.
* Add documentation for running individual tests
* Fixed String encoding in Patorni state check and error case
* Printing config as multi log line entity, makes it readable and grepable on startup
* Cosmetic changes to logs. Removed quotes from diff. Move all object diffs to text diff. Enabled padding for log level.
* Mount script with tools for easy logaccess and watching objects.
* Set proper update strategy for Postgres operator deployment.
* Move long running test to end. Move pooler test to new functions.
* Remove quote from valid K8s identifiers.
2020-10-28 10:04:33 +01:00
arminfelder 7730ecfdec
fixed case where, no ready label is defined, but node is unscheduable (#1162)
* fixed case where, no ready label is defined, but node is unscheduable
2020-10-28 09:33:52 +01:00
Felix Kunde e97235aa39
update dependencies oct 2020 (#1184)
* update dependencies oct 2020

* update codegen
2020-10-27 16:59:26 +01:00
preved911 d9f5d1c9df
changed PodEnvironmentSecret location namespace (#1177)
Signed-off-by: Ildar Valiullin <preved.911@gmail.com>
2020-10-22 08:49:30 +02:00
Felix Kunde 22fa0875e2
add maxLength constraint for CRD (#1175)
* add maxLength constraint for CRD
2020-10-22 08:44:04 +02:00
刘新 a8bfe4eb87
Remove repeated initialization of Pod ServiceAccount (#1164)
Co-authored-by: xin.liu <xin.liu@woqutech.com>
2020-10-20 14:18:22 +02:00
Dmitry Dolgov 1f5d0995a5
Lookup function installation (#1171)
* Lookup function installation

Due to reusing a previous database connection without closing it, lookup
function installation process was skipping the first database in the
list, installing twice into postgres db instead. To prevent that, make
internal initDbConnWithName to overwrite a connection object, and return
the same object only from initDbConn, which is sort of public interface.

Another solution for this would be to modify initDbConnWithName to
return a connection object and then generate one temporary connection
for each db. It sound feasible but after one attempt it seems it
requires a bit more changes around (init, close connections) and
doesn't bring anything significantly better on the table. In case if
some future changes will prove this wrong, do not hesitate to refactor.

Change retry strategy to more insistive one, namely:

* retry on the next sync even if we failed to process one database and
install pooler appliance.

* perform the whole installation unconditionally on update, since the
list of target databases could be changed.

And for the sake of making it even more robust, also log the case when
operator decides to skip installation.

Extend connection pooler e2e test with verification that all dbs have
required schema installed.
2020-10-19 16:18:58 +02:00
Dmitry Dolgov d15f2d3392
Readiness probe (#1169)
Right now there are no readiness probes defined for connection pooler,
which means after a pod restart there is a short time window (between a
container start and connection pooler starting listening to a socket)
when a service can send queries to a new pod, but connection will be
refused. The pooler container is rather lightweight and it start to
listen immediately, so the time window is small, but still.

To fix this add a readiness probe for tcp socket opened by connection
pooler.
2020-10-15 10:16:42 +02:00
Alex Stockinger 692c721854
Introduce ENABLE_JSON_LOGGING env variable (#1158) 2020-10-08 15:32:15 +02:00
Felix Kunde 21475f4547
Cleanup config examples (#1151)
* post polishing for latest PRs

* update travis and go modules

* make deprecation comments in structs less confusing

* have separate pod priority class es for operator and database pods
2020-09-30 17:24:14 +02:00
Sergey Dudoladov 2a21cc4393
Compare Postgres pod priority on Sync (#1144)
* compare Postgres pod priority on Sync

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-09-23 17:26:56 +02:00
neelasha-09 ab95eaa6ef
Fixes #1130 (#1139)
* Fixes #1130

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-09-22 17:16:05 +02:00
Rico Berger d09e418b56
Set user and group in security context (#1083)
* Set user and group in security context
2020-09-15 13:27:59 +02:00
Igor Yanchenko d8884a4003
Allow to overwrite default ExternalTrafficPolicy for the service (#1136)
* Allow to overwrite default ExternalTrafficPolicy for the service
2020-09-15 13:19:22 +02:00
hlihhovac e03e9f919a
add missing omitempty directive to the attributes of PostgresSpec (#1128)
Co-authored-by: Pavlo Golub <pavlo.golub@gmail.com>
2020-08-31 12:28:52 +02:00
Felix Kunde 3ddc56e5b9
allow delete only if annotations meet configured criteria (#1069)
* define annotations for delete protection

* change log level and reduce log lines for e2e tests

* reduce wait_for_pod_start even further
2020-08-13 16:36:22 +02:00
Felix Kunde dfd0dd90ed
set search_path for default roles (#1065)
* set search_path for default roles

* deployment back to 1.5.0

Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-08-11 10:42:31 +02:00
Felix Kunde 0508266219
Remove all secrets on delete incl. pooler (#1091)
* fix syncSecrets and remove pooler secret

* update log for deleteSecret

* use c.credentialSecretName(username)

* minor fix
2020-08-10 18:26:26 +02:00
Felix Kunde 43163cf83b
allow using both infrastructure_roles_options (#1090)
* allow using both infrastructure_roles_options

* new default values for user and role definition

* use robot_zmon as parent role

* add operator log to debug

* right name for old secret

* only extract if rolesDefs is empty

* set password1 in old infrastructure role

* fix new infra rile secret

* choose different role key for new secret

* set memberof everywhere

* reenable all tests

* reflect feedback

* remove condition for rolesDefs
2020-08-10 15:08:03 +02:00