Commit Graph

946 Commits

Author SHA1 Message Date
Felix Kunde cfd83e33c8
preserving fields only when using k8s specs (#1228)
* preserving fields when k8s specs are used with x-kubernetes-preserve-unknown-fields flag
* cleaning up merge errors in postgresql and operatorconfiguration CRD
* add operatorconfiguration CRD and sample manifests in setUpClass of e2e tests
* update generated code and go modules
2020-11-24 16:23:22 +01:00
Jan Mussler c4ae11629b
Fix connection pooler deployment selectors (#1213)
Stick with the existing pooler deployment selector labels to make it compatible with existing deployments.

Make the use of additional labels clear and avoid where not needed.

Deployment Selector and Service Selector now do not use extra labels, pod spec does.
2020-11-23 17:18:18 +01:00
Thunderbolt 580883bc59
add operator ui helm chart value imagePullSecret (#1211) 2020-11-17 10:47:35 +01:00
Jan Mussler 67d1b4b167
compile coverage report and add badges. (#1218)
Add GH badges, and added coveralls support again.
2020-11-16 12:08:28 +01:00
Jan Mussler 65d1a71cc9
Update README.md (#1217)
Removed all the badges, they don't work anyways and are off.
2020-11-16 10:30:18 +01:00
Felix Kunde 3e42e8a896
CRD: preserve unknown fields and add to all category (#1212)
* CRD: preserve unknown fields and add to all category

* allow Pg13

* left over
2020-11-16 10:29:01 +01:00
Jan Mussler a7f453352a
Use Github.com actions to run tests and e2e tests. (#1215)
* Use GH action to run tests and end 2 end tests.
* Remove travis.
2020-11-16 10:15:47 +01:00
Rafia Sabih 49158ecb68
Connection pooler for replica (#1127)
* Enable connection pooler for replica
* Refactor code for connection pooler
  - Move all the relevant code to a separate file
  - Move all the related tests to a separate file
  - Avoid using cluster where not required
  - Simplify the logic in sync and other methods
  - Cleanup of duplicated or unused code
* Fix labels for the replica pods
* Update deleteConnectionPooler to include role
* Adding test cases and other changes
   - Fix unit test and delete secret when required only
   - Make sure we use empty fresh cluster for every test case.
* enhance e2e test
* Disable pooler in complete manifest as this is source for e2e too an creates unnecessary pooler setups.

Co-authored-by: Rafia Sabih <rafia.sabih@zalando.de>
Co-authored-by: Jan Mussler <janm81@gmail.com>
2020-11-13 14:52:21 +01:00
Felix Kunde 3fed565328
check resize mode on update events (#1194)
* check resize mode on update events

* add unit test for PVC resizing

* set resize mode to pvc in charts and manifests

* add test for quantityToGigabyte

* just one debug line for syncing volumes

* extend test and update log msg
2020-11-11 13:22:43 +01:00
Sergey Dudoladov e779eab22f
Update e2e pipeline (#1202)
* clean up after test_multi_namespace test

* see the PR description for complete list of changes

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-11-11 10:21:46 +01:00
Felix Kunde b379db20ed
fix redundant appending of infrastructure roles (#1192) 2020-11-05 12:04:51 +01:00
Felix Kunde 9a824c38f4
fix identation in operatorconfiguration CRD and jsonPath case (#1195)
* fix identation in operatorconfiguration CRD

* fix jsonPath field case
2020-11-05 11:49:24 +01:00
Jan Mussler 90799d7e7f
More output from test watch script. All namespaces and deployments. (#1193) 2020-11-04 16:40:15 +01:00
Pavel Tumik db0d089e75
Fix cloning from GCS (#1176)
* Fix clone from gcs

* pass google credentials env var if using GS bucket

* remove requirement for timezone as GCS returns timestamp in local time to the region it is in

* Revert "remove requirement for timezone as GCS returns timestamp in local time to the region it is in"

This reverts commit ac4eb350d9.

* update GCS documentation

* remove sentence about logical backups

* reword pod environment configmap section

* fix documentation
2020-11-03 15:05:44 +01:00
Sergey Dudoladov 4f3bb6aa8c
Remove operator checks that prevent PG major version upgrade (#1160)
* remove checks that prevent major version upgrade

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-11-02 16:49:29 +01:00
Felix Kunde d76419565b
move to apiextensions from v1beta1 to v1 (#746)
* move to apiextensions from v1beta1 to v1

* remove metadata from CRD validation

* some forgotten change
2020-11-02 10:49:29 +01:00
Jan Mussler 7f7beba66b
Improving e2e more (#1185)
* Add curl to operator image.

* Wait for idle operator in delete.
2020-10-29 13:59:22 +01:00
Jan Mussler c694a72352
Make failure in retry a warning not an error. (#1188) 2020-10-29 13:12:25 +01:00
Felix Kunde 9a11e85d57
disable PostgresTeam by default (#1186)
* disable PostgresTeam by default

* fix version in chart
2020-10-28 17:51:37 +01:00
Jakub Warczarek e10e0fec9e
Add support in UI for custom S3 endpoints for backups (#1152)
* Support custom S3 endpoint for backups

* Log info about AWS S3 endpoint during start up
2020-10-28 10:56:50 +01:00
Felix Kunde d658b9672e
PostgresTeam CRD for advanced team management (#1165)
* PostgresTeamCRD for advanced team management

* rework internal structure to be closer to CRD

* superusers instead of admin

* add more util functions and unit tests

* fix initHumanUsers

* check for superusers when creating normal teams

* polishing and fixes

* adding the essential missing pieces

* add documentation and update rbac

* reflect some feedback

* reflect more feedback

* fixing debug logs and raise QueueResyncPeriodTPR

* add two more flags to disable CRD and its superuser support

* fix chart

* update go modules

* move to client 1.19.3 and update codegen
2020-10-28 10:40:10 +01:00
Jan Mussler 3a86dfc8bb
End 2 End tests speedup (#1180)
* Improving end 2 end tests, especially speed of execution and error, by implementing proper eventual asserts and timeouts.
* Add documentation for running individual tests
* Fixed String encoding in Patorni state check and error case
* Printing config as multi log line entity, makes it readable and grepable on startup
* Cosmetic changes to logs. Removed quotes from diff. Move all object diffs to text diff. Enabled padding for log level.
* Mount script with tools for easy logaccess and watching objects.
* Set proper update strategy for Postgres operator deployment.
* Move long running test to end. Move pooler test to new functions.
* Remove quote from valid K8s identifiers.
2020-10-28 10:04:33 +01:00
arminfelder 7730ecfdec
fixed case where, no ready label is defined, but node is unscheduable (#1162)
* fixed case where, no ready label is defined, but node is unscheduable
2020-10-28 09:33:52 +01:00
Felix Kunde e97235aa39
update dependencies oct 2020 (#1184)
* update dependencies oct 2020

* update codegen
2020-10-27 16:59:26 +01:00
preved911 d9f5d1c9df
changed PodEnvironmentSecret location namespace (#1177)
Signed-off-by: Ildar Valiullin <preved.911@gmail.com>
2020-10-22 08:49:30 +02:00
Felix Kunde 22fa0875e2
add maxLength constraint for CRD (#1175)
* add maxLength constraint for CRD
2020-10-22 08:44:04 +02:00
刘新 a8bfe4eb87
Remove repeated initialization of Pod ServiceAccount (#1164)
Co-authored-by: xin.liu <xin.liu@woqutech.com>
2020-10-20 14:18:22 +02:00
Dmitry Dolgov 1f5d0995a5
Lookup function installation (#1171)
* Lookup function installation

Due to reusing a previous database connection without closing it, lookup
function installation process was skipping the first database in the
list, installing twice into postgres db instead. To prevent that, make
internal initDbConnWithName to overwrite a connection object, and return
the same object only from initDbConn, which is sort of public interface.

Another solution for this would be to modify initDbConnWithName to
return a connection object and then generate one temporary connection
for each db. It sound feasible but after one attempt it seems it
requires a bit more changes around (init, close connections) and
doesn't bring anything significantly better on the table. In case if
some future changes will prove this wrong, do not hesitate to refactor.

Change retry strategy to more insistive one, namely:

* retry on the next sync even if we failed to process one database and
install pooler appliance.

* perform the whole installation unconditionally on update, since the
list of target databases could be changed.

And for the sake of making it even more robust, also log the case when
operator decides to skip installation.

Extend connection pooler e2e test with verification that all dbs have
required schema installed.
2020-10-19 16:18:58 +02:00
Dmitry Dolgov d15f2d3392
Readiness probe (#1169)
Right now there are no readiness probes defined for connection pooler,
which means after a pod restart there is a short time window (between a
container start and connection pooler starting listening to a socket)
when a service can send queries to a new pod, but connection will be
refused. The pooler container is rather lightweight and it start to
listen immediately, so the time window is small, but still.

To fix this add a readiness probe for tcp socket opened by connection
pooler.
2020-10-15 10:16:42 +02:00
Alex Stockinger 692c721854
Introduce ENABLE_JSON_LOGGING env variable (#1158) 2020-10-08 15:32:15 +02:00
Sergey Dudoladov 38e15183a2
update kind (#1156)
Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-10-02 09:31:55 +02:00
Felix Kunde 21475f4547
Cleanup config examples (#1151)
* post polishing for latest PRs

* update travis and go modules

* make deprecation comments in structs less confusing

* have separate pod priority class es for operator and database pods
2020-09-30 17:24:14 +02:00
Sergey Dudoladov 3b6dc4f92d
Improve e2e tests (#1111)
* icnrease vm size

* cache deps

* switch to the absolute cache path as cdp does not support shell expansion

* do not pull non-existing image

* manually install kind

* add alias to kind

* use full kind name

* one more name change

* install kind with other tools

* add bind mounts instead of copying files

* test fetching the runner image

* build image for pierone

* bump up the client-go version to match the master

* bump up go version

* install pinned version of kind before any test run

* do not overwrite local ./manifests during test run

* update the docs

* fix kind name

* update go.* files

* fix deps

* avoid unnecessary image upload

* properly install kind

* Change network to host to make it reachable within e2e runner. May not be the right solution though.

* Small changes. Also use entrypoint vs cmd.

* Bumping spilo. Load before test.

* undo incorrect merge from the master

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
Co-authored-by: Jan Mußler <janm81@gmail.com>
2020-09-25 14:14:19 +02:00
Sergey Dudoladov ffdb47f53a
remove outdated GSOC info (#1148)
Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-09-25 09:46:50 +02:00
Sergey Dudoladov 2a21cc4393
Compare Postgres pod priority on Sync (#1144)
* compare Postgres pod priority on Sync

Co-authored-by: Sergey Dudoladov <sergey.dudoladov@zalando.de>
2020-09-23 17:26:56 +02:00
neelasha-09 ab95eaa6ef
Fixes #1130 (#1139)
* Fixes #1130

Co-authored-by: Felix Kunde <felix-kunde@gmx.de>
2020-09-22 17:16:05 +02:00
Rico Berger d09e418b56
Set user and group in security context (#1083)
* Set user and group in security context
2020-09-15 13:27:59 +02:00
Igor Yanchenko d8884a4003
Allow to overwrite default ExternalTrafficPolicy for the service (#1136)
* Allow to overwrite default ExternalTrafficPolicy for the service
2020-09-15 13:19:22 +02:00
Felix Kunde 03437b6374
Update issue templates (#1051)
* Update issue templates

To help us helping them

* update the template

* some updates

* or not on
2020-09-03 08:02:46 +02:00
hlihhovac e03e9f919a
add missing omitempty directive to the attributes of PostgresSpec (#1128)
Co-authored-by: Pavlo Golub <pavlo.golub@gmail.com>
2020-08-31 12:28:52 +02:00
Felix Kunde 5e93aabea6
improve e2e test debugging (#1107)
* print operator log in most tests when they time out
2020-08-28 14:57:19 +02:00
Felix Kunde 30c86758a3
update kind and use with old storage class (#1121)
* update kind and use with old storage class
* specify standard storage class in minimal manifest
* remove existing local storage class in kind
* fix pod distribution test
* exclude k8s master from nodes of interest
2020-08-28 12:16:37 +02:00
Felix Kunde 248ce9fc78
Update to go 1.14.7 (#1122)
* update go version, dependencies, and client-go 1.18.8
2020-08-26 14:00:14 +02:00
Peter Halliday dab704c566
Add kustomize support to Postgres UI. (#1086)
Co-authored-by: Peter Halliday <peter.halliday@walgreens.com>
2020-08-26 12:06:25 +02:00
Felix Kunde 3ddc56e5b9
allow delete only if annotations meet configured criteria (#1069)
* define annotations for delete protection

* change log level and reduce log lines for e2e tests

* reduce wait_for_pod_start even further
2020-08-13 16:36:22 +02:00
Jan Mussler 0d81f972a1
Added build and node directory to gitignore file. (#1102) 2020-08-12 15:45:00 +02:00
Felix Kunde 808030ad17
update go modules (#1097) 2020-08-12 15:37:40 +02:00
Sonay Şevik fc9ee76832
UI Service port forwarding internal port is updated to 80 from 8081. (#1096)
Fix #1093
2020-08-11 15:14:39 +02:00
Felix Kunde dfd0dd90ed
set search_path for default roles (#1065)
* set search_path for default roles

* deployment back to 1.5.0

Co-authored-by: Felix Kunde <felix.kunde@zalando.de>
2020-08-11 10:42:31 +02:00
Felix Kunde 0508266219
Remove all secrets on delete incl. pooler (#1091)
* fix syncSecrets and remove pooler secret

* update log for deleteSecret

* use c.credentialSecretName(username)

* minor fix
2020-08-10 18:26:26 +02:00