Commit Graph

499 Commits

Author SHA1 Message Date
Dmitrii Dolgov 66129335fd Adjust sync logic 2020-02-19 14:53:58 +01:00
Dmitrii Dolgov 97217e27ed Delete if a new specification is nil
Use coalesce for username too.
2020-02-17 15:34:10 +01:00
Dmitrii Dolgov 35d82e5a17 Do sync also when there are no deployment 2020-02-17 14:28:38 +01:00
Dmitrii Dolgov 4add317b48 Merge branch 'master' into feature/connection-pooler 2020-02-17 13:11:57 +01:00
Dmitrii Dolgov 515bb2dfad Address review, add ConnectionPool init for sync 2020-02-17 13:07:40 +01:00
Felix Kunde 702a194c41
switch to rbac/v1 (#829)
* switch to rbac/v1
2020-02-17 11:25:07 +01:00
Dmitrii Dolgov a9d02bacc4 Address review 2020-02-14 14:20:35 +01:00
Dmitrii Dolgov 0095be0279 Fix uninitialized ConnectionPool structure usage
It's being used quite early to setup a connection pool user, and if only
enableConnectionPool specified, can be nil.
2020-02-14 13:04:24 +01:00
Dmitrii Dolgov e11f787664 Rename replicas to avoid potential confusion about terminology 2020-02-13 16:34:02 +01:00
Felix Kunde 3b10dc645d
patch/update services on type change (#824)
* use Update when disabling LoadBalancer + added e2e test
2020-02-13 16:24:15 +01:00
Felix Kunde 6ddac2f9d6 update codegen 2020-02-13 14:51:43 +01:00
Felix Kunde c9a1c8e6b4 add enableConnectionPool to validation and some nior fixes 2020-02-13 14:13:06 +01:00
Dmitrii Dolgov 6dad83325b Cleanup configuration
Add pool configuration into CRD & charts. Add preliminary documentation.
Rename NumberOfInstances to Replicas like in Deployment. Mention couple
of potential improvement points for connection pool specification.
2020-02-13 13:36:14 +01:00
Felix Kunde f2c9905123 reflect connectionPool validation in Go code and publish in manifests and chart 2020-02-13 12:54:32 +01:00
Dmitrii Dolgov 55873f06be Add test for both ways to enable connection pool 2020-02-13 11:04:11 +01:00
Jonathan Juares Beber 744c71d16b
Allow services update when changing annotations (#818)
The current implementations for `pkg.util.k8sutil.SameService` considers
only service annotations change on the default annotations created by the
operator. Custom annotations are not compared and consequently not
applied after the first service creation.

This commit introduces a complete annotations comparison between the
current service created by the operator and the new one generated based on
the configs. Also, it adds tests on the above-mentioned function.
2020-02-13 10:55:30 +01:00
Dmitrii Dolgov 6c3752068b Various improvements
Add synchronization logic. For now get rid of podTemplate, type fields.
Add crd validation & configuration part, put retry on top of lookup
function installation.
2020-02-12 17:35:43 +01:00
Dmitrii Dolgov b40ea2c426 Add more tests 2020-02-12 17:31:10 +01:00
Dmitrii Dolgov 2b2f29ff0b Add CRD configuration
With convertion for config, and start tests.
2020-02-12 17:31:10 +01:00
Dmitrii Dolgov 4c69b2b996 Improve cleaning up
Set up a proper owner reference to StatefulSet, and delete with
foreground policy to not leave orphans.
2020-02-12 17:31:10 +01:00
Dmitrii Dolgov fb43ee92d6 WIP Connection pooler support
Add an initial support for a connection pooler. The idea is to make it
generic enough to be able to switch a corresponding docker image to
change from pgbouncer to e.g. odyssey. Operator needs to create a
deployment with pooler and a service for it to access.
2020-02-12 17:31:10 +01:00
Fredrik Østrem 00f00af2e8
Fix MasterPodMoveTimeout field that cannot be unmarshalled (#816)
* Update operator_configuration_type.go

* Update operator_config.go
2020-02-11 17:16:38 +01:00
Jonathan Juares Beber ba60e15d07 Add ServiceAnnotations cluster config (#803)
The [operator parameters][1] already support the
`custom_service_annotations` config.With this parameter is possible to
define custom annotations that will be used on the services created by the
operator. The `custom_service_annotations` as all the other
[operator parameters][1] are defined on the operator level and do not allow
customization on the cluster level. A cluster may require different service
annotations, as for example, set up different cloud load balancers
timeouts, different ingress annotations, and/or enable more customizable
environments.

This commit introduces a new parameter on the cluster level, called
`serviceAnnotations`, responsible for defining custom annotations just for
the services created by the operator to the specifically defined cluster.
It allows a mix of configuration between `custom_service_annotations` and
`serviceAnnotations` where the latest one will have priority. In order to
allow custom service annotations to be used on services without
LoadBalancers (as for example, service mesh services annotations) both
`custom_service_annotations` and `serviceAnnotations` are applied
independently of load-balancing configuration. For retro-compatibility
purposes, `custom_service_annotations` is still under
[Load balancer related options][2]. The two default annotations when using
LoadBalancer services, `external-dns.alpha.kubernetes.io/hostname` and
`service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout` are
still defined by the operator.
`service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout` can
be overridden by `custom_service_annotations` or `serviceAnnotations`,
allowing a more customizable environment.
`external-dns.alpha.kubernetes.io/hostname` can not be overridden once
there is no differentiation between custom service annotations for
replicas and masters.

It updates the documentation and creates the necessary unit and e2e
tests to the above-described feature too.

[1]: https://github.com/zalando/postgres-operator/blob/master/docs/reference/operator_parameters.md
[2]: https://github.com/zalando/postgres-operator/blob/master/docs/reference/operator_parameters.md#load-balancer-related-options
2020-02-10 12:03:25 +01:00
Vito Botta a660d758a5 Add region setting for logical backups to non-AWS storage (#813)
* Add region setting for logical backups to non-AWS storage
2020-02-10 11:48:24 +01:00
Felix Kunde 1f0312a014
make minimum limits boundaries configurable (#808)
* make minimum limits boundaries configurable
* add e2e test
2020-02-03 11:43:18 +01:00
Jonathan Juares Beber fddaf0fb73 Change error computation on JSON Unmarshal and create subtests on table test scenarios (#801)
* Change error computation on JSON Unmarshall

The [Unmarshall function][1] on the encoding/JSON default library returns
different errors for different go versions. On Go 1.12, the version used
currently on the CI system it returns `json: cannot unmarshal number into
Go struct field PostgresSpec.teamId of type string`. On Go 1.13.5 it
returns `json: cannot unmarshal number into Go struct field
PostgresSpec.spec.teamId of type string`. The new version includes more
details of the whole structure being unmarshelled.

This commit introduces the same error but one level deeper on the JSON
structure. It creates consistency across different Go versions.

[1]: https://godoc.org/encoding/json#Unmarshal

* Create subtests on table test scenarios

The Run method of T allows defining subtests creating hierarchical tests.
It provides better visibility of tests in case of failure. More
details on https://golang.org/pkg/testing/.

This commit converts each test scenario on
pkg/apis/acid.zalan.do/v1/util_test.go to subtests, providing a better
visibility and the debugging environment when working with tests. The
following code snippet shows an error during test execution with
subtests:

```
--- FAIL: TestUnmarshalMaintenanceWindow (0.00s)
    --- FAIL: TestUnmarshalMaintenanceWindow/expect_error_as_'From'_is_later_than_'To' (0.00s)
```

It included a `about` field on test scenarios describing the test
purpose and/or it expected output. When a description was provided with
comments it was moved to the about field.
2020-01-27 14:43:32 +01:00
Felix Kunde 7fb163252c
standby clusters can only have 1 pod for now (#797) 2020-01-16 10:47:34 +01:00
Felix Kunde b54458ee3c
update copyright in generated code (#779)
* update year in copyright of generated code and LICENSE file
2020-01-02 17:34:49 +01:00
Felix Kunde 7af1de890c
bump operator v1.3.0 with Spilo 12 image (#770) 2019-12-17 17:13:56 +01:00
Felix Kunde 182e3bc7db
add missing fields to OperatorConfiguration CRD validation (#767) 2019-12-16 17:08:09 +01:00
Felix Kunde 97e0d6d388
extend docs and polish manifest examples (#762) 2019-12-12 17:55:41 +01:00
Felix Kunde cd110aabf4
Enforce minimum cpu and memory limits (#731)
* add validation for PG resources and volume size
* check resource requests also on UPDATE and SYNC + update docs
* if cluster was running don't error on sync
2019-12-12 16:43:55 +01:00
Felix Kunde 0628439256 fix cpu resource validation (#757) 2019-12-10 16:30:57 +01:00
Felix Kunde 107334fe71
Add global option to enable/disable init containers and sidecars (#478)
* Add global option to enable/disable init containers and sidecars
* update dependencies
2019-12-10 15:45:54 +01:00
Felix Kunde 11c2e815f7
include status subresource in validation (#744)
* include status subresource in validation
2019-12-02 15:27:47 +01:00
Felix Kunde a3b34f146f
Add CRD validation (#599)
* add CRD manifests with validation
* update documentation
* patroni slots is not an array but a nested hash map
* make deps call tools
* cover validation in docs and export it in crds.go
* add toggle to disable creation of CRD validation and document it
* use templated service account also for CRD-configured helm deployment
2019-11-28 12:02:05 +01:00
Armin Nesiren 5f87384d7f Passing endpoint, access and secret key to logical-backup container (#628)
* Added possibility to add custom annotations to LoadBalancer service.

* Added parameters for custom endpoint, access and secret key for logical backup.

* Modified dump.sh so it knows how to handle new features. Configurable S3 SSE
2019-11-26 10:40:49 +01:00
Felix Kunde 2ce602fcd7 fix errors when changing service type (#716)
* fix errors when changing service type

* nullify service and endpoint before recreation

* improve wait for delete logic and reuse config parameters
2019-11-26 10:28:32 +01:00
Felix Kunde f9487e41c1 inject cluster name label into logical backup pod (#725)
* inject cluster name label into logical backup pod
2019-11-20 13:58:41 +01:00
Felix Kunde 0b544ae43f
pass additionalSecretMount to logical backup pod (#714) 2019-11-19 18:06:55 +01:00
Thomas Runyon 535517cd1b Custom annotations 329 (#657)
* Add ability for custom annotations to database pods
2019-11-11 10:45:35 +01:00
Felix Kunde 340dc4aa3d Fix operator_configuration_type.go (#703)
fix typo
2019-10-31 12:19:41 +01:00
Eric 6e682fd6b5 Fixing spelling mistake in delete PVC function name (#691) 2019-10-18 16:41:56 +02:00
Dmitry Dolgov 647a4d3023
Remove service accounts cache (#685)
For optimization purposes operator was creating a cache map to remember
if service accounts and role binding was deployed to a namespace. This
could lead to a problem, when a namespace was deleted, since this
cache was not synchronized. For the sake of correctness remove the
cache, and check every time if required service account and rbac is
present. In the normal case this introduces an overhead of two API calls
per an event (one to get a service accounts, one to get a role binding),
which should not be a problem, unless proven otherwise.
2019-10-11 11:06:14 +02:00
Dmitry Dolgov baae1887b3 Replace glide with Go modules (#544)
* And attempt to build with modules and remove glide
* new tools.go file to get code-generator dependency + updated codegen + remove Glide files and update docs
2019-10-02 16:18:55 +02:00
Felix Kunde f0e29060b1
move StatefulSet to apps/v1 (#675) 2019-09-30 16:42:04 +02:00
Weilu Jia e00b37fc17 Handle IPv6 k8s pods in Patroni URLs (#671)
* Handle IPv6 Patroni URLs
2019-09-30 10:14:27 +02:00
Sergey Dudoladov cf97ebb2b8 fix e2e tests (#672)
* fix e2e tests
* change Spilo version everywhere
2019-09-23 17:48:53 +02:00
Felix Kunde 4a863d2280 Avoid orphaned objects on delete (#654)
* Make setSpec function work correctly when updating cluster status fails
2019-08-27 12:54:35 +02:00
Felix Kunde abdb003f40
additional printer columns for CRDs (#653)
* additional printer columns for CRDs
2019-08-16 13:22:45 +02:00