Commit Graph

233 Commits

Author SHA1 Message Date
zerg-junior 3c178f68df Warn on infrastructure-roles.yaml format violations (#177)
Emit a warning if there are unprocessed entries in the infrastructure-roles secret.
2017-12-15 17:21:41 +01:00
zerg-junior 5d5fa680a3
Merge pull request #180 from zalando-incubator/container-name
Make pod's single container name static
2017-12-15 16:13:33 +01:00
Oleksii Kliukin bf80f5225e
Introduce higher and lower bounds for the number of instances (#178)
* Introduce higher and lower bounds for the number of instances

Reduce the number of instances to the min_instances if it is lower and
to the max_instances if it is higher. -1 for either of those means there
is no lower or upper bound.

In addition, terminate the operator when there is a nonsense in the
configuration (i.e. max_instances < min_instances).

Reviewed by Jan Mußler and Sergey Dudoladov.
2017-12-15 16:02:50 +01:00
Sergey Dudoladov 52e358ba8f Make pod's single container name static 2017-12-15 15:53:53 +01:00
Oleksii Kliukin 0e255f82c6 Provide more information about variable conflicts.
They are mentioned in the documentation and the operator will emit a
warning each time the variable from the pod environment configmap is
ignored because the same variable is defined by the operator.

Some minor changes in the variable names to make the code more readable.

Per review from Sergey Dudoladov.
2017-12-14 14:39:33 +01:00
Oleksii Kliukin da4b66210a Expand variables from the PodEnvironmentConfigMap (#4)
Inject PodEnvironmentConfigMap variables inline into the
statefulset definition in order to be able to figure out
changes to the statefulset when only PodEnvironmentConfigMap
has changed.
2017-12-14 14:39:33 +01:00
Oleksii Kliukin 1c5451cd7d Spelling fix. 2017-12-14 14:39:33 +01:00
Oleksii Kliukin 55dc12e512 Examine custom environment sources when syncing.
When comparing statefulsets, make sure EnvFrom fields are compared
as well.
2017-12-14 14:39:33 +01:00
Georg Kunz e8d9c75949 Allow custom Postgres pod environment variables 2017-12-14 14:39:33 +01:00
Oleksii Kliukin 87bc47d8d0 Fixes for the case of re-creating the cluster after deletion.
- make sure that the secrets for the system users (superuser, replication)
  are not deleted when the main cluster is. Therefore, we can re-create
  the cluster, potentially forcing Patroni to restore it from the backup
  and enable Patroni to connect, since it will use the old password, not
  the newly generated random one.

- when syncing users, always check whether they are already in the DB.
  Previously, we did this only for the sync cluster case, but the new
  cluster could be actually the one restored from the backup by Patroni,
  having all or some of the users already in place.

 - delete endponts last. Patroni uses the $clustername endpoint in order
   to store the leader related metadata. If we remove it before removing
   all pods, one of those pods running Patroni will re-create it and the
   next attempt to create the cluster with the same name will stuble on
   the existing endpoint.

 - Use db.Exec instead of db.Query for queries that expect no result.
   This also fixes the issue with the DB creation, since we didn't
   release an empty Row object it was not possible to create more than
   one database for a cluster.
2017-12-13 16:49:00 +01:00
Oleksii Kliukin 1fb8cf7ea0
Avoid overwriting critical users. (#172)
* Avoid overwriting critical users.

Disallow defining new users either in the cluster manifest, teams
API or infrastructure roles with the names mentioned in the new
protected_role_names parameter (list of comma-separated names)

Additionally, forbid defining a user with the name matching either
super_username or replication_username, so that we don't overwrite
system roles required for correct working of the operator itself.

Also, clear PostgreSQL roles on each sync first in order to avoid using
the old definitions that are no longer present in the current manifest,
infrastructure roles secret or the teams API.
2017-12-05 14:27:12 +01:00
Oleksii Kliukin 022ce29314 Make an error message more verbose. 2017-12-04 10:49:25 +01:00
Oleksii Kliukin 637921cdee Tests for initHumanUsers and initinitRobotUsers.
Change the Cluster class in the process to implelement Teams API
calls and Oauth token fetches as interfaces, so that we can mock
them in the tests.
2017-12-04 10:49:25 +01:00
Oleksii Kliukin 611cfe96d6 Fix an issue when not assigning the merge result.
Add some tests.
2017-12-04 10:49:25 +01:00
Oleksii Kliukin 831ebb1f32 Fix the error reporting. 2017-12-04 10:49:25 +01:00
Oleksii Kliukin 2e226dee26 Avoid overwriting infrastrure roles.
When a role is defined in the infrastructure roles and the cluster
manifest use the infrastructure role definition and add flags
defined in the manifest.

Previously the role has been overwritten by the definition from the
manifest.  Because a random password is generated for each role from the
manifest the applications relying on the infrastructure role credentials
from the infrastructure roles secret were unable to connect.
2017-12-04 10:49:25 +01:00
Oleksii Kliukin dd0affc390 Tweak our reaction to the cluster upgrade process.
Previously, the operator started to move the pods off the nodes to be
decomissioned by watching the eol_node_label value. Every new postgres
pod has been created with the anti-affinity to that label, making sure
that the pods being moved won't land on another to be decomissioned
node.

The changes introduce another label that indicates the ready node.  The
new pod affinity will esnure that the pod is only scheduled to the node
marked as ready, discarding the previous anti-affinity.  That way the
nodes can transition from the pending-decomission to the other statuses
(drained, terminating) without having pods suddently scaled to them.

In addition, rename the label that triggers the start of the upgrade
process to node_eol_label (for consistency with node_readiness_label)
and set its default vvalue to lifecycle-status:pending-decomission.
2017-11-30 14:11:49 +01:00
Oleksii Kliukin 1ffe98ba9f Fix the connection leak and user options sync.
- fix the lack of closing the cursor for the query that returned no
rows.
- fix syncing of the user options, as previously those were not
  fetched from the database.
2017-11-27 16:46:34 +01:00
Oleksii Kliukin 086ead03f5 Warn about attempts to use escape quotes. 2017-11-22 10:43:35 +01:00
Oleksii Kliukin 975b21f633 Rename api roles configuration parameter.
Change api_roles_configuration to team_api_role_configuration
2017-11-22 10:43:35 +01:00
Oleksii Kliukin 6b2f5071f7 Special case for search_path in user options.
- search_path accepts a list of values that cannot be quoted, as
  quoting would make PostgreSQL interpret the result as a single
  value. Since we require quoting of values with commas in the
  operator's configMap in order to avoid confusing them with the
  separate map entities, we need to strip those quotes before
  passing the value to PostgreSQL.
- make ftm run
2017-11-22 10:43:35 +01:00
Oleksii Kliukin 2079d811b4 Add tests for the string splitting function. 2017-11-22 10:43:35 +01:00
Oleksii Kliukin e95f80e351 Make configMap marshaling code aware of quotes.
A value in a configMap that is a map itself
(a key:value string separated by commas) may include commans inside
quotes (i.e. search_path:"public,"$user"). The changes make marshaling
code process such cases correctly.
2017-11-22 10:43:35 +01:00
Oleksii Kliukin 2352fc9a39 go fmt run 2017-11-22 10:43:35 +01:00
Oleksii Kliukin 71f57c9fe3 Fix escaping of parameter values and extra spaces.
- document the newly introduced option (for now in the main README)
- make query error output more readable.
2017-11-22 10:43:35 +01:00
Oleksii Kliukin 415a7fdc4d Allow global configuration options for API roles.
Add options to the PgUser structure, potentially allowing to set
per-role options in the cluster definition as well.

Introduce api_roles_configuration operator option with the default
of log_statement=all
2017-11-22 10:43:35 +01:00
Oleksii Kliukin 6dcd074ea0 Allow per-cluster setting of a docker image.
Add dockerImage cluster configuration parameter that overrides global
operator defaults when set to a non-empty value.
2017-11-14 11:53:04 +01:00
Oleksii Kliukin c25e849fe4 Fix a failure to create new statefulset at sync.
Also do a fmt run.
2017-11-08 18:24:17 +01:00
Murat Kabilov 86803406db
use sync methods while updating the cluster 2017-11-03 12:00:43 +01:00
Georg Kunz 47dd766fa7 Add node toleration config to PodSpec (#151)
* Add node toleration config to PodSpec

This allows to taint nodes dedicated to Postgres and prevents other pods from running on these nodes.

* Document taint and toleration setup

And remove setting from default operator ConfigMap

* Allow to overwrite tolerations with Postgres manifest
2017-11-02 19:10:44 +01:00
Oleksii Kliukin ce960e892a
Create new databases and change owners of existing ones during sync. (#153)
* Create new databases and change owners of existing ones during sync.
2017-11-02 17:46:33 +01:00
Oleksii Kliukin 7a76be7d3e Minor fixes around PDB (pod-distruption-budget) syncing: (#147)
- Call comparison function in the case of the sync as well as for update
- Include full cluster name in PDB name
- Assign cluster labels to the PDB object
2017-10-23 12:26:59 +02:00
Murat Kabilov c17aabb642 fix pod disruption budget labels (#146) 2017-10-20 15:01:51 +02:00
Murat Kabilov 661b141849 Fix Pod Disruption Budget null pointer exception 2017-10-20 11:43:50 +02:00
Murat Kabilov a1deae198b add missing master matchLabel for the PDB (#144) 2017-10-20 11:26:40 +02:00
Oleksii Kliukin eba23279c8 Kube cluster upgrade 2017-10-19 10:49:42 +02:00
Oleksii Kliukin 1dbf259c76 Retry opening DB connections. (#140)
Make sure DB connection retry also reopens a connection after closing it
2017-10-18 16:28:00 +02:00
Oleksii Kliukin 99870d8eac Fix division by zero when connecting to the DB.
Apparently the retry function's first parameter is the duration of
a single attempt and it cannot be zero.
2017-10-18 10:44:49 +02:00
Murat Kabilov 202f2de988 Retry connecting to pg 2017-10-17 17:03:50 +02:00
Murat Kabilov 6c4cb4e9da Perform manual failover during the scale down 2017-10-16 17:41:23 +02:00
Murat Kabilov 5b29576a8e Remove redundant constants 2017-10-16 15:52:48 +02:00
Murat Kabilov 3b32265258 Set status of the cluster on sync fail/success 2017-10-12 15:10:42 +02:00
Jan Mussler cec695d48e Superuser toggle for team members
Make superuser toggleable for team members. Add and "admin" role to team members if superuser is disabled.
2017-10-12 15:01:54 +02:00
Murat Kabilov 8d5faaa5a5 return idle status when worker has nothing to do 2017-10-11 15:42:20 +02:00
Oleksii Kliukin 793defef72 Fix pod wait timeouts.
Previously, a timer had been reset on every message received through
the pod channel.
2017-10-11 14:58:37 +02:00
Murat Kabilov 83c8d6c419 Extend diagnostic api with worker status info 2017-10-11 12:26:09 +02:00
Murat Kabilov 32aa7270e6 Use round-robin strategy while assigning workers 2017-10-09 16:56:27 +02:00
Murat Kabilov 71a540ff48 Merge branch 'master' into crd 2017-10-09 11:55:18 +02:00
Murat Kabilov 2f3bb1e265 set the proper name for the crd related constants file 2017-10-09 11:01:46 +02:00
Murat Kabilov a35e9c6119 move from tpr to crd 2017-10-06 15:12:08 +02:00