Commit Graph

277 Commits

Author SHA1 Message Date
Oleksii Kliukin 1f71c8d72f
Avoid reading the namespace file every time the NamespacedName value is decoded. (#243) 2018-02-20 17:41:11 +01:00
zerg-junior b0549c3c9c
Merge pull request #225 from zalando-incubator/support-many-namespaces
Support many namespaces
2018-02-20 17:39:42 +01:00
Sergey Dudoladov e048328d6a Comment on special values for watched namespace 2018-02-20 17:26:17 +01:00
Oleksii Kliukin 99c090899f
Change the suffix delimiter to slash. (#242)
This allows using S3 API in order to simplify finding all folders that are different only by a suffix, since the suffix delimiter will not occur in the suffix itself (currently being a UID).
2018-02-20 16:31:44 +01:00
Sergey Dudoladov 66a3b6830e Call fatalf if namespace to watch does not exist 2018-02-20 16:13:48 +01:00
Oleksii Kliukin c597377617
Use cluster UID as a suffix to the WAL bucket. (#211)
Avoid reusing WAL S3 buckets of the older cluster with the same name as the existing one.

For the new cluster, the S3 bucket name will include a suffix that is equal to the UID of the PostgreSQL object describing the cluster. That way, the bucket name will stay the same for all members iff  they correspond to the same PostgreSQL cluster object.

When "clone: uid:" key is present in the cluster manifest and the cluster is cloned from an S3 bucket (currently that happens if the endTimestamp is present in the clone description) the S3 bucket to clone from is suffixed with the -uid value.
2018-02-20 15:36:43 +01:00
Sergey Dudoladov dcfc9925f6 Respond to code review 2018-02-20 14:43:02 +01:00
Sergey Dudoladov b1fae716b1 Update REST API to require namespace when fetching info about particular cluster 2018-02-16 15:32:09 +01:00
Sergey Dudoladov e3d2434420 Use '*' as an alias to denote all namespaces 2018-02-16 15:20:26 +01:00
Sergey Dudoladov 088bf70e7d Merge branch 'master' into support-many-namespaces 2018-02-16 15:06:10 +01:00
Sergey Dudoladov ec7de38f9b Make operator watch its own namespace instead of controller's one 2018-02-16 14:22:38 +01:00
Sergey Dudoladov bbe2801d69 Adjust unit tests for namespace decoding 2018-02-16 11:13:08 +01:00
Sergey Dudoladov 5e9a21456e Remove the incorrect service account check 2018-02-15 16:33:53 +01:00
Sergey Dudoladov 155ae8d50f Rename the function that checks service account existence 2018-02-15 11:14:13 +01:00
Sergey Dudoladov d5d15b7546 Look for secrets in the deployed namespace 2018-02-14 15:37:30 +01:00
Sergey Dudoladov 06fd9e33f5 Watch the namespace where operator deploys to unless told otherwise 2018-02-13 18:17:47 +01:00
Sergey Dudoladov 4c23917d42 Watch all namespaces if the relevant param is empty string / 'default' if param is unset 2018-02-12 11:47:56 +01:00
Sergey Dudoladov 066f11cbbd Streamline handling of the watched_namespace param/envvar 2018-02-09 11:39:56 +01:00
Sergey Dudoladov b5b0b027f2 Handle watched namespace set in operator config map 2018-02-08 14:51:45 +01:00
Sergey Dudoladov 86807d21ba Kill operator if the namespace to watch does not exist 2018-02-08 14:24:47 +01:00
Sergey Dudoladov 794feee3e1 Fix the bug with the operator always listening to all namespaces 2018-02-08 13:49:44 +01:00
Dmitrii Dolgov 4c1db33c27 Change the order of arguments 2018-02-08 10:43:27 +01:00
Sergey Dudoladov de2a028592 Warn if the watched namespace does not exist 2018-02-07 17:43:05 +01:00
Dmitrii Dolgov dd79fcd036 Tests for retry_utils
One can argue about how necessary they are,
but at least I remembered how to do golang.
2018-02-07 17:04:43 +01:00
Sergey Dudoladov 74fa7b9492 Restrict operator to single watched namespace via env var 2018-02-07 16:44:49 +01:00
Sergey Dudoladov f194a2ae5a Introduce changes from the PR #200 by @alexeyklyukin 2018-02-07 14:02:32 +01:00
Sergey Dudoladov 74a1e9661b Remove setting the actual watched namespace as env var (os.Setenv won't work) 2018-02-06 17:40:06 +01:00
Sergey Dudoladov 8b7bbde06e Make env var overwrite configmap setting for watching namespaces 2018-02-06 16:12:47 +01:00
Sergey Dudoladov ea84f9d577 Rename the configmap 'namespace' entry to avoid confusion with the map's owm namespace 2018-02-06 15:09:00 +01:00
Oleksii Kliukin b90a36c909
Set node_readiness_label default to an empty value. (#204)
Previously, it was set to the lifecycle-status:ready, breaking a
lot of minikube deployments. Also it was not possible befor to run
with this label set to an empty value.

Document the effect of the label in the new section of the
documentation.
2018-01-16 15:43:03 +01:00
Manuel Gómez bf4406d2a4 Consider container names in Statefulset diffs (#210)
This includes a comparison on container names being equal in the
decision of whether a Statefulset has been updated.
2018-01-16 12:06:11 +01:00
Sergey Dudoladov ec6799f34a Overwrite scalyr api key if the relevant env variable is present in the operator pod 2018-01-12 14:56:14 +01:00
Oleksii Kliukin 23011bdf9a
Migrate only master pods. Migrate single masters. (#199)
Avoid migrating replica pods, since they will be handled by the
node draining anyway (the PDB specifies that only masters are to
be kept).

Allow migration of the single-pod clusters.
2018-01-09 11:55:11 +01:00
zerg-junior bb5ce6cbbe
Merge pull request #195 from zalando-incubator/databases-rest-endpoint
Add a REST endpoint to list databases in all clusters
2018-01-09 11:53:32 +01:00
Oleksii Kliukin 8e99518eeb
Improve behavior on node decomissionining (#184)
* Trigger the node migration on the lack of  the readiness label.

* Examine the node's readiness status on node add.

Make sure we don't miss the not ready node, especially when the
operator is killed during the migration.
2018-01-04 11:53:15 +01:00
Manuel Gómez 1109cfa7a1
Add PostgreSQL pod namespace Scalyr sidecar environment (#196)
Another tiny bit of information that could be useful for log filters
once we start deploying clusters into separate namespaces.
2017-12-22 17:12:50 +01:00
Oleksii Kliukin 5c8bd04169 Sort database by name. 2017-12-22 15:48:13 +01:00
Oleksii Kliukin 6102b0368c Merge remote-tracking branch 'origin/databases-rest-endpoint' into databases-rest-endpoint
# Conflicts:
#	pkg/apiserver/apiserver.go
#	pkg/controller/status.go
2017-12-22 13:08:50 +01:00
Oleksii Kliukin 9720ac1f7e WIP: Hold the proper locks while examining the list of databases.
Introduce a new lock called specMu lock to protect the cluster spec.
This lock is held on update and sync, and when retrieving the spec in
the API code. There is no need to acquire it for cluster creation and
deletion: creation assigns the spec to the cluster before linking it to
the controller, and deletion just removes the cluster from the list in
the controller, both holding the global clustersMu Lock.
2017-12-22 13:06:11 +01:00
Sergey Dudoladov b8bf97ab76 Integrate comments from code reviews 2017-12-22 12:53:57 +01:00
Sergey Dudoladov 011458fb05 Add a REST endpoint to list databases in all clusters 2017-12-21 17:28:55 +01:00
Manuel Gómez cd9bc7bdc5
Add PostgreSQL pod name Scalyr sidecar environment (#194)
This will allow the Scalyr image to add a custom attribute to shipped
log entries that notes the name of the originating pod.
2017-12-21 16:52:27 +01:00
Manuel Gómez 15c278d4e8
Scalyr agent sidecar for log shipping (#190)
* Scalyr agent sidecar for log shipping

* Remove the default for the Scalyr image

Now the image needs to be specified explicitly to enable log shipping to
Scalyr.  This removes the problem of having to generate the config file
or publish our agent image repository.

* Add configuration variable for Scalyr server URL

Defaults to the EU address.

* Alter style

Newlines are cheap and make code easier to edit/refactor, but ok.

* Fix StatefulSet comparison logic

I broke it when I made the comparison consider all containers in the
PostgreSQL pod.
2017-12-21 15:34:26 +01:00
Oleksii Kliukin da0de8cff7
Make sure the statefulset that is deleted manually gets re-created. (#191)
* Make sure the statefulset that is deleted manually gets re-created.

Per report and analysis by Manuel Gomez.

* Move the existence checks for other objects out of the Create functions.

create{Object} for services, endpoints and PDBs refused to continue if
there is a cached definition in the cluster, however, the only place
where it makes sense is when creating a new cluster. Note that contrary
to the statefulset this doesn't fix any issues, since those definitions
were nullified correspondingly when the sync code detected there is no
object present in the Kubernetes cluster.
2017-12-21 15:20:43 +01:00
zerg-junior 3c178f68df Warn on infrastructure-roles.yaml format violations (#177)
Emit a warning if there are unprocessed entries in the infrastructure-roles secret.
2017-12-15 17:21:41 +01:00
zerg-junior 5d5fa680a3
Merge pull request #180 from zalando-incubator/container-name
Make pod's single container name static
2017-12-15 16:13:33 +01:00
Oleksii Kliukin bf80f5225e
Introduce higher and lower bounds for the number of instances (#178)
* Introduce higher and lower bounds for the number of instances

Reduce the number of instances to the min_instances if it is lower and
to the max_instances if it is higher. -1 for either of those means there
is no lower or upper bound.

In addition, terminate the operator when there is a nonsense in the
configuration (i.e. max_instances < min_instances).

Reviewed by Jan Mußler and Sergey Dudoladov.
2017-12-15 16:02:50 +01:00
Sergey Dudoladov 52e358ba8f Make pod's single container name static 2017-12-15 15:53:53 +01:00
Oleksii Kliukin 0e255f82c6 Provide more information about variable conflicts.
They are mentioned in the documentation and the operator will emit a
warning each time the variable from the pod environment configmap is
ignored because the same variable is defined by the operator.

Some minor changes in the variable names to make the code more readable.

Per review from Sergey Dudoladov.
2017-12-14 14:39:33 +01:00
Oleksii Kliukin da4b66210a Expand variables from the PodEnvironmentConfigMap (#4)
Inject PodEnvironmentConfigMap variables inline into the
statefulset definition in order to be able to figure out
changes to the statefulset when only PodEnvironmentConfigMap
has changed.
2017-12-14 14:39:33 +01:00