Sample logical backup image. (#572)

* Sample logical backup image.

Based on the earlier work by Dmitry Dolgov @erthalion

docker/logical-backup/Dockerfile

@@ -0,0 +1,33 @@
+FROM ubuntu:18.04
+LABEL maintainer="Team ACID @ Zalando <team-acid@zalando.de>"
+
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN apt-get update     \
+    && apt-get install --no-install-recommends -y \
+        apt-utils \
+        ca-certificates \
+        lsb-release \
+        pigz \
+        python3-pip \
+        python3-setuptools \
+        curl \
+        jq \
+        gnupg \
+    && pip3 install --no-cache-dir awscli --upgrade \
+    && echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list \
+    && cat /etc/apt/sources.list.d/pgdg.list \
+    && curl --silent https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
+    && apt-get update \
+    && apt-get install --no-install-recommends -y  \
+        postgresql-client-11  \
+        postgresql-client-10  \
+        postgresql-client-9.6 \
+        postgresql-client-9.5 \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY dump.sh ./
+
+ENV PG_DIR=/usr/lib/postgresql/
+
+ENTRYPOINT ["/dump.sh"]

docker/logical-backup/dump.sh

@@ -0,0 +1,94 @@
+#! /usr/bin/env bash
+
+# enable unofficial bash strict mode
+set -o errexit
+set -o nounset
+set -o pipefail
+IFS=$'\n\t'
+
+# make script trace visible via `kubectl logs`
+set -o xtrace 
+
+ALL_DB_SIZE_QUERY="select sum(pg_database_size(datname)::numeric) from pg_database;"
+PG_BIN=$PG_DIR/$PG_VERSION/bin
+DUMP_SIZE_COEFF=5
+
+TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
+K8S_API_URL=https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT/api/v1
+CERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
+
+function estimate_size {
+    "$PG_BIN"/psql -tqAc "${ALL_DB_SIZE_QUERY}"
+}
+
+function dump {
+    # settings are taken from the environment
+    "$PG_BIN"/pg_dumpall
+}
+
+function compress {
+    pigz
+}
+
+function aws_upload {
+    declare -r EXPECTED_SIZE="$1"
+
+    # mimic bucket setup from Spilo
+    # to keep logical backups at the same path as WAL
+    # NB: $LOGICAL_BACKUP_S3_BUCKET_SCOPE_SUFFIX already contains the leading "/" when set by the Postgres operator
+    PATH_TO_BACKUP=s3://$LOGICAL_BACKUP_S3_BUCKET"/spilo/"$SCOPE$LOGICAL_BACKUP_S3_BUCKET_SCOPE_SUFFIX"/logical_backups/"$(date +%s).sql.gz
+
+    if [ -z "$EXPECTED_SIZE" ]; then
+        aws s3 cp - "$PATH_TO_BACKUP" --debug --sse="AES256"
+    else
+        aws s3 cp - "$PATH_TO_BACKUP" --debug --expected-size "$EXPECTED_SIZE" --sse="AES256"
+    fi;
+}
+
+function get_pods {
+    declare -r SELECTOR="$1"
+
+    curl "${K8S_API_URL}/pods?$SELECTOR"        \
+        --cacert $CERT                          \
+        -H "Authorization: Bearer ${TOKEN}" | jq .items[].status.podIP -r
+}
+
+function get_current_pod {
+    curl "${K8S_API_URL}/pods?fieldSelector=metadata.name%3D${HOSTNAME}" \
+        --cacert $CERT                                                   \
+        -H "Authorization: Bearer ${TOKEN}"
+}
+
+declare -a search_strategy=(
+    list_all_replica_pods_current_node
+    list_all_replica_pods_any_node
+    get_master_pod
+)
+
+function list_all_replica_pods_current_node {
+    get_pods "labelSelector=version%3D${SCOPE},spilo-role%3Dreplica&fieldSelector=spec.nodeName%3D${CURRENT_NODENAME}" | head -n 1
+}
+
+function list_all_replica_pods_any_node {
+    get_pods "labelSelector=version%3D${SCOPE},spilo-role%3Dreplica" | head -n 1
+}
+
+function get_master_pod {
+    get_pods "labelSelector=version%3D${SCOPE},spilo-role%3Dmaster" | head -n 1
+}
+
+CURRENT_NODENAME=$(get_current_pod | jq .items[].spec.nodeName --raw-output)
+export CURRENT_NODENAME
+
+for search in "${search_strategy[@]}"; do
+
+    PGHOST=$(eval "$search")
+    export PGHOST
+
+    if [ -n "$PGHOST" ]; then
+        break
+    fi
+
+done
+
+dump | compress | aws_upload $(($(estimate_size) / DUMP_SIZE_COEFF))

docs/administrator.md

@@ -346,7 +346,7 @@ The operator logs reasons for a rolling update with the `info` level and a diff
 
 The operator can manage k8s cron jobs to run logical backups of Postgres clusters. The cron job periodically spawns a batch job that runs a single pod. The backup script within this pod's container can connect to a DB for a logical backup. The operator updates cron jobs during Sync if the job schedule changes; the job name acts as the job identifier. These jobs are to be enabled for each indvidual Postgres cluster by setting `enableLogicalBackup: true` in its manifest. Notes:
 
-1. The provided  `registry.opensource.zalan.do/acid/logical-backup` image implements the backup via `pg_dumpall` and upload of (compressed) results to an S3 bucket; `pg_dumpall` requires a `superuser` access to a DB and runs on the replica when possible.
+1. The [example image](../docker/logical-backup/Dockerfile) implements the backup via `pg_dumpall` and upload of compressed and encrypted results to an S3 bucket; the default image ``registry.opensource.zalan.do/acid/logical-backup`` is the same image built with the Zalando-internal CI pipeline. `pg_dumpall` requires a `superuser` access to a DB and runs on the replica when possible.  
 
 2. Due to the [limitation of Kubernetes cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) it is highly advisable to set up additional monitoring for this feature; such monitoring is outside of the scope of operator responsibilities. 
 

docs/reference/operator_parameters.md

@@ -487,7 +487,7 @@ scalyr sidecar. In the CRD-based configuration they are grouped under the
     Backup schedule in the cron format. Please take [the reference schedule format](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) into account. Default: "30 00 \* \* \*"
 
   * **logical_backup_docker_image**
-    Docker image for the pods of the cron job. Must implement backup logic and correctly handle pod and job restarts. The default image runs `pg_dumpall` (on a replica if possible) and uploads compressed results to an S3 bucket under the key `/spilo/pg_cluster_name/cluster_k8s_uuid/logical_backups` Default: "registry.opensource.zalan.do/acid/logical-backup" 
+    An image for pods of the logical backup job. The [example image](../../docker/logical-backup/Dockerfile) runs `pg_dumpall` on a replica if possible and uploads compressed results to an S3 bucket under the key `/spilo/pg_cluster_name/cluster_k8s_uuid/logical_backups`. The default image is the same image built with the Zalando-internal CI pipeline. Default: "registry.opensource.zalan.do/acid/logical-backup" 
 
   * **logical_backup_s3_bucket**
     S3 bucket to store backup results. The bucket has to be present and accessible by Postgres pods. Default: empty.