diff --git a/docs/reference/cluster_manifest.md b/docs/reference/cluster_manifest.md index 1b2d71a66..66867872c 100644 --- a/docs/reference/cluster_manifest.md +++ b/docs/reference/cluster_manifest.md @@ -473,3 +473,41 @@ Those parameters are grouped under the `tls` top-level key. relative to the "/tls/", which is mount path of the tls secret. If `caSecretName` is defined, the ca.crt path is relative to "/tlsca/", otherwise to the same "/tls/". + +## Change data capture streams + +This sections enables change data capture (CDC) streams e.g. into Zalando’s +distributed event broker [Nakadi](https://nakadi.io/). Parameters grouped +under the `streams` top-level key will be used by the operator to create a +CRD for Zalando's internal CDC operator. Each stream can have the following +properties: + +* **streamType** + Defines the sink. Either `nakadi`, `sqs` or `wal` (which is just plain wal + files). Required. + +* **database** + Name of the database from where events will be published via Postgres' + logical decoding feature. The operator will take care of updating the + database configuration (setting `wal_level: logical`, creating logical + replication slots, using output plugin `wal2json` and creating a dedicated + replication user). + +* **tables** + Defines a map of table names and event types. The CDC operator is following + the [outbox pattern](https://debezium.io/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/) + meaning changes are only consumed from an extra table that already has the + structure of the event in the target sink. The operator will assume that this + outbox table is called like `__outbox`. + +* **filter** + Streamed events can be filtered by a jsonpath expression for each table. + +* **batchSize** + Defines the size of batches in which events are consumed. + +* **sqsArn** + ARN to the SQS service used as event sink when streamType is `sqs`. + +* **queueName** + Name of the queue to be used in SQS service when streamType is `sqs`. diff --git a/hack/update-codegen.sh b/hack/update-codegen.sh index 18c71f83c..7043053ea 100755 --- a/hack/update-codegen.sh +++ b/hack/update-codegen.sh @@ -1,18 +1,13 @@ #!/usr/bin/env bash -set -eou pipefail -GOPKG="github.com/zalando/postgres-operator" -SCRIPT_ROOT="$(dirname "${BASH_SOURCE[0]}")/.." +set -o errexit +set -o nounset +set -o pipefail -rm -rf "${SCRIPT_ROOT}/generated" +SCRIPT_ROOT=$(dirname ${BASH_SOURCE})/.. +CODEGEN_PKG=${CODEGEN_PKG:-$(cd ${SCRIPT_ROOT}; ls -d -1 ./vendor/k8s.io/code-generator 2>/dev/null || echo ${GOPATH}/src/k8s.io/code-generator)} -go run k8s.io/code-generator/cmd/deepcopy-gen \ - --input-dirs ${GOPKG}/pkg/apis/acid.zalan.do/v1,${GOPKG}/pkg/apis/zalando.org/v1alpha1 \ - -O zz_generated.deepcopy \ - --bounding-dirs ${GOPKG}/pkg/apis \ - --go-header-file "${SCRIPT_ROOT}/hack/custom-boilerplate.go.txt" \ - -o "${SCRIPT_ROOT}/generated" - -cp -rv "${SCRIPT_ROOT}/generated/${GOPKG}"/* . - -rm -rf "${SCRIPT_ROOT}/generated" \ No newline at end of file +bash "${CODEGEN_PKG}/generate-groups.sh" all \ + github.com/zalando/postgres-operator/pkg/generated github.com/zalando/postgres-operator/pkg/apis \ + "acid.zalan.do:v1" \ + --go-header-file "${SCRIPT_ROOT}"/hack/custom-boilerplate.go.txt \ No newline at end of file